Shulou(Shulou.com)06/02 Report--

Redis's AOF persistence strategy is to record every command sent to the redis server and save it to the AOF file on the hard disk, similar to typing a log file to record one command.

AOF Settin

The location of the AOF file is the same as that of the RDB file, both of which are set by the dir parameter. The default file name is appendonly.aof, which can be modified by the appendfilename parameter.

AOF test

When the client sends some redis commands to the server, Redis records the executed commands to the aof file, as follows:

When the redis server is restarted, the aof file will be executed for the purpose of data recovery.

AOF file rewriting

Why rewrite it? Rewriting removes the intermediate execution of the data and retains the final data command directly.

Take Chestnut: for example, a series of commands were executed on key on the redis client.

Set count 1 / / initial value is 1

Incr count / / plus 1

Incr count / / plus 1

Decr count / / minus 1

At this time, the result is

Get count

"2"

If there is no AOF rewrite, when the AOF file is restored, Redis executes every command in the AOF file and ends up with a count of 2.

After AOF rewriting, it is equivalent to omitting the intermediate calculation process. Setting the calculated results directly into redis is equivalent to executing only one command, set count 2.

You can use the BGREWRITEAOF command to overwrite the AOF file.

Rewriting strategy

Parameter settings for override policy:

Auto-aof-rewrite-percentage 100

When the current AOF file size exceeds how many percent of the AOF file size at the time of the last override, it is rewritten again, based on the AOF file size at startup if it has not been rewritten before.

Auto-aof-rewrite-min-size 64mb

The minimum AOF file size allowed for rewriting is limited, and usually when the AOF file is small, even some redundant commands can be ignored.

Redis's excellent performance is due to the fact that it stores all its data in memory, as does memcached, but why redis stands out is largely due to Redis's excellent persistence mechanism to ensure that data is not lost when the server is restarted. Let's look at how Redis is persisted.

Redis supports persistence in two ways, one is RDB, and the other is AOF. These two methods can be used alone or in combination.

Introduction of RDB mode

RDB is done through snapshots. When certain conditions are met, Redis will automatically snapshot all data in memory and store them on the hard disk. Just like taking a picture, save everything in this moment. The conditions for taking snapshots are specified in the configuration file. There are two main parameters: time and the number of keys changed, that is, when the number of keys changed within a specified time is greater than the execution value, a snapshot will be taken. RDB is the default persistence method for Redis.

RDB mode configuration

Find the configuration file for Redis: redis.conf

1) set trigger conditions:

2) set the rdb file path

The default rdb file storage path is the current directory, and the file name is: dump.rdb. You can modify the path and filename in the configuration file, dir and dbfilename, respectively.

After Redis starts, it will read the RDB snapshot file and load the data from the hard disk into memory. Normally, the 1GB snapshot file will be loaded into memory in about 20-30 minutes.

How to take snapshots by RDB

Snapshot process for RDB:

1) Redis uses the fork function to make a copy of the current process (parent process)

2) the parent process continues to accept and process commands from the client, while the child process begins to write data in memory to temporary files on the hard disk.

3) when the child process has finished writing all the data, it will replace the old RDB file with the temporary file.

Manual snapshot:

If automatic snapshots are not triggered, you can perform manual snapshot operations on redis, and both SAVE and BGSAVE can perform manual snapshots. The difference between the two commands is that the former is performed by the main process, which blocks other requests, while the latter is performed by the help child process.

Note:

Because redis uses fork to copy a copy of the current process, then the child process will occupy the same memory resources as the main process, such as 8 GB of memory for the main process, so you must ensure 16 GB of memory at the time of backup, otherwise virtual memory will be enabled and the performance is very poor.

Compression of RDB files

When the RDB file is too large, it can be compressed. Redis enables compression by default. Of course, you can also disable compression by configuring the rdbcompression parameter.

Advantages and disadvantages of compression and non-compression:

Compression:

Advantages: reduced disk storage space

Disadvantages: consuming CPU resources

Do not compress:

Advantages: no consumption of CPU resources

Disadvantages: taking up a lot of disk space

Advantages of RDB

A RDB is a compact file that represents Redis data at a point in time. RDB files are suitable for backup. For example, you might want to archive RDB files for the last 24 hours every hour and save RDB snapshots for nearly 30 days a day. This allows you to easily recover different versions of datasets for disaster recovery.

RDB is ideal for disaster recovery as a compact single file that can be transferred to a remote data center or Amazon S3 (which may have to be encrypted).

RDB maximizes the performance of Redis, because the only thing that needs to be done when the Redis parent process persists is to fork a child process, which does all the remaining work. The parent process instance does not need to perform operations such as disk IO.

RDB is faster than AOF when restarting an instance that holds a large dataset.

Shortcomings of RDB

RDB may not be good when you need to minimize data loss when Redis stops working (such as a power outage). You can configure different Savepoints. However, you usually take a RDB snapshot every 5 minutes or more, so once Redis stops working for any reason that it doesn't shut down correctly, you have to be prepared for data loss in the next few minutes.

RDB needs to call the fork () child process frequently to persist to disk. If the dataset is large, fork () is time-consuming, and as a result, when the dataset is very large and CPU performance is not powerful enough, Redis stops serving the client for milliseconds or even seconds. AOF also requires fork (), but you can adjust how often to rewrite the log without compromising trade-off (durability).

Advantages of AOF

Using AOF Redis is more durable: you can have many different fsync strategies: no fsync, fsync per second, fsync on each request. With the default fsync per second policy, write performance is still good (fsync is done by background threads, and the main thread continues to work hard to execute write requests), even if you only lose one second of write data.

The AOF log is an appended file, so there is no need to locate and there is no corruption problem in the event of a power outage. Even if for some reason the end of the file is a halfway-written command (disk full or otherwise), the redis-check-aof tool can easily fix it.

The AOF file contains one operation after another, stored in a format that is easy to understand and parse. You can also easily export an AOF file. For example, even if you accidentally use the FLUSHALL command to clear everything, if there is no rewrite at this time, you can still save your dataset, you can just stop the server, delete the last command, and then restart Redis.

Shortcomings of AOF

For the same dataset, AOF files are usually larger than equivalent RDB files.

AOF may be slower than RDB, depending on the exact fsync policy. Usually fsync is set to once per second, performance is still very high, if fsync is turned off, it is as fast as RDB even under high load. However, even in the case of a large write load, RDB can provide maximum latency guarantee.

How to choose between RDB and AOF

In general, you should use both persistence methods to achieve the same level of data security as PostgreSQL provides.

If you are concerned about your data, but can still accept a few minutes of data loss in the event of a disaster, you can use RDB alone.

There are many users using AOF alone, but we do not encourage this, because frequent RDB snapshots are very convenient for database backups, start quickly, and avoid the bug of the AOF engine.

How to choose? That depends on the demand and the server resources.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.