Shulou(Shulou.com)05/31 Report--

Editor to share with you what RDB and AOP persistence in redis is, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Redis is an in-memory database, and the data is stored in memory, but we all know that the data in memory changes quickly and is easy to be lost. Fortunately, Redis also provides us with persistence mechanisms, namely RDB (Redis DataBase) and AOF (Append Only File).

Assuming that you already know the basic syntax of redis, there are good tutorials on some alphabet website that you can see. Basic use of the article will not be written, are some commonly used commands.

I. persistence process

Now that redis data can be saved on disk, what is the process like?

There are five processes:

(1) the client sends a write operation to the server (the data is in the client's memory).

(2) the database server receives the data of the write request (the data is in the memory of the server).

(3) the server calls the system call write to write the data to disk (the data is in the buffer of the system memory).

(4) the operating system transfers the data in the buffer to the disk controller (the data is in the disk cache).

(5) the disk controller writes the data to the physical media of the disk (the data actually falls on the disk).

These five processes are a normal save process under ideal conditions, but in most cases, our machines and so on will have a variety of failures, which are divided into two cases:

(1) if the Redis database fails, as long as the third step above is completed, it can be persisted, and the remaining two steps will be completed by the operating system for us.

(2) if there is a failure of the operating system, all the above five steps must be completed.

Here only consider the failure that may occur in the preservation process, in fact, the saved data may also be damaged, requiring a certain recovery mechanism, but it is no longer extended here. The main consideration now is how redis implements the above five steps to save the disk. It provides two policy mechanisms, namely RDB and AOF.

II. RDB mechanism

RDB actually stores data on disk in the form of snapshots. What is a snapshot? you can understand it as taking a picture of the data at the current moment and saving it.

RDB persistence refers to writing a snapshot of a dataset in memory to disk within a specified time interval. It is also the default persistence method, which is to write in-memory data to a binary file as a snapshot, and the default file name is dump.rdb.

After we install redis, all the configurations are in the redis.conf file, which stores the various configurations of the two persistence mechanisms, RDB and AOF.

Since the RDB mechanism is saved by generating a snapshot of all the data at some point, there should be a trigger mechanism to implement this process. For RDB, three mechanisms are provided: save, bgsave, and automation. Let's take a look at it separately.

1. Save trigger mode

This command blocks the current Redis server, and during the execution of the save command, Redis cannot process other commands until the RDB process is complete. The specific process is as follows:

If there is an old RDB file when the execution is complete, replace the old one with the new one. Our clients may be tens of thousands or hundreds of thousands, which is obviously not desirable.

2. Bgsave trigger mode

When executing this command, Redis performs snapshot operations asynchronously in the background, and snapshots can also respond to client requests. The specific process is as follows:

The specific operation is that the Redis process executes the fork operation to create the child process, and the RDB persistence process is responsible for the child process, which ends automatically after completion. Blocking occurs only in the fork phase, usually for a short time. Basically all RDB operations within Redis use the bgsave command.

3. Automatic trigger

The automatic trigger is done by our configuration file. In the redis.conf configuration file, there are the following configurations, which we can set:

① save: this is used to configure the RDB persistence condition that triggers Redis, that is, when the data in memory is saved to the hard disk. Such as "save m n". Indicates that bgsave is automatically triggered when there are n modifications to the dataset within m seconds.

The default configuration is as follows:

# means that if there is a change in the value of at least one key within 900s, then saving save 9000s means that if there is a change in the value of at least 10 key within 300s, then saving save 300 means that if there is a change in the value of at least 10000 key in 60 seconds, save save 60 10000

No persistence is required, so you can comment out all save lines to disable the save feature.

② stop-writes-on-bgsave-error: the default is yes. Whether Redis stops receiving data when RDB is enabled and the last time the data is saved in the background fails. This will make the user realize that the data is not correctly persisted to disk, otherwise no one will notice that a disaster has occurred. If Redis is restarted, you can start receiving data again.

③ rdbcompression; the default is yes. For snapshots stored on disk, you can set whether to compress storage.

④ rdbchecksum: the default is yes. "after storing the snapshot, we can also have redis use the CRC64 algorithm for data validation, but doing so increases performance consumption by about 10%, which can be turned off if you want to get the maximum performance improvement."

⑤ dbfilename: sets the file name of the snapshot. The default is dump.rdb.

⑥ dir: sets the path where snapshot files are stored. This configuration item must be a directory, not a file name.

We can modify these configurations to achieve the effect we want. Because the third method is configured, let's make a comparison between the first two:

4. Advantages and disadvantages of RDB

①, advantage

(1) RDB files are compact and fully backed up, which is very suitable for backup and disaster recovery.

(2) when generating the RDB file, the fork main process will fork () a child process to handle all the save work, the main process does not need to do any disk IO operation.

(3) the recovery speed of RDB is faster than that of AOF when recovering large datasets.

②, disadvantage

RDB snapshots are full backups that store binary serialization of in-memory data and are very compact in storage. When snapshot persistence is performed, a child process is opened to take charge of snapshot persistence. The child process owns the memory data of the parent process, and the parent process modifies the memory child process without reflecting it. Therefore, the modified data during snapshot persistence will not be saved, and data may be lost.

III. AOF mechanism

Full backup is always time-consuming, and sometimes we provide a more efficient way to AOF, the working mechanism is very simple, redis will append every write command received to the file through the write function. The popular understanding is logging.

1. Persistence principle

His principle is shown in the following picture:

Whenever a write command comes over, it is saved directly in our AOF file.

2. The principle of file rewriting

The AOF approach also brings another problem. Persistent files will get bigger and bigger. To compress the persistence file of aof. Redis provides the bgrewriteaof command. Save the data in memory to a temporary file as a command, and fork a new process to rewrite the file.

The operation of rewriting the aof file does not read the old aof file, but rewrites the entire in-memory database contents to a new aof file as a command, which is a bit similar to a snapshot.

3. AOF also has three triggering mechanisms.

(1) synchronous always for every modification: synchronous persistence every time data changes are immediately recorded to disk with poor performance but good data integrity

(2) synchronous everysec per second: asynchronous operation. Record per second if downtime occurs within one second, data loss occurs.

(3) different no: never synchronized

4. Advantages

(1) AOF can better protect data from loss. Generally, AOF performs a fsync operation through a background thread every 1 second, and the data is lost for up to 1 second. (2) the AOF log file does not have any disk addressing overhead, the write performance is very high, and the file is not easily damaged.

(3) even if the AOF log file is too large, the background rewrite operation will not affect the read and write of the client.

(4) AOF log file commands are recorded in a very readable manner, which is very suitable for emergency recovery of catastrophic erroneous deletions. For example, if someone accidentally clears all the data with the flushall command, as long as the background rewrite has not occurred at this time, you can immediately copy the AOF file, delete the last flushall command, and then put the AOF file back. All the data can be recovered automatically through the recovery mechanism.

5. Shortcomings

(1) for the same data, AOF log files are usually larger than RDB data snapshot files.

(2) when AOF is enabled, the supported write QPS is lower than the write QPS supported by RDB, because AOF is generally configured to fsync log files once per second. Of course, the performance of fsync per second is still very high.

(3) bug has occurred in AOF before, that is, when the data is recovered through the log recorded by AOF, the exact same data is not recovered.

4. How to choose RDB and AOF

If you choose, it is better to add the two together. Because you understand the two persistence mechanisms, the rest depends on your own needs, and different needs may not be chosen, but they are usually used in combination. There is a picture to sum up:

After comparing these features, the rest is up to you.

This is all the content of the article "what is RDB and AOP persistence in redis?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.