Shulou(Shulou.com)06/02 Report--

In this issue, the editor will bring you about how to achieve persistence in redis. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

Two ways of redis persistence

The RDB:RDB persistence mechanism is to perform periodic persistence on the data in redis.

The AOF:AOF mechanism writes each write command as a log and writes to a log file in append-only mode. When redis restarts, the entire dataset can be reconstructed by playing back the write instructions in the AOF log.

Through RDB or AOF, the data in redis memory can be persisted to disk, and then the data can be backed up to other places, such as Aliyun and other cloud services.

If the redis fails, the memory on the server and the data on the disk are lost. You can copy the previous data from the cloud service and put it in the specified directory, and then restart redis,redis. According to the data in the persistent data file, you will automatically recover the data in memory and continue to provide services.

If both RDB and AOF persistence mechanisms are used, AOF will be used to rebuild the data when redis is restarted, because the data in AOF is more complete.

Advantages and disadvantages of RDB

RDB generates multiple data files, and each data file represents the data of redis at a certain time. This method of multiple data files is very suitable for cold backup. You can send this complete data file to some remote security storage, such as Amazon's S3 cloud service. In China, it can be Aliyun's ODPS distributed storage, and regularly back up the data in redis with a predetermined backup strategy.

RDB has little impact on the read and write service provided by redis, which allows redis to maintain high performance, because the main process of redis only needs to fork one child process, and the child process can perform disk IO operation to persist RDB.

Compared with the AOF persistence mechanism, it is faster to restart and resume redis processes directly based on RDB data files.

If you want to lose as little data as possible in the event of redis failure, RDB is not as good as AOF. Generally speaking, RDB data snapshot files are generated every 5 minutes or more, so you have to accept that if the redis process goes down, the last 5 minutes of data will be lost.

Every time RDB performs RDB snapshot data file generation in the fork child process, if the data file is particularly large, it may cause the service provided by the client to be paused for milliseconds, or even seconds.

Advantages and disadvantages of AOF

AOF can better protect data from loss. Generally, AOF will perform a fsync operation through a background thread every 1 second, losing data for up to 1 second.

AOF log files are written in append-only mode, so there is no disk addressing overhead, write performance is very high, and the file is not easily damaged, even if the tail of the file is broken, it is easy to repair.

Even if the AOF log file is too large, the background rewrite operation will not affect the read and write of the client. Because in rewrite log, the instructions are compressed to create a minimum log that needs to recover data. When you create a new log file, the old log file is written as usual. When the log file ready after the new merge, you can exchange the new and old log files.

The commands for AOF log files are recorded in a very readable manner, which is ideal for emergency recovery of catastrophic erroneous deletions. For example, if someone accidentally clears all the data with the flushall command, as long as the background rewrite has not occurred at this time, you can immediately copy the AOF file, delete the last flushall command, and then put the AOF file back, and all data can be recovered automatically through the recovery mechanism.

AOF log files are usually larger than RDB data snapshot files for the same piece of data.

When AOF is enabled, the supported write QPS is lower than the write QPS supported by RDB, because AOF is generally configured to fsync log files once per second. Of course, the performance of fsync per second is still very high. (if you write in real time, then QPS will be greatly degraded and redis performance will be greatly degraded)

In the past, bug occurred in AOF, that is, through the log recorded by AOF, when the data was recovered, the exact same data was not recovered. Therefore, the more complex command log / merge/ playback method such as AOF is more fragile and easier to have bug than the way based on RDB to persist a complete data snapshot file at a time. However, AOF is to avoid the bug caused by the rewrite process, so each rewrite is not based on the old instruction log for merge, but based on the data in memory at that time to rebuild instructions, so the robustness will be much better.

How exactly should RDB and AOF choose

Don't just use RDB, because that will cause you to lose a lot of data.

Do not just use AOF, because there are two problems, first, you do cold backup through AOF, there is no RDB to do cold backup, the recovery speed is faster; second, RDB generates data snapshots each time simply and rudely, which is more robust and can avoid the bug of AOF, a complex backup and recovery mechanism.

Redis supports enabling two persistence modes at the same time. We can combine AOF and RDB persistence mechanisms, use AOF to ensure that data is not lost as the first choice for data recovery, and use RDB for cold backup to varying degrees. When AOF files are lost or corrupted, you can also use RDB for fast data recovery.

This is how to achieve persistence in the redis shared by the editor. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.