Shulou(Shulou.com)06/01 Report--

preface

Redis is an advanced key-value database. It is similar to memcached, but data can be persisted and the supported data types are rich. There are strings, lists, sets and ordered sets. Support server-side computation of set union, intersection and complement (difference), etc., but also support a variety of sorting functions. Redis can also be considered a data structure server.

All data in Redis is stored in memory and then asynchronously saved to disk from time to time (this is called "semi-persistent mode"); each data change can also be written to an append only file(aof)(this is called "fully persistent mode").

Life lies in the toss series, network, multi-thread series blog owners will continue to toss will not give up. This is a brand-new series, cache knowledge is not just a simple addition and deletion to check, I think it is necessary to comprehensive in-depth study of a wave. Record the learning process and understanding.

RDB

What is RDB?

Performs periodic persistence of data in redis by setting the check interval and backup trigger conditions in the configuration file

The advantages of RDB persistence

RDB will generate multiple data files, each data file represents a certain time in redis data, this multiple data file way, very suitable for cold backup. RDB has very little impact on the read-write services provided by redis, which can keep redis high performance, because redis main process only needs to fork a child process, and let the child process execute disk IO operations to carry out RDB persistence. Compared with AOF persistence mechanism, it is faster to restart and recover redis process directly based on RDB data file.

Disadvantages of RDB Persistence

RDB is not as good as AOF if you want to lose as little data as possible when redis fails. Generally speaking, RDB data snapshot files are generated every 5 minutes or longer, and at this time, you have to accept that once the redis process goes down, the last 5 minutes of data will be lost. This problem is also the biggest disadvantage of rdb, that is, it is not suitable for the first priority recovery scheme. If you rely on RDB to do the first priority recovery scheme, it will lead to more data loss. When RDB snapshot data file generation is performed in fork subprocess, if the data file is particularly large, it may cause the service provided by the client to be suspended for several milliseconds, or even several seconds. Generally, do not let the RDB interval be too long, otherwise the RDB file generated each time is too large. It may affect the performance of redis itself.

How to configure redis RDB persistence

redis.conf file to configure persistence

save 60 1000

Every 60s, if more than 1000 keys have changed, a new dump.rdb file is generated, which is a complete snapshot of the data in the current redis memory. This operation is also called snapshotting.

You can also manually invoke the save or bgsave commands to perform rdb snapshot generation synchronously or asynchronously. (save will block redis main thread while generating dump.rdb file, bgsave will not block redis main thread)

save can set multiple snapshotting checkpoints. At each checkpoint, it will check whether the specified number of keys has changed. If so, it will generate a new dump.rdb file.

AOF

What AOF?

The AOF mechanism records each write command as a log, writing it to a log file in append-only mode. When redis restarts, the entire dataset can be reconstructed by playing back the write command in the AOF log.

Advantages of AOF Persistence

AOF can better protect data from loss. Generally, AOF will perform an fsync operation every 1 second through a background thread (the function of fsync is to ensure that all modified contents have been correctly synchronized to the hard disk. This call will block and wait until the device reports IO completion). At most 1 second of data loss Every 1 second, perform an fsync operation to ensure that the data in oscache is written to disk. The redis process hangs and at most 1 second of data is lost. AOF log files are written in append-only mode, so there is no overhead of disk addressing, write performance is very high, and files are not easily damaged, even if the end of the file is damaged, it is easy to repair. AOF log file even if too large, background rewrite operations occur, will not affect the client read and write. Because when rewritelog, the instructions are compressed to create a minimum log that needs to recover data. When new log files are created, old log files are written as usual. When the new merged log file is ready, swap the old log file with the new one. AOF log file commands are recorded in a very readable manner, a feature that is ideal for emergency recovery from catastrophic erasures. For example, someone accidentally used the flushall command to clear all the data. As long as the background rewrite has not occurred at this time, you can immediately copy the AOF file, delete the last flushall command, and then put the AOF file back. You can automatically recover all the data through the recovery mechanism.

Disadvantages of AOF persistence mechanism

AOF log files are typically larger than RDB snapshot files for the same piece of data. When AOF is turned on, the write QPS supported will be lower than the write QPS supported by RDB, because AOF is generally configured to fsync log files once per second. Although fsync once per second, the performance is still very high, if you want to ensure that not a piece of data is lost, it is OK, AOF fsync is set to not write a piece of data, fsync once, it is finished, redis QPS will be lower. In the past, AOF had a bug, that is, when data recovery was carried out through the logs recorded by AOF, the exact data was not recovered. Therefore, the more complex method based on command log/merge/playback like AOF is more fragile and buggy than the method based on RDB persisting a complete data snapshot file at a time. However, AOF is to avoid bugs caused by the rewrite process, so each rewrite is not based on the old instruction log for merge, but based on the data in memory at that time for instruction reconstruction, so the robustness is much better. The only big disadvantage is that it will be slower when doing data recovery, and it is not convenient to do cold backup and regular backup. It may be necessary to hand-write complex scripts to do it. It is not appropriate to do cold backup. RDB recovery log, is a data file, recovery time, directly loaded into memory. AOF is different. When data recovery is done, it is actually necessary to replay and execute all the instruction logs to recover all the data in memory.

How to configure redis AOF persistence

AOF persistence, default is off, default is on RDB persistence

appendonly yes, you can open the AOF persistence mechanism, in the production environment, AOF is generally open, unless you say that it doesn't matter if you lose a few minutes of data. After turning on AOF persistence mechanism, redis will write to the log file every time it receives a write command, of course, write to os cache first, and then fsync at regular intervals.

If both AOF and RDB are enabled, when redis restarts, priority is given to data recovery through AOF, because aof data is relatively complete.

You can configure the fsync policy of AOF. There are three policies to choose from:

always: Every time you write a piece of data, immediately fsync the write log corresponding to this data to the disk. The performance is very, very poor, and the throughput is very low. Make sure that none of the data in redis is lost. That's the only way to do it. everysec: fsync the data in os cache to the disk every second. This is the most commonly used configuration. The production environment is generally configured like this. The performance is very high. QPS can still reach tens of thousands of no: Just redis is responsible for writing data to os cache and then let go, and then os itself will have its own strategy from time to time to brush data to disk, uncontrollable

summary

The above is all the content of this article, I hope the content of this article for everyone's study or work has a certain reference learning value, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.