Shulou(Shulou.com)05/31 Report--

This article focuses on "what is the method of Redis persistence". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn "what is the method of Redis persistence"?

RDB persistence Redis supports both RDB and AOF persistence mechanisms. Persistence can avoid data loss caused by abnormal exit of the process or downmachine, and data recovery can be achieved by using the previous persistence file in the next restart.

RDB persistence is to persist by creating snapshots (compressed binaries) to save all the data at a certain point in time. RDB persistence is the default persistence method for Redis. The trigger of RDB persistence includes manual trigger and automatic trigger.

Manual trigger

Save, executing the save command on the command line, will create a rdb file saving snapshot synchronously, which will block the main process of the server. Do not use the

Bgsave, which executes the bgsave command on the command line, creates the rdb file to save the snapshot asynchronously through a child process of fork. Except for fork blocking, when the child process creates the rdb file, the main process can continue to process the request

Automatic trigger

Configure save m n timing trigger in redis.conf. For example, save 9001 means that it will be triggered if there is at least one update within 900s.

In the case of master-slave replication, if the slave node performs a full copy operation, the master node automatically executes bgsave to generate RDB files and sends them to the slave node

When you execute the debug reload command to reload Redis

Execute shutdown with AOF persistence not enabled

RDB persistence configuration in redis.conf

# the bgsave command save 900 1 # is executed as long as one of the following conditions is met. # there is at least one write operation in 900s save 300 10save 60 1000 disable RBD persistence, and you can add save "" # whether the main process stops writing when the backup process goes wrong, whether the stop-writes-on-bgsave-error yes# file is compressed or not it is recommended that no is more expensive than hard disk cost cpu resources rdbcompression no

AOF persistence

AOF (Append-Only-File) persistence is an instruction to record all changes to the state of the database, which is appended to the AOF file in the form of append. The next time the server starts, you can restore the database state before the server shutdown by loading and executing the commands saved in the AOF file.

The AOF persistence configuration in redis.conf is as follows

# disable AOF by default. To change no to the name of the yesappendonly no# append file appendfilename "appendonly.aof" # write the contents of the cache to the file by default every other second. Appendfsync everysec# automatically enables rewriting when the growth rate of the AOF file size is greater than this configuration item (here, it means more than 100% of the original size). Auto-aof-rewrite-percentage 10 automatically turns on rewriting auto-aof-rewrite-min-size 64mb when the AOF file size is larger than this configuration item

The implementation of AOF persistence consists of three steps:

Command append: appends a command to the AOF buffer

File write: buffer contents are written to AOF file

File saving: saving AOF files to disk

The frequency of the last two steps is configured through appendfsync, and the options for appendfsync include

Always, which is saved every time a command is executed, has the highest security, losing data from at most one command, but also the lowest performance (frequent disk IO)

Everysec, saved every second, is recommended to make a compromise between security and performance, losing up to one second of data.

No, which relies on the operating system to execute (usually once every 30 seconds), has the lowest security and the highest performance, and loses the data after the operating system triggered the SAVE operation on the AOF file for the last time.

AOF persists by saving commands. With the passage of time, AOF files become larger and larger. Redis uses AOF file rewriting to solve the growing problem of AOF files (it can reduce the disk occupation of files and speed up data recovery). The principle is as follows:

Call fork to create a child process

The child process reads the state of the current database to "rewrite" a new AOF file (although it is called "rewriting" here, it does not actually read any of the old files, but forms instructions based on the current state of the database)

The main process continues to write new changes to both the AOF rewrite buffer and the original AOF buffer.

The main process obtains the signal that the child process rewrites the AOF, calls the signal processing function to write the contents of the AOF rewrite buffer into the new AOF file, renames the new file, and atomically overwrites the original AOF file to complete the replacement of the old and new files.

The rewriting of AOF is also divided into manual trigger and automatic trigger.

Manual trigger: call bgrewriteaof command to trigger automatically: determine the timing of automatic trigger according to auto-aof-rewrite-min-size and auto-aof-rewrite-percentage parameters. Where auto-aof-rewrite-min-size represents the minimum file size when running AOF rewriting, and the default is 64MB. Auto-aof-rewrite-percentage represents the ratio of the current AOF file size (aof_current_size) to the AOF file size (aof_base_size) since the last override. The automatic trigger time is aof_current_size > auto-aof-rewrite-min-size & & (aof_current_size-aof_base_size) / aof_base_size > = auto-aof-rewrite-percentage

RDB vs AOF

RDB and AOF have their own advantages and disadvantages.

Advantages of RDB: compared with AOF, RDB files are relatively small and data recovery is faster (see data recovery section for reasons)

Disadvantages of RDB: when the server goes down, the data after the last RDB persistence will be lost in RBD mode; memory will be consumed when using bgsave for child processes.

The advantages of AOF: AOF only appends files, has little impact on server performance, is faster than RDB, consumes less memory, and is highly readable.

Disadvantages of AOF: the files generated are relatively large, and even if rewritten by AOF, they are still relatively large; data recovery is slower than RDB.

Recovery of database

When the server starts, if AOF persistence is not enabled, the RDB file will be automatically loaded and the main process will be blocked. If AOF persistence is enabled, the server will give priority to using AOF files to restore database state, because AOF files are usually updated more frequently than RDB files, and the saved data is more complete.

The process of redis database recovery is as follows

In terms of data recovery, RDB takes less time to start for two reasons:

There is only one record for each piece of data in the RDB file, unlike the AOF log, which may have multiple operations of a piece of data. So each piece of data only needs to be written once, and the file is relatively small.

The storage format of RDB files is consistent with the encoding format of Redis data in memory, so there is no need for data coding, so the CPU consumption is much less than the loading of AOF logs.

However, in the RDB persistence, the child process that fork comes out for dump operation will occupy the same memory as the parent process, and the copy-on-write mechanism has a great impact on performance and memory consumption. For example, 16 gigabytes of memory, Redis has already used 10 gigabytes, then the save will generate another 10 gigabytes, to 20 gigabytes, larger than the system's 16 gigabytes. At this point, swapping occurs, and if there is not enough virtual memory, it will crash, resulting in data loss. So when using redis, you must make a good capacity planning for the system memory.

Mixed persistence of RDB and AOF

Redis has supported the hybrid persistence scheme of RDB and AOF since version 4. 0. First, RDB periodically completes the backup of memory snapshots, and then AOF completes the data backup between two RDB. These two parts together constitute the persistent file. The advantage of this scheme is that it makes full use of the characteristics of fast loading of RDB, small backup files and AOF without losing data as much as possible. The disadvantage is poor compatibility, once the mixed persistence is enabled, the version before 4.0 does not recognize the persistence file, and because the first part is in RDB format, the readability is low.

Enable mixed persistence

Aof-use-rdb-preamble yes

The data recovery loading process is to load according to RDB, and then append the AOF command to write.

Suggestions for persistence scheme

If Redis is only used as a cache server, for example, the database caches after querying data, then you don't have to worry about persistence, because the cache service fails and can be recovered from the database.

If you want to provide high data security, it is recommended that you use both persistence methods. If you can accept a few minutes of data loss caused by a disaster, you can only use RDB.

The usual design idea is to use the master-slave replication mechanism to make up for the performance impact of persistence. That is, RDB and AOF are not done on Master to ensure the read and write performance of Master, while RDB and AOF (or mixed persistence mode above version 4.0) are enabled on Slave at the same time to ensure the security of data.

At this point, I believe you have a deeper understanding of "what is the method of Redis persistence". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.