Shulou(Shulou.com)06/01 Report--

This article focuses on "how to achieve Redis persistence", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to achieve Redis persistence.

Redis is an advanced key-value database. It is similar to memcached, except that data can be persisted and supports a wide range of data types. There are strings, linked lists, sets and ordered collections. Support in the server-side calculation of the collection union, intersection and complement set (difference), etc., but also supports a variety of sorting functions.

Redis supports two persistence mechanisms, RDB and AOF. Persistence can avoid data loss caused by abnormal process exit or downmachine, and data recovery can be achieved by using the previous persistence file in the next restart.

RDB persistence

RDB persistence is to persist by creating snapshots (compressed binaries) to save all the data at a certain point in time. RDB persistence is the default persistence method for Redis. The trigger of RDB persistence includes manual trigger and automatic trigger.

Manual trigger

Save, executing the save command on the command line, will create the rdb file saving snapshot synchronously, which will block the main process of the server. Do not use bgsave in the production environment, execute the bgsave command on the command line, and will create the rdb file saving snapshot asynchronously through a child process of fork. Except for blocking when fork, the main process can continue to process requests when the child process creates the rdb file.

Automatic trigger

Configure save m n timing trigger in redis.conf, for example, when save 9001 indicates that master-slave replication is triggered when there is at least one update within 900s, if the slave node performs a full copy operation, the master node automatically executes bgsave-generated RDB files and sends them to the slave node to execute shutdown when the debug reload command reloads Redis and does not enable the RDB persistence configuration in AOF persistence redis.conf

# as long as one of the following conditions is met, the bgsave command save 9001 # there is at least one write operation in 900s save 300 10save 60 1000 disable RBD persistence, you can add save "#" # whether the main process stops writing when the backup process goes wrong, whether the stop-writes-on-bgsave-error yes# file is compressed or not recommended that no is more expensive than hard disk cost cpu resources rdbcompression noAOF persistence

AOF (Append-Only-File) persistence is an instruction to record all changes to the state of the database, which is appended to the AOF file in the form of append. The next time the server starts, you can restore the database state before the server shutdown by loading and executing the commands saved in the AOF file.

The AOF persistence configuration in redis.conf is as follows

# disable AOF by default. To change no to the name of the yesappendonly no# append file appendfilename "appendonly.aof" # write the contents of the cache to the file by default every other second. Appendfsync everysec# automatically enables rewriting when the growth rate of the AOF file size is greater than this configuration item (here, it means more than 100% of the original size). Auto-aof-rewrite-percentage 10 automatically turns on rewriting auto-aof-rewrite-min-size 64mb when the AOF file size is larger than this configuration item

The implementation of AOF persistence consists of three steps:

Command append: append command to AOF buffer file write: buffer contents write to AOF file save: AOF file save to disk the frequency of the last two steps is configured by appendfsync, the options for appendfsync include

Always, saved every time a command is executed, has the highest security, losing at most the data of one command, but also the lowest performance (frequent disk IO) everysec, saved every second, recommended, compromise between security and performance, data lost for up to one second no, rely on the operating system to execute (usually about 30 seconds once), the lowest security, the highest performance AOF persists the data after the last SAVE operation on the AOF file is triggered by the operating system. With the passage of time, the AOF file will become larger and larger. Redis uses AOF file rewriting to solve the growing problem of AOF file (it can reduce the disk occupation of files and speed up data recovery). The principle is as follows:

Call fork to create a child process

The child process reads the state of the current database to "rewrite" a new AOF file (although it is called "rewriting" here, it does not actually read any of the old files, but forms instructions based on the current state of the database)

The main process continues to write new changes to both the AOF rewrite buffer and the original AOF buffer.

The main process obtains the signal that the child process rewrites the AOF, calls the signal processing function to write the contents of the AOF rewrite buffer into the new AOF file, renames the new file, and atomically overwrites the original AOF file to complete the replacement of the old and new files.

AOF rewriting is also divided into manual trigger and automatic trigger manual trigger: direct call bgrewriteaof command automatic trigger: determine the automatic trigger time according to auto-aof-rewrite-min-size and auto-aof-rewrite-percentage parameters. Where auto-aof-rewrite-min-size represents the minimum file size when running AOF rewriting, and the default is 64MB. Auto-aof-rewrite-percentage represents the ratio of the current AOF file size (aof_current_size) to the AOF file size (aof_base_size) since the last override. The automatic trigger time is aof_current_size > auto-aof-rewrite-min-size & & (aof_current_size-aof_base_size) / aof_base_size > = auto-aof-rewrite-percentageRDB vs AOF

RDB and AOF have their own advantages and disadvantages.

Advantages of RDB: compared with AOF, RDB files are relatively small, and data recovery is relatively fast (see data recovery section). Disadvantages of RDB: server downtime, RBD method will lose data after the last RDB persistence; memory consumption will be consumed when using bgsave for child processes. The advantages of AOF: AOF only appends files, has little impact on server performance, is faster than RDB, consumes less memory, and is highly readable. Disadvantages of AOF: the files generated are relatively large, and even if rewritten by AOF, they are still relatively large; data recovery is slower than RDB. Recovery of database

When the server starts, if AOF persistence is not enabled, the RDB file will be automatically loaded and the main process will be blocked. If AOF persistence is enabled, the server will give priority to using AOF files to restore database state, because AOF files are usually updated more frequently than RDB files, and the saved data is more complete.

The process of redis database recovery is as follows

In terms of data recovery, RDB takes less time to start for two reasons:

There is only one record for each piece of data in the RDB file, unlike the AOF log, which may have multiple operations of a piece of data. So each piece of data only needs to be written once, and the file is relatively small.

The storage format of RDB files is consistent with the encoding format of Redis data in memory, so there is no need for data coding, so the CPU consumption is much less than the loading of AOF logs.

However, in the RDB persistence, the child process that fork comes out for dump operation will occupy the same memory as the parent process, and the copy-on-write mechanism has a great impact on performance and memory consumption. For example, 16 gigabytes of memory, Redis has already used 10 gigabytes, then the save will generate another 10 gigabytes, to 20 gigabytes, larger than the system's 16 gigabytes. At this point, swapping occurs, and if there is not enough virtual memory, it will crash, resulting in data loss. So when using redis, you must make a good capacity planning for the system memory.

Mixed persistence of RDB and AOF

Redis has supported the hybrid persistence scheme of RDB and AOF since version 4. 0. First, RDB periodically completes the backup of memory snapshots, and then AOF completes the data backup between two RDB. These two parts together constitute the persistent file. The advantage of this scheme is that it makes full use of the characteristics of fast loading of RDB, small backup files and AOF without losing data as much as possible. The disadvantage is poor compatibility, once the mixed persistence is enabled, the version before 4.0 does not recognize the persistence file, and because the first part is in RDB format, the readability is low.

Enable mixed persistence

Aof-use-rdb-preamble yes

The data recovery loading process is to load according to RDB, and then append the AOF command to write.

According to the persistence solution, if Redis is only used as a cache server, such as a database to cache data after querying, then persistence can be ignored, because the cache service fails and can be recovered from the database. If you want to provide high data security, it is recommended that you use both persistence methods. If you can accept a few minutes of data loss caused by a disaster, you can only use RDB. The usual design idea is to use the master-slave replication mechanism to make up for the performance impact of persistence. That is, RDB and AOF are not done on Master to ensure the read and write performance of Master, while RDB and AOF (or mixed persistence mode above version 4.0) are enabled on Slave at the same time to ensure the security of data.

At this point, I believe you have a deeper understanding of "how to achieve Redis persistence". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.