Shulou(Shulou.com)06/01 Report--

This article is about what Redis persistence means. The editor thought it was very practical, so I shared it with you as a reference. Let's follow the editor and have a look.

Demonstration environment

Centos7.0 redis4.0 redis storage directory: / usr/local/redis redis.conf storage directory: / usr/local/redis/data

1. Brief introduction to persistence

All the data of redis is stored in memory, and the data will be lost when redis crashes. Redis persistence is about storing data on disk. The working mechanism of saving the data process by using the permanent storage medium and restoring the saved data at a specific time is called persistence.

What is saved by the persistence process?

The first form of snapshot, which stores the results of the data, focuses on the data, that is, RDB, which will be discussed below.

The second kind of operation procedure, the storage operation procedure, the storage structure is complex, and the focus is on the operation process of the data, that is, AOF, which will be discussed below.

2. RDB2-1 RDB startup mode-- save command

The following figure shows the configuration information of redis.conf. After executing save, a dump.rdb file will be generated.

Now we set a value, and then save, there will be a file of dump6379.rdb under / usr/local/redis/data.

2-2 RDB startup method-save directive related configuration dbfilename dump6379.rdb: set local database file name, default is dump.rdbdir: store rdb file path rdbcompression yes: set whether to compress data when storing to local database, default is yes, use lzf compression rdbchecksum yes: set whether process RDB file format verification, the verification process performs 2-3 RDB data recovery in both writing and reading file process

In fact, this data recovery basically does not need to do anything compared to other relational database recovery. You just need to restart it.

2-4 RDB-- how the save instruction works

This picture is from online video. The execution of the save instruction blocks the current redis server until the current RDB process is complete, which may cause blocking for a long time. This instruction is basically abandoned in the course of work. Will replace it all with bgsave.

2-5 RDB-- how the bgsave instruction works

When bgsave is executed in redis, a Background saving started will be returned directly.

At this time, we are looking at the log file. The bgsave command is optimized for the save blocking problem.

2-5 RDB-profile self-booting save 9001

Save 300 10

Save 60 10000

Stop-writes-on-bgsave-error yes

Save [time] [number of key changes]

In other words, if 10 key values change in 300 seconds, bgsave will be executed in the background.

3. AOF3-1 AOF concept

AOF persistence: each write command is recorded as an independent log, and the data recovery is achieved by re-executing the command in the AOF file when restarting. Compared with RDB, it can be simply described as the process of recording data generation.

The main function of AOF is to solve the real-time performance of data persistence, and it has become the mainstream way of redis persistence.

3-2 AOF data writing process

Execute a redis command

Redis's AOF flushes the command to the buffer

Then synchronize it to the .aof file configured by redis.conf according to certain policies.

3-3 three strategies for AOF writing data always: every write operation is synchronized to the AOF file with zero data error and low performance. Everysec is not recommended: synchronize the instructions in the buffer to the AOF file every second, with high data accuracy and high performance. It is recommended and is also the default configuration. But in the case of a sudden system downtime, the data within 1 second is lost no: the operating system controls the cycle of each synchronization to the AOF file, and the whole process is uncontrollable. 3-4 AOF function is enabled configuration: appendonly yes | no function: whether to enable AOF persistence function. Default is not enabled status configuration: appendfsync always | everysec | no function: AOF write data policy configuration: appenfilename filename function: AOF persistence file name, default is appendonly.aof.

Then by restarting the redis service, you can see the appendonly.aof file in the usr/local/redis/data directory, and then we execute a command on the redis client to check it out again. You can see that the data is stored in the appendonly.aof file.

Problems in writing data with 3-5 AOF

Let's first look at a case. After we repeatedly set up the name key, we open the appendonly.aof file to view, we can see that there are three operations, but these three operations we are all modified a key ah! Can't we just keep the last key? With this question in mind, we are moving on.

3-6 AOF rewrite

As commands continue to be written to AOF, the file becomes larger and larger. To solve this problem, redis introduces an AOF rewriting mechanism to compress the file volume. AOF file rewriting is the process of converting data within the redis process into write commands to synchronize to a new AOF file. To put it simply, the execution results of several commands on the same data are transformed into the execution records of the instructions corresponding to the final result data.

For example, we executed the set name instruction three times above, but in the end we only need the data that was executed for the last time. That is, we only need to execute the record for the last time.

3-7 AOF rewrite reduces disk usage, improves disk utilization, improves persistence efficiency, reduces persistence write time, improves IO performance, reduces data recovery efficiency, and improves data recovery efficiency. 3-8 data that has been timed out in the process of rewriting rules is no longer written to the file to ignore invalid instructions, and in-process data is directly generated during rewriting, so that the new AOF file value retains the write command of the final data. Such as del instruction, hdel,srem. Multiple write commands such as setting a key value for the same data are merged into one command: for example, lpush list a lpush lsit b lpush list c can be converted into lpush list a b c. However, in order to prevent the client-side buffer overflow caused by too much data, a maximum of 64 elements per instruction of type list,set,hash,zset are written to 3-9 AOF manual rewrite.

Instruction: bgrewriteaof

Then we have a 3-5 problem. We execute the bgrewriteaof instruction on the command line and then look at the appendonly.aof file.

When the execution is finished, you will find that the file becomes smaller and there is only one instruction in the file.

3-10 AOF manual rewriting working principle 3-11 AOF automatic rewriting

Configuration: auto-aof-rewrite-percentage 100 | auto-aof-rewrite-min-size 64mb trigger comparison parameter: aof_current_size | aof_base_size

When aof_current_size > auto-aof-rewrite-min-size 64mb starts rewriting

This picture comes from the network.

3-11 AOF Workflow and rewrite flow = proc

4. The difference between RDB and AOF

Very sensitive to data, it is recommended to use the default AOF persistence scheme

AOF persistence strategy uses everysecond, fsync- times per second, this strategy redis can still maintain good processing performance, when a problem occurs, the maximum loss of data within 0-1 second. Note: due to the large storage volume and slow recovery speed of AO files

For the validity of data presentation phase, it is recommended to use RDB persistence scheme.

The data can be well preserved in the stage (this stage is manually maintained by developers, operators and maintenance personnel), and the recovery speed is relatively fast. Phase point data recovery usually adopts RDB scheme Note: using RDB to achieve tight data persistence will make Redis drop very low.

Comprehensive comparison

The choice between RDB and AOF is actually a tradeoff. Each has its advantages and disadvantages. If it cannot withstand data loss within a few minutes, it is very sensitive to data. If it can withstand data loss within a few minutes, choose A0F if it can withstand data loss within a few minutes. In pursuit of the recovery speed of big data set, choose RDB disaster recovery using RDB dual insurance strategy. At the same time, restart RDB and AOF. After restart, Redis gives priority to using A0F to recover data. Reduce the amount of lost data. Thank you for reading! So much for sharing what Redis persistence means. I hope the above content can be helpful to you, so that you can learn more knowledge. If you think the article is good, you can share it and let more people see it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.