Shulou(Shulou.com)05/31 Report--

This article will explain in detail the example analysis of RDB and AOF in Redis. The editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

Redis has two persistence schemes, RDB (Redis DataBase) and AOF (Append Only File). If you want to quickly learn and use RDB and AOF, you can skip to the bottom of the article and read the summary. This chapter learns the persistence of key knowledge of Redis through configuration files, triggering snapshots, restoring data operations, demonstrating command operations, and advantages and disadvantages.

RDB detailed explanation

RDB is the default persistence scheme for Redis. If the specified number of writes are performed within the specified time interval, the data in memory is written to disk. That is, a dump.rdb file is generated in the specified directory. Redis restart recovers data by loading the dump.rdb file.

Learn about RDB from the configuration file

Open the redis.conf file and find the corresponding content of SNAPSHOTTING

1 RDB core rule configuration (key)

Save # save "" save 900 1save 300 10save 60 10000

Explanation: save, if the conditions are met, the data in memory will be synchronized to the hard disk. The official factory configuration defaults to 1 change in 900 seconds, 10 changes in 300 seconds, and 10000 changes in 60 seconds, then write a snapshot of the data in memory to disk.

If you don't want to use the RDB scheme, you can open the comments for save "". Here are three comments.

2 specify the file name of the local database, generally using the default dump.rdb

Dbfilename dump.rdb

3 specify the local database storage directory, and generally use the default configuration

Dir. /

4 data compression is enabled by default

Rdbcompression yes

Explanation: configure whether to compress data when storing to the local database. Default is yes. Redis uses LZF compression, but takes up a little bit of CPU time. If this option is turned off, it will cause the database file to become huge. It is recommended to open it.

Trigger RDB Snapshot

1 perform a specified number of writes within a specified time interval

2 execute save (blocking, just save snapshots, other waits) or bgsave (Asynchronous) command

(3) it is of little significance to execute the flushall command and empty all the data in the database.

Execute the shutdown command to ensure that the server shuts down normally and does not lose any data. It's not big.

Recover data from an RDB file

Copy the dump.rdb file to the bin directory of redis's installation directory and restart the redis service. In the actual development, generally consider the physical machine hard disk damage, choose to back up dump.rdb. You can see it from the following demonstration.

Advantages and disadvantages of RDB

Advantages:

1 is suitable for large-scale data recovery.

2 if the business does not require high data integrity and consistency, RDB is a good choice.

Disadvantages:

1 the integrity and consistency of the data is not high because RDB may be down during the last backup.

2 the backup takes up memory, because Redis creates a child process independently during the backup, writes the data to a temporary file (at this time, the data in memory is twice as large), and finally replaces the temporary file with the previous backup file.

Therefore, it is more reasonable for Redis persistence and data recovery to be performed in the dead of night.

Operation demonstration

[root@itdragon bin] # vim redis.confsave 900 1save 120 5save 60 10000 [root@itdragon bin] #. / redis-server redis.conf [root@itdragon bin] #. / redis-cli-h 127.0.0.1-p 6379127.0.1 set key1 value1OK127.0.0.1:6379 > set key1 value1OK127.0.0.1:6379 > set key2 value2OK127.0.0.1:6379 > set key3 value3OK127.0.0.1:6379 > set key4 value4OK127. 0.0.1 root@itdragon bin 6379 > set key5 value5OK127.0.0.1:6379 > set key6 value6OK127.0.0.1:6379 > SHUTDOWNnot connected > QUIT [root@itdragon bin] # cp dump.rdb dump_ bk.rdb [root @ itdragon bin] #. / redis-server redis.conf [root@itdragon bin] #. / redis-cli-h 127.0.0.1-p 6379127.0.1Plut6379 > FLUSHALL OK127.0.0.1:6379 > keys * (empty list or set) 127.0.0 .1 root@itdragon bin 6379 > SHUTDOWNnot connected > QUIT [root@itdragon bin] # cp dump_bk.rdb dump.rdbcp: overwrite `dump.rdb'? Y [root@itdragon bin] #. / redis-server redis.conf [root@itdragon bin] #. / redis-cli-h 127.0.0.1-p 6379 > keys * 1) "key5" 2) "key1" 3) "key3" 4) "key4" 5) "key6" 6) "key2"

Step 1: vim modifies the persistence configuration time. If you modify it 5 times in 120 seconds, you will persist once.

Step 2: restart the service to make the configuration effective.

Step 3: set 5 key separately, and after two minutes, a dump.rdb file will be automatically produced in the current directory of bin. (set key6 is to verify that shutdown can trigger RDB snapshots.)

Step 4: back up the current dump.rdb (simulate online work).

Step 5: execute the FLUSHALL command to empty the database data (simulate data loss).

Step 6: restart the Redis service and restore the data. Huh? ('◔ ◔'). The data is empty? This is because FLUSHALL also has the ability to trigger RDB snapshots.

Step 7: replace the backed-up dump_bk.rdb with dump.rdb and then re-Redis.

Note: both SHUTDOWN and FLUSHALL commands trigger RDB snapshots. This is a pit, please pay attention.

Other commands:

Keys * matches all key save blockages in the database to trigger RDB snapshots, causing its backup data FLUSHALL to empty the entire Redis server (hardly used) SHUTDOWN shutdown (rarely used)

AOF detailed explanation

AOF: Redis is not enabled by default. It appears to make up for the deficiency of RDB (data inconsistency), so it records each write operation in the form of a log and appends it to the file. The Redis restart will execute the write instruction from front to back according to the contents of the log file to complete the data recovery.

Learn about AOF from the configuration file

Open the redis.conf file and find the corresponding content of APPEND ONLY MODE

1 redis is off by default. If it is enabled, you need to manually change no to yes

Appendonly yes

2 specify the local database file name. The default value is appendonly.aof.

Appendfilename "appendonly.aof"

3 specify update log conditions

# appendfsync alwaysappendfsync everysec# appendfsync no

Commentary:

Always: synchronous persistence, each time the data changes will be immediately written to disk. Poor performance when data integrity is good (slow, secure)

Everysec: factory default recommendation, asynchronous recording once per second (default)

No: out of sync

4 configuration rewrite trigger mechanism

Auto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mb

Explanation: triggered when the AOF file size is twice the size of the last rewrite and the file is larger than 64m. It is generally set to 3G, 64m is too small.

Trigger AOF Snapshot

Depending on the profile trigger, it can be triggered per execution, per second, or out of sync.

Recover data from AOF files

Normally, copy the appendonly.aof file to the bin directory of redis's installation directory and restart the redis service. However, in actual development, the appendonly.aof file format may be abnormal for some reason, resulting in data restore failure, which can be repaired by the command redis-check-aof-- fix appendonly.aof. Learn from the following demonstration.

Rewriting Mechanism of AOF

As mentioned earlier, AOF works by appending writes to a file, and there will be more and more redundancy in the file. So smart Redis adds a rewriting mechanism. When the size of the AOF file exceeds the set threshold, Redis will compress the contents of the AOF file.

The principle of rewriting: Redis will fork a new process, read the data in memory and rewrite it to a temporary file. I didn't read the old file (you are so big, I still read you? O (stupid) stupid! ). Finally, replace the old aof file.

Trigger mechanism: triggered when the AOF file size is twice the size of the last rewrite and the file is larger than 64m. The "double" and "64m" here can be modified through the configuration file.

Advantages and disadvantages of AOF

Advantages: higher data integrity and consistency

Disadvantages: because AOF records a lot of content, the files will get bigger and bigger, and the data recovery will be slower and slower.

Operation demonstration

[root@itdragon bin] # vim appendonly.aofappendonly yes [root@itdragon bin] #. / redis-server redis.conf [root@itdragon bin] #. / redis-cli-h 127.0.0.1-p 6379127.0.1root@itdragon bin 6379 > keys * (empty list or set) 127.0.0.1 SHUTDOWNnot connected 6379 > set keyAOf valueAofOK127.0.0.1:6379 > FLUSHALL OK127.0.0.1:6379 > SHUTDOWNnot connected > QUIT [root@itdragon bin] # / redis-server redis.conf [root @ itdragon bin] #. / redis-cli-h 127.0.0.1-p 6379127.0.0.1 root@itdragon bin 6379 > keys * 1) "keyAOf" 127.0.0.1 root@itdragon bin 6379 > SHUTDOWNnot connected > QUIT [root@itdragon bin] # vim appendonly.aoffjewofjwojfoewifjowejfwf [root@itdragon bin] #. / redis-server redis.conf [root@itdragon bin] #. / redis-cli-h 127.0.0.1-p 6379Could not connect to Redis at 127.0.0.1-p 6379Could not connect to Redis at 127.0.0.16379: Connection refusednot Connected > QUIT [root@itdragon bin] # redis-check-aof-- fix appendonly.aof'x 3e: Expected prefix'*' Got: 'AOF analyzed: size=92, ok_up_to=62, diff=30This will shrink the AOF from 92 bytes, with 30 bytes, to 62 bytesContinue? [keyAOf N]: ySuccessfully truncated AOF [root@itdragon bin] #. / redis-server redis.conf [root@itdragon bin] #. / redis-cli-h 127.0.0.1-p 6379127.0.1purl 6379 > keys * 1) "keyAOf"

Step 1: modify the configuration file to enable AOF persistence configuration.

Step 2: restart the Redis service and enter the client that comes with Redis.

Step 3: save the value, then simulate data loss and shut down the Redis service.

Step 4: restart the service and find that the data has been restored. (an additional point: there are tutorials that show that FLUSHALL commands are written to the AOF file, causing data recovery to fail. I installed redis-4.0.2 without this problem).

Step 5: modify appendonly.aof to simulate file anomalies.

Step 6: failed to restart the Redis service. This also shows that RDB and AOF can exist at the same time, and priority is given to loading AOF files.

Step 7: verify the appendonly.aof file. Normal after restarting the Redis service.

Supplementary point: the aof check is through the redis-check-aof file, so can the rdb check through the redis-check-rdb file?

Summary Redis enables RDB persistence by default. If you perform a specified number of write operations within a specified time interval, the data in memory will be written to disk. RDB persistence is suitable for large-scale data recovery, but its data consistency and integrity are poor. Redis needs to enable AOF persistence manually. By default, write logs are appended to the AOF file every second.

The data integrity of AOF is higher than that of RDB, but there are many records, which will affect the efficiency of data recovery. Redis provides a slimming mechanism for rewriting to solve the problem of large AOF files. If you only intend to use Redis for caching, you can turn off persistence. If you plan to use Redis persistence. It is recommended that both RDB and AOF be enabled. In fact, RDB is more suitable for data backup, leaving a hindhand. There's something wrong with AOF, and RDB.

This is the end of this article on "sample Analysis of RDB and AOF in Redis". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.