Shulou(Shulou.com)06/01 Report--

What the editor shares today is an introduction to the methods and principles of Redis persistent snapshots. Maybe you are no stranger to Redis, or you have never known Redis. But don't worry, today the editor will explain the principle of Redis with the simplest description.

The so-called persistence is to keep our data from being lost, usually on our hard disk. There are two ways of persistence in Redis, one is snapshot persistence, the other is AOF persistence, each has its own advantages and disadvantages. In the project, we have to choose the specific persistence method according to the actual situation.

Snapshot persistence (RDB)

Also known as RDB persistence, persistence is achieved by taking snapshots, storing memory data at a certain time in a rdb file, and loading the data in the file when the redis service is restarted.

Configure persistent Snapshot

Snapshot persistence in redis is enabled by default, and there are relevant configuration options in the redis.conf configuration file

# # SNAPSHOTTING # # Save the DB on disk:## save # # Will save the DB if both the given number of seconds and the given# number of write operations against the DB occurred.## In the example below the behaviour will be to save : # after 900sec (15 min) if at least 1 key changed# after 300sec (5 min) if at least 10 keys changed# after 60 sec if at least 10000 keys changed## Note: you can disable saving completely by commenting out all "save" lines.## It is also possible to remove all the previously configured save# points by adding a save directive with a single empty string argument# like in the following example:## save "save 900 1 # 900 at least one key has been changed in 1 # 900 seconds Execute snapshot # By default Redis will stop accepting writes if RDB snapshots are enabled# (at least one save point) and the latest background save failed.# This will make the user aware (in a hard way) that data is not persisting# on disk properly if you describe that at least 10 key are changed within save 300 10 # 300 and execute snapshot save 60 10000 # 60 seconds if at least 10000 key are changed Otherwise chances are that no one will notice and some# disaster will happen.## If the background saving process will start working again Redis will# automatically allow writes again.## However if you have setup your proper monitoring of the Redis server# and persistence, you may want to disable this feature so that Redis will# continue to work as usual even if there are problems with disk,# permissions And so forth.stop-writes-on-bgsave-error yes # whether to continue with the write command # Compress string objects using LZF when dump .rdb databases?# For default that's set to 'yes' as it's almost always a win.# If you want to save some CPU in the saving child set it to' no' but# the dataset will likely be bigger if you have compressible values or keys.rdbcompression yes # whether to compress the snapshot file # Since version 5 of RDB a CRC64 checksum is Placed at the end of the file.# This makes the format more resistant to corruption but there is a performance# hit to pay (around 10%) when saving and loading RDB files So you can disable it# for maximum performances.## RDB files created with checksum disabled have a checksum of zero that will# tell the loading code to skip the check.rdbchecksum yes # whether to perform data verification # The filename where to dump the DBdbfilename dump.rdb # name of snapshot file storage # The working directory.## The DB will be written inside this directory, with the filename specified# above using the 'dbfilename' configuration directive.## The Append Only File will also be created inside this directory.## Note that you must specify a directory here Location where not a file name.dir / opt/redis-5.0.3# snapshot files are stored

Verification effect

1. Enter the installation directory and delete the dump.db file if any

2. Start redis, then add some data, and then close redis to exit

[root@root redis-5.0.3] # src/redis-cli 127.0.0.1 set age 18OK127.0.0.1:6379 6379 > pingPONG127.0.0.1:6379 > set name aaaOK127.0.0.1:6379 > set age 18OK127.0.0.1:6379 > incr age (integer) 19127.0.0.1 set age 18OK127.0.0.1:6379 6379 > shutdownnot connected > exit3. A dump.rdb file is generated in the directory where we installed. This is our snapshot backup file.

4. Start redis again and find that the original data is still there. This is because the data in the backup file is loaded when redis starts.

[root@root redis-5.0.3] # src/redis-server redis.conf 1211:C 13 Feb 2019 01:27:22.668 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1211:C 13 Feb 2019 01:27:22.668 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=1211 Just started1211:C 13 Feb 2019 01 Configuration loaded [root@root redis-5.0.3] # src/redis-cli 127.0.0.1 src/redis-cli 6379 > ping PONG127.0.0.1:6379 > get name "aaa" 127.0.0.1 name > keys * 1) "name" 2) "age" 5, close and exit

Delete the dump.rdb file after closing and exiting, and find that the data is gone after startup.

[root@root redis-5.0.3] # src/redis-server redis.conf 1218:C 13 Feb 2019 01:29:01.336 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo1218:C 13 Feb 2019 01:29:01.336 # Redis version=5.0.3, bits=64, commit=00000000, modified=0, pid=1218 Just started1218:C 13 Feb 2019 01 Configuration loaded # Configuration loaded [root@root redis-5.0.3] # src/redis-cli 127.0.0.1 Configuration loaded > ping PONG127.0.0.1:6379 > keys * (empty list or set) Snapshot persistence principle

Save command:

While redis is running, we can send a save command to take a snapshot. The save command is a blocking command, that is, when the server receives a save command, it will start taking snapshots, during which no other requests will be processed, and other requests will be suspended until the backup ends.

Bgsave command

The bgsave command also takes a snapshot immediately. Unlike the save command, bgsave is not a blocking command, but a fork child thread, which is then responsible for the backup operation. The parent process continues to process the client's request so that it does not cause blocking.

127.0.0.1 set age 20OK127.0.0.1:6379 6379 > pingPONG127.0.0.1:6379 > keys * (empty list or set) 127.0.0.1 set age 20OK127.0.0.1:6379 6379 > set name zhangsanOK127.0.0.1:6379 > bgsaveBackground saving started4, shutdown command

When we just want the shutdown command. The server automatically sends a save command to complete the snapshot operation. And shut down the server after the backup operation is complete. So when we shut down the server when our operation does not meet the first three situations, our data will not be lost when we turn it on again.

5. Sync command

When in a master-slave environment, the slave node wants to synchronize the data of the master node, a sync command is sent to develop a replication. At this point, the master node will send a bgsave command to fork a new thread to complete the snapshot and send the snapshot file to the slave node after the end of the bgsave command operation to complete the synchronization of the data of the master and slave node.

Advantages and disadvantages

Advantages

RDB files save redis data at a certain point in time and are suitable for backup. You can set a time point to archive RDB files so that you can easily restore the data to different versions when needed. RDB is very suitable for disaster recovery. A single file can be easily transferred to a remote server. In the case of a relatively large amount of data, RDB starts quickly.

Shortcoming

RDB is easy to cause data loss. If you set it to save every 3 minutes, and if redis does not work properly for some reason, then the data during the period from the last snapshot to the problem with Redis will be lost.

How to disable snapshot persistence

1. Comment out all save configurations in the redis.conf configuration file. Configure the additional eat command in the last save

Save ""

AOF persistence

Unlike snapshot persistence, which stores Redis's key-value pairs directly, AOF persistence records Redis's in-memory data by saving write commands executed by Redis. In theory, as long as we save all the commands that may modify Redis memory data (that is, write commands), we can restore the memory state of Redis based on these saved write commands. AOF persistence is based on this principle to achieve data persistence and data recovery.

Turn on AOF

In redis, aof is disabled by default. We need to modify the configuration file to enable aof.

Appendonly yesappendfilename "appendonly.aof" # If unsure, use "everysec". # appendfsync alwaysappendfsync everysec# appendfsync nono-appendfsync-on-rewrite noauto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mb

Turn off snapshot persistence

Save "" # save 900 1#save 300 10#save 60 10000 Authentication, restart the service

By performing a simple command operation, we can see that the contents stored in the append.aof file are the commands we executed.

Attention points of AOF persistent backup

There are three values for 1.appendfsync, which are everysec,always and no. Here, we recommend using everysec instead of always. Because always will seriously affect the performance of the server 2.everysec in the worst case will only lose 1 second of data, the impact is within a controllable range.

Advantages and disadvantages

Advantages

The method of AOF persistence provides a variety of synchronization frequencies, and even if you use the default synchronization frequency to synchronize once per second, Redis will only lose data for a second at most. The format of AOF file is readable, which also provides users with a more flexible way to deal with it. For example, if we accidentally misuse the FLUSHALL command, we can manually remove the last FLUSHALL command before the rewrite, and then use AOF to recover the data.

Shortcoming

Although AOF provides a variety of synchronization frequencies, by default, the frequency of synchronizing once per second also has high performance. However, when the load of Redis is high, RDB has better performance guarantee than AOF. RDB uses snapshots to persist the entire Redis data, while AOF simply appends commands executed each time to the AOF file, so in theory, RDB is more robust than AOF

Some suggestions for using persistence

1. If redis is only used as a cache server, we can not use any persistence.

2. In general, we will enable both persistence methods. Redis gives priority to loading AOF files to reply to the data. The advantage of RDB is that it is fast.

3. In the master-slave node, RDB, as our backup data, is only started on the salve (slave node). The synchronization time can be set a little longer, leaving only the rule (save 9001).

These are the details of the methods and principles of Redis persistent snapshots. Have you gained anything after reading them? If you want to know more, welcome to the industry information!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.