Shulou(Shulou.com)06/01 Report--

1. Data Snapshot RDB

1.1 principle

(1)RDB is to persist data at a certain time to disk, which is a way of snapshot.

(2)redis In the process of data persistence, the data will be written to a temporary file first, and when the persistence process is over, the temporary file will be used to replace the last persistent file. It is this feature that allows us to backup at any time, and redis is running in time;

(3)For RDB mode, redis forks a separate child process for persistence, while the main process does not perform any IO operations, thus ensuring redis high performance.

(4)If large-scale data recovery is required and the integrity of the data recovery is not very sensitive, then RDB is more efficient than AOF.

If you are sensitive to data integrity, then RDB is not for you, because even if you persist every 5 minutes, there will still be nearly 5 minutes of data loss when redis fails. So redis also provides another way to persist, which is AOF.

1.2 Several ways to generate snapshots

Manual trigger:

(1) The save command is used to create a backup of the current database. This command will create a dump.rdb file in the redis installation directory dir;

If you need to recover data, simply move the backup file dump.rdb to the redis installation directory and start the service. The redis directory can be obtained using the config get dir command

(2)bgsave executes in the background;

(3)shutdown save, shutdown has two options, nosave| save, if not added, the default is save;

Automatic trigger:

(4)Settings in the configuration file redis.conf:

save 900 1

save 300 10

save 60 10000

dbfilename dump.rdb

1.3 Restore testing using rdb files

Note: When restoring, you need to turn off the aof function, otherwise redis will load the appendonly.aof log file when starting, so that the restored content is not dump.rdb, but the applied aof log.

#Load test data with redis-benchmark and close aof:

src/redis-benchmark -h 127.0.0.1 -p 6379 -n 200000 -c 20 -d 4 -k 1 --csv > redis_benchmart_$(date +%Y%m%d).log 2>&1

[root@sht-sgmhadoopcm-01 redis]# src/redis-cli

127.0.0.1:6379> config get dir

1) "dir"

2) "/usr/local/redis"

127.0.0.1:6379> config get appendonly

1) "appendonly"

2) "no"

127.0.0.1:6379> keys *

1) "key1"

2) "key2"

3) "key:__rand_int__"

4) "mylist"

5) "counter:__rand_int__"

127.0.0.1:6379> shutdown save

[root@sht-sgmhadoopcm-01 redis]# mv dump.rdb dump.rdb.bak

[root@sht-sgmhadoopcm-01 redis]# src/redis-server redis.conf

[root@sht-sgmhadoopcm-01 redis]# src/redis-cli

127.0.0.1:6379> keys *

(empty list or set)

127.0.0.1:6379> shutdown nosave

Put the backup rdb file in the specified location and restart redis, so that the data is restored again.

[root@sht-sgmhadoopcm-01 redis]# mv dump.rdb.bak dump.rdb

[root@sht-sgmhadoopcm-01 redis]# src/redis-server redis.conf

[root@sht-sgmhadoopcm-01 redis]# src/redis-cli

127.0.0.1:6379> keys *

1) "key:__rand_int__"

2) "mylist"

3) "key2"

4) "key1"

5) "counter:__rand_int__"

2.AOF(append only file)

2.1 AOF

(1)That is, only appending and not changing files are allowed.

Method: appendonly yes

AOF mode is to record the executed write instructions, and execute them again according to the order from front to back when data is recovered;

AOF files are larger than RDB files for the same dataset. Moreover, the recovery speed of AOF mode is slower than RDB mode.

We can turn on AOF by configuring appendonly yes in redis.conf. If there is a write operation (such as SET), redis is appended to the end of the AOF file.

The default AOF persistence policy is fsync once per second, because redis still performs well in this case, and even if redis fails, only the last second of data will be lost.

(2)If you happen to encounter disk space full, inode full or power failure when appending logs, which leads to incomplete log writing, redis provides the redis-check-aof tool, which can be used to repair logs:

Make a backup copy of your AOF file.

Fix the original file using the redis-check-aof tool that ships with Redis: $ redis-check-aof --fix appendonly.aof

Optionally use diff -u to check what is the difference between two files.

Restart the server with the fixed file.

(3)Restore testing via appendonly.aof file

127.0.0.1:6379> config get appendonly

1) "appendonly"

2) "yes"

127.0.0.1:6379> mset key1 1 key2 2 key3 3

OK

127.0.0.1:6379> keys *

1) "key3"

2) "key1"

3) "key2"

[root@sht-sgmhadoopcm-01 redis]# cp appendonly.aof appendonly.aof.bak

127.0.0.1:6379> flushall

OK

127.0.0.1:6379> keys *

(empty list or set)

127.0.0.1:6379> shutdown

[root@sht-sgmhadoopcm-01 redis]# rm -rf appendonly.aof

[root@sht-sgmhadoopcm-01 redis]# mv appendonly.aof.bak appendonly.aof

[root@sht-sgmhadoopcm-01 redis]# src/redis-server redis.conf

[root@sht-sgmhadoopcm-01 redis]# src/redis-cli

127.0.0.1:6379> keys *

1) "key1"

2) "key2"

3) "key3"

2.2 aof file rewrite

(1)Rewrite Principle

Because of the appending method, if no processing is done, the AOF file will become larger and larger. For this reason, redis provides an AOF file rewrite mechanism, that is, when the size of the AOF file exceeds the set threshold, redis will start the content compression of the AOF file and only retain the minimum instruction set that can recover the data. If we call INCR 100 times, we have to store 100 instructions in the AOF file, but this is obviously inefficient. We can merge these 100 instructions into a SET instruction. This is how the rewrite mechanism works.

When rewriting AOF, the process of writing temporary files first and replacing them after completion is still adopted, so power failure, disk fullness and other problems will not affect the availability of AOF files.

Another benefit of the AOF approach is illustrated by a "scene representation." A classmate accidentally executed flushall when operating redis, resulting in all the data in redis memory being emptied. As long as redis configured with AOF persistence mode and the AOF file has not been rewritten, we can pause redis and edit the AOF file at the fastest speed. Delete the last line of FLUSHALL command, and then restart redis, and we can restore all the data of redis to the state before FLUSHALL. However, if the AOF file has been overwritten, there is no way to recover the data in this way.

(2)The method of triggering rewrite

The first method: trigger manually using the bgrewriteaof command;

Second method: controlled by configuration files

auto-aof-rewrite-percentage 100

auto-aof-rewrite-min-size 64mb

For example, the meaning of the above parameter setting: when appendonly.aof is less than 64M, rewrite will not be triggered. When the file size is 64M and the growth rate reaches 100%, that is, 128M, rewrite will be triggered once. At this time, redis will remember the size of the file after rewrite. If it is 80M, the next rewrite will be triggered only after the file size increases to 160M again, and so on

3 Summary:

It is recommended to enable both backup policies at the same time to ensure more secure data.

If your business can accept certain data loss, pay more attention to performance, you can only turn on RDB;

If you use redis as a cache only, you don't need to turn on RDB and AOF;

reference link

https://redis.io/topics/persistence

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.