Shulou(Shulou.com)06/01 Report--

1. Default persistence

It means that the persistence of RDB will be triggered automatically when storing one object at 900s, 10 objects at 300s, and 10000 objects at 60s.

Save 900 1

Save 300 10

Save 60 10000

Snapshot file name, customizable

Dbfilename dump.rdb

Snapshot file save directory

Dir. /

If there is an error in bgsave, whether to stop writing is generally configured as yes.

Stop-writes-on-bgsave-error yes

Enable compressed rdb

Rdbcompression yes

Check and

Rdbchecksum yes

If you want to turn off snapshot persistence, make the following modifications:

Comment Snapshot persistence rules

# save 900 1

# save 300 10

# save 60 10000

Set to empty

Save ""

Restart redis after that

Backup save instructions:

You can use the SAVE and BGSAVE commands to back up manually, and there is a difference: the SAVE command indicates that the current database is snapped to the dump file using the master process, and the BGSAVE command indicates that the master process will fork a child process for snapshot backup. The former blocks the main process, while the latter does not, so BGSAVE is generally used for manual backups.

You can use the command to shut down db without restarting the service

Config set save ""

Check to see if it is effective

Config get save

2. If you want to use aof for persistence

Turn on appendonlylog, and if it is enabled, each write operation will record a log. The binlog; equivalent of mysql is different in that each time redis starts, it reads this file to build the complete data. Even if the rdb file is deleted, the data is secure

Appendonly no

> >

Appendonly yes

The aof file save directory and file name can be customized

Dir. /

Appendfilename "appendonly.aof"

Aof policy, forcing write to disk once per second

Appendfsync everysec

Whether append is not needed during rewriting needs to be weighed according to the actual situation, because append while rewriting will have performance overhead, of course, if not append, a small amount of data may be lost, so turn off append when performance is first.

No-appendfsync-on-rewrite no

> >

No-appendfsync-on-rewrite yes

Use the command to turn off aof

Config set appendfsync no

Check to see if it is effective

Config get appendfsync

3. Comparison of persistence between RDB and AOF

RDB trigger mechanism:

Save (synchronization)

Execute a save command and a RDB file is generated

Blocking (save is blocking the main process, the client cannot connect to the redis, and after the SAVE is completed, the main process starts to work, and the client can connect)

Bgsave (Asynchronous)

Execute the bgsave command

Fork () A child process creates a RDB file in the background

Does not affect the main process, the client can link redis normally, wait for the child process fork to execute the save is completed, notify the main process, the child process shuts down.

RDB persistence refers to saving the data and operations in memory to dump.rdb files through snapshots within a specified time interval. RDB is a very compact (compact) file with relatively small files, which is very suitable for backup. Recovery can be loaded directly into memory, so it is faster to recover data. However, if the server fails within the non-backup time span, the current state can not be saved in real time, resulting in data loss. In addition, the data of the snapshot may be incomplete in case of downtime. Every time you save a RDB file, save will block Redis, and bgsave will not block Redis, but you need fork () to send a child process to perform specific persistence work. When the dataset is relatively large, fork () may be very time-consuming, causing the server to stop processing the client in so-and-so milliseconds, consuming more CPU resources, and even making this stop time as long as a full second.

AOF persistence is when each write command received is appended to the appendonly.aof file through the write function. Using the policy can maximize the data integrity in unexpected cases, fsync once per second, or fsync each time the write command is executed. AOF's default policy is to fsync once per second, under this configuration, Redis can still maintain good performance, and even if a failure occurs, it will only lose data for up to one second. (fsync executes in the background thread, so the main thread can continue to work hard on command requests.) Because AOF operates by constantly appending commands to the end of the file, recovering data means executing commands one by one, so with the increasing number of write commands, the volume of AOF files will become larger and larger, which will make the recovery process longer.

There are two ways to recover data:

RDB:

Formulate a strategy to automatically and regularly backup the dump.rdb files to the standby. The recovery process is to copy the standby files to the redis server snapshot file storage directory, name it dump.rdb, and then start the redis service.

AOF:

If when the accidental implementation of a flushall purge of all the data, open the file appendonly.aof, delete the end of the command flushall, it is best to use redis-check-aof this tool to detect whether you modified the aof file is normal, in case you start to recover data error, show that the AOF is valid aof description file is valid, then you can restart redis to restore the data.

Two ways to choose:

Do only caching, and you can use no persistence if you want the data to exist while the server is running.

If you can tolerate data loss for a period of time, you can use RDB.

If real-time data backup is required, you need to choose AOF mode.

It is recommended to enable two persistence types at the same time:

In this case, when redis is restarted, the AOF file will be loaded first to restore the original data.

Only AOF is not recommended because there is a potential BUG for AOF persistence

RDB files are used for backup purposes. It is recommended that only the rule save 9001 be retained.

The frequency of AOF rewrite should be reduced as much as possible. The default value of 64m for the base size of AOF rewriting is too small and can be set above 5G. The override can be changed to the appropriate value when the default exceeds 100% of the original size.

4. Solve the problem that the redis aof file is too large

Execute the BGREWRITEAOF command to rewrite the AOF of redis

Redis-cli BGREWRITEAOF

AOF rewrite:

(1) as the AOF file gets larger and larger, most of the commands will be repeated or can be merged (100times incr = set key 100)

(2) benefits of rewriting: reduce AOF log size, reduce memory consumption, and speed up database recovery time.

Perform an AOF file rewrite operation, which creates a volume-optimized version of the current AOF file.

Even if the BGREWRITEAOF fails, there will be no data loss, because the old AOF file will not be modified until the BGREWRITEAOF succeeds.

The default automatically triggers the rewrite rule

Auto-aof-rewrite-percentage 100

Auto-aof-rewrite-min-size 64mb

Percentage means that rewrite occurs when the current aof file size of redis increases by more than 100% compared to the size of the last rewriteaof operation.

However, when the growth rate is greater than 100%, the aof file is actually very small, so rewrite is not necessary, so you also need to set a min-size. When the growth rate of redis is greater than 100%, and the min-size is greater than 64mb, the rewriteaof operation will be performed.

5. RDB and AOF others

5.1, rewrite

If you want to turn off the rewriteaof operation, you can set percentage to 0.

Redis will perform rewrite operations at the following three times

Redis received the bgrewriteaof command sent by the client

Redis aof file growth rate and growth size reach auto-aof-rewrite

Redis received the "CONFIG SET appendonly yes" command sent by the client

5.2. Data recovery sequence

When redis restarts, it starts according to the following priorities:

If only AOF is configured, load the AOF file to recover the data when restarting

If both RBD and AOF are configured, only the AOF file is loaded to recover the data.

If only RBD is configured, the dump file will be loaded at startup to recover the data.

5.3.The repair method of AOF configuration file damage

For the wrong format of AOF files, first backup, and then use the redis-check-aof--fix command to repair, after repair, use diff-u to compare the differences of data, find out the lost data, some can be manually modified. The end of the AOF file may be incomplete, such as the AOF tail file command is not fully written due to a sudden power down of the machine. Redis provides us with aof-load-truncated configuration to be compatible with this situation, which is enabled by default. When loading AOF, it will ignore and continue to start when this problem is encountered.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.