Shulou(Shulou.com)05/31 Report--

What are the operating mechanism and advantages and disadvantages of Redis persistence, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain in detail for you, people with this need can come to learn, I hope you can gain something.

Preface

Everyone knows that Redis is an in-memory database that supports two persistence methods: RDB (Snapshot memory Snapshot) and AOF (append only file). The persistence function synchronizes the data in memory to disk to avoid data loss caused by Redis exceptions. When the Redis instance is restarted, the previously persisted files can be used for data recovery.

Next, the editor introduces the operating mechanisms of the two persistence and their advantages and disadvantages.

One RDB

RDB is the default persistence method, which periodically generates snapshots of the data in memory and saves them to disk according to a certain policy.

Each snapshot persistence is a complete write of memory data to disk, not incremental synchronization of dirty data. If there is a large amount of data and a lot of write operations, it will inevitably cause a large number of disk io operations, which may seriously affect performance.

1.1 Snapshot persistence process

1.2 trigger mechanism

1. Save command

When a client sends a save command request to Redis server for persistence, because Redis uses a main thread to handle all, the save command blocks Redis server from processing requests from other clients until data synchronization is complete.

2. Bgsave command

Unlike the save command, bgsave is executed asynchronously. After executing the bgsave command, the Redis main process will fork a child process to save the data to the rdb file. After synchronizing the data, it replaces the original file, and then informs the main process that the synchronization is complete.

3. Automatic trigger

In addition to manually triggering RDB persistence, there is also an automatic trigger mechanism within Redis.

The centralized configuration of save m n in the configuration means that when there are n modifications to the dataset within m seconds, the system automatically triggers the bgsave operation.

At least one write command within # 900s save 900 1 # 300s at least 10 write commands save 300 10 # 60s at least 10000 write commands save 60 10000

The slave node performs a full copy operation, and the master node automatically executes bgsave to generate RDB files and sends them to the slave node.

By default, when the shutdown command is executed, if AOF persistence is not enabled, the system will automatically execute the bgsave command. When you execute the debug reload command to reload Redis, the save operation is also triggered automatically.

1.3 related parameters

# when there is a problem with persisting rdb files, whether the main process accepts writing or not, yes means to stop writing. If it is no, it means that redis continues to provide services. Stop-writes-on-bgsave-error yes # whether to compress when doing snapshot mirroring. Yes: compression, but requires some cpu consumption. No: no compression, more disk space is needed. Rdbcompression yes # A CRC64 check is placed at the end of the file. There will be a 10% performance degradation when storing or loading rbd files. In order to maximize performance, you can turn off this configuration item. Rdbchecksum yes # File name of the snapshot dbfilename dump.rdb # directory dir / var/lib/redis where the snapshot is stored

1.4 advantages and disadvantages of RDB

Advantages

RDB files are small and are ideal for scheduled backups for disaster recovery.

Because memory data is directly stored in RDB files, while commands are stored in AOF files, commands need to be applied. Redis loads RDB files much faster than AOF.

Shortcoming

RDB persistence cannot achieve real-time / second persistence. Real-time persistence requires full memory brushing to disk, which is too expensive. The fork child process per second also blocks the main process, affecting performance.

RDB files are binary files, and there are multiple versions of rdb files as Redis iterates, so cross-version compatibility is not supported. The old Redis does not recognize the new RDB file format.

Two AOF

AOF (Append-only file) optimizes against the shortcomings of RDB. When using AOF persistence, Redis appends every write command received to the end of the file through the Write function, similar to MySQL's binlog. When Redis restarts, it rebuilds the contents of the entire database in memory by re-executing the write command saved in the file.

2.1 AOF persistence process

1. The client issues the bgrewriteaof command.

2. The redis main process fork child process.

3. The parent process continues to process client requests, except for writing write commands to the original aof file. At the same time, the received write commands are cached into the AOF rewrite buffer. This ensures that there will be no problems if the child process rewrite fails.

4. According to the memory snapshot, the child process writes to the new AOF file according to the command merge rules.

5. When the child process writes the memory snapshot to a temporary file, the child process signals the parent process. The parent process then writes the cached write command to the temporary file.

6. Now the parent process can replace the old aof file with a temporary file and rename it, and the write commands received later begin to append to the new aof file.

2.2 related parameters

# whether to enable AOF, disable appendonly yes by default # specify the AOF file name appendfilename appendonly.aof # Redis to support three write modes: # appendfsync always # force writing to disk every time you receive a write command. Sync_binlog=1, like MySQL, is the safest. However, the speed in this mode is also the slowest and is generally not recommended. Appendfsync everysec # forcefully writes to disk once per second to strike a balance between performance and persistence, which is recommended. # appendfsync no # relies entirely on OS writes, usually about once in 30 seconds. The performance is the best, but persistence is the least guaranteed, so it is not recommended. # during log rewriting, do not append the command, but just put it in the buffer to avoid conflict with the command appending on the DISK IO. # set to yes means that new writes are not fsync during rewrite and are temporarily stored in memory until rewrite is completed. The default is no. It is recommended that yes no-appendfsync-on-rewrite yes # automatically start a new log rewrite process when the current AOF file size is twice the size of the AOF file obtained from the last log rewrite. Auto-aof-rewrite-percentage 100 # the minimum value for the current AOF file to start a new log rewrite process to avoid frequent rewrites due to small file size when Reids is just started. Auto-aof-rewrite-min-size 64mb

2.3 Log rewriting

The AOF mechanism appends every write operation on the client to the end of the aof file. For example, if a key executes the incr,set command many times, the command will be written to the aof file many times, and the aof file will become larger and larger, and some core businesses write dozens of gigabytes of data every day.

Incr k1 1 set k2 a set k2 b incr k1 2 incr k1 3 set k2 c del k3... Incr k1 100

When restoring an Redis instance, it takes a long time to load very large aof files. To solve this problem, Redis supports aof file rewriting-the process of converting data within the Redis process into a write command that is synchronized to a new AOF file. By rewriting, you can generate a minimal collection of commands. For example, the above commands can be merged into

Incr k1 100 set k2 c

Rules for writing data

1. Expired data in the process does not need to be written

two。 The old AOF file contains invalid commands del K1, set a 1, set a 2. Rewriting is generated directly from in-process data, and the aof file retains the latest collection of commands.

3. Multiple commands can be merged into a single command. In order to prevent a single command from overflowing the client buffer, operations such as list,set,hash,zset are divided into multiple operations with 64 elements as boundaries.

Trigger mechanism

1. Manually triggers the execution of the bgrewriteaof command.

two。 Trigger automatically according to configuration

Auto-aof-rewrite-min-size means that running AOF rewrite is the smallest size of the file. The default is 64m. If it is less than 64m, it will not be rewritten automatically.

Auto-aof-rewrite-percentage represents the ratio of (aof_current_size- aof_base_size) / aof_base_size.

How much the current file size increases after aof file rewriting triggers the rewrite.

Automatic trigger time:

Aof_current_size > auto-aof-rewrite-min-size

& &

(aof_current_size-aof_base_size) / aof_base_size > = auto-aof-rewrite-percentage

Three RDB VS AOF comparison

Here are some official suggestions on which persistence method to use:

In general, if you want to provide high data security, it is recommended that you use both persistence methods. If you can accept a few minutes of data loss caused by a disaster, then you can only use RDB. Many users only use AOF, but we recommend that since RDB can take a complete snapshot of the data from time to time and provide a faster restart, it is best to use RDB as well.

Most of the instances in production will not be a single point, but master and slave, and some use slave as a persistence method to meet the needs of HA. Readers can share their problems related to redis persistence.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.