Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to understand the persistence of redis". The explanation content in this article is simple and clear, and it is easy to learn and understand. Please follow the ideas of Xiaobian to study and learn "how to understand the persistence of redis" together.

Persistence of redis:

Concept: Save data in memory to disk, ensuring that data is not lost when the machine is down or restarted.

Note: It is recommended that RDB and AOF be turned on at the same time. If both are turned on at the same time, redis will first use aof file to recover data at startup. If reading aof file fails, read rdb file to recover data.

RDB(Redis DataBase)

Concept:

Take a snapshot of the data in memory and store it on disk, that is, generate a dump.rdb file under the specified directory. RDB is the default persistence mode for redis.

After redis starts, it can be used to load the data into memory again by reading the rdb file. Under normal circumstances, the time for loading the rdb file into memory is about 20~30 seconds.

Procedure: write all the data to a temporary file and replace the rdb file with this temporary file.

Trigger mechanism:

Manual trigger:

The save command blocks the current Redis process until the RDB process is complete. If the instance consumes a large amount of memory, it will cause the redis service to be unavailable for a long time. (Not recommended)

bgsave command: redis The current process executes the fork operation to create a child process. The child process is responsible for the RDB persistence process. The parent process continues to receive and process requests from the client. Blocking occurs only in the fork phase, usually for a short time.

Note: When forking a process, the child process occupies the same memory space as the parent process.

Automatic trigger:

The bgsave command is automatically triggered when the configured condition is reached.

The bgsave command is automatically triggered after the shutdown command is executed.

Configuration file redis.conf:

#Will save the DB if both the given number of seconds and the given number of write operations against the DB occurred.

#Format: save

save 900 1 # 900 seconds, if more than one (including one) key is modified, a snapshot is triggered.

save 300 10 # 300 seconds, if more than 10 (including 10) keys have been modified, a snapshot is triggered.

save 60 10000 # 60 seconds, if more than 10000 (including 10000) keys have been modified, trigger snapshot.

# RDB will be enabled by default, RDB persistence will be disabled

save ""

#Set the name of the rdb file

dbfilename dump.rdb

#Set the working directory of redis, i.e. the directory where rdb files and aof files are located

# The working directory.

# The DB will be written inside this directory, with the filename specified above using the 'dbfilename' configuration directive.

# The Append Only File will also be created inside this directory.

dir ./

#Set whether the rdb file is compressed, the default is yes. Note: compressing rdb files reduces disk space usage, but requires additional cpu consumption.

rdbcompression yes

Advantages: Suitable for large-scale data recovery, and the speed of data loading is relatively fast.

Disadvantages: Data cannot be persisted in real time (seconds). Therefore, the integrity of the data backup is not high.

AOF(Append Only File)

Concept:

Each write sent to redis server is stored on disk, i.e. an appendonly.aof file is generated in the specified directory, and each write is appended to the end of the aof file.

After redis starts, it reads the aof file and executes the write operation in the aof file again in order to achieve the effect of data recovery.

Procedure: Command append to cache, synchronize cache data to aof file (sync), rewrite aof file (rewrite)

Rewrite the aof file:

Purpose: Remove the intermediate execution process of the data and retain the final data command, which can greatly reduce the size of the aof file, thus speeding up the speed of data recovery.

Process: redis The current process executes fork operation to create child process (cost equivalent to bgsave process), child process according to the command merge rules to write the data in memory to the new AOF file, and finally replace the existing aof file with the new aof file.

Trigger mechanism:

Manual trigger:

BGREWRITEAOF

If aof file format is abnormal, you need to fix aof file: redis-check-aof --fix appendonly.aof

Automatic trigger:

When the configured condition is reached, the operation of updating or resetting the aof file is triggered automatically.

3)Configuration file redis.conf:

# AOF and RDB can exist simultaneously, AOF persistence has better data integrity and consistency.

# AOF and RDB persistence can be enabled at the same time without problems.

# If the AOF is enabled on startup Redis will load the AOF, that is the file with the better durability guarantees.

#Turn on AOF (AOF defaults to OFF appendonly no)

appendonly yes

#Set conditions that trigger updating aof files

# appendfsync always #Synchronous persistence, i.e. updating the aof file after each write operation. Data integrity is good, but performance is poor.

appendfsync everysec #Sync once per second, which is the default condition for AOP to trigger update logs.

# appendfsync no #does not actively synchronize, the operating system determines when to synchronize (usually 30s), the best performance but persistence is not guaranteed, so this configuration is not recommended.

#Set conditions that trigger rewriting of aof files, multiple conditions are "and" relationships.

# redis will record the size of aof file after rewriting aof file (if aof file has not been rewritten, this size defaults to the size of aof file when redis starts)

auto-aof-rewrite-percentage 100 #The size of the current aof file exceeds 100% of the size of the record after the last rewrite.

auto-aof-rewrite-min-size 64mb #The current aof file is greater than or equal to 64mb. Note: Generally, it will not be set so small. It is reasonable to set it to ngb.

#Set aof file name

# appendfilename appendonly.aof

#Set the working directory of redis, i.e. the directory where rdb files and aof files are located

dir ./

Advantages: Data can be persisted in real time (seconds), so the integrity of data backup is very high.

Disadvantages: AOF records a lot of content, files will be larger and larger, data recovery will be slower and slower, so redis introduced a mechanism to rewrite aof files in AOF.

Persistence costs redis:

Concept: Child processes consume CPU, memory, hard disk, etc. resources when rewriting RDB files and rewriting AOF files.

CPU: The child process is responsible for writing the data in the process to the file in batches. This process is a CPU-intensive operation. Usually, the child process uses nearly 90% of the CPU of the single core.

Memory: When a parent forks a child, the child takes up the same memory as the parent by default. We can modify the Linux memory allocation configuration to avoid fork process failures due to insufficient physical memory.

Hard disk: disk io is under pressure to write data to rdb files or aof files.

Note: When Redis with AOF is enabled for high traffic writing, if ordinary mechanical disks are used, the write throughput is generally around 100MB/s. At this time, the bottleneck of Redis instances is mainly on AOF synchronous hard disks.

Note: redis instances should not be mixed with other cpu, memory, or hard disk sensitive services.

Thank you for reading, the above is the content of "how to understand the persistence of redis". After studying this article, I believe everyone has a deeper understanding of how to understand the persistence of redis. The specific use situation still needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.