Shulou(Shulou.com)05/31 Report--

Editor to share with you what are the advantages and disadvantages of RDB and AOF persistence in redis. I hope you will gain something after reading this article. Let's discuss it together.

Redis provides two persistence schemes: RDB and AOF:

RDB: generates a snapshot of Redis data in memory within a specified time interval, which is a binary file dumpr.rdb

AOF: records all Redis write commands except queries, and restores the data by re-executing these commands when the Redis service starts.

RDB persistence

The default Redis persists the data over a period of time to the hard disk in the form of a RDB snapshot and saves it as an dumpr.rdb binary file. [related recommendation: Redis video tutorial]

A brief introduction to how it works:

When Redis needs to be persisted, Redis fork a child process that writes data to a temporary RDB file on disk. When the child process finishes writing the temporary file, the original RDB is replaced, which has the advantage of being able to copy-on-write.

Of course, we can also manually execute save or bgsave (asynchronously) to generate RDB files.

Redis.conf default configuration

Save 900 1save 300 10save 60 10000

Within 900s, if more than one key is modified, snapshot save is initiated.

Within 300 seconds, if more than 10 key are modified, snapshot save is initiated.

Within 60 seconds, if 10, 000 key is modified, a snapshot save is initiated.

RDB Snapshot command

By default, Redis saves the database snapshot in a binary file named dump.rdb.

You can set Redis to save the dataset automatically when the condition of "at least M changes to the dataset in N seconds" is met.

You can also manually let Redis save the dataset by calling SAVE or BGSAVE.

For example, the following setting allows Redis to automatically save a dataset when it meets the condition that "at least 1000 keys have been changed in 60 seconds":

Save 60 1000

This method of persistence is called snapshot.

The principle of RDB creation

When Redis needs to save the dump.rdb file, the server does the following:

Redis calls fork () and has both parent and child processes.

The child process writes the dataset to a temporary RDB file.

When the child process finishes writing to the new RDB file, Redis replaces the original RDB file with the new RDB file and deletes the old RDB file.

This way of working allows Redis to benefit from the copy-on-write (copy-on-write) mechanism.

Advantages of RDB

RDB is a relatively compact file, which stores the data of Redis at a certain point in time, which is more suitable for backup and disaster recovery.

For example, you can back up RDB files every hour for the last 24 hours, and back up one RDB file every day of each month. In this way, even if you encounter problems, you can restore the dataset to a different version at any time.

Shortcomings of RDB

If you need to avoid losing data in the event of a server failure, RDB is not for you. Although Redis allows you to set different Savepoints to control how often RDB files are saved, because RDB files need to save the state of the entire dataset, it is not an easy operation. So you may save the RDB file at least once in 5 minutes. In this case, in the event of a downtime, you may lose data for several minutes.

AOF persistence

Using AOF for persistence, each write command is appended to the appendonly.aof file through the write function.

AOF can achieve full persistence, as long as it is turned on in the configuration file (the default is no). After appendfsync yes opens AOF, Redis will add it to the AOF file every time Redis executes a command to modify the data. when Redis restarts, it will read the AOF file for "replay" to restore to the last moment before Redis shuts down.

Configuration of AOF

You can configure how often Redis fsync data to disk.

Redis.conf default configuration

Appendfsync yesappendfsync always # is written to the AOF file every time a data modification occurs. Appendfsync everysec # synchronizes once per second, which is the default policy for AOF.

There are three options:

1, execute fsync every time a new command is appended to the AOF file: very slow and very secure.

2, fsync once per second: fast enough (almost as fast as using RDB persistence), and only 1 second of data will be lost in the event of a failure.

3, never fsync: leave the data to the operating system for processing. Faster and less secure options.

The recommended (and default) measure is fsync per second, which is a fsync strategy that combines speed and security.

The principle of AOF creation

AOF rewriting takes advantage of the write-time replication mechanism as well as RDB creating snapshots.

The following steps are performed for AOF rewriting:

Redis executes fork () and now has both parent and child processes.

The child process starts writing the contents of the new AOF file to a temporary file.

For all newly executed write commands, the parent process appends these changes to the end of the existing AOF file as it accumulates them into an in-memory cache: so that the existing AOF file is safe even if a downtime occurs in the middle of the rewrite.

When the child process finishes rewriting, it sends a signal to the parent process, which, after receiving the signal, appends all data in the memory cache to the end of the new AOF file.

Got it! Now Redis atomically replaces the old file with the new file, after which all commands are appended directly to the end of the new AOF file.

Advantages of AOF

1. Using AOF for persistence, you can set different fsync policies, such as no fsync, fsync per second, or fsync each time a write command is executed.

AOF's default policy is to fsync once per second. In this configuration, Redis can still maintain good performance, and even if a failure occurs, it will only lose data for up to one second.

The fsync executes in the background thread, so the main thread can continue to work hard on command requests.

2the redis-check-aof AOF file is a log file that only performs append operations, not the kind that is replaced after generating a new one. Even if the log contains commands that are not fully written for some reason (for example, the disk is full when writing, writing halfway, etc.), the AOF tool can easily fix this problem.

3 AOF Redis can automatically rewrite AOF in the background when the size of the AOF file becomes too large: the new AOF file after rewriting contains the minimum set of commands needed to recover the current dataset.

The whole rewrite operation is absolutely safe, because Redis rewriting is the creation of a new AOF file, and commands will continue to be appended to the existing old AOF file during the rewrite process, and the existing old AOF file will not be lost even if a downtime occurs during the rewrite process. Once the new AOF file is created, Redis switches from the old AOF file to the new AOF file and starts appending the new AOF file.

4the AOF file saves all the write operations performed to the database in an orderly manner, and these write operations are saved in the format of the Redis protocol, so the contents of the AOF file can be easily understood and analyzed (parse) very easily. Exporting (export) AOF files is also very simple: for example, if you accidentally execute _ FLUSH ALL_ (emptying the entire Redis server (removing all key from all databases). Command, but as long as the AOF file is not overwritten, simply stop the server, remove the FLUSHALL command at the end of the AOF file, and restart Redis to restore the dataset to the state it was before FLUSHALL execution.

Shortcomings of AOF

For the same dataset, the volume of the AOF file is usually larger than that of the RDB file.

Depending on the fsync strategy used, AOF may be slower than RDB. In general, the performance of fsync per second is still very high, and turning off fsync can make AOF as fast as RDB, even under heavy loads.

However, RDB can provide a more guaranteed maximum latency (latency) when dealing with large write loads.

The difference between RDB and AOF

RDB persistence means that the snapshot of the dataset in memory is written to disk within a specified time interval. The actual operation is a child process of fork, which first writes the dataset to a temporary file, and then replaces the previous file and stores it with binary compression.

AOF persistence records every write and delete operation processed by the server in the form of log, query operation is not recorded, records are appended in the form of text, and you can open the file to see the detailed operation record.

Which one should I use, RDB or AOF?

If you are very concerned about your data, but can still withstand data loss within a few minutes, then you can just use RDB persistence.

AOF appends every command executed by Redis to disk. Handling huge writes will degrade Redis's performance. I don't know if you can accept it.

Database backup and disaster recovery:

Timed generation of RDB snapshots (snapshot) is very convenient for database backups, and RDB recovers datasets faster than AOF does.

Redis supports enabling RDB and AOF at the same time. After the system is rebooted, Redis will first use AOF to recover data, so that the data will be lost to the minimum.

AOF BGREWRITEAOF rewriting

Because AOF works by constantly appending commands to the end of the file, the size of the AOF file becomes larger and larger as the number of write commands increases.

For instance

If you call INCR on a counter 100 times, the AOF file needs to use 100 records (entry) just to save the current value of the counter.

In practice, however, a single SET command is sufficient to hold the current value of the counter, and the remaining 99 records are actually redundant.

To handle this situation, Redis supports an interesting feature: AOF files can be rebuilt (rebuild) without interrupting the service client.

Execute the BG REWRITE AOF command, and Redis will generate a new AOF file that contains the minimum number of commands needed to rebuild the current dataset.

Redis 2.2 needs to execute the BGREWRITEAOF command manually; Redis 2.4 can automatically trigger AOF rewriting. For more information, please see the sample configuration file of 2.4.

Backing up Redis data

Disk failures, node failures, and so on can make your data disappear, and it is very dangerous not to back up.

Redis is very friendly for data backup because you can copy the RDB file while the server is running: once the RDB file is created, no modification is made. When the server wants to create a new RDB file, it saves the contents of the file in a temporary file. When the temporary file is written, the program uses rename (2) atomically to replace the original RDB file with the temporary file.

That is to say, it is absolutely safe to copy RDB files at any time.

Here are our suggestions:

1. Create a regular task (cron job) that backs up one RDB file to one folder every hour and one RDB file to another folder every day.

2, make sure that the backup of the snapshot comes with the appropriate date and time information, and use the find command to delete expired snapshots each time you execute a regular task script: for example, you can keep hourly snapshots for the last 48 hours and daily snapshots for the last month or two.

3. At least once a day, back up RDB outside your data center, or at least to the physical machine where you are running the Redis server.

After reading this article, I believe you have a certain understanding of "what are the advantages and disadvantages of RDB and AOF persistence in redis". If you want to know more about it, you are welcome to follow the industry information channel. Thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.