Shulou(Shulou.com)06/01 Report--

As we all know, the reason why Redis has good performance and fast read and write is that Redis is an in-memory database, and its operations are almost based on memory. However, the memory database has a big drawback, that is, when the database process crashes or the system restarts, if the in-memory data is not saved, the data will be lost. Such a database is not a reliable database.

So data persistence is the top priority of memory database. It not only provides the function of saving data to the hard disk, but also expands the data storage space with the capacity of the hard disk, so that the Redis can store more data than the memory of the machine itself.

Redis provides two persistence schemes for data persistence, RDB and AOF. Their principles and usage scenarios are quite different, so let's take a closer look at them.

RDB-- data Snapshot (Snapshot)

RDB, which provides a Snapshot of data at a certain point in time, is saved in the RDB file. It can be executed manually through the SAVE/BGSAVE command, writing the data Snapshot to the RDB file, or it can be executed regularly through configuration.

Redis can also load data from disk and read it into Redis by loading RDB files.

RDB file creation

Connect to Redis, set some values, and then execute the SAVE command.

You can then take a look at redis.conf 's persistence working directory. Enter the directory and you can see that a dump.rdb file has been saved. The file is a binary file and cannot be opened directly and normally.

As for the difference between SAVE/BGSAVE, the prefix is blocking execution, and the service will not accept requests at this time. The latter is a child process of Fork, which performs the operation of saving RDB files without affecting user requests.

P.S. Redis is single-process, so BGSAVE can only Fork a child process instead of creating a thread.

The above is a manual process. However, in production, we rarely log on to the service manually to perform operations, so more often we rely on the configuration of Redis and save RDB files regularly.

Open the redis.conf configuration file and find the configuration of SNAPSHOTTING and the settings of Save Point.

The configuration in the figure means that when there is at least one key change, the SAVE will be executed after 900 seconds. Other configurations are the same, with 10 changes and one save after 300 seconds.

In Redis, this function of automatically saving RDB is turned on by default.

RDB file loading

If you first kill the Redis process, and then restart Redis Server, you will find that there is a line like this in the log

And in Redis, you still have the three values you set before. Indicates that Redis will load data initialization when it starts.

However, the initialization data loaded here is not necessarily RDB. If AOF is enabled in Redis, the data will be initialized first from AOF, otherwise the data of RDB will be loaded.

Advantages and disadvantages of RDB

Advantages:

RDB is a snapshot at a certain point in time and is a compact single file that is more used for data backup. Backups can be made on an hourly or daily basis to facilitate data recovery from different versions. A single file can be easily transferred to a remote service for fault recovery. RDB can be persisted by Fork child processes, so that Redis can better handle user requests. In the case of a large amount of data, RDB loads faster than AOF.

Disadvantages:

If Redis does not save the RDB file in time, it will result in data loss. For example, the system suddenly lost power, but there was no time to save the data. Even if you set up more Save point, there is no guarantee that 100% of the data will not be lost. RDB often requires a CPU subprocess to execute, but if you have a large amount of data, this fork operation can be very resource-intensive. Compared with AOF, although it is also a fork, its data preservation processing can be controlled and does not need to be fully saved. AOF-- log append (Append-Only)

Another persistence scheme for Redis is AOF,Append Only File. AOF is equivalent to a log record of an operation, and each change to the data is appended to the AOF log. When the service starts, these operation logs are read and the operation is performed again to restore the original data.

AOF enabled

AOF is off by default. Open the redis.conf configuration file and find appendonly no and change it to appendonly yes.

AOF and RDB can coexist, as long as the saved file names do not conflict.

AOF fsync synchronization rules

Drop down the configuration file and see the configuration of fsync.

Fsync () is a system call function that tells the operating system to write data to the hard disk instead of caching more data to write to the hard disk. Such a call can save the data to the hard disk in time.

Redis provides three ways to call fsync

Appendfsync always, every operation record is synchronized to the hard disk, which is the least efficient and the safest. Appendfsync everysec, execute once every second to synchronize the operation record to the hard disk. Default option. Appendfsync no, which does not perform fysnc calls, allows the operating system to automatically write cached data to the hard disk, which is unreliable, but the fastest. Parsing of AOF file format

After opening AOF, you will see the appendonly.aof file in the working directory.

After executing some commands on the client, open the AOF file, and you can observe the logging of the corresponding operations.

File parsing instructions:

*, indicates the number of parameters of the command. For example, set a 1 is three parameters, so it is * 3 $, indicating the number of bytes of the parameter. For example, the parameter set is three bytes, so it is $3, and the key value an is one byte, so $1 is unsigned, indicating that it is parameter data. For example, set,a,1 is a specific data log rewrite.

Although AOF is more reliable than RDB, the disadvantage is that every write operation has to write the operation log to the file, which will result in a very redundant file.

If you want to increment a counter 100 times, if you do not rewrite, the AOF file will have a self-increment record of 100 times, such as INCR a. If a log rewrite is performed, the file retains only set a 100 instead of 100 INCR a. This has the same result, but it can greatly reduce the file size of AOF and improve the efficiency of AOF loading.

Looking back at the redis.conf configuration, there are two options for controlling rewrite.

Auto-aof-rewrite-percentage 100. automatically rewrite when the file grows by 100% (double). Auto-aof-rewrite-min-size 64mb, the minimum file size for log rewriting. If it is less than this size, it will not be rewritten automatically.

To experiment with the results of the rewrite, we first set a value of a, and then increment it several times to view the contents of the AOF file. There are a lot of INCR statements in it.

Then we manually execute BGREWRITEOF and perform log rewriting.

As you can see, multiple incr statements become a set a 6 statement, reducing the operation log of five incr a statements.

Advantages and disadvantages of AOF

Advantages:

AOF can be set to be completely out of sync, synchronized per second, same for every operation, and synchronized per second by default. Because AOF is an addition to operation instructions, it can be synchronized frequently and massively. The AOF file is a value appended log file, and even if the service goes down to write complete commands, these problems can be fixed through the redis-check-aof tool. If the AOF file is too large, Redis automatically rewrites the AOF file in the background. Rewriting will compress the AOF file to the smallest required instruction set. AOF file is an orderly storage of all write operations of the database, easy to read and easy to analyze. Even if you accidentally mismanipulate the database, it is easy to find the wrong instructions and restore to a data node. Careless FLUSHALL, for example, can be easily restored to before the execution of the command.

Shortcoming

With the same amount of data, AOF files are usually larger than RDB files. Because AOF stores instructions, and RDB is a snapshot of the results of all instructions. However, AOF will compress some space after log rewriting. AOF is less efficient than RDB when writing and loading heavily. Because of the large number of writes, AOF executes more save commands and requires a large number of re-execution commands when loading to get the final results. RDB has an advantage over this. How to choose

The above has a basic understanding of the use of RDB and AOF, the basic principles and the corresponding advantages and disadvantages. So in practice, how do we choose which persistence method to use?

Generally speaking, regardless of the hard disk size, the safest way is to use RDB and AOF at the same time, even if the AOF damage cannot be repaired, you can also use RDB to recover data.

If Redis data is not necessary in your service, for example, just as a simple cache, no cache will not cause a cache avalanche. It shows that the security and reliability of the data is not the primary consideration, so just use RDB alone.

Using AOF alone is not recommended because AOF is slower than RDB for data recovery loading. And Redis officials have also stated that AOF has a rare bug. If something goes wrong, it can't be solved well. So when using AOF, it's best to have RDB as a data backup.

According to the official will, there may be a persistence model that combines RDB with AOF in the future. Redis persistence will no longer be so troublesome. Let's wait and see.

For more technical articles and wonderful practical information, please follow us.

Blog: zackku.com

Official account of Wechat: Zack says code

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.