Shulou(Shulou.com)06/01 Report--

This article mainly introduces "what is the method of Redis persistence implementation". In daily operation, I believe many people have doubts about what is the method of Redis persistence implementation. Xiaobian consulted all kinds of materials and sorted out simple and easy operation methods. I hope to help you answer the doubts of "what is the method of Redis persistence implementation"! Next, please follow the small series to learn together!

persistence routine

OK, the persistence strategy we generally use in production is

(1)Master Close Persistence

(2)slave can open RDB, if necessary, AOF and RDB are open

This strategy can be adapted to most scenarios, most cluster architectures.

Why most scenes?

This is because there is some possibility of data loss. Redis master-slave replication is asynchronous, the master executes the command requested by the client and immediately returns the result to the client, and then synchronizes the command to the slave asynchronously. Therefore, the master may not have time to transmit the command to the slave, and the slave will be down. At this time, the slave will become the master and the data will be lost.

Fortunately, most business scenarios tolerate partial data loss. Assuming that we really encounter cache avalanche, there are fuses in the code to protect resources, so that all requests are not forwarded to the database, causing our service to crash!

ps: Cache avalanche here refers to a bunch of requests coming at the same time, and the requested key does not exist in redis, resulting in all requests being forwarded to the database.

Why most cluster architectures?

Because redis reads and writes are separated in clusters, it is not suitable for this solution.

Fortunately, due to the adoption of redis read-write separation architecture, it is necessary to consider the delay of master-slave synchronization, which increases the complexity of the system. At present, there are too few projects in the industry that adopt redis read-write separation architecture.

Why do this (1)master close persistence

The reason is simple, because no matter which persistence method will affect the performance of redis, which persistence will cause CPU to stall and affect the processing of client requests. For best read and write performance, turn master persistence off!

RDB Persistence

RDB persistence is to save a snapshot of the data in the current process to the hard disk (hence snapshot persistence) with the file suffix rdb; when Redis restarts, the snapshot file can be read to recover the data.

Then the process of RDB persistence is equivalent to executing the bgsave command. The command execution process is shown in the following figure

As shown in the figure, the main thread needs to call the system function fork() to build a child process for persistence! Unfortunately, in the process of building a child process, the parent process blocks and cannot respond to client requests!

Also, in tests, fork functions are slower on virtual machines and faster on real machines. Considering that most of them are deployed in docker containers now and rarely deployed on real machines, master does not recommend opening RDB persistence for performance reasons!

AOF Persistence

RDB persistence writes process data to a file, while AOF persistence (Append Only File persistence) records each write command executed by Redis to a separate log file.

As time passes, you will notice that the AOF file gets bigger and bigger, so redis has a rewrite mechanism to reduce the size of the AOF file. However, in the rewrite process, the parent process is also required to fork out a child process for the rewrite operation. As for the influence of fork function, mentioned above.

Another is the disk brushing policy fsync, this value is recommended with everysec, that is, Redis will make a fsync call every second by default to write the data in the buffer to disk.

However, if disk performance is erratic, fsync takes longer than 1 second to invoke. At this point, when the main thread performs AOF, it will compare the time when the last fsync succeeded; if it is less than 2s since the last time, the main thread will return directly; if it exceeds 2s, the main thread will block until the fsync synchronization is completed.

Therefore, AOF also affects the performance of redis.

ps :Linux functions, wrtie function to write data to a file, is to write data to the operating system buffer, not to disk. The fsync function forces the operating system to flush buffer data to disk.

To sum up, in order to ensure maximum read and write performance, we will turn off the persistence of master.

(2)slave can open RDB, if necessary, AOF and RDB are open

First of all, let me explain that the reason I don't recommend AOF-based data recovery is too slow.

You have to think, we have done master-slave replication, the data has been backed up, why do slaves need to open persistence?

Because one day, due to a certain project, the wires in the machine room may be cut, which will cause the master and slave machines to shut down at the same time.

Then at this time, we need to quickly restore the cluster, and RDB files are small and fast to recover, so RDB files are commonly used for disaster recovery.

Secondly, the official website also does not recommend a single AOF, the address is as follows:

https://redis.io/topics/persistence

screenshot is as follows

Therefore, if there is a certain requirement for data security, both AOF and RDB persistence are turned on.

Also, make disaster backups. Use linux scp command to copy rdb files to Cloud Virtual Machine periodically.

ps:scp is an abbreviation for secure copy, used to remotely copy files under Linux. Similar commands include cp, but cp only copies locally and cannot cross servers, and scp transmission is encrypted.

At this point, the study of "What is the method of Redis persistence implementation" is over, hoping to solve everyone's doubts. Theory and practice can better match to help everyone learn, go and try it! If you want to continue learning more relevant knowledge, please continue to pay attention to the website, Xiaobian will continue to strive to bring more practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.