Shulou(Shulou.com)05/31 Report--

This article is to share with you about the potential blocking points of AOF in Redis. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

What are the potential blocking points of AOF

1. When Redis uses the help child process to rewrite the AOF file, there is a potential risk of blocking.

1), fork child process

Fork must block the main thread at this moment (note that fork does not copy all memory data to the child process at once). Fork uses the realistic copy (Copy On Write) mechanism provided by the operating system to avoid the long-term blocking problem caused by copying a large amount of memory data to the child process at once. [related recommendation: Redis video tutorial]

However, the fork sub-process needs to copy the necessary data structures of the process, one of which is to copy the memory page table (the mapping index table of virtual memory and physical memory). This copying process consumes a lot of fork resources. Before the copy is completed, the whole process will block, and the blocking time depends on the memory size of the whole instance. The larger the instance, the larger the memory page table, and the longer the page table is blocked.

After copying the memory page table, the child process points to the same memory address space as the parent process, that is, although the child process is generated, it does not apply for the same memory size as the parent process.

So when will the parent-child process really separate memory?

"realistic replication", as its name implies, actually copies the real data in memory only when the write occurs. In this process, the parent process may also run the risk of blocking, which is the scenario described below.

2) scenarios in which the parent process has writes in AOF rewriting

The child process out of fork points to the same memory address space as the parent process, and then the child process can perform AOF rewriting to write all the data in memory to the AOF file.

But at this time, the parent process will still have traffic to write. If the parent process is operating an existing key, then the parent process will really copy the memory data corresponding to the key and apply for new memory space. Gradually, the memory data of the parent and child processes will begin to separate, and the parent and child processes will gradually have their own independent memory space. Because the memory allocation is allocated on a page-by-page basis, by default, 4k, if the parent process is operating on a bigkey at this time, it will take longer to reapply a large chunk of memory, which may lead to the risk of blocking.

In addition, if the operating system turns on the memory large page mechanism (Huge Page, page size 2m), the probability of blocking when the parent process requests memory will be greatly increased, so the Huge Page mechanism needs to be turned off on the Redis machine. Redis every time fork generates RDB or AOF rewrite is completed, you can see in Redis log how much memory space the parent process has reapplied.

3) Why not reuse the AOF log for AOF rewriting

AOF rewrites logs that do not reuse AOF itself:

One reason is that the parent-child process will inevitably have competition problems when writing the same file, and controlling the competition means that it will affect the performance of the parent process.

Second, if the AOF rewrite process fails, then the original AOF file is equivalent to being contaminated and cannot be restored. So Redis AOF rewrites a new file, and if the rewrite fails, just delete the file and will not affect the original AOF file. After the rewrite is complete, you can directly replace the old file.

Thank you for reading! This is the end of this article on "what are the potential blocking points of AOF in Redis". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.