Shulou(Shulou.com)06/02 Report--

This article introduces the knowledge of "master-slave synchronization of Redis distributed foundation". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

When using Redis, we start with a single Redis server. With the growth of business and users, some problems will be exposed, such as the response of a single server has reached the upper limit, all requests from the Redis server have exceeded the cache, and so on.

Then our simplest thing is to have a backup Redis server. When the master server dies, the slave server will replace the master server to continue to serve and improve availability.

After we have two master and slave Redis servers, when the master server dies, the slave server will be replaced to continue to serve us, and the data of our two servers will be different again after the original master server returns to normal, so how can we ensure the data consistency of these two servers? We mentioned the CAP theorem and BASE theory earlier, and we know that we need to ensure data consistency in a distributed, clustered environment. So we have to use master-slave synchronization here, or if we have a large number of servers, we can reduce the synchronization pressure on the master server, and we can use slave synchronization. Let's introduce several synchronization methods supported by Redis.

Incremental synchronization

We know that Redis incremental backups are backed up by saving execution instructions, so we can do the same during synchronization. The master node records the instructions that modify its state in the local memory buffer, and then asynchronously synchronizes the instructions in the buffer to the slave node, which is carried out on one side of the slave node.

The instruction stream of line synchronization achieves the same state as the master node, while feedback to the master node where it is synchronized (offset).

However, the buffer of memory is limited, so the Redis master node cannot record all instructions in the memory buffer. The replication memory buffer of Redis is a circular array of fixed length. If the array content is full, it will overwrite the previous content from scratch. If the slave node cannot synchronize with the master node in a short period of time because of the poor network condition, then when the network condition recovers, the instructions that are not synchronized in the master node of the Redis may have been overwritten by subsequent instructions in buffer.

The slave node will not be able to synchronize directly through the instruction stream, which requires a more complex synchronization mechanism-snapshot synchronization.

Snapshot synchronization

Snapshot synchronization is full synchronization, that is, the entire Redis database snapshot is sent to the slave node for synchronization. After success, the next action is incremental synchronization, so snapshot synchronization is a very resource-consuming synchronization method. Note here that the new slave node needs snapshot synchronization first.

Process:

First bgsave the data of the master node

Save this snapshot on disk, rewrite and start a socket thread

A snapshot can be sent to a child node through the socket thread, which is shared by all child nodes

Child node synchronization

Pay attention to a problem here: when we do snapshot synchronization, incremental synchronization is also going on, and snapshot synchronization occurs when the data of incremental synchronization is overwritten, thus forming an endless cycle repeatedly.

Diskless replication

We mentioned above that incremental synchronization is performed during snapshot synchronization, and another issue that has not been noticed here is Redis's AOF incremental synchronization.

When the master node synchronizes the snapshot, it first saves the snapshot to disk, and then shares the file to the slave node through child threads. The IO operation of the file is carried out here, which is very time-consuming. Snapshot synchronization will cause a heavy load on the system when it is stored on a non-SSD disk. At this time, the master node comes to perform the AOF backup operation, but the two cannot be carried out at the same time. Therefore, the execution of AOF operations will be delayed, which will seriously affect the execution efficiency of the master node, so diskless replication is supported after the Redis2.8 version.

Diskless replication means that the master server sends the snapshot content to the slave node directly through the socket. Generating the snapshot is a traversal process. While traversing the memory, the master node sends the serialized content to the slave node. The slave node first stores the received content in the disk file, and then loads it at one time.

Synchronous replication

Diskless replication is an asynchronous way. Redis3.0 provides a synchronous replication instruction-wait to ensure strong consistency of the system. Wait provides two parameters, the first is the number of slave nodes, and the second is time, in milliseconds.

All writes before waiting for the wait instruction are synchronized to N slave nodes for a maximum of T milliseconds. If time = 0, it means waiting indefinitely until N slave nodes are synchronized.

Note: if the time is equal to 0 and a node happens to be offline, then it will wait all the time and block the server.

> set key valueOK > wait 1 0 (integer) 1

To sum up:

Synchronization types include master-slave synchronization and slave synchronization.

Synchronization methods are: incremental synchronization, snapshot synchronization, diskless replication, synchronous replication

The buffer ring array in incremental synchronization memory will be overwritten when it is full, while the master node performs snapshot synchronization

The newly added slave node uses snapshot synchronization

Snapshot synchronization process: first execute bgsave to put snapshot information into disk, and open child threads to share snapshots and send them to slave nodes for synchronization

Snapshot synchronization is accompanied by incremental synchronization. If the incremental synchronization has been overwritten, snapshot synchronization will be performed, resulting in an endless loop.

In order to reduce the pressure on the master node, when synchronization is enabled, the snapshot data is synchronized to the slave node directly through the socket without going through the disk.

This is the end of the content of "Master-Slave synchronization of Redis distributed Foundation". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.