Shulou(Shulou.com)05/31 Report--

Today, I would like to share with you the relevant knowledge of what the two high-availability implementation schemes of Redis are. The content is detailed and the logic is clear. I believe most people still know too much about this knowledge, so share this article for your reference. I hope you can get something after reading this article.

In order to achieve high availability (High Availability, referred to as HA) in Redis, the following two methods are adopted:

Master and slave copy data.

The sentry is used to monitor the operation of the data node, and if there is a problem in the master node, the service will continue to be carried out on the top of the slave node.

Master-slave replication

The data replicated by master and slave nodes in Redis can be divided into full replication and partial replication.

The realization of the full copy function of the old version

Full copy is achieved using the snyc command, and the process is as follows:

Send sync commands from the server to the master server.

After receiving the sync command, the master server calls the bgsave command to generate a * rdb file and synchronizes the file to the slave server, so that after the slave server loads the rdb file, the state will be the same as when the master server executes the bgsave command.

The master server synchronizes the write commands saved in the command buffer to the slave server, and executes these commands from the slave server, so that the state of the slave server is consistent with the current state of the master server.

The problem with the full copy feature of the old version is that when you disconnect and reconnect from the server, you need to copy all the data even if you already have some data on the slave server, which is very inefficient, so the new version of Redis has made improvements in this part.

The realization of the full copy function of the new version

The new version of Redis uses the psync command instead of the sync command, which can achieve both full and partial synchronization.

Copy offset

The two parties performing the replication, the master and slave servers, each maintain a replication offset:

Each time the master server synchronizes N bytes of data to the slave server, it will modify its own replication offset + N.

Each time the slave server synchronizes N bytes of data from the master server, it modifies its own replication offset + N.

Copy backlog buffer

The main server maintains a fixed-length first-in-first-out queue as a replication backlog buffer with a default size of 1MB.

When the master server propagates commands, it not only synchronizes write commands to the slave server, but also writes write commands to the replication backlog buffer.

The server is running ID

Each Redis server has its own running ID, the running ID is automatically generated by the server at startup, the master server sends its own running ID to the slave server, and the slave server saves the running ID of the master server.

When synchronizing after being disconnected from server Redis and reconnected, the progress of synchronization is judged by running ID:

If the ID running on the master server saved from the server is the same as the ID running on the current master server, it is considered that this disconnection and reconnection is the previously replicated master server, and the master server can continue to attempt partial synchronization.

Otherwise, if the primary server runs ID differently before and after two times, it is considered to complete the fully synchronized process.

Psync command flow

With the previous preparation, let's begin the process of analyzing the psync command:

If the slave server has not copied any master server before, or if the slaveof no one command has been executed before, the slave server sends a psync?-1 command to the master server, requesting the master server to synchronize all the data.

Otherwise, if part of the data has been synchronized previously from the slave server, a psync command is sent from the slave server to the master server, where runid is the last run of the master server id,offset is the current replication offset from the slave server.

In the first two cases, after the primary server receives the psync command, the following three possibilities arise:

The master server returns a + fullresync reply, indicating that the master server requires full data synchronization with the slave server. Where runid is the current primary server running id, and offset is the replication offset of the current primary server.

If the master server replies + continue, it means that the master server is partially synchronized with the slave server to synchronize the missing data from the slave server.

If the master server answers-err, it means that the master server version is less than 2.8and the psync command is not recognized, and the slave server will send the sync command to the master server to perform a full data synchronization.

Overview of Sentinel Mechanism

Redis uses the Sentinel mechanism to achieve high availability (HA), which roughly works as follows:

Redis uses a set of sentinel nodes to monitor the availability of master-slave redis services.

Once the Redis master node is found to be invalid, a sentinel node will be elected as the * (leader).

Sentinel * * then selects a Redis node from the remaining slave Redis nodes to serve as the new master Redis node.

The above classifies Redis nodes into two categories:

Sentinel node (sentinel): responsible for monitoring the operation of the node.

Data node: that is, the Redis node requested by the normal service client, which can be divided into master and slave.

The above is a general process, which needs to solve the following problems:

How to monitor Redis data nodes?

How to determine the failure of an Redis data node?

How to select a Sentinel node?

What is the basis for the Sentinel node to choose a new primary Redis node?

To answer these questions one by one.

Three monitoring tasks

The Sentinel node monitors the service availability of the Redis data node through three scheduled monitoring tasks.

Info command

Every 10 seconds, each sentinel node sends info commands to the master and slave Redis data nodes to obtain new topology information.

Redis topology information includes:

The role of this node: master or slave.

The address and port information of the master-slave node.

In this way, the sentry node can automatically obtain the slave node information from the info command, so the slave node information that is added later can be automatically perceived without explicit configuration.

Synchronize information to _ _ sentinel__:hello channel

Every 2 seconds, each Sentinel node will synchronize its own information of the master node and the information of the current Sentinel node to the _ _ sentinel__:hello channel of the Redis data node. Because other Sentinel nodes also subscribe to this channel, this operation can actually exchange information between Sentinel nodes about the master node and the Sentinel node.

This actually accomplishes two things: * discover a new Sentinel node: if a new Sentinel node joins, save the information about the new Sentinel node at this time, and then establish a connection with the Sentinel node. * Exchange the status information of the master node as the basis for the subsequent objective judgment of the downline of the master node.

Do heartbeat detection to data nodes

Every second, each sentinel node sends ping commands to master, slave data nodes and other sentinel nodes to do heartbeat detection. This heartbeat detection is the basis for subsequent subjective judgment of data nodes offline.

Subjective referral and objective referral subjective referral

If the heartbeat detection task, the third of the above three monitoring tasks, does not receive a valid response after the configured down-after-milliseconds, the data node is considered to be "sdown".

Why is it called "subjective referral"? Because in a distributed system, there are multiple machines working together, various conditions may occur in the network, and the judgment of one node alone is not enough to think that a data node is offline, which requires an "objective offline".

Objective offline

When a Sentinel node thinks that the master node is subjectively offline, the Sentinel node needs to consult other Sentinel nodes through the "sentinel is-master-down-by addr" command to see if the master node is offline. If more than half of the Sentinel nodes have answered the downline, it is considered that the master node is "objectively offline".

Election Sentinel

When the master node is offline objectively, it is necessary to elect a sentinel node as a sentinel to complete the subsequent selection of a new master node.

The general idea of this election is:

Each Sentinel node applies to become a Sentinel by sending a "sentinel is-master-down-by addr" command to other Sentinel nodes.

On the other hand, when each sentinel node receives a "sentinel is-master-down-by addr" command, it is only allowed to vote for * nodes, and the command of other nodes will be rejected.

A Sentinel becomes a Sentinel if it receives more than half of the consent votes.

If the first three steps do not elect a sentry within a certain period of time, the next election will be resumed.

As you can see, the process of voting * * is very similar to the process of voting leader in raft.

Select a new master node

Among the remaining Redis slave nodes, select the new master node in the following order:

Filter out "unhealthy" data nodes: such as subjective offline, disconnected slave nodes, nodes that have not replied to the Sentinel ping command in five seconds, and slave nodes that have lost contact with the master node.

Select the slave node of slave-priority (Slave Node priority) *. If it exists, if it does not exist, then continue the later process.

Select the slave node of the replication offset * *, which means that the data on the slave node is the most complete, and if it exists, the return does not exist and continue the later process.

At this point, the state of all remaining slave nodes is the same, select the slave node with the smallest runid.

Promote a new master node

After you select a new master node, you also need a * * process to make it the new master node:

Sentinel * * issues a "slaveof no one" command to the slave node selected in the previous step, making that node the master node.

The Sentinel sends commands to the remaining slave nodes, making them slaves to the new master node.

The Sentinel node set updates the original master node to the slave node and orders it to copy the data of the new master node when it is restored.

These are all the contents of the article "what are the two high-availability implementations of Redis?" Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.