Shulou(Shulou.com)06/02 Report--

It is believed that many inexperienced people have no idea about how to achieve master-slave switching in Redis. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Redis master-slave replication actually replicates the data of the master Redis node to other slave Redis nodes for storage. When the master node has an abnormal downtime, how to switch the slave node to the master node to continue to provide services? Redis master-slave switching is mainly divided into the following two modes: manual switching and Sentinel mode. Today, let's take a look at how Redis continues to provide services through master-slave switching in the event of a failure.

Extraneous remarks

First of all, before talking about master-slave switching, let's add a knowledge point that we missed yesterday: the startup interval between starting the master node and starting different slave nodes should not be too short, because the master node needs to synchronize data to different slave nodes, which consumes a lot of resources.

Manual switching between master and slave

When a master node goes down, the easiest way is to manually switch a slave node to a master node by switching between master and slave manually, so we need to manually intervene in the manual setting. The most important thing is that the Redis service will be unavailable during manual switching. So the master-slave manual switching scheme is not a suitable master-slave switching scheme, but let's also take a look at how the master-slave manual switching is realized.

When the master node goes down, we need to manually set the slave node to the master node. Command:

Redis-cli-h-p slaveof no one

With the above command, you can temporarily set the slave node as the master node. When Redis is restarted, the master-slave switch settings will become invalid. Then change the master configuration of other slave nodes to the current master node according to the master-slave replication configuration in the previous article. When the original master node recovers from the downtime, the data of the temporary master node is saved, and the AOF file and RDB file are copied to replace the AOF file and RDB file under the original master node. Then restart the original master node Redis service and the temporary master node Redis service to restore the original master-slave relationship. But after all, the master-slave manual switching scheme is problematic and not very suitable, so the general master-slave switching will use the Sentinel mode.

Sentinel mode

In Redis, a Sentinel is a separate process that runs independently. Consists of one or more Sentinel instances that can monitor multiple master nodes and slave nodes under the master node. When the monitored master node is down due to a failure, the Sentinel instance can automatically upgrade one of the slave nodes under the master node to a new master node, and the new master node will continue to process write requests. In fact, Sentinel can be understood as a Redis server running in a special mode. After the Sentinel server is initialized, the Sentinel server will keep all the status records related to the Sentinel function. The Sentinel actually has three tasks: monitoring (Monitoring), reminder (Notification), and automatic failover (Automatic failover).

Monitoring: the Sentinel instance will constantly check whether the master / slave node is running normally.

Reminder: when an abnormal downtime occurs on a node, the Sentinel instance will send a reminder to the administrator or other applications.

Automatic failover: when the master node goes down, the Sentinel instance upgrades one of the slave nodes under the master node to the new master node, and the other slave nodes re-initiate the socket request to become the slave node of the new master node.

Configuration center: return the address of the new master node to the client, and you can normally use the new master node to process the request.

Through the brief introduction above, we can find that the Sentinel mode can actually change the master-slave mode from manual switching to automatic switching. The Sentinel will regularly send commands to allow the monitored master-slave node to return to its running state. When the Sentinel monitors that the master node is down, it will automatically select a slave node under the master node and switch to a new master node. Then notify the other slave node to change to the slave node of the new master node, so that you can successfully carry out the master-slave switch to start processing the new request. However, if we use only one sentry, that is, only one Sentinel instance is enabled for monitoring, problems will easily occur. In general, multiple Sentinel instances will be enabled for monitoring. In general, at least 3 Sentinel instances are required.

Two downtime states of nodes

There are generally two states when the sentry detects the downtime of the primary node: sdown (subjective downtime) and odown (objective downtime). If only one sentry thinks that the primary node is down, it becomes a subjective downtime. If a certain number of nodes think that the primary node is down, it becomes an objective downtime.

How do Sentinels judge subjective downtime and objective downtime

When a sentry ping the master node and no information is returned when the timeout millisecond configured by is-master-down-after-milliseconds is exceeded, the sentry thinks that the primary node is down, which becomes a subjective outage, that is, sdown. When the sentry receives a specified number of sentinels within a specified period of time and also thinks that the primary node is down, this is an objective outage, that is, odown. When there are not enough sentinels to recognize the downtime of the primary node in a certain period of time, the objective downtime state of the primary node will be removed. When the sentry who believes that the subjective downtime is down again ping and gets a valid reply, the subjective downtime of the primary node will also be removed.

Why do I need at least 3 Sentinel instances?

As mentioned just now, when a specified number of sentinels think that the primary node is down within a specified period of time, it is called objective downtime. What is the specified quantity? This specified number is actually equal to the number of Sentinels / 2 + 1. In other words, if the number of Sentinels is equal to 2, a Sentinel outage occurs. When master-slave switching is needed, because the number of Sentinels considered to be down at the master node is 2, the master-slave switch cannot be performed when the master node is down. So at least 3 Sentinel instances are required to deploy Sentinels to ensure robustness.

Sentinel mode causes data loss problem

Sentinel mode + Redis master-slave replication of this deployment structure, there is no guarantee that data will not be lost. There are two main cases of data loss in Sentinel mode:

Because the master-slave replication is an asynchronous operation, the master-slave replication may not be successful and the master node is down. At this time, the data that has not been successfully copied will be lost.

If the master node cannot connect to other slave nodes, but is actually still running. At this point, the Sentinel will switch a slave node to a new master node, but in the process, the master node is actually still running, so data that continues to be written to this master node will be lost.

Solution to data loss

Use the command:

Min-slaves-to-write 1

Min-slaves-max-lag 10

Using this set of commands, you can set that the data replication delay of at least one slave node cannot exceed 10s, that is, if the data replication delay of all slave nodes under a direct click exceeds 10s, stop the master node from receiving and processing new requests. This ensures that data loss will only be lost within 10s at most.

After reading the above, have you mastered how to achieve master-slave switching in Redis? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.