Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "what is the master-slave replication of Redis". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

How does Redis ensure high concurrency and availability?

High concurrency: it is not a problem that the throughput of redis can reach tens of thousands. If you want to improve the read and write ability of redis, you can use the master-slave architecture of redis. Redis supports the preparation mode of one master and multiple slaves. Single master is responsible for writing requests and multiple slaves are responsible for read requests. Asynchronous replication between master and slave to synchronize master data to slave.

High availability: first, use the master-slave architecture of redis to solve the unavailability caused by a single point of failure of redis. Then, if you use the master-slave architecture, you only need to add the sentinel mechanism to achieve this. When the master instance of redis goes down, master / slave switching will be carried out automatically. In order to achieve the high availability of redis.

You just said master-slave replication, can you talk about the principle of master-slave replication in detail?

In the redis master-slave architecture, master is responsible for receiving write requests. After a successful write operation, it returns to the client OK, and then sends the data asynchronously to multiple slaver for data synchronization. However, starting from redis 2.8, slave node periodically confirms the amount of data per replication.

When a slave node is started, it sends a PSYNC command to master node. If the slave node is reconnecting to the master node, then the master node will only copy the missing data to the slave; otherwise, if the slave node connects to the master node for the first time, it will trigger a full full resynchronization copy.

When you start full resynchronization, master starts a background thread, starts to generate a RDB snapshot file, and caches all write commands received from the client in memory (memory buffer). After the RDB file is generated, master sends the RDB to slave,slave, which is written to the local disk and then loaded into memory from the local disk. Master then sends in-memory cached write commands to slave,slave and synchronizes the data.

In addition, when slave node makes replication, it will not block master node normally, nor will it block its own query operation. It will use the old data set to provide services. However, when the replication is completed, the old data set needs to be deleted and the new data set needs to be loaded, and the external service will be suspended. Slave node is mainly used for horizontal expansion and separation of read and write. The expanded slave node can improve the read throughput. Slave has a lot to do with high availability.

What happens if the master-slave replication stops because of network reasons?

If there is a network failure and is disconnected, it will be reconnected automatically. Starting from redis 2.8, you can continue to copy from the breakpoint of master-slave replication, and you can continue to copy where you copied last time, instead of making a copy from scratch.

If master finds that multiple slave node are reconnecting, it simply initiates a rdb save operation that serves all slave node with one piece of data.

Master node will create a backlog,master in memory and slave will save a replica offset, and a master id,offset will be saved in backlog. If the master and slave network connections are disconnected, slave will let master continue to copy from the last replica offset.

But if no corresponding offset is found, then a full copy of the resynchronization is performed.

Okay, can you tell me what a sentry is and what it does?

Sentinel is a very important component of redis cluster architecture. The main functions are as follows

(1) Cluster monitoring, which is responsible for monitoring whether redis master and slave processes are working properly.

(2) message notification: if a redis instance fails, the sentry is responsible for sending a message to the administrator as an alarm notification.

(3) failover. If the master node fails, it will be automatically transferred to the slave node.

(4) configuration center, if a failover occurs, notify the client client of the new master address

The Sentinel itself is also distributed, running as a Sentinel cluster and working together.

(1) during failover, it is determined that a master node is down, which requires the consent of most of the sentinels, which involves the issue of distributed election.

(2) even if some of the sentinel nodes are down, the sentry cluster can still work properly, because it would be awkward if the failover system itself, which is an important part of the high availability mechanism, is a single point.

Currently using the sentinal 2 version, sentinal 2 has rewritten a lot of code compared to sentinal 1, mainly to make the failover mechanisms and algorithms more robust and simple.

Why is only 2 nodes in the redis Sentinel cluster not working properly?

If there are two sentinel instances, that is, two redis instances, one master and one slave mode.

Then the configuration quorum=1 of redis indicates that a Sentinel thinks that master is down and master is down.

But if machine 1 is down, then Sentinel 1 and master are down, although Sentinel 2 knows that master is down, but at this time, you need majority, that is, most Sentinels are running, the majority of 2 Sentinels is 2 (2's majority=2,3 's majority=2,5 's majority=3,4 's majority=2), and 2 Sentinels are running, you can allow failover.

But when Sentinel 1 is gone, there is only one sentinel left, and there is no majority to allow failover, so the failover will not be performed.

Is there any possibility of data loss when switching between master and slave?

There will be, and there are two possibilities, one is asynchronous replication, and the other is data loss caused by brain fissure.

Briefly describe the process of these two kinds of data loss.

OK, the first is easy to understand, because master-to-slave replication is asynchronous, so master may be down before some data is replicated to slave, and some of this data is lost. Although master will persist, after the Sentinel promotes slave to master, if the old master is good at this time, it will be hung as slave on the new master, and the original data will still be lost when synchronizing data from the new master.

The second, that is, a master machine suddenly disconnects from the normal network and cannot connect to other slave machines, but in fact master is still running, that is, cluster partitioning. At this point, the Sentinel may think that master is down and start an election to switch other slave to master.

At this time, there will be two master in the cluster, that is, the so-called brain fissure.

At this point, although a slave is switched to master, but the client may not have time to switch to the new master, but also continue to write data to the old master, this part of the data may be lost. Therefore, when the old master is reinstated and added to the master-slave structure, it will be hung on the new master as a slave, its own data will be emptied, the data will be copied again from the new master, and the data written to the old master will be lost.

Is there any way to solve the problem of data loss?

The problem of data loss is inevitable, but we can minimize it.

Set parameters in the configuration file of redis

Min-slaves-to-write 1

Min-slaves-max-lag 10

By default, min-slaves-to-write is 0, minslaves, maxim, lag, by default, 10.

The above configuration means that at least 1 slave is required, and the delay for data replication and synchronization cannot exceed 10 seconds. If the delay in data replication and synchronization exceeds 10 seconds once all the slave is in place, then the master will not receive any requests at this time.

The above two configurations can reduce data loss caused by asynchronous replication and brain fissure.

How exactly do you set these two parameters to reduce data loss?

Taking the above configuration as an example, these two parameters indicate that the synchronization replication delay between at least one salve and master cannot exceed 10s. Once the delay of all slave replication and synchronization reaches 10s, then master will not accept any requests.

We can reduce the value of the min-slaves-max-lag parameter to avoid a large amount of data loss in the event of a failure, and once the delay is found to exceed this value, the data will not be written to the master.

For client, we can take downgrade measures to temporarily write the data to the local cache and disk, and then rewrite the master after a period of time to ensure that the data is not lost. We can also write the data to the kafka message queue to consume the data in the kafka at regular intervals.

Through the setting of the above two parameters, we try to reduce the loss of data as much as possible, and the specific values also need to be tested in a specific environment.

This is the end of the content of "what is the master-slave copy of Redis". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.