Shulou(Shulou.com)05/31 Report--

This article mainly shows you the "sample analysis of cluster clusters in Redis", which is easy to understand and well-organized. I hope it can help you solve your doubts. Let the editor lead you to study and study the "sample analysis of cluster clusters in Redis".

1. Preface

There are two main Redis cluster modes:

Master-slave cluster, distributed cluster.

The former is mainly for high availability or read-write separation, while the latter is load balancing for better data storage.

Redis clusters provide the following two benefits

1. Automatically split the data to multiple nodes

2. When a node in the cluster fails, redis can continue to process client requests.

A redis cluster contains 16384 hash slot, and each data in the database belongs to one of the 16384 hash slots. The cluster uses the formula CRC16 (key)% 16384 to calculate which slot the key key belongs to. Each node in the cluster is responsible for handling part of the hash slot.

Master-slave replication in a cluster

Each node in the cluster has 1 to N replicas, one of which is a master node and the rest is a slave node. If the master node goes offline, the cluster will set a slave node of this master node as the new master node and continue to work. In this way, the cluster will not fail to work properly because of the offline of a master node.

Note:

1. If a master node and all its slave nodes go offline, the redis cluster will stop working. Redis clusters do not guarantee strong data consistency. Under certain circumstances, redis clusters will lose write commands that have been executed.

2. The use of asynchronous replication (asynchronous replication) is one of the reasons why redis clusters may lose write commands. Sometimes due to network reasons, if the network is disconnected for too long, the redis cluster will enable a new master node, and the data previously sent to the master node will be lost.

two。 Master-slave switching principle

The master-slave principle of Redis is similar to that of MySQL, which sets up two machines, one master and one slave. That is to say, hot standby and cold standby. While setting up the master and slave, set up two sentinel processes to detect whether the master node is down. If the master node is found to be down, the appropriate node is immediately selected from the slave node as the new master node. This is similar to VIP (virtual IP technology).

3.Redis cluster TCP port

The node of each Redis cluster needs to open two TCP connections, because these two connections require two ports, the regular Redis TCP command port used to provide services to clients (for example, 6379) and the port obtained by adding 10000 and command ports (10000 to 6379), which is the cluster port (for example, 16379).

The second large port is used for the cluster bus, even using the node-to-node communication channel of the binary protocol. Nodes use cluster bus for fault detection, configuration update, failover authorization and so on. Clients should not attempt to communicate with the cluster bus port, and to ensure the proper use of the Redis command port, make sure that these two ports are open in the firewall, otherwise the Redis cluster node will not be able to communicate.

The command port and cluster bus port offset is fixed, always 10000.

Note that in order for the Redis cluster to work properly, you need to:

1. The normal client communication port (usually 6379) for communicating with clients is open to all clients that need to reach the cluster, as well as to all other cluster nodes (using the client port for key migration).

2. The cluster bus port (client port + 10000) must be accessible from all other cluster nodes.

If you do not open these two TCP ports, your cluster will not work properly.

Cluster bus uses different binary protocols for node-to-node data exchange, which is more suitable for exchanging information between nodes with less bandwidth and processing time.

4.Redis Cluster and Docker

Currently, Redis clusters do not support NAT address environments, and in general environments where IP addresses or TCP ports are remapped.

Docker uses a technique called port mapping: a program running in a Docker container may be exposed on a different port than the one the program believes to be using. This is useful for running multiple containers on the same server using the same port at the same time.

To make Docker compatible with Redis Cluster, you need to use Docker's host networking mode. Check the-net = host option in the Docker documentation for more information.

5.Redis cluster data fragmentation

Instead of using a consistent hash, the Redis cluster is a different form of sharding, where each key is conceptually part of what we call a hash slot.

There are 16384 hash slots in the Redis cluster. In order to calculate the hash slots of a given key, we simply take the 16384 module CRC16.

Each node in the Redis cluster is responsible for a subset of the hash slot. For example, you might have a cluster with 3 nodes, where:

1. Node A contains hash slots from 0 to 5500.

2. Node B contains hash slots from 5501 to 11000.

3. Node C contains hash slots from 11001 to 16383.

This allows you to easily add and remove nodes from the cluster. For example, if I want to add a new node D, I need to move some of the hash slots in node A ~ ~ B ~ C to D. Similarly, if I want to remove node A from the cluster, I can just move the hash slot used by A to B and C, and when node A will be empty, I can remove it completely from the cluster.

Because moving hash slots from one node to another does not require downtime, adding and removing nodes or changing the percentage of hash slots occupied by nodes does not require any downtime.

As long as all key involving a single command execution (or entire transaction or Lua script execution) belong to the same hash slot, the Redis cluster supports multiple key operations. Users can use a concept called hash tags to force multiple key to be part of the same hash slot.

The Hash tag is recorded in the Redis cluster specification document, but the point is that if there is a substring within the keyword {} parentheses, only the contents inside the curly braces "{}" are hashed, for example, this {foo} key and another {foo} key are guaranteed to be in the same hash slot and can be used together in commands with multiple key as parameters.

Master-Slave Model of 6.Redis Cluster

To remain available if a subset of the master server node fails or cannot communicate with most nodes, the Redis cluster uses the master-slave model, where each hash slot ranges from 1 (master server itself) to N replicas (N-1 additional slave nodes).

In our cluster with an example of node A _ C, if node B fails, the cluster cannot continue because we can no longer provide hash slots in the range 5501-11000. However, when we create the cluster (or later), we add a slave server node for each master server node, so that the final cluster consists of Agraine Bline C as the master server node and A1 Magi B1P C1 as the slave server node. If node B fails, the system can continue to run. If node B1 replicates B, and B fails, the cluster will cause node B1 to be the new primary server node and will continue to operate correctly.

Note, however, that if nodes B and B1 fail at the same time, the Redis cluster cannot continue to run.

7.Redis Cluster consistency Assurance

Redis clusters cannot guarantee strong consistency. In practice, this means that in some cases, the Redis cluster may lose writes that the system acknowledges to the customer.

The first reason a Redis cluster may lose writes is because it uses asynchronous replication. This means that the following occurs during the write:

1. Your client writes to the master server node B.

2. Master server node B replies to your client for confirmation.

3. Master server node B propagates writes to its slave servers B1, B2 and B3.

As you can see, the master server node B does not wait for the confirmation of B1MagneB2MagneB3 before replying to the client, because this will cause serious delay losses to Redis, so if your client writes something, the master server node B acknowledges the write, and the system crashes just before sending the write to its slave server node storage, and one of the slave stations (no writes received) can be promoted to the master station. Write is lost forever.

This is very similar to what happens in most databases configured to flush data to disk every second, because past experience is related to traditional database systems and does not involve distributed systems, so you can already infer this situation. Similarly, consistency can be improved by forcing the database to refresh data on disk before replying to the client, but this usually results in extremely poor performance. This is equivalent to synchronous replication in Redis Cluster.

Basically, there is a tradeoff between performance and consistency.

The Redis cluster also supports synchronous writes when absolutely needed, through the WAIT command, which makes it much less likely to lose writes, but note that even with synchronous replication, the Redis cluster cannot achieve full consistency: it is always possible when a slave device that cannot accept writes is selected as the master device.

Another noteworthy situation is that the Redis cluster will also lose data writes, which occurs when the network is partitioned and the client is isolated from a small number of instances that contain at least one primary server.

This paper takes six node clusters composed of three master stations and three slave stations as an example, which is composed of three master stations and three slave stations. There is another customer, we will call Z1.

After the occurrence of the zoning, there may be Amagin C, Magi A1, B1 and C1 on one side of the zone, B and Z1 on the other.

Z1 can still write to B, and it will also accept writes from Z1. If the partition is restored in a short period of time, the cluster will continue normally. However, if the partition takes a long time to promote B1 to the primary device of most side partitions, the writes sent by Z1 to B will be lost.

Note that there is a maximum window (maximum window) for the amount of writes that Z1 can send to B: if the majority side of the partition has enough time to select a slave device as the master device, then each master node on the minority side will stop accepting writes.

This time value is a very important configuration instruction for the Redis cluster, called node timeout (node timeout).

After the node times out, the primary node is considered invalid and can be replaced by one of its replicas. Similarly, after the node times out, the primary node cannot perceive most of the other primary nodes, it enters an error state and stops accepting writes.

8.redis fault-tolerant mechanism

Each redis provides nodes to send ping commands to each other to test the health status of each node. When a node in the cluster with a normal connection receives a ping command from another node, it will return a pong string.

Redis voting mechanism: if a node A sends ping to B and does not get a pong return, then A will notify other nodes to send ping to B again. If more than half of the nodes in the cluster send ping to B without being returned, then B will be confirmed as game over, so in order to avoid a single point of failure, a backup node is generally provided for each node of the redis. After the B node dies, the B node server is started immediately.

The above is all the content of the article "sample Analysis of cluster clusters in Redis". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.