Shulou(Shulou.com)06/01 Report--

Redis Cluster uses virtual slot partition, and all keys are mapped to the integer slot of 0,16383 according to the hash function. The calculation formula: slot=CRC16 (key) & 16383. Slot is the basic unit of data management and migration in the cluster. The main purpose of adopting large-scale slots is to facilitate data splitting and cluster expansion. Each node is responsible for slots for a certain amount of data, as shown in the following figure:

When the cluster has five nodes, each node has an average load of about 3276 slots. Due to the use of high-quality hashing algorithm, the data mapped by each slot is usually uniform, and the data is equally divided into 5 nodes for data partition. Each node is responsible for maintaining part of the slot and the key value data mapped by the slot, as shown in the following figure:

Characteristics of Redis virtual slot partition:

Decoupling the relationship between data and nodes simplifies the difficulty of node expansion and contraction.

The node itself maintains the mapping relationship of the slot, and there is no need for the client or agent service to maintain the slot partition metadata.

Mapping query between nodes, slots and keys is supported, which is used in scenarios such as data routing and online scaling.

Data partitioning is the core of distributed storage. Understanding and flexible application of data partitioning rules is very helpful to master Redis Cluster.

Cluster function limitation

There are some limitations in the function of Redis cluster relative to the stand-alone, which need to be understood in advance by the developer page and avoided when using it. The restrictions are as follows:

1) the support for key batch operations is limited. For example, mset and mget only support batch operations for key with the same slott value. Key that maps to different slott values is not supported because performing operations such as mget, mset, and so on may exist on multiple nodes.

2) the support for key transaction operations is limited. Similarly, the transaction operation of multiple key on the same node is only supported, and the transaction function cannot be used when multiple key are distributed on different nodes.

3) key is the minimum granularity of data partition, so you cannot map a large key object such as hash, list, etc., to different nodes.

4) multiple database spaces are not supported. Redis on a stand-alone machine can support 16 databases, while only one database space, namely db0, can be used in cluster mode.

5) replication structure only supports one layer, slave node can only copy master node, and nested tree replication structure is not supported.

Set up a cluster

To start building a cluster, you need the following three steps:

1) prepare the node

2) Node handshake

3) Distribution slot

Prepare the node

In the node planning, three machines are used, and two Redis instances are deployed on the first machine, one master and one slave.

Redis_1 192.168.56.120redis_2 192.168.56.121redis_3 192.168.56.122

Redis cluster is generally composed of multiple nodes, with at least 6 node data to ensure the formation of a complete and highly available cluster. Each node needs to turn on the configuration cluster-enabled yes and let the Redis run in cluster mode. It is recommended to unify a directory for all nodes in the cluster, which is generally divided into three directories: conf, data and log, which store configuration, data and log-related files respectively. Put the configuration of the six nodes in the conf directory, and the cluster-related configurations are as follows:

# Node port port 637 enables cluster-enabled yes# node timeout in cluster mode, in millisecond cluster-node-timeout 1500times cluster internal configuration file cluster-config-file "nodes-6379.conf"

Other configurations can be consistent with the stand-alone mode. Configure the file naming rule redis- {port} .conf. Start all nodes when you are ready to configure. The command is as follows:

# redis_1redis-server conf/redis-6379.conf & redis-server conf/redis-6380.conf & # redis_2redis-server conf/redis-6379.conf & redis-server conf/redis-6380.conf & # redis_3redis-server conf/redis-6379.conf & redis-server conf/redis-6380.conf &

Check whether the log of the node is correct, as follows

5200 383261e3f0053f74c953bd07ceee36a4b5795bc3 M 05 Apr 11 383261e3f0053f74c953bd07ceee36a4b5795bc3 28 18. 931 * No cluster configuration found. Redis 4.0.13 (00000000Univer 0) 64 bitRunning in cluster modePort: 6380PID: 5200.5200 Apr M 05 Apr 1128 Ready to accept connections

The 6379 node on the redis_1 starts successfully. If there is no cluster configuration file at the first startup, it will automatically create one. The file name is controlled by the cluster-config-file parameter item. It is recommended to use the node- {port} .conf format to distinguish different nodes by using port numbers to prevent multiple nodes under the same machine from overwriting each other, resulting in abnormal cluster information. If a cluster configuration file exists at startup, the node initializes the cluster information with the contents of the configuration file, as shown in the following figure:

Redis in cluster mode adds a cluster configuration file in addition to the original configuration file. When the cluster node information changes, such as adding nodes, node offline, failover and so on. The node automatically saves the cluster state to the configuration file. It should be noted that Redis automatically maintains the cluster configuration file and does not modify it manually to prevent cluster information confusion when the node is restarted.

For example, after the 6379 instance of redis_3 is launched for the first time, the cluster configuration is as follows:

[redis@redis_3 ~] $cat data/nodes-6379.conf

275753d11365feabb366170b940bca4b8486bbd7: 0 myself,master-0 000 connected

Vars currentEpoch 0 lastVoteEpoch 0

The contents of the file record the initial state of the cluster. The most important thing here is the node ID, which is a 40-bit hexadecimal string used for a node in the standard cluster. After that, many cluster operations are completed with the help of the node ID. It is important to note that the node IDeas is different from running IDeas. The node IDeas is created only once during cluster initialization, and the cluster configuration file is loaded for reuse when the node is restarted, while the running IDeas of Redis changes each time it is restarted. Execute the cluster nodes command on the 6379 node of redis_3 to get the cluster node status:

127.0.0.1 6380 > cluster nodes

D54639285709a65bb8ca331f26d4fe1b1c8c73ca: 6380 / 16380 myself,master-2000 connected

At present, each node can only identify its own node information. We start 6 nodes, but each node does not know the existence of each other. Let's make 6 nodes suggest contact with each other by shaking hands to form a cluster.

Node handshake

Node handshake is a process in which a batch of nodes running in cluster mode communicate with each other through Gossip protocol to perceive each other. The node handshake is the first step for the cluster to communicate with each other. The client initiates the command: cluster meet {ip} {port}, as shown in the following figure

The command executed in the figure is: cluster meet 127.0.0.1 6380 lets nodes 6379 and 6380 shake hands and communicate. The cluster meet command is an asynchronous command that returns immediately after execution. Internally initiate handshake communication with the target node, as shown in the figure

1) Node 6379 locally creates 6380 node information objects and sends meet messages.

2) after receiving the meet message, node 6380 saves the 6379 node information and replies to the pong message.

3) after that, nodes 6379 and 6380 regularly communicate with each other through ping/pong messages.

The meet, ping and pong messages here are the carriers of Gossip protocol communication. After that, the node communication part is further introduced, and its main function is to exchange state data information between nodes. 6379 and 6380 nodes are advised to communicate with each other through the meet command, and the cluster structure is shown in the figure:

After executing cluster meet, you can see through cluster nodes that both instances have checked the existence of each other.

[redis@redis_1] $redis-cli-h redis_1-p 6379redis_1:6379 > cluster meet 192.168.56.120 6380OKredis_1:6379 > cluster nodes383261e3f0053f74c953bd07ceee36a4b5795bc3 192.168.56.120 cluster meet 6380mm 16380 master-015544401391581 connectedceaf2ccb751978b7334ddeca474da3d6b7aac99b 192.168.56.1206379U 16379 myself Master-00 connectedredis_1:6379 > exit [Redis @ redis_1 ~] $redis-cli-h redis_1-p 6380redis_1:6380 > cluster nodesceaf2ccb751978b7334ddeca474da3d6b7aac99b 192.168.56.120 redis-cli 6379mm 16379 master-01554440152934 0 connected383261e3f0053f74c953bd07ceee36a4b5795bc3 192.168.56.120 6380mm 16380 myself,master-00 1 connectedredis_1:6380 >

To add other nodes to the cluster using the following command, you only need to cluster meet any node in the cluster to join the new node, and the handshake status will be spread through the cluster through messages, so that other nodes will automatically discover the new node and initiate the handshake process. Finally, execute the cluster nodes command to confirm that the six nodes are aware of each other and form a cluster:

[redis@redis_1] $redis-cli-h redis_1-p 6379redis_1:6379 > cluster meet 192.168.56.121 6379OKredis_1:6379 > cluster meet 192.168.56.121 6380OKredis_1:6379 > cluster meet 192.168.56.122 6379OKredis_1:6379 > cluster meet 192.168.56.122 6380OKredis_1:6379 > cluster nodes275753d11365feabb366170b940bca4b8486bbd7 192.168.56.122 6380OKredis_1:6379 > cluster nodes275753d11365feabb366170b940bca4b8486bbd7 192.168.56.122 cluster meet 6379 "16379 master-01554440316783 4 connected383261e3f0053f74c953bd07ceee36a4b5795bc3 192.168.56.12080" 16380 master-01554440316000 1 connectedceaf2ccb751978b7334ddeca474da3d6b7aac99b 192.168.56.120Fr637915379 myself Master-0 1554440315000 5 connectedd54639285709a65bb8ca331f26d4fe1b1c8c73ca 192.168.56.122 master 6380mm 16380 master-01554440316000 0 connectedca4f05809dc9a9eda4292a33b994b4e2aab13033 192.168.56.121Val 6380 master-01554440317000 3 connecteda85bbb17a3f559d18d7f5059f14ffd9f6ba48ee1 192.168.56.121 master 6379U 16379 master-01554440315000 2 connected

The status of the cluster after the handshake is shown below:

The node recommends that the cluster does not work properly after shaking hands. Here, the cluster handles offline status, and all data reading and writing are prohibited. You can see this with the following command:

Redis_2:6379 > set hello world (error) CLUSTERDOWN Hash slot not served

You can obtain the current status of the cluster through the cluster info command:

Redis_2:6379 > cluster infocluster_state:failcluster_slots_assigned:0cluster_slots_ok:0cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:0cluster_current_epoch:5cluster_my_epoch:2cluster_stats_messages_ping_sent:315cluster_stats_messages_pong_sent:314cluster_stats_messages_meet_sent:3cluster_stats_messages_sent:632cluster_stats_messages_ping_received:312cluster_stats_messages_pong_received:318cluster_stats_messages_ Meet_received:2cluster_stats_messages_received:632

As you can see from the output, the allocated cluster_slots_assigned is 0, and because all slots currently have no nodes assigned, the cluster cannot complete the slot-to-node mapping. The cluster goes online only when all 16384 slots are allocated to the nodes.

Distribution slot

The Redis cluster maps all the data to 16384 slots. Each key is mapped to a fixed slot, and only when the node assigns slots does it respond to the key commands associated with those slots. Assign slots to nodes through the cluster addslots command. These use the bash feature to set up slots (slots) in batches, with the following commands:

[redis@redis_1 ~] $redis-cli-h redis_1-p 6379 cluster addslots {0.5461} OK [Redis @ redis_1 ~] $redis-cli-h redis_2-p 6379 cluster addslots {5462... 10922} OK [Redis @ redis_1 ~] $redis-cli-h redis_3-p 6379 cluster addslots {10923. 16383} OK

Allocate 16384 slot evenly to 6379 of the three nodes of the redis_1/2/3. Execute cluster info to check the cluster status as follows:

[redis@redis_1] $redis-cli-h redis_2-p 6379 cluster infocluster_state:okcluster_slots_assigned:16384cluster_slots_ok:16384cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:3cluster_current_epoch:5cluster_my_epoch:2cluster_stats_messages_ping_sent:1062cluster_stats_messages_pong_sent:1067cluster_stats_messages_meet_sent:3cluster_stats_messages_sent:2132cluster_stats_messages_ping_received:1065cluster _ stats_messages_pong_received:1065cluster_stats_messages_meet_received:2cluster_stats_messages_received:2132

The current cluster status is OK, and the cluster is online. All slots have been assigned to nodes. You can see the allocation relationship between nodes and slots by executing the cluster nodes command:

[redis@redis_1] $redis-cli-h redis_2-p 6379 cluster nodesa85bbb17a3f559d18d7f5059f14ffd9f6ba48ee1 192.168.56.121 myself Master-01554441448000 2 connected 5462-10922ceaf2ccb751978b7334ddeca474da3d6b7aac99b 192.168.56.120master-01554441448455 5 connected 0-5461ca4f05809dc9a9eda4292a33b994b4e2aab13033 192.168.56.121 10922ceaf2ccb751978b7334ddeca474da3d6b7aac99b 6380mm 16380 master-01554441449000 3 connected383261e3f0053f74c953bd07ceee36a4b5795bc3 192.168.56.12080A 6380th 16380 master-01554441450000 1 connectedd54639285709a65bb8ca331f26d4fe1b1c8c73ca 192.168.56.122master 6380th 16380 master-01554441450491 0 connected275753d11365feabb366170b940bca4b8486bbd7 192.168.56.122 6379379 master-01554441449470 4 connected 10923-16383

At present, there are still three nodes not in use. As a complete cluster, each node responsible for processing slots should have a slave node to ensure that it can fail over automatically when it fails. In cluster mode, the role of Redis node is divided into master node and slave node. The node started for the first time and the node of the assigned slot are the master node, and the slave node is responsible for copying the master node slot information and related data. Use the cluster replicate {nodeID} command to make a node a slave. Where the command must be executed on the corresponding slave node, and nodeID is the node ID of the master node to be copied. The command is as follows:

[redis@redis_1 ~] $redis-cli-h redis_1-p 6380 cluster replicate a85bbb17a3f559d18d7f5059f14ffd9f6ba48ee1OK [redis @ redis_1 ~] $redis-cli-h redis_2-p 6380 cluster replicate 275753d11365feabb366170b94 0bca4b8486bbd7OK [redis @ redis_1 ~] $redis-cli-h redis_3-p 6380 cluster replicate ceaf2ccb751978b7334ddeca474da3d6b7aac99bOK

After replication, the structure of the whole cluster is shown in the figure.

At this point, we manually set up a cluster according to the Redis protocol. It consists of six nodes, three master nodes are responsible for processing slots and related data, and three slave nodes are responsible for failover.

The content of the blog mainly refers to the book "Redis Development and Operation and maintenance".

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.