Shulou(Shulou.com)06/01 Report--

Editor to share with you how to expand the cluster in Redis, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

When using Redis to build a cluster environment, it is usually necessary to expand the cluster according to the needs of the business to meet the needs of the business. And this is also a common requirement of distributed storage. The expansion of cluster in Redis is mainly divided into three steps:

1. Prepare the new node

two。 Join the cluster

3. Migration slots and data

Let's describe these three steps in detail:

Prepare the new node

Just like preparing other nodes, we modify the node configuration first.

# Node Port

Port 6385

# enable cluster mode

Cluster-enabled yes

# Node timeout (in milliseconds)

Cluster-node-timeout 15000

# Cluster internal configuration file

Cluster-config-file "nodes-6385.conf"

Start the node:

. / src/redis-server redis-6385.conf

. / src/redis-server redis-6386.conf

However, at this time, the new node has not yet communicated with other nodes, so it has not been added to the cluster environment yet.

Join the cluster

We can use the cluster meet command to add the new node to the cluster.

Cluster meet 127.0.0.1 6385

Cluster meet 127.0.0.1 6386

As shown in the figure above, although the new node is added to the cluster environment, the type of the new node is master, that is, it is all the primary node. Similarly, we can use the redis-trib.rb tool to add new nodes, and this command can directly support the addition of slave nodes.

Redis-trib.rb add-node 127.0.0.1:6385 127.0.0.1:6379

Redis-trib.rb add-node 127.0.0.1:6386 127.0.0.1:6379

In a production environment, we recommend using the redis-trib.rb command to add a new node because it performs a check when a new node is added, and discards the cluster join if the new node has joined the cluster or contains data.

Migration slots and data

After joining the cluster, you need to migrate slots and related data for the new nodes, and the cluster can provide read and write services normally during the migration process. Let's introduce this aspect in detail below.

1. Slot migration plan

Slots are the basic unit of Redis cluster management data. First of all, you need to specify the migration plan of slots for the new nodes, that is, which slots of which nodes will be migrated to the new nodes. And the migration plan should ensure that each node is responsible for a similar number of slots, so as to ensure that the data of each node is uniform. After the slot migration plan is determined, the data in the slot is migrated from the source node to the target node one by one.

two。 Migrate data

The data migration process is carried out slot by slot, and the process of each slot migration is as follows.

Process description:

1) send the cluster setslot {slot} importing {sourceNodeId} command to the target node to prepare the target node to import slot data.

2) send the cluster setslot {slot} migrating {targetNodeId} command to the source node to prepare the source node to migrate the slot data.

3) the source node iterates through the cluster getkeysinslot {slot} {count} command to obtain the keys of count data slots {slot}.

4) execute migrate {targetIp} {targetPort} "0 {timeout} {keys} {keys … on the source node. } command. The acquired keys are batch migrated to the target node through the pipelining (pipeline) mechanism.

5) repeat steps 3 and 4 until all the key data under the slot are migrated to the target node.

6) send the cluster setslot {slot} node {targetNodeId} command to all master nodes in the cluster, informing that the slot is assigned to the target node.

Let's manually use the above command to migrate the slot 4096 responsible for node 6379 to the target node 6385, as follows:

1. The target node is ready to import slot 4096 data.

Cluster setslot 4096 importing Target Node ID

two。 The source node prepares the export slot 4096 data.

Cluster setslot 4096 migrating Source Node ID

3. The keys corresponding to slot 4096 are obtained in batch.

Cluster getkeysinslot 4096 100

Let's first check whether the above key is included on the 6379 node.

Let's migrate these three keys.

Let's continue to query the above three keys in the 6379 node.

Next we inform all master node slots 4096 to assign to the target node 6385.

We can check in the 6379 node to make sure that the 4096 slot is no longer the responsibility of 6379.

The above is the process of performing slot migration manually. In the actual operation, it will be very inconvenient because a large number of slots and keys will be migrated, so the redis-trib tool provides slot slicing function as follows:

Redis-trib.rb reshard host:port-form-to-slots-yes-timeout-pepeline

Let's introduce the description of the above command parameters:

Host:port: compare and pass parameters and any node address in the cluster, which is used to obtain the information of the entire cluster.

-- form: specify the id of the source node, and if there is more than one source node, use a comma division.

-- to: the target node id that needs to be migrated, and only one target node can be specified.

-- slots: the total number of slots to be migrated.

-- yes: when printing out the reshard execution plan, whether the user is required to enter yes to confirm and then execute reshard.

-- timeout: controls the timeout for each migrate operation. The default is 60000 milliseconds.

-- pipeline: controls the number of keys for each batch migration. Default is 10.

Let's use the reshard command to migrate the data from the remaining slots.

After the above command is executed, the reshard command will prompt us to enter the number of migration slots, and we will temporarily enter 4096.

When we enter the number of migration slots, we will be prompted to enter the target node ID, and we will enter the 6385 node ID.

Similarly, after we have entered the target node, we will be prompted to enter the source node ID, that is, the primary node ID, so we enter the 6379, 6380, 6381 master node ID, and the done command has ended.

When the above command is entered, the plan of all slots from the source node to the target node will be printed, and we will enter the yes command before we continue with the migration work.

After entering the yes command, view the node status through the cluster nodes command.

We have the last step below, that is, although we have added 6385 and 6386 nodes to the cluster, and have migrated some slots and data to 6385 nodes, this node does not have a corresponding slave node. So the next step is to set 6386 nodes as the slave nodes of 6385 nodes.

The above is all the contents of the article "how to expand the cluster in Redis". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.