Shulou(Shulou.com)06/02 Report--

Redis cluster cluster

When Redis-Cluster is designed, it takes into account decentralization, de-middleware, each node in the cluster is equal and peer-to-peer, and each node saves its own data and the state of the whole cluster. Each node is connected to all other nodes, and these connections remain active, which ensures that we only need to connect to any node in the cluster to get the data of other nodes.

Instead of using traditional consistent hashes to allocate data, Redis clusters allocate data in another way called hash slot. Redis cluster allocates 16384 slot by default. When we set a key, we use the CRC16 algorithm to get the slot we belong to, and then assign the key to the nodes in the hash slot. The specific algorithm is: CRC16 (key)% 16384. So when we saw set and get during the test, we jumped directly to the node where the default first configuration started.

The Redis cluster stores the data in a master node and then synchronizes the data between the master and its corresponding salve. When the data is read, the data is also obtained from the corresponding master node according to the consistent hashing algorithm. Only when a master is dead will a corresponding salve node be started to act as a master.

It should be noted that the Redis cluster is at least 3-3, which continues the design concept of Master-Slave.

1. Installation dependency

Yum-y install gcc gcc-c++ tcl ruby ruby-devel rubygems rpm-build

Execute after installing Redis: gem install redis

II. Redis configuration example

Conf directory: redis_cluster

Data directory: data

Logs directory: logs

Port instance; 6001 6001 6002 6003 6004 6005 6006

# cat redis.conf

Bind 10.10.101.100

Protected-mode yes

Port 6379

# Port configuration 6379

Tcp-backlog 511

Timeout 0

Tcp-keepalive 300

Daemonize yes

# redis running in the background

Cluster-enabled yes

# start the cluster

Cluster-config-file nodes_6379.conf

Cluster configuration starts generation for the first time

Cluster-node-timeout 5000

# request timeout

Appendonly yes

# enable aof log

Supervised no

Pidfile / var/run/redis_6379.pid

Loglevel notice

Logfile / opt/redis/logs/6379/redis.log

# Log directory

Databases 16

Save 900 1

Save 300 10

Save 60 10000

Stop-writes-on-bgsave-error yes

Rdbcompression yes

Rdbchecksum yes

Dbfilename dump.rdb

Dir / opt/redis/data/6379

Slave-serve-stale-data yes

Slave-read-only yes

Repl-diskless-sync no

Repl-diskless-sync-delay 5

Repl-disable-tcp-nodelay no

Slave-priority 100

Appendfilename "appendonly.6379.aof"

Appendfsync everysec

No-appendfsync-on-rewrite no

Auto-aof-rewrite-percentage 100

Auto-aof-rewrite-min-size 64mb

Aof-load-truncated yes

Lua-time-limit 5000

Slowlog-log-slower-than 10000

Slowlog-max-len 128

Latency-monitor-threshold 0

Notify-keyspace-events ""

Hash-max-ziplist-entries 512

Hash-max-ziplist-value 64

List-max-ziplist-size-2

List-compress-depth 0

Set-max-intset-entries 512

Zset-max-ziplist-entries 128

Zset-max-ziplist-value 64

Hll-sparse-max-bytes 3000

Activerehashing yes

Client-output-buffer-limit normal 0 0 0

Client-output-buffer-limit slave 256mb 64mb 60

Client-output-buffer-limit pubsub 32mb 8mb 60

Hz 10

Aof-rewrite-incremental-fsync yes

III. Start

Src/redis-server redis_cluster/6001/redis.conf

Src/redis-server redis_cluster/6002/redis.conf

Src/redis-server redis_cluster/6003/redis.conf

Src/redis-server redis_cluster/6004/redis.conf

Src/redis-server redis_cluster/6005/redis.conf

Src/redis-server redis_cluster/6006/redis.conf

IV. Cluster operation

Writing a node configuration file can be easily done by using the Redis cluster command line tool redis-trib: redis-trib is located in the src folder of the Redis source code, which is a Ruby program that sends special commands to the instance to create a new cluster, check the cluster, or reshared the cluster.

1. Create a cluster

Redis-trib.rb create-- replicas 1 10.10.101.100 6001 10.10.101.100 6002 10.10.101.100 6003 10.10.101.100 6004 10.10.101.100 6005 10.10.101.100

/ /-- replicas 1 means that a slave node is created for each master node in the cluster.

2. Cluster rescheduling

. / redis-trib.rb reshard 10.10.101.100:6003

/ / the number of mobile hash slots (1-16383) and the target node ID need to be specified

# src/redis-cli-h 10.10.101.100-p 6003 cluster nodes | grep myself

Root@redis: / opt/redis# redis-cli-h 10.10.101.100-p 6003 cluster nodes | grep myself

739f446f69739eb7f95331eec7c74a4e57d2eeac 10.10.101.100 myself,master-003 connected 10923-16383

3. Add a new node

The basic process of adding a new node is to add an empty node and then move some data to it. In two cases, add a master node and add a slave node (when adding a slave node, you need to set this new node as a replication of a node in the cluster).

In both cases, the first step is to add an empty node.

A. add a node to the existing cluster.

. / redis-trib.rb add-node 10.10.101.100:6379 10.10.101.100:6003

/ / use the addnode command to add a node. The first parameter is the address of the new node, and the second parameter is the IP and port of any existing node.

Root@redis: / opt/redis# src/redis-cli-c-h 10.10.101.100-p 6379

10.10.101.100Suzhou 6379 > cluster nodes

0d9cb75cf92ce44dc3a4ad48bd95b48fc0d7a5ca 10.10.101.100 0d9cb75cf92ce44dc3a4ad48bd95b48fc0d7a5ca 6379 myself,master-00 connected

40d6721c15f32d732da89871b941788c3c57dcd3 10.10.101.100 40d6721c15f32d732da89871b941788c3c57dcd3 6001 master-0 1494413014845 1 connected 0-5460

041cb360dfcd8c1e8c77d8e7db03183da553953b 10.10.101.100:6004 slave 40d6721c15f32d732da89871b941788c3c57dcd3 0 1494413014344 1 connected

C9d5f58cd5e74f2c5ebbf6213e4314f4e92fb389 10.10.101.100:6006 slave 739f446f69739eb7f95331eec7c74a4e57d2eeac 0 1494413014644 3 connected

5a606eef48bc1fd8be9ad4d7a1d2a9420d8b2552 10.10.101.100 5a606eef48bc1fd8be9ad4d7a1d2a9420d8b2552 6002 master-01494413015845 2 connected 5461-10922

F539cc2330a19f31a8dd55360e184cda2dc81151 10.10.101.100:6005 slave 5a606eef48bc1fd8be9ad4d7a1d2a9420d8b2552 0 1494413015345 2 connected

739f446f69739eb7f95331eec7c74a4e57d2eeac 10.10.101.100 739f446f69739eb7f95331eec7c74a4e57d2eeac 6003 master-01494413016345 3 connected 10923-16383

10.10.101.100DI 6379 >

The new node does not contain any data because it does not contain any hash slots.

Although the new node does not contain any hash slots, it is still a master node, so when the cluster needs to upgrade a slave node to a new master node, the new node will not be selected. As long as some hash buckets in the cluster are moved to the new node, the new node will become the real master node.

4. Add a slave node

There are two ways to add slave nodes, either using the redis-trib command as you would add a master node, or using the-slave option as in the following example:

Src/redis-trib.rb add-node-- slave 10.10.101.100RV 6379 10.10.101.100RU 6001

/ / the master node of the added slave node is not specified here, in which case the system will randomly select one of the master nodes in other replication sets as the master node of this slave node.

A. Specify the master node to add:

Src/redis-trib.rb add-node-- slave-- master-id 739f446f69739eb7f95331eec7c74a4e57d2eeac 10.10.101.100 master-id 739f446f69739eb7f95331eec7c74a4e57d2eeac 6379 10.10.101.100

B. Add a slave node to the master node 10.10.101.100virtual 6003, which has a range of 10923-16383 hash slots

Node ID 739f446f69739eb7f95331eec7c74a4e57d2eeac, we need to link the new node (the already empty master node) and execute the command:

Redis 10.10.101.100 redis 6379 > cluster replicate 739f446f69739eb7f95331eec7c74a4e57d2eeac

C. Verify the cluster node

$redis-cli-p 6379 cluster nodes | grep slave | grep 739f446f69739eb7f95331eec7c74a4e57d2eeac

F093c80dde814da99c5cf72a7dd01590792b783b 10.10.101.100:6379 slave 739f446f69739eb7f95331eec7c74a4e57d2eeac 0 1385543617702 3 connected

041cb360dfcd8c1e8c77d8e7db03183da553953b 10.10.101.100:6004 slave 40d6721c15f32d732da89871b941788c3c57dcd3 0 1494413014344 1 connected

/ / Node 739f446f69739eb7f95331eec7c74a4e57d2eeac has two slave nodes, 6004 (existing) and 6379 (newly added).

5. Remove a node

Simply use the del-node command:

Src/redis-trib del-node 10.10.101.100 6001 ``

/ / the first parameter is the address of any node, and the second node is the address of the node you want to remove.

Use the same method to remove the primary node, but make sure that the primary node is empty before removing it. If it is not empty, you need to refragment the data of this node to other master nodes.

The alternative to removing the master node is to perform fault recovery manually, and the removed master node will exist as a slave node, but in this case the number of cluster nodes will not be reduced and the data will need to be refragmented.

6. Migration from nodes

There is a situation where the master node of a slave node is changed in the Redis cluster, and the following command needs to be executed:

CLUSTER REPLICATE

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.