Shulou(Shulou.com)06/01 Report--

Blog outline:

I. the principle of Redis clustering

II. Details of Redis cluster architecture

III. Redis-cluster election

Deploy the Redis cluster environment

For a description of the performance tuning parameters of Redis, please refer to the blog post: Redis Database detailed solution and Parameter tuning of centos 7

First, the principle of Redis clustering:

1) Redis cluster architecture:

Redis Cluster uses virtual slot partitioning to map all the data into the slots of 0,16384 integers according to the algorithm. Redis Cluster is a centerless structure, and each node holds the data and the state of the entire cluster.

2) assign slotsSlave:Slave between cluster roles Master:Master to synchronize data to the Master specified by it

3) the TCP port 6379 used by the cluster node for the client connection port 16379 is used for the cluster bus

The Redis3.0 version begins to support clustering, using hash slot (hash slot), which can consolidate multiple Redis instances together to form a cluster, that is, spread data across multiple servers in the cluster.

Redis cluster (Redis cluster) is an acentric structure, as shown in the following figure, each node holds the data and the state of the entire cluster. Each node will save the information of other nodes, know the slot that other nodes are responsible for, and send heartbeat information regularly with other nodes, which can sense the abnormal nodes in the cluster in time.

When the client sends a command related to the database key to any node in the cluster, the node receiving the command calculates which slot the data to be processed by the command belongs to and checks whether the slot is assigned to itself. if the slot where the key is located happens to be assigned to the current node, then the node directly holds the command If the slot in which the key value is located is not assigned to the current node, the node returns a MOVED error to the client, directs the client to redirect the correct node, and sends the command you wanted to execute again.

The cluster roles are master and slave. Slots is allocated between master with a total of 16384 slot. Slave synchronizes data to the master specified by it for backup. When one of the master fails to provide service, the slave of that master is promoted to master to ensure the integrity of the intercluster slot. When one of the master and its slave fails, resulting in incomplete slot and cluster failure, it is necessary for the operation and maintenance staff to deal with it.

After the cluster is built, each node in the cluster will periodically send PING messages to other nodes. If the node receiving the PING message does not return the PONG message within the specified time, then the node sending the PING message will mark it as suspected offline (PFAIL). Each node exchanges the status information of each node in the cluster by sending messages to each other. If more than half of the primary nodes already in a cluster report a primary node x as suspected offline, then the primary node x will be marked as FAIL and a FAIL message about the primary node x will be broadcast to the cluster, and all nodes that receive the FAIL message will immediately mark the primary node x as offline.

When we need to reduce or increase the number of servers in the cluster, we need to change the slot that has been assigned to one node (source node) to another node (target node). And move the key-value pairs contained in the relevant slots from the source node to the target node.

The refragmentation of the Redis cluster is performed by Redis's cluster management software redis-trib, which does not support automatic fragmentation and requires you to calculate how much Slot is migrated from which nodes. During the refragmentation process, the cluster does not need to go offline, and both the source and destination nodes can continue to process command requests.

2. Details of Redis cluster architecture:

1. All Redis nodes are interconnected with each other (PING-PONG mechanism). Binary protocol is used internally to give priority to transmission speed and bandwidth.

2. The fail of nodes will not take effect until more than half of the primary (master) nodes in the cluster detect failures.

3. The client is directly connected to the redis node and does not need an intermediate agent (proxy) layer. The client does not need to connect all the nodes in the cluster, but can connect to any available node in the cluster.

4. Redis-cluster maps all physical nodes to [0-1638] slot, and cluster is responsible for maintaining node.

Slot

Key .

III. Redis-cluster election:

The election process is that all master in the cluster participate. If more than half of the master nodes time out to communicate with the current master node (cluster-node-timeout), the current master node is considered to be dead. The following two situations are when the entire cluster is unavailable (cluster_state:fail), when the cluster is unavailable, all operations to the cluster are not available, and a ((error) CLUSTERDOWN Thecluster is down) error is received:

Error 1: if any master in the cluster dies and the current master does not have slave, the cluster enters the fail state, and it can also be understood that if the slot mapping [0-16383] of the cluster is incomplete, it will enter the fail state.

Error 2: if more than half of the master in the cluster is down, the cluster enters the fail state with or without slave.

By default, the node of each cluster uses two TCP ports, one is 6379, the other is 16379bot 6379 serving client connections, and 16379 is used for cluster bus, which is the node-to-node communication channel using the binary protocol. Nodes use cluster bus for fault detection, configuration update, failover authorization, and so on.

4. Deploy the Redis cluster environment:

1. Environmental preparation:

1) six servers, three for master and three for slave. The IP address of centos 7 here is 192.168.1.1 (the number of servers participating in the cluster is preferably even, and each master will automatically correspond to a slave. If it is odd, the cluster cannot achieve redundancy, because there must be a master that does not have a corresponding slave. Once the master goes down, the whole cluster will lose some data.)

2) prepare the required source code package: download link, extract code: 4kky, configure local yum, need the system image to install the running environment of ruby.

3) configure the network and firewall to release the traffic from ports 6379 and 16379 of TCP. I have directly turned off the firewall here.

4) all redis servers must ensure that there is no data, preferably a new installation, because if there is any data, an error will be reported when clustering later. I have not delved into how to solve the error.

2. Start deployment:

① 192.168.1.1:

[root@localhost ~] # mount / root/redis-5.0.iso / media mount: / dev/loop1 write protection [root@localhost ~] # cd / media [root@localhost media] # cp * / usr/src/ # will be mounted read-only to copy all the files in iso to the specified directory [root@localhost media] # cd / usr/src [root@localhost src] # ls # to see if the relevant packages have been copied over debug kernels redis-3.2.0.gem redis-5.0.5.tar.gz [root@localhost Src] # tar zxf redis-5.0.5.tar.gz # unpack [root@localhost src] # cd redis-5.0.5/ # switch to the decompressed directory [root@localhost redis-5.0.5] # make & & make install # compile and install [root@localhost redis-5.0.5] # cd utils/ # to the subdirectory [root@localhost utils] #. / install_server.sh # generate service-related configuration files # the next step is to specify the location of the relevant configuration files Just press enter all the way. [root@localhost utils] # cd / etc/init.d/ # optimize redis control start and stop mode [root@localhost init.d] # mv redis_6379 redis [root@localhost init.d] # chkconfig-- add redis # add redis as a system service [root@localhost init.d] # systemctl restart redis # restart service To test whether [root@localhost init.d] # systemctl restart redis [root@localhost init.d] # vim / etc/redis/6379.conf # edits the main configuration file, change the following configuration items bind 127.0.0.1 192.168.1.1 # to add native IP. Change appendonly yes # to "yes" to log after each update operation, write data synchronously to daemonize yes # to see if the log file location is specified for "yes" logfile / var/log/redis_6379.log # You can keep the default cluster-enabled yes # startup cluster cluster-config-file nodes-6379.conf # cluster profile cluster-node-timeout 15000 # cluster node timeout Unit is millisecond cluster-require-full-coverage no # change the default "yes" to "no" port 6379 # default listening port number [root@localhost init.d] # systemctl restart redis # restart the service to make the configuration effective [root@localhost init.d] # netstat-anpt | grep redis # as long as 6379 and 16379 are listening, OK. Tcp 0 192.168.1.1 tcp 6379 0.0.0.0 tcp * LISTEN 7139/redis-server 1 [root@localhost] # yum-y install ruby rubygems # operating environment and client side [root@localhost] # cd / usr/src/ [root@localhost src] # ls # change to the directory containing redis-3.2.0.gem debug kernels redis-3.2.0.gem redis-5.0.5 redis-5.0.5.tar.gz [root@localhost src] # gem install redis--version 3.2.0 # execute the gem command Successfully installed redis-3.2.0Parsing documentation for redis-3.2.0Installing ri documentation for redis-3.2.01 gem installed

② 192.168.1.2:

[root@localhost ~] # scp root@192.168.1.1:/usr/src/redis-5.0.5.tar.gz / usr/src# copy the redis source code package of host 1.1 to The authenticity of host 192.168.1.1 (192.168.1.1) can t be established.ECDSA key fingerprint is SHA256:3U5q5gBxEOkHLFZZueGnEXGV/LZA9M4+1rM1lLvkD1g.ECDSA key fingerprint is MD5:ad:a1:9b:f7:e3:41:bf:5f:da:cd:5e:3f:74 E0:8a:b9.Are you sure you want to continue connecting (yes/no)? Yes # enter yesWarning: Permanently added '192.168.1.1' (ECDSA) to the list of known hosts.root@192.168.1.1 s password: # enter the user password of host 1.1 redis-5.0.5.tar.gz 100% 1929KB 68.5MB/s 00:00 [root@localhost ~] # cd / usr/src [root@localhost Src] # tar zxf redis-5.0.5.tar.gz # unpack [root@localhost src] # cd redis-5.0.5/ # enter the extracted directory [root@localhost redis-5.0.5] # make & & make install # compile and install [root@localhost redis-5.0.5] # cd utils/ # change to the subdirectory [root@localhost utils] #. / install_server. Sh # generate service-related configuration files # the next step is to specify the location of the relevant configuration files Just press enter all the way. [root@localhost utils] # cd / etc/init.d/ # optimize redis control start and stop mode [root@localhost init.d] # mv redis_6379 redis [root@localhost init.d] # chkconfig-- add redis # add redis as a system service [root@localhost init.d] # systemctl restart redis # restart service To test whether it works [root@localhost init.d] # scp root@192.168.1.1:/etc/redis/6379.conf / etc/redis/ # copy the redis configuration file of host 1.1 over root@192.168.1.1 s password: 6379.conf 100% 60KB 34.5MB/s 00:00 [root@localhost init.d] # scp root@192 .168.1.1: / etc/redis/6379.conf / etc/redis/ # Edit the configuration redis configuration file bind 127.0.0.1 192.168.1.2 # change the original 1.1 to 1.2 to [root@localhost init.d] # systemctl restart redis # restart the service Make the configuration file effective [root@localhost init.d] # netstat-natp | grep redis # check whether 6379 and 16379 are listening to tcp 0 0192.168.1.2 tcp 6379 0.0.0.0 * LISTEN 7479/redis-server 1 tcp 0 0192.168.1.2 natp 16379 0.0.0.0 * LISTEN 7479/redis-server 1

Now that 192.168.1.2 is also configured, go to 192.168.1.3-6 and repeat the configuration of this host. It is important to note that when changing the main configuration file copied from 1.1, the corresponding IP address can be replaced with the local IP address, and all other configuration commands are the same.

After all the remaining servers are configured, go back and configure host 192.168.1.1:

[root@localhost src] # redis-cli-- cluster create 192.168.1.1 cluster create 6379\ 192.168.1.3 cluster create 6379\ 192.168.1.3\ 192.168.1.4 cluster create 6379\ 192.168.1.5 cluster-replicas creates the cluster and adds all nodes to the cluster. " \ "indicates that the line break continues to enter, and the"\ "can be omitted. But blanks > > Performing hash slots allocation on 6 nodes...Master [0]-> Slots 0-5460Master [1]-> Slots 5461-10922Master [2]-> Slots 10923-16383Adding replica 192.168.1.5 Slots 6379 to 192.168.1.1:6379Adding replica 192.168.1.6 to 192.168.1.4 to 192.168.1.4 to 192.168.1.3purl 6379M: 758dc679fc1d46df195b4c16abcfde1d743b8008 192.168.1.1 5460Master 6379 slots: [0-5460] (5461 slots) masterM: 4b1d119d0a8179afcc10bf5891e5c04e6a7054ba 192.168.1.2 4b1d119d0a8179afcc10bf5891e5c04e6a7054ba slots: [5461-10922] (5462 slots) masterM: 58d342b59e98c9f9f469332b08c36b6ddd456bf6 192.168.1.3 slots 6379 slots: [10923-168.1.3] (5461 slots) masterS: bb95619aff3d613137733eeb55ff2d303a8720a1 192.168.1.4 slots 6379 replicates 58d342b59e98c9f9f469332b08c36b6ddd456bf6S: 22d77a79272847f29e5db381b98ef6ccc4a169a7 192.168.1.5 slots replicates 758dc679fc1d46df195b4c16abcfde1d743b8008S: ebc138bbea070c6496ae266b1a84936754603ad2 192.168.1.6 replicates 4b1d119d0a8179afcc10bf5891e5c04e6a7054baCan I set the above configuration? (type 'yes' to accept): yes # enter "yes" > Nodes configuration updated > Assign a different config epoch to each node > Sending CLUSTER MEET messages to join the clusterWaiting for the cluster to join. > > Performing Cluster Check (using node 192.168.1.1Nodes configuration updated 6379) M: 758dc679fc1d46df195b4c16abcfde1d743b8008 192.168.1.1 slots 6379 slots: [0-5460] (5461 slots) master 1 additional replica (s) S: ebc138bbea070c6496ae266b1a84936754603ad2 192.168.1.66379 slots: (0 Slots) slave replicates 4b1d119d0a8179afcc10bf5891e5c04e6a7054baM: 4b1d119d0a8179afcc10bf5891e5c04e6a7054ba 192.168.1.2 master 6379 slots: [5461-10922] (5462 slots) master 1 additional replica (s) M: 58d342b59e98c9f9f469332b08c36b6ddd456bf6 192.168.1.3 master 1 additional replica (s) slots: [10923-168.1.3 master 1 additional replica (s) S: bb95619aff3d613137733eeb55ff2d303a8720a1 192.168.1.4lav 6379 slots: (0 slots) slave replicates 58d342b59e98c9f9f469332b08c36b6ddd456bf6S: 22d77a79272847f29e5db381b98ef6ccc4a169a7 192.168.1.5 slots 6379 slots: (0 slots) slave replicates 758dc679fc1d46df195b4c16abcfde1d743b8008 ] All nodes agree about slots configuration. > Check for open slots... > Check slots coverage... [OK] All 16384 slots covered. [root@localhost src] # redis-cli-- cluster check 192.168.1.1 Check slots coverage... 6379 # View the cluster state 192.168.1.1 Check slots coverage... 6379 (758dc679...)-> 0 keys | 5461 slots | 1 slaves.192.168.1.2:6379 (4b1d119d...)-> 0 keys | 5462 slots | 1 slaves.192.168. 1.3 keys in 6379 (58d342b5...)-> 0 keys | 5461 slots | 1 slaves. [OK] 0 keys in 3 masters.0.00 keys per slot on average. > Performing Cluster Check (using node 192.168.1.1 keys in 6379) M: 758dc679fc1d46df195b4c16abcfde1d743b8008 192.168.1.1keys in 6379 slots: [0-5460] (5461 slots) master 1 additional replica (s) S: ebc138bbea070c6496ae266b1a84936754603ad2 192.168.1.6 master slots: (0 slots) slave replicates 4b1d119d0a8179afcc10bf5891e5c04e6a7054baM: 4b1d119d0a8179afcc10bf5891e5c04e6a7054ba 192.168 .1.2 master 6379 slots: [5461-10922] (5462 slots) master 1 additional replica (s) M: 58d342b59e98c9f9f469332b08c36b6ddd456bf6 192.168.1.3 master 6379 slots: [10923-16383] (5461 slots) master 1 additional replica (s) S: bb95619aff3d613137733eeb55ff2d303a8720a1 192.168.1.4 additional replica 6379 slots: (0 slots) slave replicates 58d342b59e98c9f9f469332b08c36b6ddd456bf6S: 22d77a79272847f29e5db381b98ef6ccc4a169a7 192.168.1.5 Check for 6379 slots: (0 slots) slave replicates 758dc679fc1d46df195b4c16abcfde1d743b8008 [OK] All nodes agree about slots configuration. > Check for > Open slots... > Check slots coverage... [OK] All 16384 slots covered. [root@localhost src] # redis-cli-h 192.168.1.1-p 6379-c # login to the cluster requires the "- c" option 192.168.1.1All 6379 > set lv 11 # insert data OK192.168.1.1:6379 > get lv "11" 192.168.1.1 All 6379 > exit [root@localhost src] # redis -cli-h 192.168.1.3-p 6379-c # Log in to other master192.168.1.3:6379 > get lv # to view the data inserted at 1.1 You can still query it, indicating that the cluster is valid-> Redirected to slot [4118] located at 192.168.1.1

The difference between redis-3.x.x and redis-5.x.x in creating a cluster:

Instead of using the command, redis-3.x.x uses the redis-trib.rb command with the following syntax:

[root@localhost ~] # redis-trib.rb create-- replicas 1 192.168.1.1 replicas 6379.. 192.168.1.6:6379#redis-3.x.x create a cluster. [root@localhost src] # redis-trib.rb check 192.168.1.1 redis-trib.rb check cluster status # redis-trib.rb cannot be used directly, you need to do the following to copy the script to local.. / bin directly using [root@localhost src] # cd / usr/src/redis-5.0.5/src/ [root@localhost src] # cp redis-trib.rb / usr/local/bin/ # for direct use. If not, you need to execute the file using ". /" in the directory

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.