Shulou(Shulou.com)06/02 Report--

A brief introduction to Redis

Redis is a cache middleware used on a large scale. Because of its powerful, efficient and convenient functions, it has been widely used.

Redis released 3.0.0 in 2015, and redis cluster has been officially supported. When redis cluster is designed, it takes decentralization and middleware into account. In other words, each node in the cluster has an equal relationship, and each node saves its own data and the state of the entire cluster. Each node is connected to all other nodes, and these connections remain active, which ensures that we only need to connect to any node in the cluster to get the data of other nodes.

II. Introduction of Redis cluster

A Redis cluster is a facility that can share data among multiple Redis nodes. Redis clusters do not support redis commands that need to process multiple keys at the same time, because executing these commands requires moving data between multiple nodes, and in high load situations, these commands degrade the performance of the redis cluster and lead to unpredictable behavior.

Redis clusters provide a certain degree of availability through partitioning, and the cluster can continue to process command requests even if some nodes in the cluster fail or are unable to communicate.

Third, the principle of Redis cluster

How does Redis distribute these nodes and data reasonably?

Instead of using the traditional consistent hash to allocate data, Redis uses another way called hash slot to allocate data. Redis cluster allocates 16384 slot by default. When we set a key, we use the CRC16 algorithm (cyclic redundancy check code) to obtain the slot, and then divide the key into the node mountains of the hash slot. The specific algorithm: CRC16 (key) 384.

Note: there must be more than three primary nodes to create the cluster, otherwise the cluster fails.

When the Redis cluster is working, when one of the master nodes goes down, other master nodes will vote and elect, and more than half of the voting results will determine that the master node is down. If more than half of the master node is down, the status of the whole cluster is fail. This is why at least three primary nodes are required to build a Redis cluster environment.

Benefits of Redis clustering:

1) the ability to automatically split split data to multiple nodes; 2) when part of the nodes in the cluster fail or are unable to communicate, they still have the ability to continue to process command requests; 3) all rredis nodes are interconnected with each other, and binary protocol is used internally to optimize transmission speed and bandwidth; binary protocol: message headers and message entities are given during transmission. If the header length is fixed. The message entity can be obtained by parsing the message header; 4) the fail of the node takes effect only when the failure is detected by more than half of the nodes in the cluster; 5) the client is directly connected to the redis node and does not need an intermediate proxy layer, and the client does not need to connect to all the nodes in the cluster, but to connect to any available node in the cluster.

The allocation of hash slots adopted by Redis is both good and bad, and the advantage is that it is very clear. For example, I want to add a node called Dredis cluster, which takes a part of the slot from the front of each node to D. It would look something like this:

Node A covers 1365-5460 nodes, B covers 6827-10922 nodes, C covers 12288-16383 nodes, D covers 01364, 5461-6826, 10923-12287 similarly deletes a node, which can be deleted after the move is completed.

So redis cluster is such a shape, as shown in the figure:

Fourth, Redis-cluster master-slave mode

In order to ensure the high availability of data, Redis-cluster adds the master-slave mode, in which one master node corresponds to one or more slave nodes. The master node provides data access, and the slave node obtains data backup from the master node; when the master node is down, one of the slave nodes will be selected to act as the master node to ensure that the cluster will not hang up.

V. case implementation

For more than one Linux to implement redis cluster, you can refer to Redis database cluster.

(1) case environment

Because a single server is used to implement a cluster environment, it is distinguished according to its port!

(2) case implementation

Download Redis Cluster

1) create six nodes [root@docker ~] # tar zxf redis-4.0.14.tar.gz-C / usr/local [root@docker ~] # cd / usr/local/redis-4.0.14/ [root@docker redis-4.0.14] # make & & make install / / compile and install Redis [root @ redis ~] # mkdir-p / usr/local/cluster/700 {0.5} / / because multiple nodes are created on a single machine So create this directory in advance. Used to store the configuration information of each node [root@docker ~] # cp / usr/local/redis-4.0.14/redis.conf / usr/local/cluster/7000// copy the original configuration file of redis [root@docker ~] # vim / usr/local/cluster/7000/redis.conf / / compile the configuration file of 7000 nodes 92 port 7000lap / modify the listening end Port 158 pidfile / var/run/redis_7000.pid / / modify pid file name 672 appendonly yes / / enable aof persistence 676 appendfilename "appendonly-7000.aof" / / modify persistent configuration file name 814 cluster-enabled yes / / enable redis cluster mode 822 cluster-config-file nodes-7000.conf / / modify the configuration of the cluster Set the filename cluster-node-timeout 5000 / / modify the waiting time of the cluster [root@docker ~] # cd / usr/local/cluster/7000 [root@docker 7000] # redis-server redis.conf / / you must enter the appropriate directory to start the redis service

However, when you start, an error like this will occur, as shown in the figure:

[root@redis 7000] # vim / usr/local/cluster/7000/redis.conf / / write a configuration file for 7000 nodes 136daemonize yes / / start the daemon to run [root@redis 7000] # echo 512 > / proc/sys/net/core/somaxconn [root@redis 7000] # echo "vm.overcommit_memory = 1" > > / etc/sysctl.conf [root@redis 7000] # sysctl-pvm in the background .overcommit _ memory = 1 [root@redis 7000] # echo never > / sys/kernel/mm/transparent_hugepage/enabled// simply optimize the redis service according to the warning message just prompted 2) start each node [root@redis 7000] # redis-server redis.conf / / enter its directory to run the redis service [root@redis 7000] # cp redis.conf.. / 7001 [root@redis 7000] # cp redis. Conf.. / 7002 [root@redis 7000] # cp redis.conf.. / 7003 [root@redis 7000] # cp redis.conf.. / 7004 [root@redis 7000] # cp redis.conf.. / 7005ax / because six nodes need to be enabled So its configuration file needs to be divided into six [root@redis 7000] # sed-is / 70007002/redis.conf 7001xg.. / 7001/redis.conf [root@redis 7000] # sed-is / 7000Universe g.. / 7002/redis.conf [root@redis 7000] # sed-is / 7000max 7003max g.. / 7003/redis.conf [root@redis 7000] # sed-is / 7000Unip 7004max.. / 7004/redis.conf [root@redis 7000] # sed-is / 7000Universe g.. / 7005/redis.conf / / because all six nodes are distinguished by port So you only need to modify the content named by the port number in the configuration file [root@redis 7000] # cd.. / 7001 & & redis-server redis.conf [root@redis 7001] # cd.. / 7002 & & redis-server redis.conf [root@redis 7002] # cd.. / 7003 & & redis-server redis.conf [root@redis 7003] # cd.. / 7004 & & redis-server redis.conf [root@redis 7004] # cd.. / 7005 & & redis-server redis.conf / / all enter their corresponding directory startup service [root@redis 7005] # netstat-anpt | grep redis tcp 00 127.0.0.1 anpt 7004 0.0.0.0 LISTEN 5615/redis-server 1 tcp 00 127.0.0.1 anpt 7005 0.0.0.0 * LISTEN 5620/redis -server 1 tcp 00 127.0.0.1 LISTEN 5524/redis-server * LISTEN 5524/redis-server 1 tcp 00 127.0.0.1 LISTEN 5524/redis-server 17001 0.0.0.0 LISTEN 5592/redis-server 1 tcp 00 127.0.0.1 LISTEN 5524/redis-server 17002 0.0. 0.0LISTEN 5610/redis-server * LISTEN 5605/redis-server 1 tcp 00 127.0.0.1 LISTEN 5610/redis-server 1 tcp 00 127.0.0.1 LISTEN 5615/redis-server 1 tcp 0 0 127.0.0.1 LISTEN 5524/redis-server 17005 0.0.0.0 LISTEN 5620/redis-server * LISTEN 5524/redis-server 1 tcp 00 127.0.0.1 LISTEN 5524/redis-server 7001 0.0.0.0 * LISTEN 5592/redis-server 1 tcp 00 127.0.0.1 LISTEN 5605/redis-server * LISTEN 5605/redis-server 1 tcp 00 127.0.0.1 LISTEN 5605/redis-server 7003 0.0.0.0 tcp * LISTEN 5610/redis-server 1 / / View ports are already listening The remaining 17000 ports are used for communication between clusters. 3) assign hash slots to each node. And join the cluster environment [root@redis 7005] # redis-cli-p 7000 / / randomly specify a port to enter redis terminal 127.0.0.1 redis 7000 > CLUSTER INFO / / View cluster details cluster_state:fail / / find that it is now in the state of fail (failure). / / omit part 127.0.0.1 myself 7000 > CLUSTER NODES / / View cluster node myself: 7000mm 17000 Master-000 connected// found that only 7000 nodes joined the cluster: 127.0.0.1 redis-cli 7000 > exit [root@redis 7005] # redis-cli-h 127.0.0.1-p 7000 cluster addslots {0.5461} OK [root@redis 7005] # redis-cli-h 127.0.0.1-p 7001 cluster addslots {5462... 10922} OK [root@redis 7005] # redis-cli-h 127.0.0.1-p 7002 cluster addslots {10923... 16383} OK// assigns hash slot 127.0.0.1 7000OK127.0.0.1:7000 7000 > CLUSTER MEET 127.0.0.1 7000OK127.0.0.1:7000 > CLUSTER MEET 127.0.0.1 7001OK127.0.0.1:7000 > CLUSTER MEET 127.0.0.1 7002OK127.0.0.1:7000 > CLUSTER MEET 127.0.0.1 7003OK127.0.0.1:7000 > CLUSTER MEET 127.0.0.1 7004OK127.0.0.1:7000 > CLUSTER for each cluster stage MEET 127.0.0.1 7005OK// is used to connect different Redis nodes with cluster support enabled. To enter the working cluster [root@redis 7005] # redis-cli-p 7000127.0.0.1 CLUSTER INFOcluster_state:ok 7000 > CLUSTER INFOcluster_state:ok / / check the cluster status again and find that it is already "ok". / / omit some contents: 127.0.0.1v7000 > CLUSTER NODES / / View the cluster node information again

As shown in the figure:

The default environment is that 7000 is the master of 7003, 7001 is the master of 7004, and 7002 is the master of 7005. Do the following:

[root@redis 7005] # redis-cli-p 7003 / / Node 127.0.0.1 ID 7003 > CLUSTER REPLICATE d554512885b2679d432d0d6b011c9ea56ea1ebebOK / / this is ID number 127.0.0.1 ID 7003 > exit [root@redis 7005] # redis-cli-p 7004 / / Node connecting 7004 127.0.0.1 ID 7004 > CLUSTER REPLICATE 182a55ee8b38afbf5b48209979f220dc4da7c14aOK / / this is ID number .0.0.1: 7004 > exit [root@redis 7005] # redis-cli-p 7005 / / Node of connection 7005 127.0.0.1 redis-cli 7005 > CLUSTER REPLICATE 82b62bda5b31d8ee27aee8d7320663ee86276e9fOK / / this is the ID number of 7002 127.0.0.1 redis-cli 7005 > CLUSTER NODES / / View the details of the cluster node again

As shown in the figure:

4) access data for testing [root@redis 7005] # redis-cli-p 7000 / / enter the node to insert data for testing 127.0.0.1 set name lzj 7000 > set name lzj (error) MOVED 5798 127.0.0.1 set name lzj 7001 / / an error is found It is suggested that there should be 7001 nodes to store this data 127.0.0.1redis-cli 7000 > exit [root@redis 7005] # redis-cli-p 7000-c / / using the "- c" option to indicate entry into the cluster 127.0.0.1 redis-cli 7000 > set name lzj / / reinsert data-> Redirected to slot [5798] located at 127.0.0.1:7001OK / / Prompt to store the data on the 7001 node 127.0.0.1 get name 7001 > the value corresponding to the get name / / get key "lzj" 5) simulate the failure of a single node [root@redis 7001] # cd.. / 7001 [root@redis 7001] # redis-cli-p 7001 shutdown// simulate 7001 node failure [root@redis 7001] # redis-cli-p 7000127.0.0.1 7000 > CLUSTER NODES / / enter the node to view the status of the cluster node 127.0.0.1 7004 > get name "lzj"

As shown in the figure:

[root@redis 7001] # redis-server redis.conf / / enter the directory corresponding to 7001, start the redis node [root@redis 7001] # redis-cli-p 7000 127.0.0.17000 > CLUSTER NODES / / enter the cluster to view the cluster status

As shown in the figure:

6) install ruby software

Download the software package required for ruby

[root@redis ~] # yum-y install rpm-build openssl openssl-devel// installation of ruby depends on [root@redis] # tar zxf ruby-2.3.1.tar.gz-C / usr/srccd [root@redis ~] # cd / usr/src/ruby-2.3.1/ [root@redis ruby-2.3.1] #. / configure-- prefix=/usr/local/ruby & & make & & make install// compilation and installation of ruby It may take a long time [root@redis] # ln-s / usr/local/ruby/bin/* / usr/local/bin// to create symbolic links for gem commands [root@redis] # ln-s / usr/local/redis-4.0.14/src/* / usr/local/bin// to create symbolic links for common redis commands [root@redis ~] # gem install redis-3.3.0.gem / / use the gem command to install Redis Cluster must-have package 7) add master node 7006 [root@redis ~] # redis-trib.rb check 127.0.0.1VOG 7000max / check the status of cluster node

As shown in the figure:

[root@redis ~] # mkdir / usr/local/cluster/7006 [root@redis ~] # cd / usr/local/cluster/7006// create the corresponding directory and go to [root@redis 7006] # cp.. / 7000/redis.conf. [root@redis 7006] # sed-is / 7006ampg redis.conf / / copy configuration file, And modify [root@redis 7006] # redis-server redis.conf / / modify the redis service of 7006 nodes [root@redis ~] # redis-trib.rb add-node 127.0.0.1 redis 7006 127.0.0.1 redis / add 7006 nodes to the 7000-node cluster environment (default is master state) [root@redis ~] # redis-trib.rb check 127.0.0.1UX 7000amp / check the cluster status again

As shown in the figure:

Screenshot, accidentally blocked, you can find that 7006 this node is no slot point, can not store data!

[root@redis ~] # redis-trib.rb reshard 127.0.0.1 How many slots do you want to move 7000 / / assign 7000 slot points of this cluster How many slots do you want to move (from 1 to 16384)? 4096 / / operate on how many nodes What is the receiving node ID?7add42108b3fc3e9c0af2846fb06ceaf8b172c87 / / to 7006 nodes What you enter here is the ID number of 7006 nodes, Source node # 1:all / / assign [root@redis ~] # redis-trib.rb check 127.0.0.1 1:all 7000amp from all nodes / check the cluster status again

As shown in the figure:

8) add a slave node 7007 [root@redis ~] # mkdir / usr/local/cluster/7007 [root@redis ~] # cd / usr/local/cluster/7007// to create the corresponding directory and enter [root@redis 7007] # cp.. / 7000/redis.conf. [root@redis 7007] # sed-is / 7000GUBG redis.conf// copy configuration file and modify [root@redis 7007] # redis-server redis.conf// launch Section 7007 Point Redis service [root@redis ~] # redis-trib.rb add-node-- slave 127.0.0.1 slave 7007 127.0.0.1 slave 7000Universe / add 7007 nodes to the cluster And specify it as slave [root@redis ~] # redis-trib.rb check 127.0.0.1 redis-trib.rb check 7000amp / check the cluster status

As shown in the figure:

9) add slave node 7008 [root@redis ~] # mkdir / usr/local/cluster/7008 [root@redis ~] # cd / usr/local/cluster/7008 [root@redis 7008] # cp.. / 7000/redis.conf. [root@redis 7008] # sed-is / 7008 redis.conf g redis.conf [root@redis 7008] # redis-server redis.conf / / create the corresponding directory and modify the configuration file Start the service [root@redis ~] # redis-trib.rb add-node-- slave-- master-id 7add42108b3fc3e9c0af2846fb06ceaf8b172c87 127.0.0.1 slave 7008 127.0.0.1 slave 7000lash / add a node to the cluster, specify it as salve, and specify the primary node ID number (ID of 7006 nodes) [root@redis ~] # redis-trib.rb check 127.0.0.1 slave 7000Lap / View cluster status

As shown in the figure:

10) delete 7000 nodes [root@redis ~] # redis-trib.rb reshard 127.0.0.1 How many slots do you want to move 7000 / / operate on cluster nodes How many slots do you want to move (from 1 to 16384)? 4096 / / operate on multiple nodes What is the receiving node ID? 7add42108b3fc3e9c0af2846fb06ceaf8b172c87 / / enter here to delete Slot node to which node (7006 node) Source node d554512885b2679d432d0d6b011c9ea56ea1ebeb / / enter the deleted node Source node doneDo you want to proceed with the proposed reshard plan (yes/no)? Yes// confirms that [root@redis ~] # redis-trib.rb del-node 127.0.0.1 redis-trib.rb del-node 7000 d554512885b2679d432d0d6b011c9ea56ea1ebeb// deletes 7000 nodes in the 7000 cluster [root@redis ~] # redis-trib.rb check 127.0.0.1 redis-trib.rb del-node 7001 / / View the cluster status

As shown in the figure:

-this is the end of this article. Thank you for reading-

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.