Shulou(Shulou.com)06/03 Report--

I believe that by reading the blog post: non-relational database (NoSQL)-- detailed explanation of Redis installation and deployment

We can have a preliminary understanding of the advantages of Redis database and its installation method. Today we will take a look at how to scale Redis database horizontally to meet greater access requirements.

Redis version 3.0 and later began to support cluster, using hash slot (hash slot), which allows multiple Redis instances to be grouped together to form a cluster, that is, to spread data across multiple machines in the cluster.

Redis clustering principle

Redis Cluster is an acentric structure in which each node holds data and the state of the entire cluster. Each node will save the information of other nodes, know the slot that other nodes are responsible for, and send heartbeat information regularly with other nodes, which can timely sense the abnormal nodes in the cluster. As shown in the figure:

When the client sends a command related to the database key to any node in the cluster, the node receiving the command calculates which slot the database key to be processed by the command belongs to and checks whether the slot is assigned to it. If the slot of the key is assigned to the current node, the node executes the command directly; if the slot of the key is not assigned to the current node, the node returns a MOVED error to the client, directing the client to the correct node and sending the command you wanted to execute again.

The cluster roles are Master and Slave. Slots is allocated between Master, with a total of 16384 slot. Slave synchronizes data to the Master specified by it for backup. When one of the Master is unable to provide services, the Slave of that Master is promoted to Master. To ensure the integrity of the cluster key slot. When one of the Master and its Slave fails, it will cause the slot to be incomplete and the cluster to fail, which requires manual repair.

After the cluster is built, each node in the cluster will periodically send PING messages to other nodes. If the node receiving the PING message does not return the PONG message within the specified time, then the node sending the PING message will mark it as suspected offline (PFAIL). Each node exchanges the status information of each node in the cluster by sending messages to each other. If more than half of the primary nodes in a cluster report a node X as suspected offline, the primary node X will be marked as FAIL and a FAIL message about the primary node X will be broadcast to the cluster, and all nodes that receive the FAIL message will immediately mark the primary node X as offline.

When we need to reduce or increase the number of machines in the cluster, we need to change the slot that has been assigned to one node (the source node) to another node (the target node), and move the key-value pair to which the relevant slot belongs from the source node to the target node.

The resharding of the Redis cluster is performed by Redis's cluster management software redis-trib, which does not support automatic sharding and requires you to calculate how much Slot is migrated from which nodes. During the resharding process, the cluster does not need to be offline, and both the source and destination nodes can continue to process command requests.

Architectural details

1. All redis nodes are interconnected with each other, and binary protocols are used internally to optimize transmission speed and bandwidth

two。 The failure of a node fails when more than half of the primary nodes in the cluster detect failure.

3. The client is directly connected to the redis node and does not need an intermediate proxy layer. The client does not need to connect all the nodes in the cluster, but can connect to any available node in the cluster.

4.redis-cluster maps all physical nodes to [0-16383] slot, and cluster is responsible for maintaining nodeslotkey

Redis-cluster elections:

In the election process, all master in the cluster participate. If more than half of the master nodes time out to communicate with the current master node, the current master node is considered to be dead. As shown in the figure:

The following two situations make the cluster unavailable:

If any master of the cluster dies and there is no slave in the current master, the cluster enters the fail state, which can also be understood as the fail state entered when the cluster's slot mapping [0-16383] is incomplete. If half of the master in the cluster dies, the cluster enters the fail state regardless of whether there is slave or not.

When the cluster is unavailable, all operations on the cluster are not available and a ((error) CLUSTERDOWN The cluster is down) error is received.

By default, each cluster node uses two TCP ports, one is 6379 and the other is 16379:

Port 6379 serves client connections; port 16379 is used for cluster bus

Even using the node-to-node communication channel of the binary protocol. Nodes use cluster bus for fault detection, configuration update, failover authorization, and so on. If the firewall is turned on, you need to open this port (this blog uses a forced shutdown of the firewall). Redis cluster deployment

Once you understand the principle of Redis clustering, it becomes very easy to build redis clusters. In this experiment, 6 servers are used to build Redis cluster, of which 3 are master,3 and salve. The IP address of the six servers is 192.168.1.1, and the servers are all centos 7 systems. The experimental topography will not be uploaded here.

The specific steps for Redis cluster deployment are as follows:

1. Install Redis and modify the configuration file

For more information on how to build a Redis server and how to obtain its software package, please refer to the blog post: non-relational database (NoSQL)-- detailed explanation of Redis installation and deployment

Install according to the blog post. Redis needs to be installed on all 6 servers. After installation, each server needs to be modified to the real local IP address and cancel the loopback address (127.0.0.1). The specific modifications are as follows:

[root@localhost ~] # vim / etc/redis/6379.conf bind 192.168.1.1 / / changed to native IP address daemonize yeslogfile / var/log/redis_6379.logcluster-enabled yes / / enable cluster cluster-config-file nodes-6379.confcluster-node-timeout 15000cluster-require-full-coverage no// about clustering these entries exist by default It's just a comment, just cancel the comment! [root@localhost ~] # / etc/init.d/redis_6379 restart// restart the redis service Stopping... Redis stoppedStarting Redis server... [root@localhost ~] # netstat-anpt | grep 6379tcp 0 0192.168.1.1 tcp 6379 0.0.0.0 tcp 0 0192.168.1.1 redis 16379 0.0.0. 0 LISTEN 6497/redis-server 1 tcp 0 0 127.0.0.1 TIME_WAIT-/ / 6379 port and port 16379 are listening

The first redis server has been set up, and the remaining 5 are done the same way, except that the configuration files are all their own IP addresses. Don't get confused!

two。 Create a cluster using a script

A script for ruby is used to create a cluster. Before creating a cluster, you need to install the running environment of ruby and the Redis client of ruby, which can be done on one of the servers. The gem command is provided by the redis-3.2.0 gem package downloaded in advance and can be used by uploading directly to the server!

Gem package network disk link: https://pan.baidu.com/s/1Fx7YI-ZwZoHOPm13HYeFKw

Extraction code: cj0x

[root@localhost ~] # yum-y install ruby rubygems// running environment for installing ruby [root@localhost ~] # gem install redis--version 3.2.0Successfully installed redis-3.2.0Parsing documentation for redis-3.2.0Installing ri documentation for redis-3.2.01 gem installed// installing ruby Redis client tool-- gem command

Use a script to create a cluster:

[root@localhost] # cd / usr/src/redis-3.2.9/src [root@localhost src] #. / redis-trib.rb create-- replicas 1 192.168.1.1V 6379 192.168.1.2Rd 6379 192.168.1.3V 6379 192.168.1.4R 6379 192.168.1.5V 6379 192.168.1.6 / / omit part of the content Can I set the above configuration? (type 'yes' to accept): yes. / / omit part of the content [OK] All nodes agree about slots configuration. > > Check for open slots... > Check slots coverage... [OK] All 16384 slots covered.// to create a cluster [root@localhost src] #. / redis-trib.rb check 192.168.1.1Check for open slots... 6379amp / check the status of the cluster (you can clearly see which host is master, Which host is salve) > Performing Cluster Check (using node 192.168.1.1) slots:0-5460 (5461 slots) master 1 additional replica (s) S: 00ed4b0da4f0444c7e2a54a44c2060dd2c51a19a 192.168.1.4 master 1 additional replica (s) slots: (0 slots) slave replicates afb2a0876b9c4c1c19e2bc492e398765bed0a311S: 081d87a0d26895605e4c237c5429d3ae6e01f7b2 192.168.1.56379 slots: (0 slots) slave replicates 050d71e6ad9bbf0a2a90b743d5a9bb9fb77052bbS: 00bf8cb3a48a696d9bfc4b124234335633dc14d0 192.168.1.6 slots: (0 slots) Slave replicates bec4c3401ced5a43439568f5530d79dd2a911512M: 050d71e6ad9bbf0a2a90b743d5a9bb9fb77052bb 192.168.1.2 master 6379 slots:5461-10922 (5462 slots) master 1 additional replica (s) M: bec4c3401ced5a43439568f5530d79dd2a911512 192.168.1.3 additional replica slots:10923-16383 (5461 slots) master 1 additional replica (s) [OK] All nodes agree about slots configuration. > > Check for open slots... > > Check slots coverage... [OK] All 16384slots covered.3. Test cluster

To log in to the cluster and set the key value test, you need to use the "- c" parameter to activate the cluster, as follows:

[root@localhost ~] # redis-cli-h 192.168.1.1-p 6379-c 192.168.1.1 set K11-> Redirected to slot [12706] located at 192.168.1.3:6379OK192.168.1.3:6379 > get K1 "1"

Automatically synchronize data on its redis server to view the results:

[root@localhost ~] # redis-cli-h 192.168.1.5-p 6379-c 192.168.1.5 get 6379 > get K1-> Redirected to slot [12706] located at 192.168.1.3 get 6379 "1" / / hints are synchronized from the 192.168.1.3 server

The experiment is complete!

-this is the end of this article. Thank you for reading-

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.