Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of how to install a six-node cluster in Redis, the content is detailed and easy to understand, the operation is simple and fast, and has a certain reference value. I believe you will gain something after reading this article on how to install a six-node cluster in Redis. Let's take a look.

Brief introduction

Redis cluster is a native clustering solution, and at present, great progress has been made in terms of high availability and stability. According to statistics and observation, more and more companies and communities adopt redis cluster architecture, which has become the standard of reality. Its main feature is decentralization and no need for proxy agent. One of the main design goals is to achieve linear scalability (linear scalability).

The redis cluster server alone cannot perform the officially promised functions. In a broad sense, redis cluster should include both redis server and client-side implementations such as jedis. They are a whole.

Distributed storage is nothing more than processing shards and replicas. For redis cluster, the core concept is slot, which gives you a basic understanding of how clusters are managed.

Advantages and disadvantages

When you understand these features, it is actually easier in operation and maintenance. Let's first take a look at the obvious advantages and disadvantages.

Advantages

1. No additional Sentinel clusters are needed, which provides users with a consistent solution and reduces the cost of learning.

2. Decentralized architecture, peer-to-peer, and the cluster can support thousands of nodes.

3. The concept of slot is abstracted and the operation and maintenance operation is carried out for slot.

4. The copy function can realize automatic failover without human intervention in most cases.

Shortcoming

1. The client needs to cache part of the data and implement the Cluster protocol, which is relatively complex.

2. The data is replicated asynchronously, so the strong consistency of the data can not be guaranteed.

3. Resource isolation is difficult and frequent traffic is uneven, especially when multiple businesses share clusters. The data do not know where, for hot data, can not be completed through special optimization.

4. The slave library is completely cold and cannot share the reading operation, which is really a waste of the wife. Extra work is needed.

5. MultiOp and Pipeline support is limited, so be careful if the old code is upgraded.

6. Data migration is based on key rather than slot, and the process is slow.

Basic principles

From the slot to the key, the positioning process is obviously a double-layer route.

Routing of key

Redis cluster has nothing to do with the commonly used consistent hash, which mainly uses the concept of hash slot. When you need to access a key in it, the redis client first calculates a value for the key using the crc16 algorithm, and then mod the value.

Crc16 (key) mod 16384

So, each key falls on one of the hash slots. 16384 is equivalent to 2 ^ 14 (16k). When a redis node sends a heartbeat packet, it needs to put all the slot information in this heartbeat packet, so if you want to make every effort to optimize it, you can see why the default number of slots is 16384.

Server-side simple principle

As mentioned above, redis cluster defines a total of 16384 slots, and all cluster operations are encoded around this slot. The server uses a simple array to store this information.

For operations that determine whether there is a presence or not, using bitmap to store is the most space-saving. Redis cluster uses an array called slot to hold whether the current node holds the slot.

The length of the array is 16384 Byte, so you can use 0 or 1 to identify whether this node has a slot.

In fact, only need the first data ClusterState to complete the operation, save another dimension of the Slot array, can be easily encoded and stored. In addition to recording the slots it is responsible for processing in two places (slots and numslots of the clusterNode structure), a node also sends its own slots array to other nodes in the cluster through messages to tell other nodes about the slots they currently have.

The cluster is online (ok) when all 16384 slots in the database are processed; conversely, if any slot in the database is not processed, the cluster is offline (fail).

When the client sends the relevant command to the node, the node that receives the command calculates which slot the key to be processed by the command belongs to and checks whether the slot is assigned to it. If it is not your own, it will direct the client to the correct node.

Therefore, the client can connect to any machine in the cluster and complete the operation.

Prepare to install a 6-node cluster

If we are going to assemble a 3-slice cluster, each shard has a copy. Then the total number of node instances required is 3 / 2 / 6. Redis can be started by specifying a configuration file, and all we do is modify the configuration file.

Copy 6 default configuration files.

For i in {0.. 5} docp redis.conf redis-700$ i.confdone

Modify the contents of the configuration file. In the case of redis-7000.conf, we want to enable its cluster mode.

Cluster-enabled yesport 7000cluster-config-file nodes-7000.conf

Nodes-7000.conf saves some cluster information to the current node, so it needs to be independent.

Start & shut down

Again, we use a script to start it.

For i in {0.. 5} donohup. / redis-server redis-700$ i.conf & done

For demonstration purposes, we turned it off violently.

Ps-ef | grep redis | awk'{print $2}'| xargs kill-9 combined cluster

We use redis-cli to combine clusters. Redis will automate this process. This series of processes are combined by sending instructions to each node.

. / redis-cli-- cluster create 127.0.0.1 cluster-replicas 7000 127.0.0.1 cluster-replicas 1 several advanced principle node failures

Each node in the cluster periodically sends ping messages to other nodes in the cluster to detect whether the other party is online. If the node receiving the ping message does not return the pong message within the specified time, then the node sending the ping message will mark the node receiving the ping message as suspected PFAIL.

If more than half of the nodes in a cluster report a primary node x as suspected offline, then the primary node x will be marked as FAIL, and the node marked x as FAIL will broadcast a FAIL message about x to the cluster, and all nodes that receive the FAIL message will immediately mark x as FAIL.

You can notice this process, which is similar to the node judgment of es and zk, where more than half of the nodes are judged, so the number of primary nodes is generally odd. Since there is no minimum group configuration, there will theoretically be a brain fissure (not encountered yet).

Master-slave switching

When a node finds that its master node enters the fail state, it will choose one of the slave nodes of the node, execute the slaveof no one command, and become the master node.

After the new node completes its slot assignment, it broadcasts a pong message to the cluster so that other nodes are immediately aware of their changes. It tells others: I am already the master node, I have taken over the problematic node and become its stand-in.

The management of clusters within redis makes extensive use of these defined instructions. So these instructions are not only for us to use from the command line, but also for redis itself.

Data synchronization

When a slave is connected to the master, a sync instruction is sent. After receiving this instruction, master will start the save process in the background. After execution, master transfers the entire database file to slave, thus completing the first full synchronization.

Next, master will send the change instructions it receives to slave in turn, thus achieving the final synchronization of the data. Since redis 2.8, the breakpoint continuation of master-slave replication has been supported. If the network connection is disconnected during the master-slave replication process, you can continue to copy where you copied last time, instead of making a copy from scratch.

Data loss

Asynchronous replication is used between nodes in redis cluster, and there is no concept of ack like kafka. Nodes exchange status information through gossip protocol, and use voting mechanism to complete the role promotion from Slave to Master, which is doomed to take time to complete. It is easy to have a window in the process of failure, resulting in the loss of written data. For example, the following two situations.

First, the command has reached master, and when the data is not synchronized to slave,master, it will reply ok to the client. If the primary node goes down at this time, this data will be lost. Redis will avoid a lot of problems by doing so, but it is intolerable for a system that requires high data reliability.

Second, because the routing table is stored on the client, there is a problem of timeliness. If the partition makes a node unreachable, a slave node is promoted, but the original master node is available again at this time (failover is not complete). At this time, once the routing table of the client is not updated, it will write the data to the wrong node, resulting in data loss.

So redis cluster works well under normal circumstances, and in extreme cases, there is no solution to the problem of some value loss.

Complex operation and maintenance

The operation and maintenance of redis cluster is very complicated, although it has been abstracted, but the process is still not simple. Some instructions can only be used with confidence after a detailed understanding of its implementation principle.

Some commands that will be used to expand capacity. In the process of actual use, you may need to enter these commands frequently many times, and monitor its status during the input process, so it is basically impossible to run these commands manually.

There are two entrances to the operation and maintenance staff. One is to use redis-cli to connect to any one, and then send the command to start with cluster, most of which operate on the slot. When you start to assemble a cluster, you call these commands repeatedly for specific logic execution.

Another entry is to use the redis-cli command, plus the-- cluster parameter and instruction. This form is mainly used to manage and control cluster node information, such as adding and deleting nodes and so on. So it is recommended to use this method.

Redis cluster provides very complex commands that are difficult to manipulate and remember. A tool similar to CacheCloud is recommended for management.

Here are a few examples.

By sending a CLUSTER MEET command to node A, the client can ask the node A receiving the command to add another node B to the cluster where node A currently resides:

CLUSTER MEET 127.0.0.1 7006

With the cluster addslots command, one or more slots can be assigned to a node.

127.0.0.1VL 7000 > CLUSTER ADDSLOTS 01234. . . 5000

Set up the slave node.

CLUSTER REPLICATE redis-cli-cluster

Redis-trib.rb is an official management tool for Redis Cluster, but the latest version has recommended using redis-cli for operation.

Add a new node to the cluster

Redis-cli-- cluster add-node 127.0.0.1 cluster-replicas 7006 127.0.0.1

Remove a node from a cluster

Redis-cli-- cluster del-node 127.0.0.1 54abb85ea9874af495057b6f95e0af5776b35a52 7006

Migrate slots to a new node

Redis-cli-- cluster reshard 127.0.0.1 cluster-from 54abb85ea9874af495057b6f95e0af5776b35a52 7006-- cluster-to 895e1d1f589dfdac34f8bdf149360fe9ca8a24eb-- cluster-slots

There are many similar orders.

Create: create cluster check: check cluster info: view cluster information fix: repair cluster reshard: online migration slot rebalance: balance the number of cluster nodes slot add-node: add new nodes del-node: delete nodes set-timeout: set node timeout call: execute commands on all nodes in the cluster import: import external redis data into the cluster

Overview of other solutions master-slave mode

The earliest support of redis is the Mmurs model, that is, one master and multiple slaves. Redis stand-alone qps can reach 10 wks, but it is still not enough in some high traffic scenarios. Generally, the separation of read and write is used to increase slave and reduce the pressure on the host.

Since it is a master-slave architecture, it is faced with the problem of synchronization. The synchronization of redis master-slave mode is divided into full synchronization and partial synchronization. When you first create a slave, it is inevitable to have a full synchronization. After the full synchronization is finished, enter the incremental synchronization phase. This is no different from redis cluster.

This model is relatively stable, but some extra work needs to be done. Users need to develop their own master-slave switching function, that is, use sentinels to detect the health status of each instance, and then change the cluster status through instructions.

When the size of the cluster increases, the master-slave mode will soon encounter a bottleneck. Therefore, it is generally extended using the client-side hash approach, including consistent hashes similar to memcached.

The routing of client-side hash can be very complex, and this meta information is usually maintained by publishing jar packages or configuration, which also adds a lot of uncertainty to the online environment.

However, by adding a feature similar to ZK proactive notification, maintaining the configuration in the cloud can significantly reduce risk. Thousands of redis nodes that the author has maintained are managed in this way.

Agent mode

Code patterns, such as codis, were very popular before redis cluster. The agent layer simulates itself as a redis, receives requests from the client, and then slices and migrates the data according to the custom routing logic, while the business side does not need to change any code. In addition to smooth capacity expansion, some master-slave switching and FailOver functions are also completed in the proxy layer, and the client may not even have any awareness. This kind of program is also called distributed middleware.

A typical implementation is shown below, and the redis cluster behind it can even be hybrid.

But the shortcomings of this approach are also obvious. First of all, it introduces a new agent layer, which is complex in structure and operation and maintenance. A lot of coding is required, such as failover, read-write separation, data migration, and so on. In addition, the addition of the proxy layer also has a corresponding loss to the performance.

Multiple proxy generally use front-end lvs for load balancing design. If there are few machines in the proxy layer and the traffic of the back-end redis is very high, then the network card will become the main bottleneck.

Nginx can also be used as a proxy layer for redis, which is more professionally called Smart Proxy. This approach is more partial, if you are familiar with nginx, it is an elegant approach.

Use restrictions and pits

The speed of redis is very fast. Generally speaking, the faster the thing, the greater the consequences when something goes wrong.

Not long ago, I wrote a specification for redis: "Redis specification, which is probably the most pertinent." Specification and architecture, suitable for their own company environment, is the best, but will provide some basic ideas.

What is strictly forbidden is usually the place where the predecessors stepped on the pit. In addition to the content of this specification, for redis-cluster, add the following points.

1. Redis cluster claims to support 1k nodes, but you'd better not do so. When the number of nodes increases to 10, you can feel some jitter of the cluster. Such a large cluster proves that your business is already very strong, so consider client fragmentation.

2. Hot spots must be avoided. If all the traffic reaches a certain node, the consequences are generally very serious.

3. Do not put redis in large key, it will generate a large number of slow queries and affect normal queries.

4. If you are not using it as storage, the cache must be set to expire. The feeling of not taking a shit in the manger is very annoying.

5. For large traffic, do not open aof, just open rdb.

6. Redis cluster operations, less pipeline, less multi-key, they will produce a large number of unpredictable results.

This is the end of the article on "how to install a six-node cluster in Redis". Thank you for reading! I believe that everyone has a certain understanding of the knowledge of "how to install a six-node cluster in Redis". If you want to learn more, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.