Shulou(Shulou.com)06/01 Report--

Be careful! Versions above redis3.0 are required, because redis clustering features are added above 3.0.

1.redis introduction

1.1 what is redis

Redis is an open source high performance key-value pair (key-value) non-relational database developed in C language. A variety of key data types are used to meet the storage needs in different scenarios. Currently, the supported key data types are:

String, hash, list, collection, ordered collection

2.2 Application scenario

Caching (data queries, short links, news content, commodity content, etc.). (most used)

Session separation in distributed cluster architecture.

A list of online friends in chat rooms.

Task queue. (second kill, rush purchase, 12306, etc.)

Application rankings.

Website visit statistics.

Data expiration processing (can be accurate to milliseconds)

Introduction of 2.Redis Cluster

Architecture of 2.1Redis Cluster

There are 16384 hash slots built into the Redis cluster. Redis-cluster maps all physical nodes to [0-16383] slot, and cluster is responsible for maintenance. When you need to place a key-value in a Redis cluster, redis first uses the crc16 algorithm for key to calculate a result, and then calculates the remainder of the result to 16384, so that each key corresponds to a hash slot numbered between 0 and 16383, and redis will map the hash slot to different nodes according to the number of nodes.

2.2 characteristics of Redis clusters

When the Redis cluster starts, it automatically does sharding among multiple nodes, and provides the availability between shards: that is, when some redis nodes fail or the network is interrupted, the cluster has slave nodes that can replace the master node to continue to work, but if a large area of nodes fail, the cluster will not be available.

The Redis cluster provides:

Automatically split 16384 data slots into multiple Redis nodes

When some nodes fail or are unreachable, the cluster can still continue to work.

2.3 TCP port of Redis cluster

Each node of the cluster needs to establish two TCP connections to listen on these two ports:

Client port (usually 6379): it needs to be open to all clients and cluster nodes to receive client instructions, and the cluster node needs to transfer data to the client through this port.

Cluster bus port (typically 6379 / 10000): only needs to be open to all nodes in the cluster for communication between nodes through the binary protocol. Each node detects the faulty node and updates the configuration through the cluster bus, but the client cannot use this port.

2.4 fragmentation of Redis cluster data

The Redis cluster uses hash slots, with 16384 hash slots. The algorithm for determining which slot a key is assigned to: calculate the CRC16 of the key, and the result is 16384.

Each node in the cluster is responsible for part of the hash slot. For example, if there are 3 nodes in the cluster, then:

The hash slot range of node A storage is: 0-5500 node B storage: 5501-11000 node C storage hash slot range: 11001-16384

This distribution facilitates the addition and deletion of nodes. For example, you need to add a new node D, and you only need to move some of the hash slot data in A, B, C to the D node. Similarly, if you want to delete node An in the cluster, you only need to move the data of the hash slot of node A to nodes B and C. when all the data of node An is removed, node A can be completely deleted from the cluster.

Because moving hash slots from one node to another does not require downtime, adding or deleting nodes, or changing hash slots on nodes, does not require downtime.

If multiple key belong to a single hash slot, the cluster supports simultaneous manipulation of these key through a single command (or transaction, or lua script). With the concept of "hash tag", users can assign multiple key to the same hash slot. If the key contains curly braces "{}", only the strings in the curly braces will participate in the hash, such as "this {foo}" and "another {foo}" these two key will be assigned to the same hash slot, so you can operate on them at the same time in the same command.

Master-Slave Mode of 2.5 Redis Cluster

Each hash slot has a master node and multiple slave nodes.

For example: if there are six nodes, it is divided into three master nodes, and three A1Magi B1Magol C1 are the corresponding slave nodes. When A fails, the cluster will promote A1 master node, and A1 will inherit the data of A node. In fact, A1 is equivalent to a copy of A, allowing the cluster to continue to work.

2.5.1 redis-cluster voting: fault tolerance

The main results are as follows: (1) all the master nodes in the cluster participate in the voting process. If more than half of the master nodes communicate with the failed master node more than (cluster-node-timeout), the current master node is considered to be dead.

(2): when is the entire cluster unavailable (cluster_state:fail)?

A: if any master node of the cluster dies and there are no slave nodes. The cluster enters the fail state, which can also be understood as entering the fail state when the cluster's slot mapping [0-16383] is not completed.

B: if more than half of the master nodes of the cluster are down, the cluster will enter fail state regardless of whether there are slave nodes or not.

Ps: when the cluster is not available, all operations on the cluster are not available. A ((error) CLUSTERDOWN The cluster is down) error was received.

2.6Conformance guarantee of Redis Cluster

Redis clusters cannot guarantee strong consistency. Some operations that have confirmed the success of the write to the client will be lost in some uncertain circumstances.

The first reason for the loss of write operations is that data is synchronized asynchronously between master and slave nodes.

A write operation is a process like this:

1) the client initiates the write operation to the master node B) the master node B responds to the client write operation successfully. 3) the master node B synchronizes the write operation to its slave node B1Magi B2 and B3.

As can be seen from the above process, master node B does not wait for slave node B1MagneB2MagneB3 to reply to the client's result of this operation after it has been written. Therefore, if the master node B fails after notifying the client that the write operation is successful, but before synchronizing to the slave node, one of the slave nodes that did not receive the write operation will be promoted to the master node, and the write operation will be lost forever.

Node timeout (node timeout): very important for the cluster, when the timeout of this node is reached, the master node is considered to be down and can be replaced by one of its slave nodes. Similarly, when the node times out, if the primary node still cannot contact the other primary node, it will enter an error state and no longer accept write operations.

2.7 Parameter configuration of Redis cluster

Some parameters in redis.conf indicate:

Cluster-enabled:

If you configure "yes", the clustering feature is enabled, and this redis instance is a node of the cluster, otherwise, it is a normal single redis instance.

Cluster-config-file:

Note: although the name of this configuration is "Cluster configuration File", this configuration file cannot be edited manually. It is a file automatically maintained by cluster nodes. It is mainly used to record which nodes are in the cluster, their status and some persistence parameters, so as to facilitate the recovery of these states when rebooting. This file is usually updated after a request is received.

Cluster-node-timeout:

This is the maximum time that a node in the cluster can be lost, after which the node will be considered a failure. If the master node is still unreachable beyond this time, its slave node will initiate the failover and upgrade to the master node. Note that if any node is still not connected to most of the primary nodes within this time, the node will stop receiving any requests.

Cluster-slave-validity-factor:

If set to 0, the slave node attempts to upgrade to the master node no matter how long the slave node loses contact with the master node. If set to a positive number, the time obtained by multiplying cluster-node-timeout by cluster-slave-validity-factor is the longest time that the slave node data is valid after the slave node lost contact with the master node. After this time, the slave node will not start the failover. Assuming cluster-node-timeout=5,cluster-slave-validity-factor=10, if the slave node loses contact with the master node for more than 50 seconds, the slave node cannot become the master node. Note that if this parameter is set to non-0, the cluster may not work properly because a master node is lost but not on top of the slave node. In this case, the cluster will resume operation only when the original master node returns to the cluster.

Cluster-migration-barrier

The minimum number of slave nodes required by the master node, which will be migrated only when the master node fails. For more details, you can see the copy migration section later in this tutorial.

Cluster-require-full-coverage

If this parameter is set to "yes" (default) when the node where some key resides is not available

The entire cluster stops accepting operations; if this parameter is set to "no", the cluster still provides read operations for the key on the reachable node.

The above is the detailed explanation and integration of the Redis cluster introduced by the editor. I hope it will be helpful to you. If you have any questions, please leave me a message and the editor will reply you in time. Thank you very much for your support to the website!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.