Shulou(Shulou.com)06/01 Report--

Today, the editor shares with you a detailed explanation of the cluster mode of redis. I believe many people do not know much about it. In order to let you know more about the cluster mode of redis, I have summarized the following contents for you. Let's look down together. I'm sure you'll get something.

There are generally 5 types of Redis clusters:

1, master-slave replication

2, Sentinel mode

3Give Redis official Cluster cluster model (server side)

4Jedis sharding cluster (client sharding)

5, using middleware agents, such as codis of pea pods, etc.

After introducing their models, let's analyze their principles:

Master-slave replication (Master-Slave Replication):

How master-slave replication (Master-Slave Replication) works: after Slave starts the slave node service and connects to the Master, it will actively send a SYNC command. After receiving the synchronization command, the master node of the Master service will start the background save process and collect all the commands received to modify the dataset. After the background process is completed, Master will transfer the whole database file to Slave to complete a complete synchronization. The Slave slave node service saves the database file data and loads it into memory after receiving it. Since then, the Master master node continues to transmit all the collected modification commands and new modification commands to Slaves,Slave in turn. These data modification commands will be executed this time to achieve the final data synchronization.

If the link between Master and Slave is disconnected, Slave can automatically reconnect the Master, but after a successful connection, a full synchronization will be performed automatically.

Master-slave replication configuration

Modify the configuration file of the slave node: slaveof masterip masterport

If you set a password, set it: masterauth master-password

Sentinel mode:

This mode was provided from Redis version 2.6, but at that time, the mode in this version was unstable, and the Sentinel mode did not stabilize until after Redis version 2.8. both master-slave mode and Sentinel mode have a problem, which cannot be expanded horizontally, and the high availability of both modes is limited by the memory of the Master master node.

The Sentinel (Sentinel) process is used to monitor the working status of the Master master server in the redis cluster. When the Master master server fails, it can switch between Master and Slave servers to ensure the high availability of the system.

Role of the Sentinel (Sentinel) process

Monitoring: the sentinel will constantly check to see if your Master and Slave are working properly.

Notification: when there is a problem with a monitored Redis node, the sentinel can send a notification to the administrator or other application through API.

Automatic failover (Automatic failover): when a Master does not work properly, the sentinel starts an automatic failover operation, which upgrades one of the Slave of the failed Master to the new Master and causes the other Slave of the failed Master to copy the new Master; instead. When the client tries to connect to the failed Master, the cluster will also return the address of the new Master to the client, so that the cluster can replace the failed Master with the current Master. After switching between Master and Slave servers, the contents of Master's redis.conf, Slave's redis.conf and sentinel.conf 's configuration files will be changed accordingly, that is, there will be an extra line of slaveof's configuration in the redis.conf configuration file of the Master master server, and the monitoring target of sentinel.conf will be changed accordingly.

How the Sentinel (Sentinel) process works

Each Sentinel process sends a PING command to the Master master server, Slave slave server, and other Sentinel processes throughout the cluster at a frequency of once per second.

If the time from the last valid reply to a PING command to an instance (instance) exceeds the value specified by the down-after-milliseconds option, the instance will be marked as subjective SDOWN by the Sentinel (Sentinel) process.

If a Master master server is marked as subjective offline (SDOWN), all Sentinel processes monitoring the Master master server should confirm that the Master master server has indeed entered the subjective offline state at a frequency of once per second.

When a sufficient number of Sentinel (Sentinel) processes (greater than or equal to the value specified in the profile) confirm that the Master master server has entered the subjective offline state (SDOWN) within the specified time range, the Master master server will be marked as objective offline (ODOWN).

In general, each Sentinel process sends INFO commands to all Master master servers and Slave slave servers in the cluster every 10 seconds.

When the Master master server is marked as objective offline (ODOWN) by the Sentinel process, the frequency of INFO commands sent by the Sentinel process to all Slave slaves of the offline Master master server will be changed from once per second to once per second.

If there are not enough Sentinel processes to allow the Master master server to go offline, the objective offline state of the Master master server will be removed. If the Master master server re-sends the PING command to the Sentinel process to return a valid reply, the subjective offline state of the Master master server will be removed.

Redis official Cluster cluster model

Redis Cluster is a server Sharding technology that is officially available in version 3. 0.

In this figure, each blue circle represents a redis server node. Any two of them are connected to each other. The client can connect to any node and then access any node in the cluster. Access it and other operations.

Redis cluster data fragmentation

On every node of the redis, there are two things, one is the slot (slot) can be understood as a variable that can store two values, the value range of this variable is: 0-16383. Another is cluster. I personally understand this cluster as a plug-in for cluster management. When our access key arrives, redis will get a result according to crc16's algorithm, and then calculate the remainder of the result to 16384, so that each key will correspond to a hash slot numbered between 0 and 16383, through this value, to find the node corresponding to the corresponding slot, and then automatically jump to the corresponding node for access operation.

Jedis sharding cluster

Redis Sharding can be said to be widely used in the industry before Redis cluster came out, and its main idea is to use hash algorithm to hash hash the key storing data, so that a specific key will be defined to a specific node.

Fortunately, the Java Redis client driver Jedis already supports the Redis Sharding feature, namely ShardedJedis and ShardedJedisPool combined with cache pools

The Redis Sharding implementation of Jedis has the following characteristics:

The consistent hash algorithm is used to hashing key and node name at the same time, and then map and match. The algorithm is MURMUR_HASH. The main reason for using a consistent hash instead of a simple similar hash module mapping is that when nodes are added or decreased, there is no rehashing caused by rematching. The consistent hash only affects the key allocation of neighboring nodes, and the impact is small.

In order to avoid the pressure of node allocation caused by consistent hashing only affecting neighboring nodes, ShardedJedis will virtualize 160 virtual nodes to hash each Redis node according to its name (no, Jedis will give it a default name). According to the weight weight, 160x virtual nodes can also be virtualized. By mapping and matching with virtual nodes, when adding or decreasing Redis nodes, key can be moved and redistributed more evenly among Redis nodes, rather than only neighboring nodes are affected.

ShardedJedis supports the keyTagPattern pattern, that is, a part of the keyTag of the key is extracted as sharding, so that a set of associated key can be put into the same Redis node by reasonably naming the key, which is important to avoid accessing related data across nodes.

Using middleware agent

The role of middleware is to calculate a value by calculating the key of the data we need to store in redis through a set of algorithms. Then find the corresponding redis node according to this value, and store the data in the node of the redis.

There are several kinds of middleware commonly used.

Twemproxy

Codis

Nginx

These are the details of the cluster mode of redis. Is there anything to gain after reading it? If you want to know more about it, welcome to the industry information!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.