Shulou(Shulou.com)06/01 Report--

The editor will share with you what the cluster plan is in redis. I hope you will gain a lot after reading this article. Let's discuss it together.

The amount of Redis data is increasing day by day, and more and more companies are using it, which is not only used for caching, but also tends to store this piece, which will promote the development of clusters. Each company is also collecting suitable cluster solutions. At present, the following cluster architectures are mostly used in the industry, most of which use sharding technology to solve a series of problems caused by the increase of single instance memory.

This article briefly introduces five options:

Official cluster scheme

Twemproxy proxy scheme

Sentinel mode

Codis

Client slicing

Official cluser scheme

Since redis version 3.0, redis-cluster clusters are supported. Redis-cluster uses a non-central structure. Each node stores data and the entire cluster state, and each node is connected to other nodes. Redis-cluster is a server side slicing technology.

Redis-cluster architecture diagram

Redis-cluster features:

Each node communicates with a node called the trunked bus (cluster bus). They use a special port number, which is the external service port number plus 10000. Therefore, it is necessary to maintain the information of each node of the cluster, otherwise the whole cluster will be unavailable, and a special binary protocol is used to optimize the transmission speed and bandwidth.

Redis-cluster maps all physical nodes to slot (slots), and cluster is responsible for maintaining node--slot--value.

The cluster is pre-divided into 16384 buckets. When you need to insert data into the redis cluster, decide which bucket to put a key into according to the value of CRC16 (KEY) mod 16384.

The client is directly connected to the redis node, so it is not necessary to connect all the nodes in the cluster, but to connect to any available node in the cluster.

Redis-trib.rb script (rub language) is a cluster management tool, such as automatically adding nodes, planning slots, migrating data, and so on.

The fail of a node takes effect only when it is detected by more than half of the nodes in the cluster.

The whole cluster is regarded as a whole, and the client can connect to any node to operate. When the key of the client operation is not assigned to that node, the redis will return the redirection instruction and point to the correct node.

In order to increase the accessibility of the cluster, the officially recommended solution is to configure node as a master-slave structure, that is, a master master node with n slave slave nodes. If the primary node fails, redis cluster will select one of the slave nodes to become a master node according to the election algorithm, and the whole cluster will continue to provide services.

Twemproxy proxy scheme

Twemproxy Agent Architecture Diagram:

Redis proxy middleware twemproxy is a technology that uses middleware to do slicing. Twemproxy is between the client and the server. After some processing (sharding), the request sent by the client is forwarded to the real redis server at the back end. In other words, the client does not directly access the redis server, but indirectly through the twemproxy proxy middleware. It reduces the number of connections to back-end servers directly connected by clients, and supports horizontal expansion of server clusters.

The internal processing of twemproxy middleware is stateless, and it can be easily clustered itself, thus avoiding a single point of stress or failure.

Twemproxy, also known as nutcracker, originated from the lightweight agent of redis and memcached clusters in Twitter system.

Github source code address (https://github.com/twitter/twemproxy)

From the architecture diagram above, we can see that twemproxy is a single point, and it is easy to put a lot of pressure on it, so it is usually combined with keepalived to achieve the high availability of twemproy. At this time, usually only one twemproxy is working, and the other is on standby. When one is dead, the vip drifts automatically and the standby takes over. Information on the use of keepalived can be found on the Internet.

Sentinel mode

Sentinel Sentinel

Sentinel (Sentinel) is a high availability solution for Redis: a Sentinel system consisting of one or more Sentinel instances can monitor any number of master servers and all slave servers under these master servers, and automatically upgrade a slave server under the offline master server to a new master server when the monitored master server goes offline.

For example:

After the Server1 is offline:

Upgrade Server2 to the new primary server:

The way Sentinel works

Each Sentinel sends a PING command to Master, Slave, and other Sentinel instances that it knows at a frequency of once per second.

If the time from the last valid reply to a PING command to an instance exceeds the value specified by the down-after-milliseconds option, the instance will be marked by Sentinel as subjectively offline.

If a Master is marked as subjectively offline, all Sentinel that are monitoring the Master should confirm that the Master has indeed entered the subjective offline state at a frequency of once per second.

When there is a sufficient number of Sentinel (greater than or equal to the value specified in the profile) to confirm that the Master has indeed entered the subjective offline state within the specified time range, the Master will be marked as objective offline.

In general, each Sentinel sends INFO commands to all Master and Slave it knows at a frequency of every 10 seconds.

When Master is marked as objective offline by Sentinel, the frequency at which Sentinel sends INFO commands to all Slave of offline Master is changed from once in 10 seconds to once per second.

If there is not a sufficient number of Sentinel to agree that Master has been offline, the objective offline status of Master will be removed. If Master returns a valid value to the PING command of Sentinel, the subjective offline state of Master will be removed.

Codis

Codis is a distributed Redis solution, which is open source by Pea Pod. For the upper application, there is no obvious difference between connecting codis proxy and connecting native redis server. The upper application can be used like a stand-alone redis, and the bottom layer of codis will handle request forwarding, non-stop data migration and other work. All the latter things are transparent to the front client. You can simply think that what is connected behind is a redis service with unlimited memory.

Client slicing

The logic of the partition is implemented on the client, and it is up to the client to choose which node to request to. The solution can refer to the consistent hash, which is usually suitable for scenarios where the user has complete control over the behavior of the client.

Conclusion: there is no best plan, only the most suitable one.

After reading this article, I believe you have a certain understanding of what cluster scheme is in redis. You want to know more about it. Welcome to follow the industry information channel. Thank you for your reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.