Shulou(Shulou.com)05/31 Report--

Editor to share with you an example of Redis cluster architecture and comparison, I believe that most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

1 、 Redis3.0

Advantages

a. No central node

b. Data is distributed over multiple Redis instances according to slot storage

c. Smooth capacity expansion / reduction nodes

d. Automatic failover (nodes exchange status information through Gossip protocol and voting mechanism to upgrade the role from Slave to Master)

e. Reduce the operation and maintenance cost and improve the scalability and high availability of the system.

(recommended (free): redis)

Shortcoming

a. Heavily dependent on external Redis-Trib

b. Lack of monitoring and management

c. Need to rely on Smart Client (connection maintenance, caching routing tables, MultiOp and Pipeline support)

D. the detection of Failover nodes is too slow, so it is not as timely as "central node ZooKeeper".

E. the cost of Gossip messages

f. It is impossible to distinguish hot and cold data according to statistics

G. Slave "cold standby", which can not relieve the pressure of reading

2. Proxy + Redis Cluster

Advantages

Smart Client:

a. Compared with the use of proxy, it reduces the consumption of one-layer network transmission and is more efficient.

b. Do not rely on third-party middleware, the implementation method and code are under their own control and can be adjusted at any time.

Proxy:

a. Provides a set of HTTP Restful interfaces to isolate the underlying storage. Completely transparent to the client, cross-language calls.

b. Upgrade maintenance is relatively easy, maintenance Redis Cluster, only need to smoothly upgrade Proxy.

c. Hierarchical storage, the underlying storage to do cold and hot heterogeneous storage.

d. Permission control, Proxy can control the whitelist through the secret key to filter out some illegal requests. And it can also control and filter the oversized Value requested by the user.

e. Security, you can block out some dangerous commands, such as Keys, Save, Flush All, etc.

f. Capacity control, capacity limit according to different user capacity applications.

g. Logical isolation of resources, which is carried out according to the Key of different users and prefixed.

h. Monitoring burial points, monitoring and other information for different interfaces.

Shortcoming

Smart Client:

a. The immaturity of the client affects the stability of the application and improves the difficulty of development.

B. MultiOp and Pipeline support is limited.

c. Connection maintenance, Smart client maintenance of Socket connecting to each node in the cluster.

Proxy:

a. There is one more forwarding in the proxy layer, resulting in a loss of performance.

B. higher requirements for operation and maintenance during capacity expansion / reduction, and it is difficult to achieve smooth capacity expansion and reduction.

3. Technology selection

The following paragraph is found in the official redis document:

The redis-cli cluster support is very basic so it alwaysuses the fact that Redis Cluster nodes are able to redirect a client to theright node. A serious client is able to do better than that, and cache the map betweenhash slots and nodes addresses, to directly use theright connection to theright node. The map is refreshed only when something changed in the clusterconfiguration, for example after a failover or after the system administratorchanged the cluster layout by adding or removing nodes.

The main idea is that the official client of redis cluster is poorly functional and relies on redis node redirection to find the redis instance where the data resides in the cluster. A better client is needed to implement consistent hash,failover and cluster management functions. Therefore, it is not a wise choice to use the official redis cluster client. This article provides three options for your reference. If there is anything unreasonable, you are welcome to discuss it with me.

Scenario 1 is developed using nginx (OpenResty mode)

The reasons are as follows:

a. Single Master multi-Work mode, each Work, like Redis, is a single-process, single-threaded mode, and is based on Epoll event-driven mode.

B. Nginx uses an asynchronous non-blocking way to process requests, an efficient asynchronous framework.

c. It takes up less memory and has its own set of memory pool management. Aggregating a large number of applications with small memory can be faster than Malloc. Reduce memory fragmentation and prevent memory leaks. Reduce the complexity of memory management.

d. In order to improve the access speed of Nginx, Nginx uses its own set of connection pools.

e. The most important thing is to support custom module development.

f. In the industry, the word-of-mouth for Nginx,Redis can be called two artifacts. The performance is very good.

Option 2 codis (agent-based redis clustering scheme adopted by Pea Pod)

Refer to the official codis document https://github.com/CodisLabs/codis

Codis is a complete set of caching solutions, including high availability, data fragmentation, monitoring, dynamic expansion etc.

It is Apps- > Agent-> rediscluster, which is basically adopted after a certain scale.

Scenario 3 develops redis smart client independently.

It mainly realizes the functions of redis slots management, failover and consistent hash.

4. Redis 3.0Cluster

Redis 3.0 cluster adopts P2P mode and is completely decentralized. Redis divides all Key into 16384 slot, and each Redis instance is responsible for some of the slot. All the information in the cluster (nodes, ports, slot, etc.) is updated through regular data exchanges between nodes.

The Redis client makes a request in any Redis instance and directs the client to access the desired instance through the redirect command if the required data is not in that instance.

The workflow of the Redis 3.0 cluster is shown in the following figure.

The machines in the Redis cluster exchange data regularly, and the workflow is as follows:

(1) the Redis client accesses certain data on the Redis2 instance

(2) when the data is found in Redis2, send a redirect command to Redis client in the instance of Redis3.

(3) after receiving the redirect command, the Redis client accesses the Redis3 instance to obtain the required data.

The clustering scheme for Redis 3.0 has the following two problems:

1) A Redis instance has a completely decentralized design of "data storage" and "route redirection". The advantage of this is that deployment is very simple, just deploy Redis directly, and don't have as many components and dependencies as Codis. But the problem is that it is difficult to upgrade the business painlessly. If there is any serious Bug in the Redis cluster, you can only roll back the entire Redis cluster.

2) the protocol has been greatly modified, and the corresponding Redis client also needs to be upgraded. Who can ensure that there is no Bug after upgrading the Redis client? And for businesses that have been running on a large scale online, it is also troublesome to upgrade the Redis client in the code.

5 、 Redis4.0

(1) all redis nodes are interconnected with each other (PING-PONG mechanism), and binary protocol is used internally to optimize transmission speed and bandwidth.

(2) the Fail of a node takes effect only when more than half of the nodes in the cluster detect failure.

(3) the client is directly connected to the redis node and does not need an intermediate proxy layer. The client does not need to connect to all the nodes in the cluster, just connect to any available node in the cluster

(4) redis-cluster maps all physical nodes to [0-16383] slot (slots), and cluster is responsible for maintaining nodeslotvalue.

These are all the contents of the article "examples of Redis Cluster Architecture and comparison". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.