Shulou(Shulou.com)06/01 Report--

Understand and build redis clusters from scratch

Some developers only use redis in their applications, such as caching for data results. And there are a lot of good redis client tools (redisson) that can basically do a lot of things without paying attention to the redis command. So you may not pay enough attention to the following issues:

How to recover from disasters? That is, if a redis node has a problem, how to ensure the high availability of the service?

How to expand horizontally? When the amount of data is very large, how to solve the performance problem of a single redis

At least how many machines does a cluster need? Or several redis nodes

What technologies and tools are used to build clusters?

How to recover from disasters?

Redis provides a master-slave hot standby mechanism, which synchronizes the data of the master server to the slave server, monitors the status of the master server in real time through the sentinel and is responsible for electing the master server. When an exception is found in the primary server, the primary server is re-elected according to a certain algorithm and the problem server is removed from the available list, and finally the client is notified. Master-slave is an one-to-many tree structure, as shown in the following figure:

Sentinel

Sentinel, the Chinese name of sentinel, is a highly available architecture tool from redis. It is an independent process that can monitor more than one redis cluster at the same time.

Sentinel cluster

Based on the consideration of high availability, the Sentinel itself also needs to support the cluster, if there is only one Sentinel, there will be a single point of problem.

Sentinel decision

The Sentinel has a quantity configuration, and the master-slave switch is made when how many Sentinels think a master service is unavailable at the same time, for example, there are a total of 5 Sentinels, and when 3 Sentinels think the service is unavailable, they decide to switch between master and slave. This can avoid some misswitching and reduce the handover cost, such as instantaneous network anomalies.

How to expand horizontally?

No matter for products such as redis or other databases, when the data capacity of a single node reaches a certain limit, the external capacity of the service will become weaker and weaker. Redis provides redis-trib.rb for clustering functionality in a higher version, or you can use a third-party tool, twemproxy.

Decentralization, each node is equal.

The centralization is not considered in the design of the redis cluster, which can avoid the single point of the central node. Each node can control the status of the entire cluster, and any node connected to it can access all the key, just like the redis of a single node.

Cluster schematic diagram

If you understand the painting by yourself, you can point out if there is any misunderstanding.

The relationship between key and redis nodes

Hasy solt is introduced, which is understood as hash slot in Chinese. There are 16384 in total, and the key we operate confirms which slot the key falls on through the modulus algorithm.

HASH_SLOT = CRC16 (key) mod 16384

There is a relationship between the hash slot and the node, so we can assign the key to a specific redis node.

The detailed relationship can be studied again, for example, node An is responsible for the hash slot numbered 0-5000 and node B is responsible for 5001-1000.

Build step by step

Began to build a three-master and three-slave cluster, the system is ubuntu, using the cluster tool redis-trib.rb provided by redis.

Install the latest redis

Create redis_cluster directories and create 6 directories from 7000 to 7005

Copy the redis.conf under the redis directory to the 6 directories created above

Modify the redis.conf file separately, and make similar changes to 6 files.

Port 7000 / / Port 7000 bind 127.0.0.1 / / default ip is 127.0.0.1 need to be changed to ipdaemonize yes / / background running pidfile / var/run/redis_7000.pid accessible to other node machines / / pidfile file corresponds to 7000cluster-enabled yes / / enable cluster cluster-config-file nodes_7000.conf / / configuration of cluster cluster-node-timeout 15000 / / request timeout defaults to 15 seconds It can be set by yourself.

What bind needs to pay attention to is that it needs to be configured as an ip that other machines can access, otherwise there will be problems in creating clusters and client connections.

Start 6 redis

Redis-server redis_cluster/7000/redis.confredis-server redis_cluster/7001/redis.confredis-server redis_cluster/7002/redis.confredis-server redis_cluster/7003/redis.confredis-server redis_cluster/7004/redis.confredis-server redis_cluster/7005/redis.conf

Create a cluster

There is a redis-trib.rb in redis's src directory, copy it to / usr/local/bin, and execute the following script:

Redis-trib.rb create-- replicas 1 127.0.0.1VL 7000 127.0.0.1V 7001 127.0.1V 7002 127.0.0.1V 7003 127.0.1V 7003 127.0.1V 7004 127.0.1V 7005

The 1 after replicas represents the number of slave servers, which can be understood as the first three master servers and the last three as slaves, that is, three pairs of master and slave servers.

You will be prompted to install ruby during execution, and after the installation is complete, you will be prompted to install gem redis.

It took a long time to install gem redis, but it was finally found that the installation through apt-get install was not successful because some websites could not be accessed in China, and finally the installation was successful by downloading the source code.

Execute the script to create the cluster again, and the following prompt appears:

Enter yes to continue

How many primary servers are required at a minimum?

It may be based on some conventions that cluster agreements have the ability to provide services only when the number of available nodes is more than half. First of all, the number must be odd, but it must be greater than 1, so the minimum number of primary servers is 3.

Test cluster

To connect to the client, since all my nodes are local, I don't need to enter ip, but I need to add the parameter-c. Redis-cli-c-p 7000

After the connection is successful, add a key

Set mykey 123

There is a prompt pointing to port 7002, indicating that although we are connecting to an instance of 7000, the hash algorithm will eventually assign key to an instance of 7002.

Then connect port 7005 to query key, and test whether any instance can query key.

Get mykey

Display points to port 7002

These are the details of the graphics and text of the redis high availability solution, please pay attention to other related articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.