Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Installation process of redis5.0 cluster

2025-01-26 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Database >

Share

Shulou(Shulou.com)05/31 Report--

The main content of this article is "the installation process of redis5.0 cluster". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "the installation process of redis5.0 cluster".

Introduction to redis cluster

Redis cluster does not use consistent hash, but introduces the concept of hash slot.

The Redis cluster has 16384 hash slots, and each key passes the CRC16 check and modulates 16384 to decide which slot to place. Each node of the cluster is responsible for part of the hash slot. For example, if the current cluster has three nodes, then:

Node A contains hash slots 0 to 5500.

Node B contains hash slots 5501 to 11000.

Node C contains hash slots 11001 to 16384.

This structure is easy to add or remove nodes. For example, if I want to add a new node D, I need to get some slots from nodes A, B, C to D. If I want to remove node A, I need to move the slots in A to B and C nodes, and then remove A nodes without any slots from the cluster. Since moving hash slots from one node to another does not stop service, no matter adding, deleting or changing the number of hash slots of a node will not cause the cluster to be unavailable.

The figure depicts a cluster of six redis instances

Port 6379 is the client communication port.

Port 16379 is a cluster bus port.

The cluster is divided into 16384 data slots, distributed in three main redis.

There are no slots from the redis, will not participate in cluster voting, and will not help speed up the reading of data, just as a backup of the host.

An average of 1/3 of the 16384 data slots are distributed among the three master nodes, and there is no duplicate data in each node, only with its own slave computer to help with redundancy.

Note: redis cluster must have at least three master nodes and at least three slave nodes for redundancy.

In order to make the cluster still available when some nodes fail or most nodes can not communicate, the cluster uses the master-slave replication model, and each node will have a replica.

In our example, if node B fails in the absence of a replication model, the whole cluster will be unavailable because of the lack of slots in the range of 5501-11000.

However, when the cluster is created (or after a period of time), we add a slave node, A1 Magi B1 and C1, for each node, then the whole cluster consists of three master nodes and three slave nodes, so that after node B fails, the cluster will elect B1 to continue to serve the new master node, and the whole cluster will not be unavailable because the slot cannot be found.

However, when both B and B 1 fail, the cluster is not available.

Redis consistency guarantee

Redis does not guarantee strong consistency of data. This means that in practice, the cluster may lose write operations under certain conditions.

The first reason is that the cluster uses asynchronous replication. Write operation process:

The client writes a command to master node B.

Master node B returns the command status to the client.

The master node copies the write operation to him and has to slave nodes B 1, B 2 and B 3.

The copy of the command by the master node occurs after the command reply is returned, because if each command request needs to wait for the copy operation to complete, then the speed at which the master node processes command requests will be greatly reduced-we have to make a tradeoff between performance and consistency. Note: Redis clusters may provide synchronous write methods in the future.

Another situation where commands may be lost in a Redis cluster is when a network partition appears in the cluster and a client is isolated from a small number of instances, including at least one master node.

For example, suppose the cluster consists of six nodes A, B, C, A1, B1, and C1, in which A, B, C are master nodes, and A1, B1, and C1 are the subordinate nodes of An and Magi. There is also a client Z1 that assumes that network partitioning occurs in the cluster, then the cluster may be divided into two sides, most of which contain nodes A, C, A1, B1 and C1, while a small part of the cluster contains node B and client Z1.

Z1 can still write to master node B. if the network partition occurs for a short time, the cluster will continue to operate normally. If the partition is long enough for most parties to elect B1 as the new master, then the data written by Z1 to B will be lost.

Note that during a network split, there is a limit to the maximum time that client Z1 can send write commands to primary node B. this time limit is called node timeout (cluster-node-timeout) and is an important configuration option for Redis clusters:

Prepare two virtual machines

Node1:172.18.22.100node2:172.18.22.101 downloads the latest version of redis 5.0installation package

Https://redis.io/download

I downloaded redis-5.0.3.tar.gz.

Compile and install (node1) [root@node1 ~] # cd / opt/ [root@node1 opt] # tar-xvf redis-5.0.3.tar.gz [root@node1 opt] # cd redis-5.0.3/ [root@node1 opt] # make MALLOC=lib [root@node1 opt] # make install modify the configuration file Copy to the node1 server on another machine: [root@node1 ~] # cd / opt/ [root@node1 opt] # tar-xvf redis-5.0.3.tar.gz [root@node1 opt] # cd redis-5.0.3/ [root@node1 redis-5.0.3] # mkdir-p redis_cluster/7000/ data [root @ node1 redis-5.0.3] # mkdir-p redis_cluster/7001/ data [root @ node1 redis-5.0.3] # mkdir-p redis_cluster/7002/data

Create a redis.conf file in the above three directories, and the areas that need to be modified are marked below:

# redis.conf default configuration needs to be modified in the case of multiple instances of daemonize yes#, such as redis_6380.pidpidfile / var/run/redis_7000.pid# multiple instances, such as 6380port 7000tcp-backlog 511bind 0.0.0.0timeout 0tcp-keepalive 0loglevel notice# multiple instances. For example, 6380.loglogfile / var/log/redis_7000.logdatabases 16save 900 1save 300 10save 60 10000stop-writes-on-bgsave-error yesrdbcompression yesrdbchecksum yes# multiple instances need to be modified For example, in the case of multiple instances of dump.6380.rdbdbfilename dump.7000.rdbslave-serve-stale-data yesslave-read-only yesrepl-diskless-sync norepl-diskless-sync-delay 5repl-disable-tcp-nodelay noslave-priority 100appendonly yes###, you need to modify For example, appendonly_6380.aofappendfilename "appendonly_7000.aof" appendfsync everysecno-appendfsync-on-rewrite noauto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mbaof-load-truncated yeslua-time-limit 5000slowlog-log-slower-than 10000slowlog-max-len 128latency-monitor-threshold 0notify-keyspace-events "hash-max-ziplist-entries 512hash-max-ziplist-value 64list-max-ziplist-entries 512list-max-ziplist-value 64set-max-intset-entries 512zset-max-ziplist-entries 128zsetMaxMaziplist- Value 64hll-sparse-max-bytes 3000activerehashing yesclient-output-buffer-limit normal 00 0client-output-buffer-limit slave 256mb 64mb 60client-output-buffer-limit pubsub 32mb 8mb 60hz 1 "system configuration # vim / etc/sysctl.conf#vm.overcommit_memory =" Custom configuration aof-rewrite-incremental-fsync yesmaxmemory 4096mbmaxmemory-policy allkeys-lru## multiple instances need to be modified For example, when / data/6380dir / opt/redis-5.0.3/redis_cluster/7000/data# cluster is configured with multiple instances of cluster-enabled yes##, you need to modify the cluster-config-file / opt/redis-5.0.3/redis_cluster/7000/nodes.confcluster-node-timeout 500 slave master interval defaults to 10 seconds # replication timeout # repl-timeout 6 remote distance From # config set client-output-buffer-limit "slave 536870912 536870912 0" # config set repl-backlog-size 209715200

[root@node1 redis-5.0.3] # scp-pr / opt/redis-5.0.3/ node2:/opt/

The parameter cluster-node-timeout is very important. If the setting is too small, automatic master-slave switching will occur frequently. If the setting time is too long, master downtime will occur, so master-slave switching cannot be carried out in time.

Once a master node enters the FAIL state, if the master node has one or more slave nodes, one of the slave nodes will be upgraded to the master node, while the other slave nodes will begin to copy the new master node.

Start the command, execute on node1 and node2

[root@node1 redis-5.0.3] # / opt/redis-5.0.3/src/redis-server / opt/redis-5.0.3/redis_cluster/7000/ redis.confs [root @ node1 redis-5.0.3] # / opt/redis-5.0.3/src/redis-server / opt/redis-5.0.3/redis_cluster/7001/redis.conf [root@node1 redis-5.0.3] # / opt/redis-5 .0.3/src/redis-server / opt/redis-5.0.3/redis_cluster/7002/redis.conf remarks: [root@node1 redis-5.0.3] # / opt/redis-5.0.3/src/redis-cli-h localhost-p 7000 shutdown [root@node1 redis-5.0.3] # / opt/redis-5.0.3/src/redis-cli-h localhost-p 7000-clocalhost:7000 > pingPONG

Note:-c starts the cluster mode

Create cluster command [root@node1 opt] # / opt/redis-5.0.3/src/redis-cli-- cluster create 172.18.22.100 cluster create 7000 172.18.22.100 cluster create 7001 172.18.22.100 cluster create 7002 172.18.22.101 7002 172.18.22.101 cluster create 7001 172.18.22.101

Enter yes once in the middle.

Option-replicas 1 indicates that we want to create a slave node for each master node in the cluster.

View cluster information

[root@node1 opt] # / opt/redis-5.0.3/src/redis-cli-h localhost-p 7001localhost:7001 > cluster infocluster_state:okcluster_slots_assigned:16384cluster_slots_ok:16384cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:3cluster_current_epoch:6cluster_my_epoch:2cluster_stats_messages_ping_sent:1302cluster_stats_messages_pong_sent:946cluster_stats_messages_meet_sent:4cluster_stats_messages_sent : 2252cluster_stats_messages_ping_received:945cluster_stats_messages_pong_received:1306cluster_stats_messages_meet_received:1cluster_stats_messages_received:2252localhost:7001 > localhost:7001 > CLUSTER NODESed9ac7cfe6847f39dfcf56ad4244c4213a0965b1 172.18.22.100 slave f881aa4c6fdce6f7ac058600ca712fd9df110b1a 01551346045000 4 connectedfb56d260650dfad25e6b54c1519e9530e1b33b62 172.18.22.100 CLUSTER NODESed9ac7cfe6847f39dfcf56ad4244c4213a0965b1 7001mm 17001 myself Master-0 1551346046000 2 connected 10923-1638309f371cfd0933a343dd6d84159a6f8ad508b1eff 172.18.22.100 master-01551346047131 1 connected 0-5460a7573d643ede9facb1e9b3b629a1604ed88b6a61 172.18.22.101 Vera 7002 "17002 slave fb56d260650dfad25e6b54c1519e9530e1b33b62 01551346046000 6 connectedf881aa4c6fdce6f7ac058600ca712fd9df110b1a 172.18.22.101Rose 7000000" 17000 master-01551346046629 4 connected 5461-109228d3b210324f24aac0c146363e95d3c87a7eb474c 172.18.22.101 Univ 700117001 slave 09f371cfd0933a343dd6d84159a6f8ad508b1eff 01551346045627 5 connected Cluster Verification

Connect the 7000 nodes of the cluster on the first machine and 7002 nodes on the other, with the connection method redis-cli-h localhost-c-p 7002.

Add the parameter-C to connect to the cluster.

Execute the command set hello world on node 7000, and the result is as follows:

Localhost:7000 > set hello worldOKlocalhost:7000 > get hello "world"

Then, on another port 7002, check the content where key is hello, get hello, and the execution result is as follows:

Localhost:7002 > get hello- > Redirected to slot [866] located at 172.18.22.100 7000 "world"

It indicates that the cluster is operating normally.

A brief description of the principle.

When redis cluster is designed, it takes into account decentralization and middleware, that is to say, every node in the cluster is equal and equal.

Each node holds its own data and the state of the entire cluster. Each node is connected to all the other nodes, and these connections remain active, which ensures that I

We only need to connect to any node in the cluster to get the data of other nodes.

Instead of using traditional consistent hashes to allocate data, Redis clusters allocate data in another way called hash slot.

Redis cluster allocates 16384 slot by default. When we set a key, we use the CRC16 algorithm to get the slot we belong to, and then use this key

Assigned to the nodes of the hash slot, the specific algorithm is: CRC16 (key)% 16384. So when we saw set and get in the test, we jumped directly.

To the node at port 7000.

The Redis cluster stores the data in a master node and then synchronizes the data between the master and its corresponding salve. When reading data

The data is also obtained from the corresponding master node according to the consistent hashing algorithm. Only when a master is down will a corresponding salve node be started.

Act as master.

It should be noted that there must be 3 or more master nodes, otherwise the cluster will fail, and when the number of surviving master nodes is less than half of the total number of nodes

The entire cluster cannot provide services.

At this point, I believe you have a deeper understanding of the "installation process of redis5.0 cluster". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 222

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Database

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report