Shulou(Shulou.com)05/31 Report--

Editor to share with you a sample analysis of Redis Sentinel installation and configuration. I hope you will gain something after reading this article. Let's discuss it together.

Overview of Redis Sentinel

We know that Redis is similar to MySQL database with master-slave replication structure. In a product environment, if crash occurs in master, we hope that slave can be automatically upgraded to provide services in place of the business. How to achieve this function? Redis sentinel cluster can help us to achieve this function.

Redis Sentinel is the official Redis native high availability solution. The Redis Sentinel deployment architecture mainly consists of two parts: Redis Sentinel cluster and Redis master-slave cluster, in which Redis Sentinel cluster is a distributed cluster composed of several Sentinel nodes.

Fault discovery, failover, configuration center, and client notification can be achieved. The number of nodes in Redis Sentinel should meet the odd number of 2n+1 (n > = 1) (at least 3 are officially recommended).

Characteristics of Redis Sentinel

(1) the failover between master and slave is monitored through sentinel. If there are five sentinel, as long as two sentinel think master crash, failover will be carried out in the configuration parameters, but the sentinel carrying out failover must be authorized by at least 3 sentinel before failover can be implemented.

(2) sentinel cluster will not execute failover concurrently by multiple sentinel at the same time. If the first sentinel for failover fails, the other will re-failover within a certain period of time, and so on.

(3) after failover, sentinel will get the latest configuration version number of master, and then broadcast to other sentinel, so a sentinel cluster that can communicate with each other will eventually adopt the highest version number and the same configuration.

(4) Redis Sentinel version1 starts with Redis2.6, Redis Sentinel version 2 starts with Redis 2.8.It is recommended to use Sentinel 2.

(5) Redis-Sentinel is a high availability (HA) solution officially recommended by Redis. Redis-sentinel itself is also an independent process, which can monitor multiple master-slave clusters and switch automatically when master is found to be down. Sentinel can monitor any number of master servers (reuse), as well as slave servers under the master server, and automatically perform failover operations when the monitored master server goes offline.

SDOWN and ODOWN

SDOWN (subjective downtime) is when sentinel subjectively detects that the state of master is down.

ODOWN (objective downtime) requires that most sentinel think master is down.

Switching from SDOWN to ODOWN does not require any consistency algorithm, only a gossip protocol. If a sentinel receives enough sentinel messages telling it that a master has been dropped, the SDOWN state will become ODOWN state. If the master becomes available later, this state will be cleaned up accordingly.

Sentinel.conf related parameters

Port 26379

# Port number of sentinel

Sentinel monitor mymaster 127.0.0.1 6379 2

The master name of # sentinel monitoring is mymaster by default. The final number 2 indicates that if two sentinel think the master is dead, then the master is considered unavailable.

Note: we can have a set of Sentinel Cluster monitor multiple master clusters by configuring different Redis master-slave names

Sentinel down-after-milliseconds mymaster 30000

# by default, 30 seconds. Sentinel uses ping to determine whether master is alive. If master returns pong to sentinel within 30 seconds, master is considered good, otherwise sentinel considers master unavailable.

Sentinel parallel-syncs mymaster 1

# when the Sentinel node set agrees on the failure determination of the master node, the Sentinel leader node will perform a failover operation and select a new master node, and the original slave node will initiate a replication operation to the new master node, limiting the number of slave nodes to the new master node to 1 at a time

Sentinel failover-timeout mymaster 180000

# failover timeout is 3min

Authentication in Redis Sentinel

When a master is configured to require a password to connect, both the client and the slave need to provide a password when connecting

Master sets its own password through requirepass. It cannot connect to this master without providing a password.

Slave uses masterauth to set the password when accessing master

But when using sentinel, because a master may become a slave and a slave may become a master, you need to set both configuration items.

Install Redis Sentinel

(1) Redis sentinel architecture diagram and node environment

RoelHostIPPortSentinel1sht-sgmhadoopnn-01172.16.101.5426379Sentinel2sht-sgmhadoopnn-01172.16.101.5526379Sentinel3sht-sgmhadoopnn-02172.16.101.5626379Mastersht-sgmhadoopdn-01172.16.101.586379Slave1sht-sgmhadoopdn-02172.16.101.596379Slave2sht-sgmhadoopdn-03172.16.101.606379

(2) configure Redis master-slave replication

[root@sht-sgmhadoopdn-01 redis] # vim redis.confbind 172.16.101.58 [root@sht-sgmhadoopdn-01 redis] # src/redis-server redis.conf [root@sht-sgmhadoopdn-02 redis] # vim redis.confbind 172.16.101.59slaveof 172.16.101.58 6379 [root@sht-sgmhadoopdn-02 redis] # src/redis-server redis.conf [root@sht-sgmhadoopdn-03 redis] # vim redis.confbind 172.16.101.60slaveof 172.16.101.58 6379 [root@sht-sgmhadoopdn-03 redis] # src/redis-server redis.conf

Check the settings of master-slave replication

[root@sht-sgmhadoopdn-01 redis] # src/redis-cli-h 172.16.101.58172.16.101.58pur6379 > client listid=3 addr=172.16.101.59:35718 fd=7 name= age=26 idle=0 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=replconfid=4 addr=172.16.101.60:33986 fd=8 name= age=22 idle=0 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=replconfid=5 addr=172.16.101.58:38875 fd=9 name= age=4 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 Events=r cmd=client172.16.101.58:6379 > info replication# Replicationrole:masterconnected_slaves:2slave0:ip=172.16.101.59 Port=6379,state=online,offset=57,lag=0slave1:ip=172.16.101.60,port=6379,state=online,offset=57,lag=0master_repl_offset:57repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:2repl_backlog_histlen:56

(3) configure sentinel cluster

The sentinel.conf file configuration of the three sentinel nodes is the same. If they are on the same host, different port numbers are required.

[root@sht-sgmhadoopcm-01 redis] # vim sentinel.confport 26379daemonize yesprotected-mode nologfile "sentinel.log" dir / usr/local/redissentinel monitor mymaster 172.16.101.58 6379 2sentinel down-after-milliseconds mymaster 30000sentinel parallel-syncs mymaster 1

There are two ways to start a sentinel node:

Src/redis-sentinel sentinel.confsrc/redis-server sentinel.conf-- sentinel [root@sht-sgmhadoopcm-01 redis] # src/redis-sentinel sentinel.conf [root@sht-sgmhadoopnn-01 redis] # src/redis-sentinel sentinel.conf [root@sht-sgmhadoopnn-02 redis] # src/redis-sentinel sentinel.conf [root@sht-sgmhadoopcm-01 redis] # ps-ef | grep redis | grep-v greproot 7541 1 0 22:33? 00:00:00 src/redis-sentinel *: 26379 [sentinel]

(4) check the status of the whole cluster

[root@sht-sgmhadoopcm-01 redis] # src/redis-cli-h 172.16.101.54-p 26379172.16.101.54 client listid=3 addr=172.16.101.55:43182 fd=13 name=sentinel-ab45fe6c-cmd age=138 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=pingid=4 addr=172.16.101.56:60016 fd=15 name=sentinel-e32f20c0-cmd age=136 idle=1 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=pingid=5 addr=172.16.101.54:35342 fd=17 name= age=26 idle=0 flags=N db=0 sub=0 Psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client172.16.101.54:26379 > info sentinel# Sentinelsentinel_masters:1sentinel_tilt:0sentinel_running_scripts:0sentinel_scripts_queue_length:0sentinel_simulate_failure_flags:0master0:name=mymaster Status=ok,address=172.16.101.58:6379,slaves=2 Sentinels=3 [root@sht-sgmhadoopdn-01 redis] # src/redis-cli-h 172.16.101.58-p 6379172.16.101.58 client listid=16 addr=172.16.101.54:56510 fd=10 name=sentinel-30393e76-pubsub age=326 idle=0 flags=N db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=subscribeid=17 addr=172.16.101.54:56508 fd=11 name=sentinel-30393e76-cmd age=326 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=pingid=18 addr=172.16.101.55:57444 fd=12 name=sentinel-ab45fe6c-cmd 6379 Age=177 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=publishid=19 addr=172.16.101.55:57446 fd=13 name=sentinel-ab45fe6c-pubsub age=177 idle=0 flags=N db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=subscribeid=3 addr=172.16.101.59:35718 fd=7 name= age=3936 idle=0 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=replconfid=4 addr=172.16.101.60:33986 fd=8 name= age=3932 idle=0 flags=S db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=replconfid=20 addr=172.16.101.56:55648 fd=14 name=sentinel -e32f20c0-cmd age=173 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=pingid=21 addr=172.16.101.56:55650 fd=15 name=sentinel-e32f20c0-pubsub age=173 idle=0 flags=N db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=subscribeid=5 addr=172.16.101.58:38875 fd=9 name= age=3914 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client

When we start the master-slave node and the sentinel node, the sentinel.conf configuration file automatically adds or modifies parameters

[root@sht-sgmhadoopcm-01 redis] # cat sentinel.confsentinel myid 30393e76e002cb64db92fb8bcb88d79f2d85a82bsentinel config-epoch mymaster 0sentinel leader-epoch mymaster 0# Generated by CONFIG REWRITEsentinel known-slave mymaster 172.16.101.60 6379sentinel known-slave mymaster 172.16.101.59 6379sentinel known-sentinel mymaster 172.16.101.55 26379 ab45fe6c0f010473ce3b7b4d2120e1a83776b736sentinel known-sentinel mymaster 172.16.101.56 26379 e32f20c0f315e712c9921371f15729246f3816a0sentinel current-epoch 0 [root@sht-sgmhadoopnn-01 redis] # cat sentinel.confsentinel myid ab45fe6c0f010473ce3b7b4d2120e1a83776b736sentinel config-epoch mymaster 0sentinel leader-epoch mymaster 0# Generated by CONFIG REWRITEsentinel known- Slave mymaster 172.16.101.60 6379sentinel known-slave mymaster 172.16.101.59 6379sentinel known-sentinel mymaster 172.16.101.56 26379 e32f20c0f315e712c9921371f15729246f3816a0sentinel known-sentinel mymaster 172.16.101.54 26379 30393e76e002cb64db92fb8bcb88d79f2d85a82bsentinel current-epoch 0 [root@sht-sgmhadoopnn-02 redis] # cat sentinel.confsentinel myid e32f20c0f315e712c9921371f15729246f3816a0sentinel config-epoch mymaster 0sentinel leader-epoch mymaster 0# Generated by CONFIG REWRITEsentinel known-slave mymaster 172.16.101.60 6379sentinel known-slave mymaster 172.16.101.59 6379sentinel known-sentinel mymaster 172.16.101.54 26379 30393e76e002cb64db92fb8bcb88d79f2d85a82bsentinel known-sentinel mymaster 172.16.101.55 26379 ab45fe6c0f010473ce3b7b4d2120e1a83776b736sentinel current-epoch 0

Test automatic failover

[root@sht-sgmhadoopdn-01 redis] # ps-ef | grep redisroot 15128 10 21:17? 00:00:05 src/redis-server 172.16.101.58 src/redis-server 172.16.101.58 sentinel 6379 [root@sht-sgmhadoopdn-01 redis] # kill-9 15128 [root@sht-sgmhadoopcm-01 redis] # tail-f sentinel.log7541:X 05 Aug 22 Vera 55 Vera 48.052 # + sdown master mymaster 172.16.101.58 6379 # sentinel subjectively considers master crash 7541 sentinel X 05 Aug 22 Aug 55 master crash 48.143 # + odown master mymaster 172.16.101.58 6379 # quorum 2 quorum 2 # as long as there are two master crash nodes Then it is objectively considered that master crash7541:X 05 Aug 22 new-epoch 17541 new-epoch 17541 try-failover master mymaster 172.16.101.58 63797541 new-epoch 17541 try-failover master mymaster 05 vote-for-leader 30393e76e002cb64db92fb8bcb88d79f2d85a82b 17541VX 05 Aug 22 55 Aug 5548.166 # ab45fe6c0f010473ce3b7b4d2120e1a83776b736 voted for ab45fe6c0f010473ce3b7b4d2120e1a83776b736 17541 Aug 2255 purl 48.173 # e32f20c0f315e712c9921371f15729246f3816a0 voted for ab45fe6c0f010473ce3b7b4d2120e1a83776b736 17541 e32f20c0f315e712c9921371f15729246f3816a0 voted for ab45fe6c0f010473ce3b7b4d2120e1a83776b736 05 Aug 22 55 purl 48.544 # + config-update-from sentinel ab45fe6c0f010473ce3b7b4d2120e1a83776b736 172.16.101.55 26379 @ mymaster 172.16.101.58 63797541 slave slave X 05 Aug 2241 switch-master mymaster 172.16.101.58 6379 172.16.101.60 63797541 switch-master mymaster 172.16.101.59 slave slave 172.16.101.59 slave slave 172.16.101.59 slave slave 172.16.101.60 63797541slave slave 172.16.101.60 63797541MAX 05 Aug 221451 slave slave 172.16.101.58 slave slave 172.16.101.58 58 6379 @ mymaster 172.16.101.60 6379 # wait 30 seconds from this step to the successful execution of failover in the next step This is due to the parameter sentinel down-after-milliseconds mymaster control, there is no response within 30s master sentinel will be the real failover 7541 mymaster X 05 Aug 22 56 mymaster 18.562 # + 172.16.101.58 mymaster 172.16.101.58

Master and slave have changed, and IP60 has become the new master,IP58 and slave.

[root@sht-sgmhadoopcm-01 redis] # src/redis-cli-h 172.16.101.54-p 26379172.16.101.54 mymaster 26379 > sentinel masters1) 1) "name" 2) "mymaster" 3) "ip" 4) "172.16.101.60" .172.16.101.54: 26379 > sentinel slaves mymaster1) 1) "name" 2) "172.16.101.58 Suzhou 6379" 3) "ip" 4) "172.16.101.58" 9) "flags" 10) "s_down" Slave,disconnected "2) 1)" name "2)" 172.16.101.59 ip "3)" ip "4)" 172.16.101.59 "9)" flags "10)" slave "

After restarting the repaired old master, it will automatically become the slave library of the new master.

[root@sht-sgmhadoopdn-01 redis] # src/redis-server redis.conf [root@sht-sgmhadoopcm-01 redis] # tail-f sentinel.log7541:X 05 Aug 23lav 11VO 10.556 #-sdown slave 172.16.101.58 sentinel.log7541:X 172.16.101.58 6379 @ mymaster 172.16.101.60 63797541 Vane X 05 Aug 23v 11v 20.518 * + convert-to-slave slave 172.16.101.58 R 6379 172.16.101.58 6379 @ mymaster 172.16.101.60 6379

Failover process Analysis:

Each Sentinel detects the master is down with an + sdown event.

This event is later escalated to + odown, which means that multiple Sentinels agree about the fact the master is not reachable.

Sentinels vote a Sentinel that will start the first failover attempt.

The failover happens.

The sentinel node periodically checks whether the master of the redis is alive through ping, once the master crash

First of all, sentinel will subjectively think of master crash, and then the three sentinel will communicate with each other. As long as two sentinel nodes think of master crash, they will objectively consider master crash.

Then three sentinel nodes will vote, and a sentinel with two votes will execute the failover

Finally, if there is no response within 30s of master, sentinel will become the real failover.

Once the old dead master is fixed and restarted, it will exist as a slave library for the new master

FAQ

Error1:

[root@sht-sgmhadoopcm-01 redis] # src/redis-cli-h 172.16.101.54-p 26379172.16.101.54 ping (error) DENIED Redis is running in protected mode because protected mode is enabled, no bind address was specified, no authentication password is requested to clients. In this mode connections are only accepted from the loopback interface. If you want to connect from external computers to Redis you may adopt one of the following solutions: 1) Just disable protected mode sending the command 'CONFIG SET protected-mode no' from the loopback interface by connecting to Redis from the same host the server is running, however MAKE SURE Redis is not publicly accessible from internet if you do so. Use CONFIG REWRITE to make this change permanent. 2) Alternatively you can just disable the protected mode by editing the Redis configuration file, and setting the protected mode option to 'no', and then restarting the server. 3) If you started the server manually just for testing, restart it with the'--protected-mode no' option.4) Setup a bind address or an authentication password. NOTE: You only need to do one of the above things in order for the server to start accepting connections from the outside. Solution: [root@sht-sgmhadoopcm-01 redis] # vim sentinel.confprotected-mode no has finished reading this article. I believe you have some understanding of "sample Analysis of Redis Sentinel installation and configuration". If you want to know more about it, please follow the industry information channel. Thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.