Shulou(Shulou.com)06/01 Report--

Introduction of Sentinel Cluster

Redis's sentinel, which is often used to manage multiple Redis servers, performs the following three main tasks:

Monitoring: the sentinel will constantly check to see if your Master and Slave are working properly.

Notification: when there is a problem with a monitored Redis, the sentinel can send a notification to the administrator or other application through API.

Automatic failover (Automatic failover): when a Master does not work properly, the sentinel will start an automatic failover operation, which will upgrade one of the Slave after the Master failure to the new Master and let the other Slave after the Master failure copy the new Master; instead. When the client tries to connect to the failed Master, the cluster will also return the address of the new Master to the client, so that the cluster can use the new Master instead of the failed Master.

Sentinel is a distributed system where you can run multiple sentinel processes in an architecture that use the gossip protocol (gossipprotocols) to receive information about whether the Master is offline, and the voting protocol (agreement protocols) to decide whether to perform an automatic failover and which Slave to choose as the new Master.

Each sentinel sends regular messages to other sentinel, master, and slave to confirm whether the other person is "alive". If he finds that the other person does not respond within a specified time (configurable), he or she temporarily thinks that the other person has hung up (the so-called "subjective downtime" Subjective Down, or sdown).

If most of the sentinel in the "Sentinel Group" reports that a Master is not responding, the system thinks that the Master is "completely dead" (that is, the objective real downmachine, Objective Down, referred to as odown). Through a certain vote algorithm, one of the remaining slave nodes is promoted to master, and then the relevant configuration is automatically modified.

Although sentinel is released as a separate executable redis-sentinel, it is really just a Redis server running in a special mode, and you can start sentinel with the-- sentinel option when starting a normal Redis server.

Server planning

Redis installation

Download the official installation package: https://redis.io/download

1. Redis support package installation

Because Redis is written in C language, the system needs to install gcc [root@rocketmq-nameserver1] # yum install-y gcc automake autoconf libtool make2, decompress the downloaded Redis installation package, enter the directory, compile, and operate on three servers respectively. [root@rocketmq-nameserver1 redis-5.0.4] # make install this process will take about 2 minutes. Wait patiently. Note: if this error occurs in the compilation process, you need to use this command make MALLOC=libc install to solve the problem. 3. Copy the Redis command file to the / usr/bin directory [root@rocketmq-nameserver1 redis-5.0.4] # cd src/ [root@rocketmq-nameserver1 src] # cp redis-cli redis-sentinel redis-server / usr/bin

Redis configuration

Filter Redis profile redis.conf

[root@rocketmq-nameserver1 redis-5.0.4] # grep-Ev "^ # | ^ $" redis.conf

Bind 192.168.2.177

Protected-mode yes

Port 6379

Tcp-backlog 511

Timeout 0

Tcp-keepalive 300

Daemonize yes

Supervised no

Pidfile / wdata/redis/data/redis.pid

Loglevel notice

Logfile "/ wdata/redis/logs/redis.log"

Databases 16

Always-show-logo yes

Save 900 1

Save 300 10

Save 60 10000

Stop-writes-on-bgsave-error yes

Rdbcompression yes

Rdbchecksum yes

Dbfilename dump.rdb

Dir / wdata/redis

Replicaof 192.168.2.177 6379

Replica-serve-stale-data yes

Replica-read-only yes

Repl-diskless-sync no

Repl-diskless-sync-delay 5

Repl-disable-tcp-nodelay no

Replica-priority 100

Lazyfree-lazy-eviction no

Lazyfree-lazy-expire no

Lazyfree-lazy-server-del no

Replica-lazy-flush no

Appendonly no

Appendfilename "appendonly.aof"

Appendfsync everysec

No-appendfsync-on-rewrite no

Auto-aof-rewrite-percentage 100

Auto-aof-rewrite-min-size 64mb

Aof-load-truncated yes

Aof-use-rdb-preamble yes

Lua-time-limit 5000

Slowlog-log-slower-than 10000

Slowlog-max-len 128

Latency-monitor-threshold 0

Notify-keyspace-events ""

Hash-max-ziplist-entries 512

Hash-max-ziplist-value 64

List-max-ziplist-size-2

List-compress-depth 0

Set-max-intset-entries 512

Zset-max-ziplist-entries 128

Zset-max-ziplist-value 64

Hll-sparse-max-bytes 3000

Stream-node-max-bytes 4096

Stream-node-max-entries 100

Activerehashing yes

Client-output-buffer-limit normal 0 0 0

Client-output-buffer-limit replica 256mb 64mb 60

Client-output-buffer-limit pubsub 32mb 8mb 60

Hz 10

Dynamic-hz yes

Aof-rewrite-incremental-fsync yes

Rdb-save-incremental-fsync yes

We only need to modify the red part of the Redis configuration file, and other options can be modified according to our own needs. The following two items are

Daemonize yes: sets Redis to run in the background when it starts

Replicaof 192.168.2.177 6379: set the cluster Master server address and port. Note that this is annotated on the 192.168.2.177 main Redis server

Copy the modified configuration file to the other two servers

For i in 178180; do scp redis.conf root@192.168.2.$i:/wdata/redis/config; done

Sentinel configuration

Filter sentinel profile sentinel.conf

[root@rocketmq-nameserver1 redis-5.0.4] # grep-Ev "^ # | ^ $" sentinel.conf

Bind 192.168.2.177

Port 26379

Daemonize yes

Pidfile / wdata/redis-sentinel.pid

Logfile "/ wdata/redis/logs/sentinel.log"

Dir / wdata/redis

Sentinel monitor mymaster 192.168.2.177 6379 2

Sentinel down-after-milliseconds mymaster 30000

Sentinel parallel-syncs mymaster 1

Sentinel failover-timeout mymaster 180000

Sentinel deny-scripts-reconfig yes

In general, we only need to modify the part marked red above, and the other options are modified according to the actual needs. The meaning of the following intention is

Sentinel monitor mymaster 192.168.2.177 6379 2: set the primary server for Sentinel monitoring to 192.168.2.177 and port 6379 to signify failover if both Sentinels consider the primary server inaccessible.

Distribute the modified sentinel configuration file to the other two servers.

For i in 178180; do scp sentinel.conf root@192.168.2.$i:/wdata/redis/config; done

Start Redis

Execute each of the three servers

[root@rocketmq-nameserver1 ~] # service redis start

Check to see if Redis starts successfully

[root@rocketmq-nameserver1 ~] # ps-ef | grep redis

Or

[root@rocketmq-nameserver1 ~] # service redis status

Activate the sentinel.

Execute each of the three servers

[root@rocketmq-nameserver1 ~] # service sentinel start

Check to see if the sentinel started successfully.

[root@rocketmq-nameserver1 ~] # ps-ef | grep sentinel

Or

[root@rocketmq-nameserver1 ~] # service sentinel status

View Sentinel cluster status

Our Redis service and sentinel service have been started successfully, and then we need to verify that the cluster is working.

[root@rocketmq-nameserver1 ~] # redis-cli-h 192.168.2.177-p 26379192.177name 26379 > SENTINEL sentinels mymaster1) 1) "name" 2) "648ced6a4a5126ffe053c7190a7787ce8507122d" 3) "ip" 4) "192.168.2.178" 5) "port" 6) "26379" 7) "runid" 8) "648ced6a4a5126ffe053c7190a7787ce8507122d" 9) "flags" 10) "sentinel" 11) "link-pending-commands" 12) "0" 13) "link-refcount" 14) "1" 15) "last-ping-sent" 16) "0" 17) "last-ok-ping-reply" 18) "203" 19) "last-ping-reply" 20) "203" 21) "down-after-milliseconds" 22) "30000" 23) "last-hello-message" 24) "425" 25) "voted-leader" 26) "27)" voted-leader-epoch "28)" 0 "2) 1)" name "2)" 25133c581dc5a5dcc41ed40d720bf417b70d6449 "3)" ip "4)" 192.168.2.180 "5)" port "6)" 26379 "7)" runid "8)" 25133c581dc5a5dcc41ed40d720bf417b70d6449 "9)" flags " 10) "sentinel" 11) "link-pending-commands" 12) "0" 13) "link-refcount" 14) "1" 15) "last-ping-sent" 16) "0" 17) "last-ok-ping-reply" 18) "20" last-ping-reply "20)" 203 "21)" down-after-milliseconds "22)" 30000 "23)" last -hello-message "24)" 870 "25)" voted-leader "26)"? "27)" voted-leader-epoch "28)" 0 "

As can be seen from the output, the other two sentinels are operating normally.

192.168.2.177mymaster 26379 > SENTINEL masters1) 1) "name" 2) "mymaster" 3) "ip" 4) "192.168.2.177" 5) "port" 6) "6379" 7) "runid" 8) "f2c92c055ec129186fec93d0b394a99ad120fd1d" 9) "flags" 10) "master" 11) "link-pending-commands" 12) "0" 13) "link-refcount" 14) "1" 15) "last-ping-sent" 16) "0" 17) "last-ok-ping-reply" 18) "800" 19) "last-ping-reply" 20) "800" 21) "down-after-milliseconds" 22) "30000" 23) "info-refresh" 24) "10034" 25) "role-reported" 26) "master" 27) "role-reported-time" 28) "120535" 29) "config-epoch" 30) "0" 31) "num-slaves" 32) "2" 33) "num-other-sentinels" 34) "2" 35) "quorum" 36) "2" 37) "failover-timeout" 38) "180000" 39) "parallel-syncs" 40) "1"

As can be seen from the output above, the main Redis is normal.

192.168.2.177ip 26379 > SENTINEL slaves mymaster1) 1) "name" 2) "192.168.2.180" 6379 "3)" ip "4)" 192.168.2.180 "5)" port "6)" 6379 "7)" runid "8)" 2675617f208ace5c13161e133819be75f7079946 "9)" flags "10)" slave "11)" link-pending-commands " 12) "0" 13) "link-refcount" 14) "1" 15) "last-ping-sent" 16) "0" 17) "last-ok-ping-reply" 18) "last-ping-reply" 20) "down-after-milliseconds" 22) "30000" 23) "info-refresh" 24 ) "4606" 25) "role-reported" 26) "slave" 27) "role-reported-time" 28) "235584" 29) "master-link-down-time" 30) "0" 31) "master-link-status" 32) "ok" 33) "master-host" 34) "192.168.2.177" 35) "master-port" 36) "6379" 37) "slave-priority" 38) "100" 39) "slave-repl-offset" 40) "47120" 2) 1) "name" 2) "192.168.2.178 name" 3) "ip" 4) "192.168.2.178" 5) "port" 6) "6379" 7) "runid" 8 ) "803d8178c44a4380243523248dca0838c7964299" 9) "flags" 10) "slave" 11) "link-pending-commands" 12) "0" 13) "link-refcount" 14) "1" 15) "last-ping-sent" 16) "0" 17) "last-ok-ping-reply" 18) "448" 19) "last-ping-reply" 20) "21)" down-after-milliseconds "22)" 30000 "23)" info-refresh "24)" 4606 "25)" role-reported "26)" slave "27)" role-reported-time "28)" 235627 "29)" master-link-down-time "30)" 0 "31)" master-link-status "32)" ok " 33) "master-host" 34) "192.168.2.177" 35) "master-port" 36) "6379" 37) "slave-priority" 38) "100" 39) "slave-repl-offset" 40) "47120"

The output from the above shows that the Redis from the service is normal.

Sentinel cluster common commands

SENTINEL masters # lists all monitored master And the current master status SENTINEL master # list specified masterSENTINEL slaves # list all slave for a given master and slave status SENTINEL sentinels # list all sentinelSENTINEL get-master-addr-by-name for monitoring the specified master # IP address and port number SENTINEL reset of the server with the given master name returned # reset all master states that match pattern expressions SENTINEL failover # when msater fails Force an automatic failover to start without asking other Sentinel comments, but it will send the other sentinel an up-to-date configuration according to which the other sentinel will update the SENTINEL ckquorum # check whether the current sentinel configuration can reach the number required for the failover master This command can be used to detect whether the sentinel deployment is normal, and return okSENTINEL flushconfig normally # forcing sentinel to write the runtime configuration to disk, including the current sentinel status

Redis startup script

#! / bin/sh#chkconfig: 2345 55 25## Simple Redis init.d script conceived to work on Linux systems# as it does use of the / proc filesystem.### BEGIN INIT INFO# Provides: redis_6379# Default-Start: 2345 # Default-Stop: 0 1 6# Short-Description: Redis data structure server# Description: Redis data structure server. See https://redis.io### END INIT INFOsource / etc/init.d/functionsREDISPORT=6379EXEC=/usr/bin/redis-serverCLIEXEC=/usr/bin/redis-cliPIDFILE=/wdata/redis/data/redis.pidCONF= "/ wdata/redis/config/redis.conf" AUTH= "BIND_IP='192.168.2.177'start () {if [- f $PIDFILE] then echo" $PIDFILE exists Process is already running or crashed "else echo" Starting Redis server... "$EXEC $CONF fi if [" $? "=" 0 "] then echo" Redis is running... " Else echo "Redis not running!" Fi} stop () {if [!-f $PIDFILE] then echo "$PIDFILE does not exist, process is not running" else PID=$ (cat $PIDFILE) echo "Stopping..." # $CLIEXEC-h $BIND_IP-a $AUTH-p $REDISPORT shutdown $CLIEXEC-h $BIND_IP-p $REDISPORT shutdown while [- x / proc/$ {PID}] do echo "Waiting for Redis to shutdown..." Sleep 1 done echo "Redis stopped." Fi} status () {ps-ef | grep redis-server | grep-v grep > / dev/null 2 > & 1 if [$?-eq 0]; then echo "redis server is running." Else echo "redis server is stopped." Fi} case "$1" in start) start;; stop) stop;; restart) stop start;; status) status; *) echo "Please use start or stop as first argument";; esac

Sentinel startup script

#! / bin/sh#chkconfig: 2345 55 25## Simple Sentinel init.d script conceived to work on Linux systems# as it does use of the / proc filesystem.### BEGIN INIT INFO# Provides: redis_6379# Default-Start: 2345 # Default-Stop: 0 1 6# Short-Description: Sentinel data structure server# Description: Sentinel data structure server. See https://redis.io### END INIT INFOsource / etc/init.d/functionsREDISPORT=26379EXEC=/usr/bin/redis-sentinelCLIEXEC=/usr/bin/redis-cliPIDFILE=/wdata/redis/data/redis-sentinel.pidCONF= "/ wdata/redis/config/sentinel.conf" AUTH= "BIND_IP='192.168.2.177'start () {if [- f $PIDFILE] then echo" $PIDFILE exists Process is already running or crashed "else echo" Starting Sentinel server... "$EXEC $CONF fi if [" $? "=" 0 "] then echo" Sentinel is running... " Else echo "Sentinel not running!" Fi} stop () {if [!-f $PIDFILE] then echo "$PIDFILE does not exist, process is not running" else PID=$ (cat $PIDFILE) echo "Stopping..." # $CLIEXEC-h $BIND_IP-a $AUTH-p $REDISPORT shutdown $CLIEXEC-h $BIND_IP-p $REDISPORT shutdown while [- x / proc/$ {PID}] do echo "Waiting for Sentinel to shutdown..." Sleep 1 done echo "Sentinel stopped." Fi} status () {ps-ef | grep redis-sentinel | grep-v grep > / dev/null 2 > & 1 if [$?-eq 0]; then echo "Sentinel server is running." Else echo "Sentinel server is stopped." Fi} case "$1" in start) start;; stop) stop;; restart) stop start;; status) status; *) echo "Please use start or stop as first argument";; esac

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.