Shulou(Shulou.com)06/01 Report--

Installation management of redis-cluster

Declaration:

This article is only allowed for personal study and communication. If there are any mistakes, please correct them.

Document version: Version 1.0

Modification record: 2015-10-30

Environment introduction

System environment: RedHat Enterprise Linux Server release 6.2 (Santiago)

Kernel version: Linuxzxt-02.com 2.6.32-220.el6.x86_64 # 1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64x86_64 x86room64 GNU/Linux

Software version: redis-3.0.5

Hostname: redis-01.com, redis-02.com

Host IP:192.168.1.193 192.168.1.176

Install the required software environment:

Zlib, ruby (must be above 1.9.2), rubygem, gem-redis

Note: before reading a text document, you need to know that this document and most of the similar documents currently on the web are from http://redis.io. If you can read English well, it is recommended that you read the official Reis documentation. Http://redis.io/topics/cluster-spec

Http://redis.io/topics/cluster-tutorial

Introduction to redis-cluster:

A Redis cluster is a facility (installation) that can share data among multiple Redis nodes. Redis clusters do not support Redis commands that need to process multiple keys at the same time, because executing these commands requires moving data between multiple Redis nodes, and in high load situations, these commands degrade the performance of the Redis cluster and lead to unpredictable behavior.

Redis clusters provide a degree of availability through partition: even if some nodes in the cluster fail or are unable to communicate, the cluster can continue to process command requests.

Redis clustering provides the following two benefits:

The ability to automatically split data into multiple nodes.

The ability to continue to process command requests when some nodes in the cluster fail or are unable to communicate

Data fragmentation of Redis Cluster

Instead of using consistent hash, Redis clusters introduce the concept of hash slots.

The Redis cluster has 16384 hash slots, and each key passes the CRC16 check and modulates 16384 to decide which slot to place. Each node of the cluster is responsible for part of the hash slot. For example, if the current cluster has three nodes, then:

Node A contains hash slots 0 to 5500

Node B contains hash slots 5501 to 11000.

Node C contains hash slots 11001 to 16384.

This structure is easy to add or remove nodes. For example, if I want to add a new node D, I need to get some slots from nodes A, B, C to D. If I want to remove node A, I need to move the slot in A to the B and C nodes, and then remove the A node without any slots from the cluster.

Since moving hash slots from one node to another does not stop service, no matter adding, deleting or changing the number of hash slots of a node will not cause the cluster to be unavailable.

Master-Slave replication Model of Redis Cluster

In order to make the cluster still available when some nodes fail or most nodes can not communicate, the cluster uses the master-slave replication model, and each node will have a replica.

In our example, if node B fails in the absence of a replication model, the whole cluster will be unavailable because of the lack of slots in the range of 5501-11000.

However, when the cluster is created (or after a period of time), we add a slave node, A1 Magi B1, C1 for each node, then the whole cluster consists of three master nodes and three slave nodes, so that after node B fails, the cluster will elect B1 to continue to serve the new master node, and the whole cluster will not be unavailable because the slot cannot be found.

However, when both B and B 1 fail, the cluster is not available.

Redis consistency guarantee

Redis does not guarantee strong consistency of data. This means that in practice, the cluster may lose write operations under certain conditions.

The first reason is that the cluster uses asynchronous replication. Write operation process:

The client writes a command to master node B.

Master node B returns the command status to the client.

The master node replicates the write operation to him to get slave nodes B1, B2 and B3.

The copy of the command by the master node occurs after the command reply is returned, because if each command request needs to wait for the copy operation to complete, then the speed at which the master node processes command requests will be greatly reduced-we have to make a tradeoff between performance and consistency.

Note: Redis clusters may provide synchronous write methods in the future.

Another situation where commands may be lost in a Redis cluster is when a network partition appears in the cluster and a client is isolated from a small number of instances, including at least one master node. .

For example, suppose the cluster consists of six nodes A, B, C, A1, B1 and C1, in which A, B and C are master nodes, A1, B1 and C1 are subordinate nodes of A _ Magi B _ 1 C, and there is a client Z1

Assuming that network partitioning occurs in the cluster, the cluster may be divided into two sides, most of which contain nodes A, C, A1, B1 and C1, while a small part of the cluster contains node B and client Z1.

Z1 can still write to master node B. if the network partition occurs for a short time, the cluster will continue to operate normally. If the partition is long enough for most parties to elect B1 as the new master, then the data written by Z1 to B will be lost.

Note that during a network split, there is a limit to the maximum time that client Z1 can send write commands to primary node B. this time limit is called node timeout (node timeout) and is an important configuration option for Redis clusters:

Deployment environment

Redis-cluster requires at least 3 nodes. Because of the test environment, I use two virtual machines to install multiple nodes to simulate the redis cluster.

Host node assignment:

Node1 192.168.1.193:6379

Node2 192.168.1.193:6479

Node3 192.168.1.193:6579

Standby node:

Node1 192.168.1.176:6379

Node2 192.168.1.176:6479

Node3 192.168.1.176:6579

Install redis cluste and dependent software: 1) install zlib software

Zlib software can be installed by yum or compiled according to your mood.

Yum installation

[zxt@redis-01 ~] $yum install-y zlib* [ZXT @ redis-01 ~] $rpm-qa | grep zlib zlib-1.2.3-27.el6.x86_64zlib-devel-1.2.3-27.el6.x86_64

Compile and install:

# download: http://www.zlib.net/tar zxf zlib-1.2.7.tar.gzcd zlib. / configure make make install

2) install ruby: version (1.9.2)

To run Redis cluter, you must install ruby and version 1.9.2 or above. Do not use yum installation here, because the default version of the RedHat6.2 system yum installation is 1.8.7

# ruby-2.1.7.tar.gz tar zxvf ruby-2.1.7.tar.gzcd ruby-2.1.7./configure-prefix=/usr/local/ruby make make install cp ruby / usr/local/bin

3) install rubygem:version (1.8.5) # rubygems-1.8.5.tgz tar zxvf rubygems-1.8.5.tgzcd rubygems-1.8.5ruby setup.rb cp bin/gem / usr/local/bin4) install gem-redis:version (3.0.5)

Install the interfaces of redis and ruby necessary to run the redis cluster.

Gem install redis-version 3.0.5

# due to some reasons, the gem source cannot be accessed or the download may fail. Here are two solutions:

Method 1: download and install manually

# download address: http://rubygems.org/gems/redis/versions/3.0.0 gem install-l / soft/redis-3.0.5.gem

Method 2: replace the gem source

Gem sources-- remove https://rubygems.org/gem sources-a http://ruby.sdutlinux.org/gem sources-lumped eggs * CURRENT SOURCES*** https://ruby.taobao.org

Commonly used sources

Http://rubygems.org/http://gems.github.comhttp://gems.rubyforge.orghttp://ruby.sdutlinux.org/ # domestic should find a more reliable, suitable for installing most common gem5) install redistar xzfredis-3.0.5.tar.gzcp-r redis-3.0.5/ opt/app/ln-s / opt/app/redis-3.0.5/ / opt/rediscd / opt/redismake testmakemake install

Description:

After the make install command is executed, the executable files, redis-server, redis-cli, redis-benchmark, redis-check-aof, and redis-check-dump, are generated in the / usr/local/bin directory. Their functions are as follows:

Daemon launcher for redis-server:Redis server

Redis-cli:Redis command line operation tool. You can also use telnet to operate according to its plain text protocol

Redis-benchmark:Redis performance testing tool to test the read and write performance of Redis on the current system

Redis-check-aof: data repair

Redis-check-dump: check the export utility

Configure redis cluster:

1) configure the cluster initialization script:

Cd / opt/redis cp/opt/redis/src/redis-trib.rb / usr/local/bin

2) configure redis cluster profile

Daemonize yes# runs as a background process redis. If pidfile/opt/redis/run/redis_6379.pid# runs Reids as a background process, you need to specify the pid file and path. Port 637 specifies the redis listening port. Tcp-backlog 51 runs in a highly concurrent environment to avoid the slow connection problem of the client. Bind 0.0.0.bind the host IP. (set to 4 zeros here to facilitate program calls). The timeout of a timeout client connection (in seconds). Tcp-keepalive 6: on Linux, specify a value (seconds) for the time it takes to send ACKs. Note that it takes twice as long to close the connection. The default is 0loglevel notice# logging level, and there are 4 optional values, debug,verbose (default), notice,warninglogfile "/ var/log/redis/redis_6379.log" # log file address databases "available database number save 900 1save 300 10save 60 1000" saves data to disk based on a given time interval and number of writes, in seconds stop-writes-on-bgsave-erroryes# background storage error stops writing. Whether rdbcompression yes# compresses data when it is stored in a local database (persisted to a dump.rdb file). The default is whether yesrdbchecksum yes# validates rdb files. Dbfilenamedump_6379.rdb# local persistent database file name, default is the path where files are placed in dump.rdbdir / opt/redis/data# database mirrored backup. # slaveof # enable this parameter when setting this database as a slave database for other databases. # password of masterauth#slave service connection master slave-serve-stale-datayesslave-read-only yes#slave read-only repl-diskless-syncnorepl-diskless-sync-delay5repl-disable-tcp-nodelaynoslave-priority 100#requirepassfoobared# setting client connection secret code appendonly yes# Open aof persistent appendfilename "appendonly_6379.aof" # aof file name The default is appendonly.aofappendfsync everysec# once per second aof write no-appendfsync-on-rewriteyes# off fsync auto-aof-rewrite-percentage100# for new writes on aof rewrite and redis instances deployed on the same machine Rub the auto-aof-rewrite apart because the memory footprint in the cluster environment is basically the same. # prevent all redis processes from doing aof rewrite in an instant fork on the same machine Takes up a lot of memory (ps:cluster must turn on aof) auto-aof-rewrite-min-size64mbaof-load-truncatedyeslua-time-limit 5000cluster-enabled yes# opens redis cluster cluster-config-filenodes-6379.conf# cluster node configuration file (starts automatic generation) cluster-node-timeout15000# node interconnect timeout threshold cluster-migration-barrier1slowlog-log-slower-than10000slowlog-max-len 128latency-monitor-threshold0notify-keyspace-events "" hash-max-ziplist-entries512hash-max- Ziplist-value64list-max-ziplist-entries512list-max-ziplist-value64set-max-intset-entries512zset-max-ziplist-entries128zset-max-ziplist-value64hll-sparse-max-bytes3000activerehashing yesclient-output-buffer-limitnormal 00 0client-output-buffer-limitslave 256mb 64mb 60client-output-buffer-limitpubsub 32mb 8mb 60hz 10aof-rewrite-incremental-fsyncyes

Note: other nodes only need to copy the configuration file to change all port numbers.

Rediscluster operation and maintenance operation 1. Start the 6 redis instance nodes 192.1.168.1.193cd / opt/redisredis-serverredis_6379.confredis-serverredis_6479.confredis-serverredis_6579.conf192.1.168.1.193cd / opt/redisredis-serverredis_6379.confredis-serverredis_6479.confredis-serverredis_6579.conf respectively.

After startup, use the netstat-lntp command to see if the port is listening, as shown in the figure, the startup is normal.

2. Use its own redis-trib.rb to build a cluster redis-trib.rb create--replicas 1 192.168.1.193149192.168.1.193479192.168.1.1936579 192.168.1.176Group6379192.168.1.193 6479 192.168.1.1936579

Note:

# build the create subcommand of redis-trib.rb

#-replicas specifies how many Master nodes are assigned to each Slave node in Redis Cluster

# Node roles are determined by sequence, with master followed by slave (for ease of identification, the port of slave is 1000 larger than that of master)

3. Check the cluster status

# checksubcommand of redis-trib.rb

# ip:port can be any node of the cluster

Redis-trib.rb check 192.168.1.193:6379

Finally, the following information is output without any warnings or errors (as shown in the figure), indicating that the cluster started successfully and is in an ok state.

4. Add a new master node

Create an empty node (empty node) and move some hash slots to the empty node.

A), create a new node profile:

In order to make it easier to distinguish, start another virtual machine here and add a pair of nodes:

192.168.1.187:6379

192.168.1.187:6479

Cd / opt/redisscp redis_6379.conf 192.168.1.187:/opt/redis/cp redis_6379.conf redis_6479.confsed-ie ie 6379 Universe 6479 Universe g redis_6479.conf

B) start the new node:

Redis-server redis_6379.conf

C) add new nodes to the cluster:

Redis-trib.rbadd-node 192.168.1.187:6379 192.168.1.193:6379

Add-node adds a node to the cluster, the first is the new node ip:port, and the second is any existing node ip:port

Note: when adding a new node, the new node cannot contain any data, otherwise the addition will fail.

Because it does not contain any hash slots. The newly added add point is a master node, which will not be selected when the cluster needs to upgrade a slave node to a new master node, and the new master node will not participate in the election and failover because it does not contain any hash slots.

D), manually add hash slots for the new node:

Redis-trib.rb reshard 192.168.1.187:6379

# Select the number of hash slots to migrate according to the prompts

How many slots do you want to move (from 1 to 16384)? 1000

# Select the node-id to accept these hash slots

What is the receiving node ID? 36c46361327dbb15d098a0c3794ac3d72869e508

# Select the source of hash slot:

# all means to reassign from all master

# or the master node id of the hash slot to be extracted from the data, and end with done

Please enter all the source node IDs.

Type 'all' to use all the nodes as sourcenodes for the hash slots.

Type 'done' once you entered all thesource nodes IDs.

Source node # 1:all

# after printing the moved hash slot, enter yes to start moving the hash slot and the corresponding data.

# Do you want to proceed with the proposed reshard plan (yes/no)? yes

# end

You can use the command to view the allocation of comparative hash slots (see figure)

Redis-trib.rbcheck 192.168.1.176:6379

At this point, a new master node has been added. Execute the command to check the status of the nodes in the current cluster.

Redis-trib.rbcheck 192.168.1.176:6379redis-cli-c-p 6379 cluster nodes5, add a new slave node

A): the first three steps are the same as adding master

Note: the newly added node cluster is assigned to master by default but the node does not have any slots assigned (see figure)

B) step 4: redis-cli connects to the new node shell, and enter the command: clusterreplicate corresponds to the node-id of master (here we enter the id of node 192.168.1.187).

Redis-cli-h 192.168.1.187-p 6479clusterreplicate 36c46361327dbb15d098a0c3794ac3d72869e508 exit

You can also check the cluster status to determine whether the addition is successful.

Note: when adding slave online, you need to bgsave the entire master data and pass it to slave, and then the slave loads the rdb file into memory. A large amount of Master memory and network IO are consumed in the process of rdb generation and transfer. It is not recommended that the memory of a single instance is too large. Be careful online.

5. Delete a slave node

Law one:

# redis-tribdel-node ip:port''redis-trib.rbdel-node 192.168.1.187purl 6479 4655fccff00ef4a7b99c10ffd590c8328ec6db8d

Method 2: just stop or kill off the node.

Redis-cli-p 6479 shutdownorkill-9 `cat / opt/redis/run/redis_ 6479.pid`6, delete a master node

A): before deleting a master node, first use reshard to remove all slot of the master, and then delete the current node

# migrate the current master of 192.168.1.187virtual6379 to 192.168.1.176virtual6579

Redis-trib.rbreshard 192.168.1.176:6579

# Select the number of hash slots to migrate according to the prompts

How manyslots do you want to move (from 1 to 16384)? 1000

(number of all hash slots of deleted master)

# Select 192.168.1.176VR 6579 to accept these hash slots

What isthe receiving node ID? E1c06dd4682a37eb6773d6cb1d5709034a3f2769 (ps: 192.168.1.176 node-id of node-id 6579)

Pleaseenter all the source node IDs.

Type'all' to use all the nodes as source nodes for the hash slots.

Type'done' once you entered all the source nodes IDs.

Sourcenode # 36c46361327dbb15d098a0c3794ac3d72869e508 (node-id of deleted master)

Sourcenode # 2:done

# after printing the moved hash slot, enter yes to start moving the hash slot and the corresponding data.

# Do youwant to proceed with the proposed reshard plan (yes/no)? Yes

B): delete empty master node

Redis-trib.rbdel-node 192.168.1.187 6379 '36c46361327dbb15d098a0c3794ac3d72869e508' 3, redis cluste common commands

1. View the cluster status

Redis-trib.rbcheck 127.0.0.1:6379

2. View cluster information

Redis-cli-c-p 6379 cluster nodes

3. Check the master-slave situation of the cluster

Redis-cli-c-p 6379 info Replication

4. View cluster information

Redis-cli-c-p 6379 cluster info Appendix I: brief introduction to redis Cluster commands system Management related instructions

Cluster

CLUSTER INFO prints information about the cluster

CLUSTER NODES lists all the nodes (node) currently known to the cluster, as well as relevant information about those nodes.

Node

CLUSTER MEET adds the nodes specified by ip and port to the cluster, making it part of the cluster.

CLUSTER FORGET removes the nodes specified by node_id from the cluster.

CLUSTER REPLICATE sets the current node as a slave to the node specified by node_id.

CLUSTER SAVECONFIG saves the configuration file of the node to the hard disk.

Slot (slot)

CLUSTER ADDSLOTS [slot...] Assign one or more slots (slot) to the current node (assign).

CLUSTER DELSLOTS [slot...] Removes the assignment of one or more slots to the current node.

CLUSTER FLUSHSLOTS removes all slots assigned to the current node, making the current node a node with no slots assigned.

CLUSTER SETSLOT NODE assigns the slot slot to the node specified by node_id, and if the slot is already assigned to another node, let the other node delete the slot > before assigning it.

CLUSTER SETSLOT MIGRATING migrates the slot slot of this node to the node specified by node_id.

CLUSTER SETSLOT IMPORTING imports slot slot into this node from the node specified by node_id.

CLUSTER SETSLOT STABLE cancels the import (import) or migration (migrate) of slot slot.

Key

CLUSTER KEYSLOT calculates which slot the key key should be placed on.

CLUSTER COUNTKEYSINSLOT returns the number of key-value pairs currently contained in slot slot.

CLUSTER GETKEYSINSLOT returns the keys in count slot slots.

Reference materials and extended knowledge:

Http://hot66hot.iteye.com/blog/2050676/

Redis instructions:

Http://redis.io/commands

A website for discussing related issues created by redis enthusiasts:

Http://www.rediscookbook.org/

Why use Redis and its product positioning:

Http://www.infoq.com/cn/articles/tq-why-choose-redis

Redis memory usage optimization and storage:

Http://www.infoq.com/cn/articles/tq-redis-memory-usage-optimization-storage

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.