Shulou(Shulou.com)06/01 Report--

Installation management of redis-cluster

Environment introduction

System environment: Red Hat Enterprise Linux Server release 6.2 (Santiago)

Kernel version: Linux zxt-02.com 2.6.32-220.el6.x86_64 # 1 SMP Wed Nov 9 08:03:13 EST 2011 x86'64 GNU/Linux

Software version: redis-3.0.5

Hostname: redis-01.com, redis-02.com

Host IP:192.168.1.193 192.168.1.176

Install the required software environment:

Zlib, ruby (must be above 1.9.2), rubygem, gem-redis

Note: before reading a text document, you need to know that this document and most of the similar documents currently on the web are from http://redis.io. If you can read English well, it is recommended that you read the official Reis documentation. Http://redis.io/topics/cluster-spec

Http://redis.io/topics/cluster-tutorial

Introduction to redis-cluster:

A Redis cluster is a facility (installation) that can share data among multiple Redis nodes. Redis clusters do not support Redis commands that need to process multiple keys at the same time, because executing these commands requires moving data between multiple Redis nodes, and in high load situations, these commands degrade the performance of the Redis cluster and lead to unpredictable behavior.

Redis clusters provide a degree of availability through partition: even if some nodes in the cluster fail or are unable to communicate, the cluster can continue to process command requests.

Redis clustering provides the following two benefits:

The ability to automatically split data into multiple nodes.

The ability to continue to process command requests when some nodes in the cluster fail or are unable to communicate

Data fragmentation of Redis Cluster

Redis cluster does not use consistent hash, but introduces the concept of hash slot.

The Redis cluster has 16384 hash slots, and each key passes the CRC16 check and modulates 16384 to decide which slot to place. Each node of the cluster is responsible for part of the hash slot. For example, if the current cluster has three nodes, then:

L node A contains hash slots 0 to 5500

Node B contains hash slots 5501 to 11000.

Node C contains hash slots 11001 to 16384.

This structure is easy to add or remove nodes. For example, if I want to add a new node D, I need to get some slots from nodes A, B, C to D. If I want to remove node A, I need to move the slot in A to the B and C nodes, and then remove the A node without any slots from the cluster.

Since moving hash slots from one node to another does not stop service, no matter adding, deleting or changing the number of hash slots of a node will not cause the cluster to be unavailable.

Master-Slave replication Model of Redis Cluster

In order to make the cluster still available when some nodes fail or most nodes can not communicate, the cluster uses the master-slave replication model, and each node will have a replica.

In our example, if node B fails in the absence of a replication model, the whole cluster will be unavailable because of the lack of slots in the range of 5501-11000.

However, when the cluster is created (or after a period of time), we add a slave node, A1 Magi B1, C1 for each node, then the whole cluster consists of three master nodes and three slave nodes, so that after node B fails, the cluster will elect B1 to continue to serve the new master node, and the whole cluster will not be unavailable because the slot cannot be found.

However, when both B and B 1 fail, the cluster is not available.

Redis consistency guarantee

Redis does not guarantee strong consistency of data. This means that in practice, the cluster may lose write operations under certain conditions.

The first reason is that the cluster uses asynchronous replication. Write operation process:

The client writes a command to master node B.

Master node B returns the command status to the client.

The master node copies the write operation to him and has to slave nodes B 1, B 2 and B 3.

The copy of the command by the master node occurs after the command reply is returned, because if each command request needs to wait for the copy operation to complete, then the speed at which the master node processes command requests will be greatly reduced-we have to make a tradeoff between performance and consistency.

Note: Redis clusters may provide synchronous write methods in the future.

Another situation where commands may be lost in a Redis cluster is when a network partition appears in the cluster and a client is isolated from a small number of instances, including at least one master node. .

For example, suppose the cluster consists of six nodes A, B, C, A1, B1 and C1, in which A, B and C are master nodes, A1, B1 and C1 are subordinate nodes of A _ Magi B _ 1 C, and there is a client Z1

Assuming that network partitioning occurs in the cluster, the cluster may be divided into two sides, most of which contain nodes A, C, A1, B1 and C1, while a small part of the cluster contains node B and client Z1.

Z1 can still write to master node B. if the network partition occurs for a short time, the cluster will continue to operate normally. If the partition is long enough for most parties to elect B1 as the new master, then the data written by Z1 to B will be lost.

Note that during a network split, there is a limit to the maximum time that client Z1 can send write commands to primary node B. this time limit is called node timeout (node timeout) and is an important configuration option for Redis clusters:

Deployment environment

Redis-cluster requires at least 3 nodes. Because of the test environment, I use two virtual machines to install multiple nodes to simulate the redis cluster.

Node assignment:

Node1 192.168.1.193:6379node2 192.168.1.193:6479node3 192.168.1.193 6579 standby node: node1 192.168.1.176:6379node2 192.168.1.176:6479node3 192.168.1.176VR 6579

Install redis cluste and dependent software: 1) install zlib software; Zlib software can be installed by yum or compiled according to mood

Yum installation

[zxt@redis-01 ~] $yum install-y zlib* [ZXT @ redis-01 ~] $rpm-qa | grep zlib zlib-1.2.3-27.el6.x86_64zlib-devel-1.2.3-27.el6.x86_64

Compile and install:

# download: http://www.zlib.net/tar zxf zlib-1.2.7.tar.gzcd zlib. / configure make make install

2) install ruby: version (1.9.2)

To run Redis cluter, you must install ruby and version 1.9.2 or above. Do not use yum installation here, because the default version of the RedHat6.2 system yum installation is 1.8.7

# ruby-2.1.7.tar.gz tar zxvf ruby-2.1.7.tar.gzcd ruby-2.1.7./configure-prefix=/usr/local/ruby make make install cp ruby / usr/local/bin

3) install rubygem:version (1.8.5) # rubygems-1.8.5.tgz tar zxvf rubygems-1.8.5.tgzcd rubygems-1.8.5ruby setup.rb cp bin/gem / usr/local/bin4) install gem-redis:version (3.0.5)

Install the interfaces of redis and ruby necessary to run the redis cluster.

Gem install redis-version 3.0.5

# due to some reasons, the gem source cannot be accessed or the download may fail. Here are two solutions:

Method 1: download and install manually

# download address: http://rubygems.org/gems/redis/versions/3.0.0

Gem install-l / soft/redis-3.0.5.gem

Method 2: replace the gem source

Gem sources-- remove https://rubygems.org/gem sources-a http://ruby.sdutlinux.org/gem sources-lumped cards * CURRENT SOURCES * https://ruby.taobao.org

Commonly used sources

Http://rubygems.org/

Http://gems.github.com

Http://gems.rubyforge.org

Http://ruby.sdutlinux.org/

Https://ruby.taobao.org domestic should find a more reliable, suitable for the installation of most common gem

5) install redistar xzf redis-3.0.5.tar.gzcp-r redis-3.0.5/ opt/app/ln-s / opt/app/redis-3.0.5/ / opt/rediscd / opt/redismake testmakemake install

Description:

After the make install command is executed, the executable files, redis-server, redis-cli, redis-benchmark, redis-check-aof, and redis-check-dump, are generated in the / usr/local/bin directory. Their functions are as follows:

Daemon launcher for redis-server:Redis server

Redis-cli:Redis command line operation tool. You can also use telnet to operate according to its plain text protocol

Redis-benchmark:Redis performance testing tool to test the read and write performance of Redis on the current system

Redis-check-aof: data repair

Redis-check-dump: check the export utility

Configure redis cluster:

1) configure the cluster initialization script:

Cd / opt/redis cp / opt/redis/src/redis-trib.rb / usr/local/bin

2) configure redis cluster profile

Daemonize yes# runs as a background process redis. Pidfile / opt/redis/run/redis_6379.pid# if you run Reids as a background process, you need to specify the pid file and path. Port 637 specify the redis listening port. Tcp-backlog 51clients in a highly concurrent environment, in order to avoid slow client connection problems. Bind 0.0.0.bind host IP. (set to 4 zeros here to facilitate program calls). The timeout of a timeout client connection (in seconds). Tcp-keepalive 6: on Linux, specify a value (seconds) for the time it takes to send ACKs. Note that it takes twice as long to close the connection. The default is 0loglevel notice# logging level, and there are 4 optional values, debug,verbose (default), notice,warninglogfile "/ var/log/redis/redis_6379.log" # log file address databases "available database number save 900 1save 300 10save 60 1000" saves data to disk based on a given time interval and number of writes, in seconds stop-writes-on-bgsave-error yes# background storage error stops writing. Whether rdbcompression yes# compresses data when it is stored in a local database (persisted to a dump.rdb file). The default is whether yesrdbchecksum yes# validates rdb files. Dbfilename dump_6379.rdb# local persistent database file name, default is the path where files are placed in dump.rdbdir / opt/redis/data# database mirrored backup. # slaveof # enable this parameter when setting this database as a slave database for other databases. # masterauth # password of slave service connection master slave-serve-stale-data yesslave-read-only yes#slave read-only repl-diskless-sync norepl-diskless-sync-delay 5repl-disable-tcp-nodelay noslave-priority 100#requirepass foobared# setting client connection password appendonly yes# Open aof persistent appendfilename "appendonly_6379.aof" # aof filename The default is appendonly.aofappendfsync everysec# once per second aof write no-appendfsync-on-rewrite yes# off fsync auto-aof-rewrite-percentage 10 write operations for new writes on aof rewrite and redis instances deployed on the same machine Rub the auto-aof-rewrite apart because the memory footprint in the cluster environment is basically the same. # prevent all redis processes from doing aof rewrite in an instant fork on the same machine Takes up a lot of memory (ps:cluster must turn on aof) auto-aof-rewrite-min-size 64mbaof-load-truncated yeslua-time-limit 5000cluster-enabled yes# opens redis cluster cluster-config-file nodes-6379.conf# cluster node configuration file (starts automatic generation) cluster-node-timeout 1500 node interconnect timeout threshold cluster-migration-barrier 1slowlog-log-slower-than 10000slowlog-max-len 128latency-monitor-threshold 0notify-keyspace-events "" hash-max -ziplist-entries 512hash-max-ziplist-value 64list-max-ziplist-entries 512list-max-ziplist-value 64set-max-intset-entries 512zset-max-ziplist-entries 128zset-max-ziplist-value 64hll-sparse-max-bytes 3000activerehashing yesclient-output-buffer-limit normal 00 0client-output-buffer-limit slave 256mb 64mb 60client-output-buffer-limit pubsub 32mb 8mb 60hz 10aof-rewrite-incremental-fsync yes

Note: other nodes only need to copy the configuration file to change all port numbers.

Redis cluster operation and maintenance operation 1. Start the 6 redis instance nodes 192.1.168.1.193cd / opt/redisredis-server redis_6379.confredis-server redis_6479.confredis-server redis_6579.conf192.1.168.1.193cd / opt/redisredis-server redis_6379.confredis-server redis_6479.confredis-server redis_6579.conf respectively.

After startup, use the netstat-lntp command to see if the port is not listening, as shown in the figure, the startup is normal.

2. Use your own redis-trib.rb to build a cluster redis-trib.rb create-- replicas 1 192.168.1.1931 replicas 6379 192.168.1.193 replicas 6479192.168.1.1937 6579 192.168.1.176 replicas 6379 192.168.1.193 6479192.168.1.193

Note:

# build the create subcommand of redis-trib.rb

#-replicas specifies how many Master nodes are assigned to each Slave node in Redis Cluster

# Node roles are determined by sequence, with master followed by slave (for ease of identification, the port of slave is 1000 larger than that of master)

3. Check the cluster status

The checksub-command # ip:port of # redis-trib.rb can be any node of the cluster redis-trib.rb check 192.168.1.193 redis-trib.rb check 6379

Finally, the following information is output without any warnings or errors (as shown in the figure), indicating that the cluster started successfully and is in an ok state.

4. Add a new master node

Create an empty node (empty node) and move some hash slots to the empty node.

A), create a new node profile:

In order to make it easier to distinguish, start another virtual machine here and add a pair of nodes:

192.168.1.187:6379192.168.1.187:6479cd / opt/redisscp redis_6379.conf 192.168.1.187:/opt/redis/cp redis_6379.conf redis_6479.confsed-ie ie 6379 Universe 6479 Universe g redis_6479.conf

B) start the new node:

Redis-server redis_6379.conf

C) add new nodes to the cluster:

Redis-trib.rb add-node 192.168.1.187 ip:port add-node adds a node to the cluster, the first is the new node ip:port, and the second is any existing node ip:port

Note: when adding a new node, the new node cannot contain any data, otherwise the addition will fail.

Because it does not contain any hash slots. The newly added add point is a master node, which will not be selected when the cluster needs to upgrade a slave node to a new master node, and the new master node will not participate in the election and failover because it does not contain any hash slots.

D), manually add hash slots for the new node:

Redis-trib.rb reshard 192.168.1.187redis-trib.rb reshard 6379 # Select the number of hash slots to migrate How many slots do you want to move (from 1 to 16384)? 1000

# Select the node-id What is the receiving node ID? 36c46361327dbb15d098a0c3794ac3d72869e508 to accept these hash slots

# choose the source of the hash slot: # all means to reassign from all master, # or the master node id of the data to extract the hash slot, and finally end the Please enter all the source node IDs with done. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs. Source node # 1:all

# after printing the moved hash slot, enter yes to start moving the hash slot and the corresponding data. # Do you want to proceed with the proposed reshard plan (yes/no)? Yes

# end

You can use the command to view the allocation of comparative hash slots (see figure)

Redis-trib.rb check 192.168.1.176:6379

At this point, a new master node has been added. Execute the command to check the status of the nodes in the current cluster.

Redis-trib.rb check 192.168.1.176:6379redis-cli-c-p 6379 cluster nodes5, add a new slave node

A): the first three steps are the same as adding master

Note: the newly added node cluster is assigned to master by default but the node does not have any slots assigned (see figure)

B) step 4: redis-cli connects to the new node shell, and enter the command: cluster replicate corresponds to the node-id of master (here we enter the id of node 192.168.1.187).

Redis-cli-h 192.168.1.187-p 6479cluster replicate 36c46361327dbb15d098a0c3794ac3d72869e508 exit

You can also check the cluster status to determine whether the addition is successful.

Note: when adding slave online, you need to bgsave the entire master data and pass it to slave, and then the slave loads the rdb file into memory. A large amount of Master memory and network IO are consumed in the process of rdb generation and transfer. It is not recommended that the memory of a single instance is too large. Be careful online.

5. Delete a slave node method 1: # redis-trib del-node ip:port''redis-trib.rb del-node 192.168.1.187 redis-trib.rb del-node 6479 4655fccff00ef4a7b99c10ffd590c8328ec6db8d

Law II:

You can Redis-cli-p 6479 shutdownorkill-9 `cat / opt/redis/run/redis_ 6479.pid`6 and delete a master node by directly stopping or kill dropping the node

A): before deleting a master node, first use reshard to remove all slot of the master, and then delete the current node

# migrate the current master to 192.168.1.176 redis-trib.rb reshard 6579 on 192.168.1.176 to 6579, select the number of hash slots to be migrated according to the prompt How many slots do you want to move (from 1 to 16384)? 1000 (number of all hash slots of deleted master) # Select the 192.168.1.176:6579What is the receiving node ID to accept these hash slots? E1c06dd4682a37eb6773d6cb1d5709034a3f2769 (ps: 192.168.1.176 node-id 6579) Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs. Source node # 36c46361327dbb15d098a0c3794ac3d72869e508 (node-id of deleted master) Source node # 2:done

# after printing the moved hash slot, enter yes to start moving the hash slot and the corresponding data. # Do you want to proceed with the proposed reshard plan (yes/no)? Yes

B): delete empty master node

Redis-trib.rb del-node 192.168.1.187purl 6379 '36c46361327dbb15d098a0c3794ac3d72869e508'

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.