Shulou(Shulou.com)06/02 Report--

Original articles, welcome to reprint. Reprint please indicate: reproduced from IT Story Association, thank you!

Original link address: "actual combat" open source project dockerization operation and maintenance deployment-redis cache (6)

Now ordinary projects will use redis to do cache, but also can not help but have old friends have not used, I will say it together. Source code: https://github.com/limingios/netFuture/tree/master/redis-cluster

# redis

Official website

Https://redis.io/

Redis is an open source (BSD licensed) in-memory data structure store that is used as a database, cache, and message broker. It supports data structures such as strings, hashes, lists, collections, sort sets with range queries, bitmaps, logs, geospatial indexes with radius queries, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of disk persistence, and provides high availability through Redis Sentinel and Redis Cluster automatic partitioning.

History

In 2008, Salvatore Sanfilippo, the founder of Merzia, an Italian startup, customized a database in order to avoid the low performance of MySQL, which was developed in 2009. This is Redis.

In just a few years, the amount of user data has soared. At home, such as Sina Weibo, Jiepang and Zhihu, and abroad, such as GitHub, Blizzard, etc., are Redis users. The largest Redis cache in the world was created by the Sina Weibo team. When it's hot news. Redis can achieve up to 100000 reads and writes per second.

Cache introduction

Cache uses memory to keep data, and the read and write speed is much faster than the hard disk cache, which can reduce IO operations and reduce IO pressure.

WeChat red packet is a good example. When sending red packets, the red packet information is saved in the cache, and the robber also takes it from the cache. Hundreds of millions of people came to rob on the day of the Spring Festival to maintain the stability of the system.

General applications are divided into common use and personalization, which may be obtained from the database. But what is commonly used may be obtained from the cache.

Introduction to Redis Cluster

Redis currently has the following clustering schemes:

RedisCluster: officially recommended, there is no central point (the primary node is not the central node, but the one that holds the most data, the latest one, and the master node disappears after synchronization). Codis: middleware, there is a central node (the central node is dead, completely finished). Twemproxy: a middleware product with a central node.

RedisCluster

There is no central node, and the client is directly connected to the redis node. There is no need for an intermediate proxy layer (very similar to PXC). The data can be stored in fragments (the content stored by each node is different). Later, you can add or remove the schematic diagram of the Redis node built this time.

Master-slave synchronization

As mentioned above, the data of RedisCluster is stored in pieces, and some of the data will be lost if the redis is hung up. In order to avoid this problem, the mechanism of master-slave synchronization must be introduced.

Data replication in a Redis cluster is achieved through master-slave synchronization. The advantage of master node (Master) distributing data to slave node (Slave) master-slave synchronization is high availability, Rredis node has redundant design.

Redis cluster is highly available

The Redis cluster should contain an odd number of Master, at least 3. If one of them is hung up, the remaining odd number can hold at least half of the election. It is easy to select to the master node. Every Master in a Redis cluster should have a Slave. Why doesn't Redis build a load balancer?

Because the front and rear ends separate the project, and when the backend requests, the backend has already done load balancing for the request, so Redis does not need to do load balancing.

Set up cluster application IP address service configuration installation application installation mode docker-mysql192.168.66.101docker-redis-cluster dual-core 8g memory docker-redis-clusterdocker

(1)。 Virtual machine vagrant describes the steps of installation

Vagrant up

(2)。 Machine window/mac is enabled for remote login under root users

Su-# password vagrant# set PasswordAuthentication yesvi / etc/ssh/sshd_configsudo systemctl restart sshd

Create a folder and configure mkdir redis-clustercd redis-clustermkdir r1cd r1vi redis.confmkdir datacd ~

Redis.conf

Five places are configured.

Daemonize yes

Run cluster-enabled yes as a background process

Open cluster cluster-config-file 150000

Timeout appendonly yes

Open the AOF mode and save the file as requirepass idig8.com.

Authentication password cluster-config-file nodes.conf

Cluster configuration file directly look at the github I submitted the source code to create a total of 6 redis cluster creation containers (R1)

Want to add security verification, but does not take effect, check daemonize yes, his role is to enable daemon mode, in this mode, redis will run in the background, and the process pid number will be written to the file set by redis.conf option pidfile, and redis will always run unless the process is manually kill. So go into this container and manually select load configuration file.

Docker run-it-d\-v / root/redis-cluster/r1/redis.conf:/etc/redis/redis.conf\-- name R1-p 5001root/redis-cluster/r1/redis.conf:/etc/redis/redis.conf 6379\-- net=net2\-- ip 172.19.0.2\ zhugeaming1314/redis bash configuration launches docker exec-it R1 bashcd / usr/redis/src./redis-server / etc/redis/redis.conf

Create container (R2) docker run-it-d\-v / root/redis-cluster/r2/redis.conf:/etc/redis/redis.conf\-- name R2-p 5002root/redis-cluster/r2/redis.conf:/etc/redis/redis.conf 6379\-- net=net2\-- ip 172.19.0.3\ zhugeaming1314/redis bash configuration launch docker exec-it R2 bashcd / usr/redis/src./redis-server / etc/redis/redis.conf

Create container (R3) docker run-it-d\-v / root/redis-cluster/r3/redis.conf:/etc/redis/redis.conf\-- name R3-p 5003root/redis-cluster/r3/redis.conf:/etc/redis/redis.conf 6379\-- net=net2\-- ip 172.19.0.4\ zhugeaming1314/redis bash configuration launch docker exec-it R3 bashcd / usr/redis/src./redis-server / etc/redis/redis.conf

Create container (R4) docker run-it-d\-v / root/redis-cluster/r4/redis.conf:/etc/redis/redis.conf\-- name R4-p 5004root/redis-cluster/r4/redis.conf:/etc/redis/redis.conf 6379\-- net=net2\-- ip 172.19.0.5\ zhugeaming1314/redis bash

Configuration startup

Docker exec-it R4 bashcd / usr/redis/src./redis-server / etc/redis/redis.conf

Create container (R5) docker run-it-d\-v / root/redis-cluster/r5/redis.conf:/etc/redis/redis.conf\-- name R5-p 5005 root/redis-cluster/r5/redis.conf:/etc/redis/redis.conf 6379\-- net=net2\-- ip 172.19.0.6\ zhugeaming1314/redis bash

Configuration startup

Docker exec-it R5 bashcd / usr/redis/src./redis-server / etc/redis/redis.conf

Create container (R6) docker run-it-d\-v / root/redis-cluster/r6/redis.conf:/etc/redis/redis.conf\-- name R6-p 5006 root/redis-cluster/r6/redis.conf:/etc/redis/redis.conf 6379\-- net=net2\-- ip 172.19.0.7\ zhugeaming1314/redis bash configuration launch docker exec-it R6 bashcd / usr/redis/src./redis-server / etc/redis/redis.conf

Redis-trib.rb

There is a cluster tool redis-trib.rb in redis, which requires a lot of instructions to operate redis-trib. It is recommended to use the mirror image provided by me, everything is installed in it, just operate according to my command.

Create a cluster command: docker exec-it R1 bashcd / usr/redismkdir clustercd srccp redis-trib.rb.. / clustercd.. / cluster./redis-trib.rb create.. / cluster./redis-trib.rb create-- replicas 1 172.19.0.2purl 6379 172.19.0.3 docker exec 6379 172.19.0.4 purl 6379 172.19.0.5 purl 6379 172.19.0.6

View cluster information docker exec-it R1 bash/usr/redis/src/redis-cli-ccluster nodes

! / upload-images.jianshu.io/upload_images/11223715-0123418e8224c25c.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

Error reported when installing the cluster

Can't connect to node 172.19.* redis.conf file

Bind 127.0.0.1 comment out requirepass idig8.com comment out redis cluster password setting password setting (recommended)

Method 1: modify all redis.conf files in the Redis cluster to join:

Masterauth idig8.comrequirepass idig8.com

Description: this way requires restarting each node

Method 2: enter each instance to set it:

. / redis-cli-c-p 6379 config set masterauth idig8.comconfig set requirepass idig8.comconfig rewrite

After that, use. / redis-cli-c-p 6379, respectively. .. Command to set a password for each node.

Note: the password of each node must be the same, or the Redirected will fail. This method is recommended, which writes the password to the redis.conf without restarting.

Use method 2 to change the password,. / redis-trib.rb check 172.19.0.2 Sorry 6379 may be reported to [ERR] Sorry, can't connect to node 172.19.0.2 Sorry 6379, because 6379 of the redis.conf did not find the password configuration.

If you need to use various commands of redis-trib.rb after setting the password

For example, if:. / redis-trib.rb check 127.0.0.1 Sorry 6379, an error will be reported: ERR] Sorry, can't connect to node 127.0.0.1 Sorry 6379

Workaround: vim / usr/local/rvm/gems/ruby-2.3.3/gems/redis-4.0.0/lib/redis/client.rb, then modify passord

Class Client DEFAULTS = {: url = > lambda {ENV ["REDIS_URL"]},: scheme = > "redis",: host = > "127.0.0.1",: port = > 6379,: path = > nil,: timeout = > 5.0,: password = > "idig8.com",: db = > 0,: driver = > nil,: id = > nil,: tcp_keepalive = > 0 : reconnect_attempts = > 1,: inherit_socket = > false}

Note: the client.rb path can be found through the find command: find /-name 'client.rb'

Access to the cluster with password

. / redis-cli-c-p 6379Mura idig8.com

PS: the entire redis cluster has been installed, with 3 master3 salve. If 1 master fails, the corresponding slave will be automatically upgraded to master, and the original master will become slave if restarted. I try to use the official docker image redis to install the whole process. Add the configuration file to the docker run command to start. This method is problematic. There will still be an error when creating the cluster. It is recommended to use my image, so it is more stable. And it comes with redis-trib.rb.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.