Shulou(Shulou.com)06/01 Report--

Editor to share with you how to use the redis cluster function in node, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Noun interpretation-- Cluster

Cluster generally refers to the server cluster, which is different from the distributed system, which brings many servers together to carry out the same service, which seems to the client that there is only one server. The cluster can use multiple computers for parallel computing to achieve high computing speed, and it can also use multiple computers for backup, so that any one machine can break down and the whole system can still run normally. (before redis3.0, Sentinel mode is generally used, but the configuration of Sentinel is slightly complex, and performance and high availability are mediocre.)

Redis cluster requirements

Because the voting fault tolerance mechanism requires more than half of the nodes to think that a node is dead, so two nodes cannot form a cluster, so the Redis cluster needs at least three nodes.

To ensure the high availability of the cluster, each node needs to have a slave node (that is, a backup node), so the Redis cluster needs at least six servers. (three masters, three slaves, three deposits and three fetches, high availability, backup)

Of course, we can't use so many servers when debugging locally, so we can simulate and run six redis instances locally. In fact, the Redis cluster in the production environment is basically the same as here.

Set up a local redis cluster under mac environment. Download and install redis

You can choose to install it on the official website or with a named line.

# install brew install redis# and start redis-server# to enter the redis client redis-cli2. Configure the cluster environment through redis

The first step is to find the location of the redis configuration file

Brew list redis # View the location of the redis installation

Cd / opt/homebrew/Cellar/redis/6.2.4 # enter the folder where the version number is located according to the location

Open. # Open a folder

Open homebrew.mxcl.redis.plist with Xcode.app to find the location of redis.conf, as shown below:

Create six service profiles

Cd / opt/homebrew/etc/ (directory of configuration files found in the previous step)

# mkdir-p redis/cluster/7000mkdir-p redis/cluster/7001mkdir-p redis/cluster/7002mkdir-p redis/cluster/7003mkdir-p redis/cluster/7004mkdir-p redis/cluster/7005 is required under / opt/homebrew/etc/ path

Modify the configuration file

The configuration file under the / opt/homebrew/etc/redis.conf path does not need to be modified, just copy it to the redis/cluster/7000 directory created above, and then modify it, as follows

Copy a configuration file to modify it first.

Cd / opt/homebrew/etc/ # enter the configuration file directory cp redis.conf redis/cluster/7000/7000.confcode redis/cluster/7000/7000.conf # open it with an editor or open a configuration file with vim to modify it

After entering the 7000.conf, modify the following properties

# Redis port number (7000-7005 each configuration file needs to be modified) port 7000 # enable cluster mode to run cluster-enabled yes # internal configuration file configuration file path, default nodes-6379.conf (7000-7005 each profile needs to be modified) cluster-config-file nodes-7000.conf # timeout of communication between nodes cluster-node-timeout 5000 # data persistence appendonly yes

Copy 7000.conf to the directory of each redis service

Cd / opt/homebrew/etc/redis/cluster # enter the configuration file directory cp 7000/7000.conf 7001/7001.confcp 7000/7000.conf 7002/7002.confcp 7000/7000.conf 7003/7003.confcp 7000/7000.conf 7004/7004.confcp 7000/7000.conf 7005/7005.conf

Then modify the port and cluster-config-file properties of each configuration file in 7001.conf-7005.conf

Note: each profile must be configured with different port and cluster-config-file values (otherwise the cluster will not take effect), which is distinguished by port.

The directory of the configuration file can be found through the find / opt/homebrew-name nodes-7000.conf command

3. Start and stop the cluster service

Since we have configured six services, it is impossible to start or stop one by one. We need to implement it with the help of shell scripts.

Go to the / opt/homebrew/etc/redis/cluster directory and create the start.sh and stop.sh files

# start.sh file #! / bin/shredis-server / opt/homebrew/etc/redis/cluster/7000/7000.conf & redis-server / opt/homebrew/etc/redis/cluster/7001/7001.conf & redis-server / opt/homebrew/etc/redis/cluster/7002/7002.conf & redis-server / opt/homebrew/etc/redis/cluster/7003/7003.conf & redis-server / opt/homebrew/etc/redis/cluster/7004/7004.conf & redis-server / opt/homebrew/etc / redis/cluster/7005/7005.conf & # stop.sh file #! / bin/shredis-cli-p 7000 shutdown & redis-cli-p 7001 shutdown & redis-cli-p 7002 shutdown & redis-cli-p 7003 shutdown & redis-cli-p 7004 shutdown & redis-cli-p 7005 shutdown &

Execute. / start.sh or. / stop.sh to start and stop the service

Execute ps-ef | grep redis to view the started redis service

Note: the first execution of. / start.sh requires the authorization of sudo chmod + x start.sh to execute.

4. Related command redis-cli-p 7000 # single client startup redis-server 7000/7000.conf # start a single server redis-cli-p 7000 shutdown # close server sudo chmod + x start.sh # enable script execution permissions # set redis master-slave relationship (three masters and three slaves) redis-cli-- cluster create-- cluster-replicas 1 127.0.0.17000 127.0.0.1redis-cli 7001 127.0.0. 1keys 7002 127.0.0.1 127.0.0.1:7005cluster nodes 7003 127.0.0.1 127.0.0.1:7005cluster nodes # View cluster nodes (execute by a client) cluster info # view cluster information (execute by a client) view all key values: keys * Delete the value of the specified index: del key clears the data of the entire Redis server: flushall clears all key:flushdb clients in the current repository access to the cluster using the ioredis framework

Redis.Cluster provides the function of automatic sharding on multiple Redis nodes. Using the six redis servers built above, and then starting node redis.js locally, you can test the effectiveness of the cluster. Ioredis

/ redis.jsconst Redis = require ("ioredis"); const cluster = new Redis.Cluster ([{port: 7000, host: "127.0.0.1",}, {port: 7001, host: "127.0.0.1",},]); cluster.set ("foo", "bar"); cluster.get ("foo", (err, res) = > {/ / res = 'bar'})) Use bull framework (redis queue) import Queue from 'bull'// to create redis queue instance const instance = new Queue (' custom', {prefix:'{myprefix}', createClient (type) {/ / cluster cluster instance as above return cluster}}) / / add data to redis queue (producer) instance.add ('request', {... params}) {removeOnComplete: false}) .catch (e = > {console.error (e)}) / / Consumer callback instance.process ('request', 5, async (job, done) = > {console.log (' get data of current consumption:', job.data) / / perform asynchronous operation await new Promise ((resolve) = > resolve ()) done ()})

There is a problem when using the bull framework to connect to the ioredis cluster: the callback function corresponding to the data push to the redis queue may be triggered multiple times. Currently, it is impossible to determine whether it is the problem of the use or the framework itself (if you know anything, please leave a message).

Alternative to clustering: without the need for data synchronization and data migration, multiple redis instances can be used on the client side, and the data can be evenly distributed to one of the redis with Math.random (), thus solving the bottleneck problem of single instance hardware (cpu, etc.).

Problem handling

1. How to report an error when connecting redis in Mac system?

Console error message: Could not connect to Redis at 127.0.0.1 6379: Connection refused

Reason: server did not open or failed to start

Solution: you need to start redis server redis-server first

Reference link

Https://blog.csdn.net/qq_23347459/article/details/104257529

2. Does the client start and read and write to report errors?

Error prompt: ClusterAllFailedError: Failed to refresh slots cache.

Reason: the cluster-config-file attribute in the configuration file under each service is the same.

Handling: changing to a unique attribute value

Reference link 1

Https://stackoverflow.com/questions/57350961/ioredis-clusterallfailederror-failed-to-refresh-slots-cache

Ref. 2

Https://github.com/luin/ioredis/issues/711

3. Failed to create master-slave redis statement?

Execute the statement: redis-cli-- cluster create-- cluster-replicas 1 127.0.0.1VL 7000 127.0.1JV 7001 127.0.0.1JV 7002 127.0.1JV 7003 127.0.1Rd 7004 127.0.0.1Vue 7005

Error: [ERR] Node 127.0.0.1 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0

Reason: no data emptying and cluster reset when the create statement is executed

Processing: clear data and reset clusters, clear rdb and aof files

Refer to clear redis data

Https://stackoverflow.com/questions/37206993/redis-server-cluster-not-working

# take port 7000 services as an example 7001-7005 repeat the following operations: $redis-cli-p 7000127.0.0.1 exit# 7000 > flushall127.0.0.1:7000 > cluster reset127.0.0.1:7000 > exit# use find to find the rdb and aof files (also in the rdb directory) find / opt/homebrew-name dump.rdb# re-execute the creation statement successfully redis-cli-- cluster create-- cluster-replicas 1 127.0.0.1 exit# 7000 127.0.0.1 exit# 7001 127.0 .0.1 7002 127.0.0.1V 7003 127.0.0.1V 7004 127.0.0.1V 7005 and above are all the contents of this article entitled "how to use redis clustering in node" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.