Shulou(Shulou.com)06/01 Report--

In this article, the editor introduces in detail "how to build a redis cluster quickly with python". The content is detailed, the steps are clear, and the details are handled properly. I hope this article "how to quickly build a redis cluster with python" can help you solve your doubts.

Redis communication protocol

List the main points to facilitate the understanding of the following program.

Redis listens for incoming connections on TCP port 6379 (the default port, which can be modified in configuration), and every Redis command or data transferred between the client and the server ends with rn.

Reply (a protocol that the server can recover from the client)

Redis replies to commands with different reply types. It may check the reply type from the first byte sent by the server:

* reply with a single line (status reply), and the first byte of the reply will be "+"

* error message, the first byte of the reply will be "-"

* integer number, the first byte of the reply will be ":"

* batch reply, the first byte of reply will be "$"

* multiple batch replies, and the first byte of the reply will be "*"

Bulk Strings (batch reply)

The batch reply is used by the server to return a single binary security string.

C: GET mykey

S: $6rnfoobarrn

The server sends the first line of reply, which starts with "$" followed by the actual number of bytes to be sent, followed by CRLF, then sends the actual data, followed by 2 bytes of additional data for the final CRLF. The exact sequence sent by the server is as follows:

"$6rnfoobarrn"

If the requested value does not exist, the batch reply uses a special value of-1 as the data length, for example:

C: GET nonexistingkey

S: $- 1

When the requested object does not exist, the client library API does not return an empty string, but an empty object. For example, the Ruby library returns' nil', and the C library returns NULL (or set the specified flag in the reply object), and so on.

Binary system

To put it simply, binary is included, so when dealing with C language, you can't use str functions, such as strlen, strcpy, etc., because they all judge the end of a string.

Redis cluster is super simple to build redis cluster.

The official website also describes how to build a redis cluster, which is troublesome because using centos6.5, if you use a newer centos, it may be better.

Data fragmentation of Redis Cluster

Redis cluster does not use consistent hash, but introduces the concept of hash slot.

The Redis cluster has 16384 hash slots, and each key passes the CRC16 check and modulates 16384 to decide which slot to place. Each node of the cluster is responsible for part of the hash slot. For example, if the current cluster has three nodes, then:

* Node A contains hash slots 0 to 5500.

* Node B contains hash slots 5501 to 11000.

* Node C contains hash slots 11001 to 16384.

This structure is easy to add or remove nodes. For example, if I want to add a new node D, I need to get some slots from nodes A, B, C to D. If I want to remove node A, I need to move the slots in A to B and C nodes, and then remove A nodes without any slots from the cluster. Since moving hash slots from one node to another does not stop service, no matter adding, deleting or changing the number of hash slots of a node will not cause the cluster to be unavailable.

Client and server in Redis cluster protocol

In a Redis cluster, the node is responsible for storing data and recording the status of the cluster (including the mapping of key values to the correct node). The cluster node can also automatically find other nodes, detect the nodes that are working properly, and select the master node from the slave node when needed.

In order to perform these tasks, all cluster nodes are connected through TCP (TCP bus? To establish communication with a binary protocol (cluster connection, cluster bus) Each node is connected to every other node on the cluster through a cluster connection (cluster bus). Nodes use a gossip protocol to propagate cluster information so that they can discover new nodes, send ping packets (to ensure that all nodes are working properly), and send cluster messages when certain situations occur. Cluster connections are also used to publish or subscribe messages in the cluster.

Because cluster nodes cannot proxy (proxy) requests, clients redirect commands to other nodes when they receive redirect errors (redirections errors)-MOVED and-ASK. In theory, the client is free to send requests to all nodes in the cluster and redirect the requests to other nodes when needed, so the client does not need to save the cluster state. However, the client can cache the mapping between key values and nodes, which can significantly improve the efficiency of command execution.

-MOVED

To put it simply, in the case of returning-MOVED, the client connects to node A to request to process key, but in fact, key is actually in node B, so it returns-MOVED. The protocol is as follows:-MOVED 3999 127.0.0.1 key 6381

Don't worry about-ASK.

Implementation of redis client in C language

The code is as follows:

# include # include ssize_t sock_write_loop (int fd, const void * vptr, size_t n) {size_t nleft = 0; ssize_t nwritten = 0There Const char * ptr; ptr = (char *) vptr; nleft = n while (nleft > 0) {if ((nwritten = write (fd, ptr, nleft)) 0 & sock_read_wait (fd, timeout)

< 0 )return (-1);elsereturn (read(fd, vptr, len));}int sock_connect_nore(const char *IPaddr , int port , int timeout){ // char temp[4096];int sock_fd = 0, n = 0, errcode = 0;struct sockaddr_in servaddr;if( IPaddr == NULL ) {return -1; }if( (sock_fd = socket(AF_INET, SOCK_STREAM, 0) ) < 0 ) {return -1; } memset(&servaddr, 0, sizeof(servaddr)); servaddr.sin_family = AF_INET; servaddr.sin_port = htons(port);//changed by navy 2003.3.3 for support domain addr//if( (servaddr.sin_addr.s_addr = inet_addr(IPaddr) ) == -1 )if( (errcode = inet_pton(AF_INET, IPaddr, &servaddr.sin_addr) ) h_addrtype != AF_INET && pHost->

H_addrtype! = AF_INET6) {close (sock_fd); return-1;} / / currently only take the first IP address if ((inet_ntop (pHost- > h_addrtype, * (pHost- > h_addr_list), sHostIp, sizeof (sHostIp)) = = NULL) {close (sock_fd); return-1 } if ((errcode = inet_pton (AF_INET, sHostIp, & servaddr.sin_addr)) h_addrtype! = AF_INET6) {close (sock_fd); return-1 } / / currently only take the first IP address if ((inet_ntop (pHost- > h_addrtype, * (pHost- > h_addr_list), sHostIp, sizeof (sHostIp)) = = NULL) {close (sock_fd); return-1;} if ((errcode = inet_pton (AF_INET, sHostIp, & servaddr.sin_addr))

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.