Shulou(Shulou.com)06/01 Report--

This article mainly introduces what the redis protocol refers to, the article is very detailed, has a certain reference value, interested friends must read it!

The Redis client interacts with the Redis server using a protocol called RESP (REdis Serialization Protocol, redis Serialization Protocol). Although this protocol is designed for Redis, it can also be used in software systems of other client-server architectures. (translation Note: from some public sources, Momo's IM protocol design refers to the Redis protocol.)

RESP weighs the following aspects:

The realization should be simple and parsing should be fast and convenient for people to read.

RESP can serialize different data types, such as integers, strings, arrays, and design special types for errors. The client sends it to the Redis server for execution in the form of a request for an array of string parameters, and Redis returns the data type associated with the command.

RESP is binary secure (binary-safe) and does not need to parse bulk data sent by one process to another because it uses the length prefix to transmit bulk data.

Note: the protocol mentioned here is only used for client-server communication. Redis Cluster uses different binary protocols for message exchange between node.

Network layer

The client communicates with Redis by establishing a TCP connection with port 6379.

Although RESP is not technically TCP-related, for Redis the protocol is only used for TCP (or other streaming protocols such as Unix domain protocols). Memcached, on the other hand, supports both tcp and udp, but in fact the production environment basically only uses tcp. I think this is overdesigned and may be used by hackers to do memcached udp reflection attacks.

Request response model

Redis receives commands made up of different parameters. When the command is received, it is processed and the response is sent to the client.

This is the simplest model, with two exceptions:

Redis supports pipelining (mentioned later). So the client can send more than one command at a time and then wait for a response. When the client subscribes to a Pub/Sub channel, the protocol changes its meaning to a push protocol, that is, the client does not have to send a command, because the server automatically sends a new message to the client after receiving the message (to the channel subscribed to by the client).

Apart from these two points, the Redis protocol is a simple request-response protocol.

RESP protocol description

The RESP protocol was introduced in Redis 1.2, but it is now the standard interaction protocol for Redis 2.0. You should use this protocol when implementing Redis clients.

RESP is actually a serialization protocol that supports the following types: Simple Strings, Errors, Integers, Bulk Strings, and Arrays.

As a request response protocol, RESP is used in Redis in the following ways:

The client sends commands to the Redis server in the form of an RESP Bulk Strings array. The server returns the corresponding RESP implementation according to different command implementations.

In RESP, some data types are determined by the first byte:

The first byte of the SImple Strings response is "+" for the Errors response is "-" for the Integers response the first byte is ":" for the Bulk Strings response the first byte is "$" for the Arrays response the first byte is "*"

In addition, RESP can use a special Bulk String or array to represent Null values, which will be mentioned later.

In RESP, different parts of the protocol are always separated by "\ r\ n" (CRLF).

RESP Simple Strings

Simple Strings is encoded by a plus sign followed by a string that does not contain CR or LF characters (no line breaks) and ends with CRLF ("\ r\ n").

SImple Strings transmits non-binary secure strings with minimal cost. For example, many Redis commands respond to "OK" when successful, which is 5 bytes encoded in RESP Simple String:

"+ OK\ r\ n"

To transfer binary-safe strings, use RESP Bulk Strings.

When Redis responds to a Simple String, the client library should return the string from the first "+" character to the end of the string to the caller, excluding CRLF bytes.

RESP Errors

RESP has a special data type for error. Error is actually like RESP Simple String, but the first string is a "-" instead of a plus sign. The real difference between Simple Strings and Errors in RESP is that errors is treated as an exception by the client, and the string that makes up the Error type is the string itself. The basic format is:

"- Error message\ r\ n"

Error responses are sent only when an error occurs, such as when you manipulate the wrong data type, or the command does not exist. When an Error response is received, the client should throw an exception.

The following is an example of an error response:

-ERR unknown command 'foobar'-WRONGTYPE Operation against a key holding the wrong kind of value

The first word between "-" and the first space or new line indicates the type of error returned. This is just a convention of Redis itself, not the format specified by RESP Error.

For example, ERR is a generic error, while WRONGTYPE is a more specific error that indicates that the client is trying to manipulate the wrong data type. This is called Error Prefix (Error prefix), from which the client can learn that the server returns the wrong type without relying on the exact message description, which can change at any time.

A client implementation may return different types of exceptions to different error, or return a string that represents an error to the caller. However, this feature is not necessary because it is of little use, and some streamlined client implementations may simply return a general error condition, such as false.

RESP Integers

This type is a string that represents an integer that ends in CRLF and begins with a ":" byte. For example, ": 0\ r\ n" or ": 1000\ r\ n" are integer responses.

Many Redis commands return RESP Integers, such as INCR, LLEN, and LASTSAVE.

The returned integer has no special meaning, it is the added number of INCR, the UNIX timestamp of LASTSAVE, and so on. But the returned integer can be guaranteed to be within the range of 64-bit signed integers.

Integer responses are also widely used to represent true or false. Commands such as EXISTS and SISMEMBER return 1 for true and 0 for false.

The following command returns an integer: SETNX, DEL, EXISTS, INCR, INCRBY, DECR, DECRBY, DBSIZE, LASTSAVE, RENAMENX, MOVE, LLEN, SADD, SREM, SISMEMBER, SCARD.

RESP Bulk Strings

Bulk Strings is used to represent a binary-safe string with a maximum length of 512m.

The encoding format of Bulk Strings is as follows:

"$" is followed by the number of string bytes (prefix length), ending with the actual string CRLF that ends with CRLF

So the string "foobar" is encoded as:

"$6\ r\ nfoobar\ r\ n"

Empty string:

"$0\ r\ n\ r\ n"

RESP Bulk String can also represent values that do not exist in a special format that represents null values. The length value of this special format is-1, and there is no data, so Null is expressed as:

"$- 1\ r\ n"

This is called Null Bulk String.

When the server returns Null Bulk String, the client API should not return an empty string, but a nil object. For example, the Ruby library should return 'nil', while the C library should return NULL (or set a special tag on the returned object), and so on.

RESP Arrays

The client sends commands to the Redis server using RESP Arrays. Similarly, some Redis commands use RESP Arrays as the return type when they want to return a collection of elements. An example is the LRANGE command that returns a list of elements.

RESP Arrays is sent using the following format:

"*" is the first byte, followed by the number of elements in the array, and then CRLF. Then there are the elements represented by each RESP type in the array.

For example, an empty array is represented as:

"* 0\ r\ n"

The array with two RESP Bulk Strings "foo" and "bar" is encoded as follows:

"* 2\ r\ nroom3\ r\ nfoo\ r\ nroom3\ r\ nbar\ r\ n"

As you can see, after the * CRLF in front of the array, the other data types in the array are concatenated one after another. For example, an array with three integers is encoded as follows:

"* 3\ r\ nVO1\ r\ nVOL2\ r\ nR3\ r\ n"

An array can contain mixed types, and it does not require all elements to be of the same type. For example, an array with four interges and one bulk string can be encoded as:

* 5\ r\ n 1\ r\ nVO2\ r\ nVOL3\ r\ nVOL4\ r\ nPROG6\ r\ nfoobar\ r\ n

The response is divided into multiple lines for clarity.

The first line * 5\ r\ n sent by the server indicates that it is followed by five responses, and then the response of each representative element is sent.

The concept of Null array also exists, which is an alternative to null values (Null Bulk String is usually used, but we have two formats for historical reasons).

For example, when the BLPOP command times out, it returns a Null array of length-1, as follows:

"*-1\ r\ n"

When the server returns a Null array, the client library API should return a null object instead of an empty array. It is necessary to distinguish between returning an empty list and other situations, such as when the BLPOP command times out.

RESP allows an array of arrays. For example, an array with two arrays is encoded as follows:

Efficient parsing of Redis protocol

Although the Redis protocol is very readable and easy to implement, it can be as efficient as binary protocols.

RESP uses length prefixes to transmit bulk data, so there is no need to scan for special symbols in the data payload like JSON, or enclose the data payload in quotation marks.

Bulk and Multi Bulk length processing can process one character at a time, while scanning CR characters, such as the following C code:

# include int main (void) {unsigned char * p = "$123\ r\ n"; int len = 0; paired r', and the len is in bulk_len; while (* p! ='\ r') {len = (len*10) + (* p -'0'); paired;} / * Now p points at'\ r', and the len is in bulk_len. * / printf ("% d\ n", len); return 0;}

When the first CR is identified, the subsequent LF can be ignored. The bulk data can then be read at once without the need to analyze the data load. The last remaining CR and LF strings can be discarded and not processed.

When compared with the binary protocol, the Redis protocol is simple enough to be implemented in most high-level languages to reduce the number of bug of client software.

Note:

1. The CR and LF in the protocol are equivalent to delimiters, and the existence of multiple CRLF between commands should not affect subsequent parsing, because multiple CRLF should be ignored. For example:

$> (printf "PING\ r\ nPING\ r\ nPING\ r\ n\ r\ n\ n\ rPING\ r\ n"; sleep 1) | nc localhost 6379+PONG+PONG+PONG+PONG

two。 Compared with memcached's protocol, redis's protocol is indeed designed to be more concise:

(1) consistent form of request. Redis requests are sent to the Bluk String array, different commands only have different number of elements of the array, all commands can first read the entire array and then parse the parameters of the array according to different commands; unlike the mc protocol, different requests have different command formats, so different commands have to be processed differently in the process of reading network byte stream, increasing the difficulty of protocol parsing.

(2) the length prefix is the key to efficiently parsing the protocol. Field length information is not exclusive to binary protocols, and text protocols can also be available.

The above is what the redis agreement refers to all the content, thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.