Shulou(Shulou.com)06/01 Report--

This article mainly introduces why Redis should avoid big key, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

Avoid big key

Redis executes commands in a single thread, which means that the Redis operation "big key" is at risk of blocking.

Big key usually means that the value stored by Redis is too large. These include:

A single value is too large. Such as 200m String.

Too many collection elements. Such as List, Hash, Set, ZSet, there are hundreds or tens of millions of data.

For example, suppose we have a 200m String key named "foo".

Execute the following command

127.0.0.1 purl 6379 > GET foo

When the result is returned, Redis allocates 200m of memory and performs a copy of the memcpy.

Void _ addReplyProtoToList (client * c, const char * s, size_t len) {. If (len) {/ * Create a new node, make sure it is allocated to at * least PROTO_REPLY_CHUNK_BYTES * / size_t size = len

< PROTO_REPLY_CHUNK_BYTES? PROTO_REPLY_CHUNK_BYTES: len; // 分配内存（例子中为 200m） tail = zmalloc(size + sizeof(clientReplyBlock)); /* take over the allocation's internal fragmentation */ tail->

Size = zmalloc_usable_size (tail)-sizeof (clientReplyBlock); tail- > used = len; / / memory copy memcpy (tail- > buf, s, len); listAddNodeTail (c-> reply, tail); c-> reply_bytes + = tail- > size; closeClientOnOutputBufferLimitReached (c, 1);}}

The output buf of Redis is 16k.

/ / server.h#define PROTO_REPLY_CHUNK_BYTES (16024) / * 16k output buffer * / typedef struct client {. Char buf [proto _ REPLY_CHUNK_BYTES];} client

This means that Redis cannot return response data at a time and needs to register "writable events" to trigger multiple write system calls.

Here are two time-consuming points:

Allocate large memory (it is also possible to free memory, such as the DEL command)

Trigger multiple writable events (frequent execution of system calls, such as write, epoll_wait)

So, how do you find big key?

If there are simple commands in slow log, such as GET, SET, DEL, there is a good chance that big key will appear.

127.0.1 SLOWLOG GET 6379 > GET 3) (integer) 201323 / / Unit subtle 4) 1) "GET" 2) "foo"

Second, you can find big key through the Redis analysis tool.

$redis-cli-- bigkeys-I 0.1... [00.00%] Biggest string found so far'"foo" 'with 209715200 bytes- summary-Sampled 1 keys in the keyspaceful Total key length in bytes is 3 (avg len 3.00) Biggest string found' "foo" 'has 209715200 bytes1 strings with 209715200 bytes (100.0000% of keys, avg size 209715200.00) 0 lists with 0 items (00.0000% of keys, avg size 0.00) 0 hashs with 0 fields (00.0000% of keys) Avg size 0.00) 0 streams with 0 entries (00.005% of keys, avg size 0.00) 0 sets with 0 members (00.005% of keys, avg size 0.00) 0zsets with 0 members (00.0000% of keys, avg size 0.00)

For big key, here are some suggestions:

1. Try to avoid big key in the business. When big key appears, you have to judge whether the design is reasonable, or whether there is bug.

two。 Split the big key into several small key.

3. Use alternate commands.

If the Redis version is greater than 4. 0, you can use the UNLINK command instead of DEL. If the Redis version is greater than 6. 0, the lazy-free mechanism can be enabled. The memory operation is freed and executed by the background thread.

LRANGE, HGETALL, etc., are replaced by LSCAN and HSCAN.

But I still recommend avoiding big key in your business.

Thank you for reading this article carefully. I hope the article "Why Redis should avoid big key" shared by the editor will be helpful to everyone. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.