Shulou(Shulou.com)06/01 Report--

Preface

There is a classic problem in Redis. In the case of a huge amount of data, doing something similar to finding Key information that conforms to certain rules, there are two ways.

First, the keys command is simple and rough. Because of the single thread of Redis, the keys command is executed in a blocking way, and the keys is implemented in a traversal way with a complexity of O (n). The more key in the Redis library, the greater the cost of finding and implementing, and the longer the blocking time.

The second is the scan command, which can find the key value in a non-blocking way. In most cases, it can replace the keys command, and it is more optional.

The following writes 100000 pieces of test data in key***:value*** format (ps: with pipline, one stroke per stroke, each stroke is completed in seconds)

#-*-coding: utf-8-*-#! / usr/bin/env python3import redisimport sysimport datetimedef create_testdata (): r = redis.StrictRedis (host='***.***', port=****, db=0, password='root') counter = 0 with r.pipeline (transaction=False) as p: for i in range (0, 100000): p.set ('key' + str (I)) "value" + str (I) counter = counter + 1 if (counter = = 10000): p.execute () counter = 0 print ("set by pipline loop") if _ _ name__ = = "_ main__": create_testdata ()

For example, what are the key at the beginning of key111?

If you use the keys command, execute keys key1111*, to find out all at once.

Similarly, if you use the scan command, use scan 0 match key1111* count 20

The syntax of scan is: SCAN cursor [MATCH pattern] [COUNT count] The default COUNT value is 10.

The SCAN command is a cursor-based iterator. This means that each time the command is called, you need to use the cursor returned by the last call as the cursor parameter of the call, thus continuing the previous iteration process.

Here we use the scan 0 match key1111* count 20 command to complete this query, but it is a bit surprising that there is no query result at first, depending on the principle of the scan command.

When scan traverses key, 0 represents the first time, key1111* represents matching according to the pattern starting with key1111, and 20 in count 20 does not represent the output of qualified key, but limits the number of dictionary slots that the server traverses at a time (approximately equal to).

So, what is the slot data? Is this slot the slot in the Redis cluster? The answer is no. In fact, the above picture has already given the answer.

If the number of "dictionary slots" mentioned above is the number of slot in the cluster, and you know that the number of slot in the cluster is 16384, then after traversing 16384 slots, you must be able to traverse all the key information. It is clear that when the number of dictionary slots traversed is 20000, the cursor still does not finish the traversal result, so this dictionary slot is not equal to the concept of slot in the cluster.

After testing, in scan, how much count value can be completely match to a qualified key, which is related to the number of key of a specific object.

If you use more than the number of key count to scan, you will find all the eligible key at once. For example, when the number of key is 10W, if you traverse 20w dictionary slots at a time, you will definitely get a complete result.

The scan instruction is a series of instructions that traverse a specified collection of containers in addition to traversing all key.

Zscan traverses zset collection elements

Hscan traverses the elements of the hash dictionary,

Sscan iterates through the elements of the set collection.

The first argument to the SSCAN, HSCAN, and ZSCAN commands is always a database key (a specified key).

In addition, when you use redis desktop manager, when you refresh a library, the console automatically refreshes the scan command, so you know what it is doing.

Reference: http://jinguoxing.github.io/redis/2018/09/04/redis-scan/

Summary

The above is the whole content of this article. I hope the content of this article has a certain reference and learning value for everyone's study or work. Thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.