Shulou(Shulou.com)05/31 Report--

In this issue, the editor will bring you about how to use Scan commands in Redis. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

1. Overview

The SCAN command and the similar SSCAN, HSCAN, and ZSCAN commands are used to incrementally iterate over dataset elements:

The SCAN command iterates over the database keys in the current database.

The SSCAN command iterates over the elements in the collection (Set).

The HSCAN command is used to iterate over the fields in the Hash and the corresponding values.

The ZSCAN command is used to iterate over the elements in an ordered set (Sorted Set) and the corresponding scores.

Because these commands can be iterated incrementally and only a small number of elements are returned for each call, these commands can be used in a production environment without having to worry about problems like using KEYS and SMEMBERS commands. Invoking these commands on the big data set of keys or elements can block the server for a long time (or even seconds). Blocking commands such as SMEMBERS can provide all the elements in the dataset at a given time, but the SCAN series commands provide only limited guarantees for the elements returned, because the dataset may change as we iterate incrementally.

The SCAN,SSCAN,HSCAN and ZSCAN commands work very similarly, so this article will cover these four commands. The difference lies in the SSCAN,HSCAN and ZSCAN commands, and the first parameter is the name of the key that holds the Set,Hash or Sorted Set value. The SCAN command does not require any key name arguments because it iterates over all the keys in the current database, so the object of the iteration is the database itself.

two。 Basic usage

SCAN is a cursor-based iterator. This means that each time the command is called, the server returns an updated new cursor that the user needs to use as the cursor parameter of the SCAN command in the next call. When the cursor parameter of the SCAN command is set to 0, the server starts a new iteration and terminates the iteration when the server returns a new cursor of 0 to the user. The following is an example of a SCAN iteration:

Redis 127.0.0.1 key:4 6379 > scan 01) "17" 2) 1) "key:12" 2) "key:8" 3) "key:4" 4) "key:14" 5) "key:16" 6) "key:17" 7) "key:15" 8) "key:10" 9) "key:3" 10) "key:7" 11) "key:1" redis 127.0.0.16379 > scan 171) "0" 2) 1) "key:5" 2) "key:18" 3) "key:0" 4) "key:2" 5) "key:19" 6) "key:13" 7) "key:6" 8) "key:9" 9) "key:11"

In the above example, the first call uses 0 as the cursor to start a new iteration. The second call uses the cursor returned by the previous call, the first element value returned by the command, that is, 17. As you can see from the above example, the SCAN command returns an array of two values: the first value is the new cursor to be used in the next call, and the second value is the array containing the returned elements.

Since the cursor returned in the second call is 0, the server sends a signal to the caller informing the caller that the iteration is complete and traversing the dataset. Iterate from the cursor value 0, and then call SCAN until the returned cursor is 0 again, indicating a full iteration.

3. Guarantee

The SCAN command, as well as other incremental iteration commands, provides a series of guarantees for the user throughout the complete iteration:

All elements are traversed from the beginning of the full iteration to the end of the full iteration; this means that if a given element is in the dataset at the beginning of the iteration and still exists at the end of the iteration, the SCAN is returned to the user at a certain iteration.

Elements that do not exist between the beginning of the full iteration and the end of the full iteration are never returned; therefore, if an element is deleted before the iteration begins and is never added back to the dataset during subsequent iterations, SCAN will never return that element.

However, because SCAN has only a few associated states (only cursors), it has the following disadvantages:

The same element may be returned multiple times. The problem of repeating elements needs to be dealt with by our own application. For example, consider using the elements returned by the iteration for idempotent operations (which can be repeated multiple times).

If an element is added to the dataset or removed from the dataset during the iteration, the element may or may not be returned.

4. Number of returns per execution

The functions of the SCAN family do not guarantee that the number of elements returned by each call will be within a given range. Each call may return 0 elements, but as long as the returned cursor is not 0, the client assumes that the iteration is not over (even if 0 elements are returned, it cannot indicate the end of the iteration). The number of elements returned will conform to certain rules:

When iterating over a large dataset, SCAN may return up to dozens of elements.

When iterating over a small dataset and encoding data structures inside (small Set, Hashe, and Sorted Set), all elements of the dataset can be returned in a single call.

However, the user can use the COUNT parameter to adjust the order of magnitude of the elements returned by each call.

5. COUNT parameters

Although SCAN cannot guarantee the number of elements returned per iteration, you can use the COUNT parameter to adjust based on experience. Basically, the function of the COUNT parameter is to let the user tell the iteration command how many elements should be returned from the dataset in each iteration. Although the COUNT parameter is only a hint on the implementation of the iterative command, in most cases, this prompt meets our expectations:

The COUNT default value is 10.

When iterating over a sufficiently large database, Set, Hash, or Sorted Set implemented by a hash table, if the user does not use the MATCH parameter, each call returns COUNT elements, or slightly more elements than COUNT.

When iterating over a Set encoded as IntSet (a small dataset made up of only integer values) or Hash, and a Sorted Set encoded as ZipList (a small Hash or Set made up of different values), the value specified by the COUNT parameter is usually ignored and all elements contained in the dataset are returned to the user on the first call.

It is not necessary to use the same COUNT value for each iteration. Users can change the COUNT value according to their needs in each iteration, as long as remember to use the cursor returned in the last iteration to the next iteration.

6. MATCH parameters

We can also iterate over elements by matching a Glob-style pattern, similar to the KEYS command. We just need to append the MATCH parameter to the SCAN command.

The following is an example of iterating using the MATCH parameter:

Redis 127.0.0.1 foo foobar feelsgood (integer) 6redis 127.0.0.1 6redis 127.0.0.1 6redis 6379 > sscan myset 0 match Fleet 1) "0" 2) 1) "foo" 2) "feelsgood" 3) "foobar" redis 127.0.0.1

We need to note that the MATCH filter is applied after retrieving elements from the dataset and before returning the data to the client. This means that if the pattern matches few elements in the dataset, the SCAN command may not return elements in many iterations. An example is as follows:

Redis 127.0.0.1 MATCH 6379 > key:911 0 MATCH * 1131) 1) "key:911" redis 127.0.0.1 redis 6379 > scan 288 MATCH * 1131) "224" 2) (empty list or set) redis 127.0.1 MATCH * 1131) "80" 2) (empty list or set) redis 127.0.1 MATCH > scan 80 MATCH * 111) (176 "2) Empty list or set) redis 127.0.0.1 MATCH * 11 * COUNT 10001) "0" 2) 1) "key:611" 2) "key:711" 3) "key:118" 4) "key:117" 5) "key:311" 6) "key:112" 7) "key:111" 8) "key:110" 9) "key:113" 10) "key:211" 11) "key:411" 12) "key:115" 13) "key:116" 14) "key:114" 15) "key:119" 16) "key:811" 17) "key:511" 18) "key:11" redis 127.0.0.1

As mentioned above, most calls do not return elements, while the last call uses a COUNT of 1000, forcing the command to scan more of the iteration, resulting in more elements returned by the command.

7. TYPE parameters

Starting with version 6. 0, we can use this parameter to require the SCAN command to return only objects that match a given type, allowing us to traverse the database to find a specific type of key. SCAN can use the TYPE parameter, but things like HSCAN or ZSCAN are not available.

The type parameter is the same as the string name returned by the TYPE command. It is important to note that some Redis types (such as GeoHashes, HyperLogLogs, Bitmap, Bitfields, etc.) are internally implemented using other Redis types (such as String or Zset), so the SCAN command cannot distinguish them from other keys of the same type. For example, ZSET and GEOHASH:

Redis 127.0.0.1 1redis 6379 > GEOADD geokey 00 value (integer) 1redis 127.0.0.1 1redis 6379 > ZADD zkey 1000 value (integer) 1redis 127.0.0.1 value 6379 > TYPE geokeyzsetredis 127.0.0.1 SCAN 6379 > TYPE zkeyzsetredis 127.0.0.1 SCAN 0 TYPE zset1) "0" 2) 1) "geokey" 2) "zkey"

Importantly, the TYPE filter is applied after retrieving elements from the database, so this parameter does not reduce the load required for the server to complete the full iteration, and for rare types, we may not receive any elements.

8. Multiple parallel iterations

Different clients may iterate through the same dataset at the same time, and the client needs to pass in a cursor for each iteration and get a new cursor at the end of the iteration, which contains all the states of the iteration, so the server does not need to record any state for the iteration.

9. Terminate the iteration in the middle

Because the server side does not record the state, and all the states of the iteration are saved in the cursor, the caller is free to terminate the iteration without sending a notification to the server. An infinite number of iterations can be started and never terminated without any issue.

10. Call SCAN with the wrong cursor

Calling SCAN with incorrect, negative, out-of-range cursors or other invalid cursors can result in undefined behavior, but never a crash. What is undefined means that the SCAN will no longer guarantee that the return element is guaranteed.

The only valid cursors are:

The cursor value at the beginning of the iteration is 0.

The cursor returned by SCAN was last called to continue the iteration.

11. Termination guarantee

The SCAN algorithm can be terminated only if the size of the iterative dataset is kept within the given maximum upper limit (constant size); otherwise, iterating over the growing dataset may result in SCAN never terminating the iteration (endless loop).

It is easy to see that if the dataset grows, more and more work will need to be done to access all possible elements, and whether or not to end an iteration depends on the number of calls to SCAN, the value of the COUNT parameter, and the growth rate of the dataset.

twelve。 Return value

Both the SCAN,SSCAN,HSCAN and ZSCAN commands return a reply with two elements, the first of which represents the unsigned 64-bit integer of the cursor, and the second is the iterated array of elements:

An array of SCAN elements is a list of keys.

An array of SSCAN elements is a list of Set members.

The HSCAN element array contains two elements, a field and a value, corresponding to each return element of the Hash.

The ZSCAN element array contains two elements, a member and its associated score, corresponding to each return element in the Sorted Set.

The above is how the Scan command shared by the editor is used in Redis. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.