Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to use the Redis command". In daily operation, I believe many people have doubts about how to use the Redis command. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "how to use the Redis command". Next, please follow the editor to study!

Cause of the problem

Ah Fan is responsible for the application is a management background application, rights management uses the Shiro framework, because there are multiple nodes, need to use distributed Session, so here use Redis to store Session information.

Since Shiro does not provide Redis storage Session components directly, A Fan has to use shiro-redis, an open source component of Github.

Because the Shiro framework needs to periodically verify that the Session is valid, the underlying Shiro will call SessionDAO#getActiveSessions to get all the Session information.

Shiro-redis happens to inherit the interface SessionDAO, and the underlying layer uses the keys command to find all the stored Session key of Redis.

Public Set keys (byte [] pattern) {

CheckAndInit ()

Set keys = null

Jedis jedis = jedisPool.getResource ()

Try {

Keys = jedis.keys (pattern)

} finally {

Jedis.close ()

}

Return keys

}

Find the cause of the problem and the solution is relatively simple. Find the solution on github and upgrade shiro-redis to the latest version.

In this version, shiro-redis fixes this problem by using the scan command instead of keys.

Public Set keys (byte [] pattern) {

Set keys = null

Jedis jedis = jedisPool.getResource ()

Try {

Keys = new HashSet ()

ScanParams params = new ScanParams ()

Params.count (count)

Params.match (pattern)

Byte [] cursor = ScanParams.SCAN_POINTER_START_BINARY

ScanResult scanResult

Do {

ScanResult = jedis.scan (cursor,params)

Keys.addAll (scanResult.getResult ())

Cursor = scanResult.getCursorAsBytes ()

} while (scanResult.getStringCursor () .compareTo (ScanParams.SCAN_POINTER_START) > 0)

} finally {

Jedis.close ()

}

Return keys

}

Although the problem was solved successfully, Ah Fan was still a little confused.

Why does the keys instruction cause other commands to slow down?

Why is the Keys instruction query so slow?

Why is there nothing wrong with the Scan instruction?

How Redis executes commands

First of all, let's look at the first question, why does the keys instruction cause other commands to slow down?

To answer this question, let's first take a look at how the Redis client executes a command:

From the client's point of view, there are three steps to executing a command:

Send a command, execute the command and return the result.

But this is only what the client thinks it is, but in fact, at the same time, there may be a lot of clients sending commands to Redis, and Redis we all know that it uses a single-threaded model.

In order to process the request commands of all clients at the same time, the queue is used inside the Redis to execute.

So it actually takes four steps for the client to execute a command:

Send a command queue to execute the command to return the result

Because Redis executes commands in a single thread, tasks can only be sequentially fetched from the queue.

As long as the execution of the command in this process is too slow, other tasks in the queue have to wait, which seems to the external client that Redis is blocked and does not get a response.

Therefore, do not use Redis procedures to execute instructions that need to run for a long time, which may cause Redis blocking and affect the execution of other instructions.

KEYS principle

Next, let's answer the second question, why is the Keys instruction query so slow?

Before you answer this question, please recall the underlying storage structure of Redis.

It doesn't matter if you don't know about your friends. You can take a look back at Ah Fan's previous article "Ali interviewer: HashMap is familiar, right? OK, let's talk about Redis dictionary. ".

Here A Fan copies the content of the previous article, and the bottom layer of Redis uses the structure of dictionary, which is similar to that of Java HashMap.

The keys command needs to return all the keys in Redis that match the given pattern pattern. To do this, Redis has to traverse the underlying array of the ht [0] hash table in the dictionary, with a time complexity of "O (N)" (N is the number of key in Redis).

If the number of key in Redis is small, then the execution speed will still be very fast. When the number of Redis key increases gradually, rising to the level of millions, tens of millions, or even hundreds of millions, then the execution speed will be very slow.

The following is a local experiment done by Ah Fan, using the lua script to add 10W key to Redis, and then using keys to query all the keys, this query will probably block for more than ten seconds.

Eval "for iTunes 1100000 do redis.call ('set',i,i+1) end" 0SCAN principle

Finally, let's take a look at the third question, why is there nothing wrong with the scan instruction?

This is because the scan command uses a cool techs-"cursor-based iterator".

Each time the scan command is called, Redis returns a new cursor and a certain number of key to the user. The next time you want to continue to get the remaining key, you need to pass this cursor into the scan command to continue the previous iteration.

In a nutshell, the scan command uses paging to query redis.

The following is an example of an iterative process for the scan command:

The scan command uses cursors to cleverly split a full query into multiple times and reduce the complexity of the query.

Although the time complexity of the scan command is the same as that of keys, it is "O (N)", but because the scan command only needs to return a small amount of key, it can be executed very quickly.

Finally, although the scan command addresses the keys deficiency, it also introduces some other defects:

The same element may be returned multiple times, which requires our application to add the ability to deal with repeating elements. If an element is added to redis during the iteration, or is deleted during the iteration, the element will be returned or may not be.

These defects need to be taken into account in our development.

In addition to scan, redis has several other incremental iteration commands:

Sscan: used to iterate the database keys in the current database to solve the possible blocking problem of smembers. The hscan command is used to iterate the key-value pairs in the hash key to solve the possible blocking problem of hgetall. Zscan: the command is used to iterate over elements in an ordered collection (including element members and element scores) and to generate zrange that may cause blocking problems. At this point, the study on "how to use the Redis command" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.