Shulou(Shulou.com)05/31 Report--

This article mainly introduces the relevant knowledge of how to realize the distributed lock in Redis, the content is detailed and easy to understand, the operation is simple and fast, and it has certain reference value. I believe you will gain something after reading this article on how to realize the distributed lock in Redis. Let's take a look at it.

What is a distributed lock?

When it comes to Redis, the first function that comes to mind is the ability to cache data. in addition, Redis is often used to do distributed locks because of its single process and high performance.

As we all know, the role of locks in programs is a synchronization tool to ensure that shared resources can only be accessed by one thread at a time. We are familiar with locks in Java, such as synchronized and Lock, but Java locks can only be effective on a single machine, and distributed cluster environment is powerless. At this time, we need to use distributed locks.

Distributed locks, as the name implies, are locks used in distributed project development, which can be used to control synchronous access to shared resources between distributed systems. generally speaking, distributed locks need to meet the following characteristics:

1. Mutual exclusion: only one application can acquire a distributed lock for the same piece of data at any time.

2. High availability: in a distributed scenario, the downtime of a small number of servers does not affect normal use. In this case, services providing distributed locks need to be deployed in a cluster manner.

3. Prevent lock timeout: if the client does not release the lock actively, the server will automatically release the lock after a certain period of time to prevent deadlock when the client is down or the network is unreachable.

4. Exclusivity: locking and unlocking must be carried out by the same server, that is, the holder of the lock can release the lock, and the lock you add cannot be unlocked by someone else.

There are many tools in the industry that can achieve distributed locking effect, but there are only a few operations: lock, unlock, and prevent lock timeout.

Since this article is talking about Redis distributed locks, we take it for granted that we will extend it with the knowledge of Redis.

The command to implement the lock

Let's first introduce some commands of Redis.

1. SETNX, usage is SETNX key value

SETNX is an abbreviation for "SET if Not eXists" (or SET if it does not exist). It returns 1 if the setting is successful, or 0 if it is set.

Setnx usage

As you can see, when the value of key to lock is set to "Java", another value will fail, which looks simple and seems to monopolize the lock, but there is a fatal problem, that is, key does not have an expiration time, so that unless you manually delete the key or set the expiration time after acquiring the lock, other threads will never get the lock.

In that case, we can always add an expiration time to the key and let the thread perform two steps when it acquires the lock:

`Expeire Key `SETNX Key 1`EXPIRE

There is also a problem with this scheme, because the acquisition of the lock and setting the expiration time are divided into two steps, not atomic operation, it is possible to acquire the lock successfully but the setting time fails, so it will be in vain.

But don't worry. Redis officials have already considered this kind of thing for us, so it leads to the following order.

2. SETEX, usage SETEX key seconds value

Associate the value value to key and set the lifetime of key to seconds (in seconds). If key already exists, the SETEX command overrides the old value.

This command is similar to the following two commands:

`value``expire key seconds # set time to living`

These two steps are atomic and will be completed at the same time.

Setex usage

3. PSETEX, usage PSETEX key milliseconds value

This command is similar to the SETEX command, but it sets the lifetime of the key in milliseconds, rather than seconds, as the SETEX command does.

However, starting with Redis version 2.6.12, the SET command can achieve the same effect as the SETNX, SETEX, and PSETEX commands through parameters.

Like this order.

`SET key value NX EX seconds`

With the addition of NX and EX parameters, the effect is equivalent to SETEX, which is the most common way for Redis to acquire lock writing.

How to release the lock

The command to release the lock is simple, just delete the key, but as we said earlier, because the distributed lock must be released by the lock holder itself, we must first make sure that the thread that currently releases the lock is the holder, and then delete it. In this way, it becomes two steps, which seems to violate atomicity. What should we do?

Don't panic, we can assemble the two-step operation with a lua script, like this:

`if redis.call ("get", KEYS [1]) = = ARGV [1] ``then`` return redis.call ("del", KEYS [1]) ``else``return 0``end`

KEYS [1] is the name of the current key, and ARGV [1] can be the ID of the current thread (or other unfixed values, which can identify the thread to which it belongs), so as to prevent threads holding expired locks or other threads from mistakenly deleting existing locks.

Code implementation

After knowing the principle, we can write code to implement the function of Redis distributed lock, because the purpose of this article is to explain the principle, not to teach you how to write distributed lock, so I use pseudo code to implement it.

The first is the utility class for redis locks, which contains the basic methods for locking and unlocking:

``public class RedisLockUtil {`` private String LOCK_KEY = "redis_lock"; hold time of ``/ / key, 5ms`` private long EXPIRE_TIME = 5;`` / / wait timeout, 1s`` private long TIME_OUT = 1000; ``/ / redis command parameter, equivalent to the command set of nx and px ``private SetParams params = SetParams.setParams (). Nx (). Px (EXPIRE_TIME) ``/ / redis connection pool, connected to the local redis client`` JedisPool jedisPool = new JedisPool ("127.0.0.1", 6379); ``/ *`` * `lock ``*`` * @ param id`` * thread's id, or other fields that can identify the current thread and are not duplicated ``* @ Secretn` * / ```public boolean lock (String id) {``Long start = System.currentTimeMillis ();`` Jedis jedis = jedisPool.getResource () If the ```for (;) {``for (;) {`` / / SET command returns OK, it proves that the lock was acquired successfully ```String lock = jedis.set (LOCK_KEY, id, params); ``if ("OK" .equals (lock)) {``return true; `} ``/ / otherwise, if the lock is not acquired within the TIME_OUT time, the acquisition fails ``long l = System.currentTimeMillis ()-start; ``if (l > = TIME_OUT) {``return false ``}`` try {``/ / hibernate for a while, otherwise repeated execution of the loop will always fail ``Thread.sleep (100);``} catch (InterruptedException e) {``e.printStackTrace (); ``} finally {`` jedis.close () ``}``} ``/ * *`` * unlock the id of a thread ``*`` * @ param id`` *, or other fields that identify the current thread and do not repeat ``* @ Secretn`` * /`` public boolean unlock (String id) {``Jedis jedis = jedisPool.getResource () ``/ delete key's lua script`` String script = "if redis.call ('get',KEYS [1]) = = ARGV [1] then" + "return redis.call (' del',KEYS [1])" + "else" ``+ "return 0" + "end"; ``try {`` String result = ``jedis.eval (script, Collections.singletonList (LOCK_KEY), Collections.singletonList (id)). ToString (); ``return "1" .equals (result); ``} finally {`jedis.close () ``} `

The specific code function comments have been clearly written, and then we can write a demo class to test the effect:

`public class RedisLockTest {``private static RedisLockUtil demo = new RedisLockUtil (); ``private static Integer NUM = 101; ``public static void main (String [] args) {``for (int I = 0; I)

{``String id = Thread.currentThread (). GetId () + "; ``boolean isLock = demo.lock (id); if ``try {`` / / gets the lock, subtract the shared parameter by ``if (isLock) {``NUM--; (NUM); ``}``} finally {`/ / release lock must be placed in finally``demo.unlock (id); ``} ``}). Start (); `}``}`

We create 100 threads to simulate concurrency, and the result of execution is as follows:

Code execution result

It can be seen that the effect of the lock has been achieved, and thread safety can be guaranteed.

Of course, the above code is only a simple implementation of the effect, the function is certainly incomplete, a sound distributed lock there are many aspects to consider, the actual design is not so easy.

Our purpose is only to learn and understand the principle, handwriting an industrial-grade distributed lock tool is not realistic, nor necessary, a lot of similar open source tools (Redisson), the principles are similar, and have long been tested by industry counterparts, directly used on the line.

Although the function is realized, in fact, in terms of design, such a distributed lock has great defects, which is what this article wants to focus on.

Defects of distributed locks

1. Lock failure caused by long-term blocking of the client

Client 1 gets the lock, which is blocked for a long time due to network problems or GC and other reasons, and then the lock expires before the business program has finished executing it. At this time, client 2 can also get the lock normally, which may lead to thread safety problems.

Client blocking for a long time

So how do you prevent such an exception? Let's not talk about the solution first, but we'll discuss it after introducing other defects.

2. Clock drift of redis server

If the machine clock of the redis server jumps forward, it will cause the key to time out prematurely. For example, after client 1 gets the lock, the key expires at 12:02, but the clock of the redis server itself is 2 minutes faster than that of the client, resulting in key invalidation at 12:00. At this time, if client 1 has not released the lock. It can lead to the problem that multiple clients hold the same lock at the same time.

III. Security of single point instance

If the redis is in single master mode, when the machine goes down, all clients will not be able to acquire locks. In order to improve availability, a slave may be added to the master, but because the master-slave synchronization of redis is asynchronous, it may occur that after client 1 sets the lock, master dies and slave is upgraded to master. Because of the feature of asynchronous replication, the lock set by client 1 has been lost. At this point, client 2 can also successfully set locks, resulting in client 1 and client 2 having locks at the same time.

In order to solve the Redis single point problem, the author of redis proposed the RedLock algorithm.

RedLock algorithm

The premise of this algorithm is that Redis must be deployed with multiple nodes, which can effectively prevent single point of failure. The specific implementation idea is as follows:

1. Get the current timestamp (ms)

2. First set the valid duration of key (TTL), after which it will be released automatically, and then client (client) tries to use the same key and value to set all redis instances, setting a timeout much shorter than TTL every time you link redis instances, so as not to wait too long for the closed redis service. And try to get the next instance of redis.

For example, if the TTL (that is, the expiration time) is 5s, the timeout for acquiring the lock can be set to 50ms, so if the lock cannot be acquired in the 50ms, the lock is discarded and the next lock is attempted.

3. Client subtracts the time of the first step from the time after all the locks that can be acquired, as well as the clock drift error of the redis server, and then the time difference is less than the TTL time and the number of instances successfully set the lock > = N Redis 2 + 1 (N is the number of Redis instances), then the lock is successful.

For example, if TTL is 5s, it takes 2s to connect redis to get all the locks, and then subtract the clock drift (assuming the error is about 1s), then the real effective duration of the lock is only 2s.

4. If the client fails to acquire the lock for some reason, it will begin to unlock all redis instances.

According to this algorithm, if we assume that there are five Redis instances, then client only needs to acquire more than 3 of the locks as a success, which is demonstrated by a flowchart like this:

Effective duration of key

All right, the algorithm is also introduced. From the point of view of design, there is no doubt that the idea of RedLock algorithm is to effectively prevent the single point of failure of Redis, and the error of server clock drift is also taken into account in the design of TTL, which makes the security of distributed lock much better.

But is that really the case? Anyway, my personal words feel mediocre.

First of all, we can see that in the RedLock algorithm, the valid time of the lock will be subtracted from the time it takes to connect to the Redis instance. If this process takes too long because of network problems, then the effective time left for the lock will be greatly reduced. The time for the client to access the shared resources is very short, and it is likely that the lock will expire during the process of processing. Moreover, the effective time of the lock also needs to be subtracted from the server's clock drift, but how much should be reduced? if this value is not set properly, it is easy to cause problems.

Then the second point, although this algorithm takes into account the problem of using multiple nodes to prevent Redis single point of failure, but if there is a node crash restart, it is still possible for multiple clients to acquire locks at the same time.

Suppose there are five Redis nodes: a, B, C, D, E, and client 1 and 2 are locked respectively

Client 1 successfully locked A _ Magol B _ C and acquired the lock successfully (but D and E were not locked).

The master of node C hung up, and then the lock was not synchronized to slave,slave. After upgrading to master, the lock added by client 1 was lost.

At this time, client 2 acquires the lock, locks C _ MagneD _ E, and acquires the lock successfully.

In this way, client 1 and client 2 get the lock at the same time, and the hidden danger of program security still exists. In addition, if time drift occurs in one of these nodes, it may also lead to lock security problems.

Therefore, although the availability and reliability are improved through multi-instance deployment, RedLock does not completely solve the hidden danger of Redis single point failure, nor does it solve the problem of lock timeout failure caused by clock drift and client blocking for a long time, and the hidden danger of lock security still exists.

This is the end of the article on "how to implement distributed locks in Redis". Thank you for reading! I believe you all have a certain understanding of the knowledge of "how to implement distributed locks in Redis". If you want to learn more, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.