Shulou(Shulou.com)05/31 Report--

This article mainly introduces the relevant knowledge of "Redis distributed lock case analysis". The editor shows you the operation process through the actual case, and the operation method is simple, fast and practical. I hope this article "Redis distributed lock case analysis" can help you solve the problem.

Overview of distributed Lock

In a multithreaded environment, in order to ensure that a block of code can only be accessed by one thread at a time, we can generally use synchronized syntax and ReetrantLock in Java to guarantee, which is actually the way of local locking. But now the company is popular distributed architecture, in a distributed environment, how to ensure that the threads of different nodes execute synchronously? Therefore, it leads to the distributed lock, which is a way to control the mutually exclusive access to shared resources between distributed systems.

In a distributed system, multiple services are deployed on multiple machines. When a client initiates a data insertion request, if there is no guarantee of distributed locking mechanism, then multiple services on multiple machines may perform concurrent insertion operations, resulting in repeated data insertion, which will cause problems for some businesses that do not allow excess data. The distributed locking mechanism is to solve similar problems and to ensure mutually exclusive access to shared resources among multiple services. If one service preempts the distributed lock and other services do not acquire the lock, no subsequent operations will be carried out. The general meaning is as shown in the following figure:

Characteristics of distributed Lock

Distributed locks generally have the following characteristics:

Mutex: only one thread can hold a lock at a time

Reentrancy: the same thread on the same node can acquire the lock again after acquiring the lock

Lock timeout: supports lock timeout like locks in J.U.C to prevent deadlock

High performance and high availability: locking and unlocking requires high efficiency, as well as high availability to prevent distributed lock failures

With blocking and non-blocking: can be awakened from the blocking state in time

Implementation of distributed Lock

We generally implement distributed locks in the following ways:

Based on database

Based on Redis

Based on zookeeper

Problems of Redis ordinary distributed Lock

When it comes to Redis distributed locks, most people think of setnx+lua (redis guarantees that no other actions are performed when lua scripts are executed, ensuring the atomicity of operations), or that set key value px milliseconds nx is known. The core implementation commands of the latter method are as follows:

-acquire lock (unique_value can be UUID, etc.) SET resource_name unique_value NX PX 30000-release lock (in lua script, be sure to compare value to prevent misunderstanding of lock) if redis.call ("get", KEYS [1]) = ARGV [1] then return redis.call ("del", KEYS [1]) else return 0end

There are three main points in this implementation (where the probability of interview is very high):

The set command uses set key value px milliseconds nx

Value should be unique

Verify the value value when releasing the lock. Do not misunderstand the lock.

In fact, the biggest disadvantage of this kind of lock is that it only works on one Redis node when locking. Even if the Redis is highly available through sentinel, if the master-slave switch occurs on this master node for some reason, then the lock will be lost:

Got the lock on the master node of Redis

But the locked key has not been synchronized to the slave node

Master failure, failover occurs, slave node is upgraded to master node

Causing the lock to be lost.

In order to avoid the problem of single point of failure, Redis author antirez proposed a more advanced implementation of distributed lock: Redlock based on distributed environment. Redlock is also the only way to make the interviewer's best part of all the distributed lock implementations of Redis.

Redis Advanced distributed Lock: Redlock

The redlock algorithm proposed by antirez looks like this:

In the distributed environment of Redis, we assume that there are N Redis master. These nodes are completely independent of each other, and there is no master-slave replication or other cluster coordination mechanism. We ensure that locks will be acquired and released in the same way on N instances as in the case of a single instance of Redis. Now let's assume that there are five Redis master nodes, and we need to run these Redis instances on five servers to ensure that they don't all go down at the same time.

To get the lock, the client should do the following:

Gets the current Unix time in milliseconds.

Try to acquire the lock from five instances in turn, using the same key and a unique value (such as UUID). When requesting a lock from Redis, the client should set a network connection and response timeout, which should be less than the lock expiration time. For example, if your lock automatic failure time TTL is 10 seconds, the timeout time should be between 5 and 50 milliseconds. This prevents the client from waiting for the response result when the server-side Redis has been hung up. If the server does not respond within the specified time, the client should try to request a lock from another Redis instance as soon as possible.

The client uses the current time minus the time to start acquiring the lock (the time recorded in step 1) to get the time to acquire the lock. The lock is successful only if and only if the lock is obtained from most of the Redis nodes, and the time used is less than the lock failure time.

If the lock is taken, the true effective time of the key is equal to the effective time minus the time it takes to acquire the lock (the result of the calculation in step 3).

If, for some reason, the acquisition of the lock fails (the lock has not been obtained on at least one Redis instance or the lock time has exceeded the valid time), the client should unlock it on all Redis instances (even if some Redis instances are not locked at all to prevent some nodes from acquiring the lock but the client does not get a response so that the lock cannot be re-acquired for the next period of time).

Clock drift is not discussed here

Redlock source code

Redisson has encapsulated the redlock algorithm, followed by a brief introduction to its usage and an analysis of the core source code (assuming 5 redis instances).

1. Redlock depends on org.redisson redisson 3.3.22. Redlock usage

First, let's take a look at the distributed locking usage implemented by the redlock algorithm encapsulated by redission, which is very simple, similar to ReentrantLock:

Config config = new Config (); config.useSentinelServers (). AddSentinelAddress ("127.0.0.1 getFairLock 6369", "127.0.0.1 RLock redLock 6379", "127.0.0.1pur6389") .setMasterName ("masterName") .setPassword ("password"). SetDatabase (0); RedissonClient redissonClient = Redisson.create (config); / / you can also getFairLock (), getReadWriteLock () RLock redLock = redissonClient.getLock ("REDLOCK_KEY"); boolean isLock Try {isLock = redLock.tryLock (); / / if 500ms cannot get the lock, it is considered a failure to acquire the lock. 10000ms, that is, 10s is the lock failure time. IsLock = redLock.tryLock (10000, TimeUnit.MILLISECONDS); if (isLock) {/ / TODO if get lock success, do something;}} catch (Exception e) {} finally {/ / in any case, unlock redLock.unlock ();} 3. Redlock unique ID

One of the most important points in implementing distributed locks is that the value of set should be unique. How does the value of redisson ensure the uniqueness of value? The answer is UUID+threadId. The entry is in redissonClient.getLock ("REDLOCK_KEY"), and the source code is in Redisson.java and RedissonLock.java:

Protected final UUID id = UUID.randomUUID (); String getLockName (long threadId) {return id + ":" + threadId;} 4. Redlock acquires lock

The code for acquiring lock is redLock.tryLock () or redLock.tryLock (10000, 10000, TimeUnit.MILLISECONDS). The final core source code of both is the following code, except that the default lease time (leaseTime) for the former to acquire lock is LOCK_EXPIRATION_INTERVAL_SECONDS, that is, 30s:

RFuture tryLockInnerAsync (long leaseTime, TimeUnit unit, long threadId, RedisStrictCommand command) {internalLockLeaseTime = unit.toMillis (leaseTime) / / the command return commandExecutor.evalWriteAsync (getName (), LongCodec.INSTANCE, command, / / the KEY of the distributed lock cannot exist when acquiring the lock to 5 redis instances. If it does not exist, execute the hset command (hset REDLOCK_KEY uuid+threadId 1) And set the expiration time (also the lease time of the lock) "if (redis.call ('exists', KEYS [1]) = = 0) then" + "redis.call (' hset', KEYS [1], ARGV [2], 1) through pexpire "+" redis.call ('pexpire', KEYS [1], ARGV [1]); "+" return nil; "+" end "+ / / if the KEY of the distributed lock already exists and the value matches, indicating that it is the lock held by the current thread, then the number of reentrants is increased by 1, and the failure time is set" if ('hexists', KEYS [1], ARGV [2]) = = 1) then "+" redis.call (' hincrby', KEYS [1], ARGV [2], 1) "+" redis.call ('pexpire', KEYS [1], ARGV [1]); "+" return nil; "+" end; "+ / / obtain the number of milliseconds of KEY failure time for distributed locks" return redis.call (' pttl', KEYS [1]) ", / / these three parameters correspond to KEYS [1], ARGV [1] and ARGV [2] Collections.singletonList (getName ()), internalLockLeaseTime, getLockName (threadId);}

In the command to acquire the lock

KEYS [1] is Collections.singletonList (getName ()), which represents the key of a distributed lock, namely REDLOCK_KEY.

ARGV [1] is internalLockLeaseTime, that is, the lease time of the lock. The default is 30s.

ARGV [2] is getLockName (threadId), which is the only value of set when acquiring a lock, that is, UUID+threadId:

5. Redlock release lock

The code to release the lock is redLock.unlock (), and the core source code is as follows:

Protected RFuture unlockInnerAsync (long threadId) {/ / execute the following command to all five redis instances: return commandExecutor.evalWriteAsync (getName (), LongCodec.INSTANCE, RedisCommands.EVAL_BOOLEAN, / / if the distributed lock KEY does not exist Then post a message to channel: "if (redis.call ('exists', KEYS [1]) = = 0) then" + "redis.call (' publish', KEYS [2], ARGV [1]) "+" return 1; "+" end; "+ / / if the distributed lock exists but the value does not match, indicating that the lock has been occupied, then directly return" if ('hexists', KEYS [1], ARGV [3]) = 0) then "+" return nil; "+" end "+ / / if the current thread owns the distributed lock, subtract the number of reentrants by 1" local counter = redis.call ('hincrby', KEYS [1], ARGV [3],-1) "if the value of + / / reentrant times minus 1 is greater than 0, which means that the distributed lock has been reentered, then only set the expiration time, and you cannot delete" if (counter > 0) then "+" redis.call ('pexpire', KEYS [1], ARGV [2]); "+" return 0 If the value of "+" else + / / after the number of reentrants minus 1 is 0, indicating that the distributed lock has only been acquired once, then delete the KEY and issue the unlock message "redis.call ('del', KEYS [1]);" + "redis.call (' publish', KEYS [2], ARGV [1]) "+" return 1; "+" end; "+" return nil; ", / / these five parameters correspond to KEYS [1], KEYS [2], ARGV [1], ARGV [2] and ARGV [3] Arrays.asList (getName (), getChannelName ()), LockPubSub.unlockMessage, internalLockLeaseTime, getLockName (threadId));} distributed locking wheels implemented by Redis

Let's use the combination of SpringBoot + Jedis + AOP to implement a simple distributed lock.

1. Custom annotation

Customize an annotation, and the annotated method executes the logic of acquiring distributed locks

@ Target (ElementType.METHOD) @ Retention (RetentionPolicy.RUNTIME) @ Documented@Inheritedpublic @ interface RedisLock {/ * Business key * * @ return * / String key (); / * * the number of seconds for the lock to expire. Default is 5 seconds * * @ return * / int expire () default 5 / * attempt to add a lock, and the maximum waiting time is * * @ return * / long waitTime () default Long.MIN_VALUE; / * the timeout unit of the lock * * @ return * / TimeUnit timeUnit () default TimeUnit.SECONDS;} 2. AOP interceptor implementation

In AOP, we perform the logic of acquiring and releasing distributed locks, as follows:

@ Aspect@Componentpublic class LockMethodAspect {@ Autowired private RedisLockHelper redisLockHelper; @ Autowired private JedisUtil jedisUtil; private Logger logger = LoggerFactory.getLogger (LockMethodAspect.class); @ Around ("@ annotation (com.redis.lock.annotation.RedisLock)") public Object around (ProceedingJoinPoint joinPoint) {Jedis jedis = jedisUtil.getJedis (); MethodSignature signature = (MethodSignature) joinPoint.getSignature (); Method method = signature.getMethod (); RedisLock redisLock = method.getAnnotation (RedisLock.class) String value = UUID.randomUUID (). ToString (); String key = redisLock.key (); try {final boolean islock = redisLockHelper.lock (jedis,key, value, redisLock.expire (), redisLock.timeUnit ()); logger.info ("isLock: {}", islock); if (! islock) {logger.error ("failed to acquire lock") Throw new RuntimeException ("failed to acquire lock");} try {return joinPoint.proceed ();} catch (Throwable throwable) {throw new RuntimeException ("system exception");}} finally {logger.info ("release lock") RedisLockHelper.unlock (jedis,key, value); jedis.close ();} 3. Redis implements the distributed lock core class @ Componentpublic class RedisLockHelper {private long sleepTime = 100 / * * directly use setnx + expire to obtain distributed locks * non-atomicity * * @ param key * @ param value * @ param timeout * @ return * / public boolean lock_setnx (Jedis jedis,String key, String value, int timeout) {Long result = jedis.setnx (key, value) / / result = 1, setting succeeded, otherwise setting failed if (result = = 1L) {return jedis.expire (key, timeout) = = 1L;} else {return false }} / * use Lua script The script uses the setnex+expire command to lock * * @ param jedis * @ param key * @ param UniqueId * @ param seconds * @ return * / public boolean Lock_with_lua (Jedis jedis,String key, String UniqueId, int seconds) {String lua_scripts = "if redis.call ('setnx',KEYS [1], ARGV [1]) = 1 then" + "redis.call (' expire') KEYS [1], ARGV [2]) return 1 else return 0 end " List keys = new ArrayList (); List values = new ArrayList (); keys.add (key); values.add (UniqueId); values.add (String.valueOf (seconds)); Object result = jedis.eval (lua_scripts, keys, values); / / determine success return result.equals (1L) } / * * in Redis 2.6.12 and later, use the set key value [NX] [EX] command * * @ param key * @ param value * @ param timeout * @ return * / public boolean lock (Jedis jedis,String key, String value, int timeout, TimeUnit timeUnit) {long seconds = timeUnit.toSeconds (timeout) Return "OK" .equals (jedis.set (key, value, "NX", "EX", seconds)) } / * Custom timeout for acquiring locks * * @ param jedis * @ param key * @ param value * @ param waitTime * @ param timeUnit * @ return * @ throws InterruptedException * / public boolean lock_with_waitTime (Jedis jedis,String key, String value, int timeout, long waitTime TimeUnit timeUnit) throws InterruptedException {long seconds = timeUnit.toSeconds (timeout) While (waitTime > = 0) {String result = jedis.set (key, value, "nx", "ex", seconds); if ("OK" .equals (result)) {return true;} waitTime-= sleepTime; Thread.sleep (sleepTime);} return false Wrong unlock method: delete key * * @ param key * / public void unlock_with_del (Jedis jedis,String key) {jedis.del (key) } / * * use Lua script for unlocking manipulation Verify the value value when unlocking: * * @ param jedis * @ param key * @ param value * @ return * / public boolean unlock (Jedis jedis,String key,String value) {String luaScript = "if redis.call ('get',KEYS [1]) = = ARGV [1] then" + "return redis.call (' del',KEYS [1]) else return 0 end" Return jedis.eval (luaScript, Collections.singletonList (key), Collections.singletonList (value)) .equals (1L);}} 4. Controller layer control

Define a TestController to test the distributed lock we implemented

@ RestControllerpublic class TestController {@ RedisLock (key = "redis_lock") @ GetMapping ("/ index") public String index () {return "index";}} that's all for "Redis distributed lock instance analysis". Thank you for reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.