Shulou(Shulou.com)05/31 Report--

This article introduces the knowledge of "how to implement distributed locks in Redis". Many people will encounter this dilemma in the operation of actual cases, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Why do you need distributed locks?

Why do you need distributed locks?

The purpose of using distributed locks is to ensure that only one client can operate on shared resources at a time.

We often encounter concurrency problems in logical processing of distributed applications. [related recommendation: Redis video tutorial]

For example, if an operation wants to modify the user's state, it needs to read out the user's status first, modify it in memory, and then save it back. If such operations are done at the same time, concurrency problems arise because reading and saving state are not atomic.

At this point, distributed locks are used to limit the concurrent execution of the program. As a caching middleware system, redis can provide this distributed locking mechanism.

Its essence is to occupy a pit in the redis. When other processes are about to occupy the pit and find that it has been occupied, just wait and try again later.

In general, distributed locks available in a production environment need to meet the following requirements:

Mutex and mutual exclusion are the basic characteristics of locks. Only one thread can hold the lock at a time and perform critical operations.

Timeout release is another essential feature of locks, which can be compared with the innodb_lock_wait_timeout configuration in the MySQL InnoDB engine to prevent unnecessary thread waiting and waste of resources.

Reentrability, in a distributed environment, if the same thread on the same node acquires the lock, the second request can still succeed.

Mode of realization

Using SETNX to implement

SETNX is used as follows: SETNX key value. Only if the key key does not exist, set the value of the key key to value. If the key key exists, SETNX does not do anything.

Boolean result = jedis.setnx ("lock-key", true) = = 1L X if (result) {try {/ / do something} finally {jedis.del ("lock-key");}}

A fatal problem with this scheme is that a thread cannot perform the unlock operation normally due to some abnormal factors (such as downtime) after acquiring the lock, so the lock will never be released.

To do this, we can add a timeout to the lock

The effect of executing SET key value EX seconds is the same as executing SETEX key seconds value

The effect of executing SET key value PX milliseconds is the same as executing PSETEX key milliseconds value

String result = jedis.set ("lock-key", true, 5); if ("OK" .equals (result)) {try {/ / do something} finally {jedis.del ("lock-key");}}

The plan looks perfect, but in fact there will be problems.

Imagine that thread An acquires the lock and sets the expiration time to 10s, and then takes 15s to execute the business logic. At this time, the lock acquired by thread A has already been automatically released by Redis's expiration mechanism.

After thread An acquires the lock and 10 seconds later, the changed lock may have been acquired by other threads. When thread A finishes executing the business logic and is ready to unlock (DEL key), it is possible to delete the lock that has been acquired by other threads.

So the best way is to determine whether the lock is your own when unlocking, we can set value to a unique value uniqueValue when setting key (it can be random value, UUID, or combination of machine number + thread number, signature, etc.).

When unlocking, that is, when deleting the key, first determine whether the value corresponding to the key is equal to the previously set value, and delete the key only if it is equal.

String velue= String.valueOf (System.currentTimeMillis ()) String result = jedis.set ("lock-key", velue, 5); if ("OK" .equals (result)) {try {/ / do something} finally {/ / nonatomic operation if (jedis.get ("lock-key") = = value) {jedis.del ("lock-key");}

Here we can see the problem at a glance: GET and DEL are two separate operations, and exceptions may occur between the execution of GET and before the execution of DEL.

If we just make sure that the unlocked code is atomic, we can solve the problem.

Here we introduce a new method, which is the Lua script. The example is as follows:

If redis.call ("get", KEYS [1]) = ARGV [1] then return redis.call ("del", KEYS [1]) else return 0end

Where ARGV [1] represents the unique value specified when setting the key.

Because of the atomicity of the Lua script, commands from other clients need to wait for the Lua script to be executed during the execution of the Lua script.

Ensure that the expiration time is greater than the business execution time

To prevent multiple threads from executing business code at the same time, you need to ensure that the expiration time is greater than the business execution time

Add a property of boolean type isOpenExpirationRenewal to identify whether to enable scheduled refresh expiration time

Add a scheduleExpirationRenewal method to start the thread that refreshes the expiration time

After successfully acquiring the lock, the locking code sets isOpenExpirationRenewal to true and calls the scheduleExpirationRenewal method to start the thread that refreshes the expiration time.

The unlock code adds one line of code, sets the isOpenExpirationRenewal property to false, and stops thread polling that refreshes the expiration time

Redisson implementation

If the lock is acquired successfully, a scheduled task will be opened, and the scheduled task will be checked regularly to renew

The time difference of each call is internalLockLeaseTime / 3, which is only 10 seconds.

By default, the locking time is 30 seconds. If the locked service is not finished, the lock will be renewed for 30-10 = 20 seconds, and the lock will be reset to 30 seconds.

RedLock

In the cluster, when the master node dies, the slave node will take its place, but there is no obvious perception on the client. The first client successfully applied for a lock in the master node, but before the lock was synchronized to the slave node, the master node suddenly hung up. Then the slave node becomes the master node, and there is no lock inside the new node, so when another client comes to request a lock, it is approved immediately. This will cause the same lock in the system to be held by two clients at the same time, resulting in insecurity.

Redlock algorithm is to solve this problem.

With Redlock, you need to provide multiple Redis instances that were previously independent of each other and have no master-slave relationship. Like many distributed algorithms, redlock uses most mechanisms

When locking, it sends set instructions to more than half of the nodes. As long as the set of more than half of the nodes is successful, it is considered to be successful. When you release the lock, you need to send a del instruction to all nodes. However, the Redlock algorithm also needs to consider many details, such as error retry, clock drift and so on. At the same time, because Redlock needs to read and write to multiple nodes, it means that the performance of Redis will be lower than that of single instance.

Redlock algorithm is a high availability mode introduced on the basis of a single Redis node. Redlock is based on N completely independent Redis nodes, usually an odd number greater than 3 (usually N can be set to 5), which can basically ensure that each node in the cluster will not downtime at the same time.

Assuming that the current cluster has five nodes, the client running the Redlock algorithm performs the following steps in turn to complete the lock acquisition operation

The client records the current system time in milliseconds

Try to acquire the lock using the same key from 5 Redis instances in turn. When requesting the lock from Redis, the client should set a network connection and response timeout, which should be less than the lock failure time to avoid problems due to network failure.

The client obtains the lock usage time by subtracting the lock acquisition time from the current time. Lock acquisition is successful only if and only if the lock is acquired from more than half of the Redis nodes, and when the use time is less than the lock failure time.

If the lock is acquired, the true valid time of the key is equal to the valid time minus the time it takes to acquire the lock, reducing the probability of timeout

If it fails to acquire the lock, the client should unlock it on all Redis instances, even the node that failed in the previous step to prevent the inconsistency caused by the loss of the server response message but the successful addition of the actual data.

In other words, assuming that the lock expires in 30 seconds, and it takes 31 seconds for the three nodes to add the lock, it naturally fails.

In the Java client Redisson officially recommended by Redis, there is a built-in implementation of RedLock

RedLock question:

RedLock only ensures the high availability of the lock, not the correctness of the lock.

RedLock is a distributed system that relies heavily on the system clock.

Martin's criticism of RedLock:

RedLock is too heavy for efficiency scenarios.

For scenarios with high requirements for correctness, RedLock can not guarantee correctness.

This is the end of "how to implement distributed locks in Redis". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.