Shulou(Shulou.com)06/01 Report--

In the past, the scheduled tasks we used were only deployed on a single machine, in order to solve the single point problem, in order to ensure that a task is executed by only one machine, we need to consider the lock problem, so we took the time to study this problem. How exactly do you implement a distributed lock?

The essence of lock is mutual exclusion, which ensures that a client can hold the same lock at any time. If you consider using redis to implement a distributed lock, the simplest solution is to create a key value in the instance and delete the key value when the lock is released. But a reliable and perfect distributed lock needs to consider a lot of details, so let's take a look at how to write a correct distributed lock.

Stand-alone distributed lock SETNX

So we implement a simple lock directly based on redis's setNX (SET if Not eXists) command. Directly add the pseudo code

Lock acquisition:

SET resource_name my_random_value NX PX 30000

Release of lock:

If redis.call ("get", KEYS [1]) = ARGV [1] then return redis.call ("del", KEYS [1]) else return 0 end

There are a few details to pay attention to:

First of all, we need to set the timeout when we acquire the lock. The timeout is set to prevent the client from crashing, or the lock is held after a network problem. The whole system is deadlocked.

Use the setNX command to ensure that the query and write steps are atomic

When the lock is released, we determine that KEYS [1]) = ARGV [1], where KEYS [1] is the value,ARGV taken from the redis [1] is the my_random_value generated above. The reason for the above judgment is to ensure that the holder of the lock is released. Let's assume that this step of verification is not performed:

Client An acquires the lock and the latecomer thread hangs. The time is greater than the expiration time of the lock. After the lock expires, client B acquires the lock. After client A restores, it issues the del command to redis after handling the relevant events. The lock is released by client C to acquire the lock. At this point, two clients in a system hold locks at the same time.

The key to this problem is that the lock held by client B is released by client A.

The release of locks must use lua scripts to ensure the atomicity of the operation. The release of the lock includes three steps: get, judgment, and del. If the atomicity of the three steps cannot be guaranteed, distributed locks will have concurrency problems.

With the above details in mind, a distributed lock with a single redis node is achieved.

There is still a single point of redis in this distributed lock. You might say that Redis is the architecture of master-slave, so just switch to slave in the event of a failure, but Redis replication is asynchronous.

If client A gets the lock on master. Master went down before master synchronized data to slave. Client B gets the lock from slave again.

In this way, due to the downtime of Master, multiple people hold locks at the same time. If your system is available for a short period of time, more than one person holds the lock. This simple solution can solve the problem.

But if we solve this problem. The official of Redis provides a solution for Redlock.

The realization of RedLock

In order to solve the problem of Redis single point. The author of Redis proposed a solution for RedLock. The scheme is very ingenious and concise.

The core idea of RedLock is to use multiple Redis Master to be redundant at the same time, and these nodes are completely independent, and there is no need to synchronize the data between these nodes.

Suppose we have N Redis nodes, N should be an odd number greater than 2. The implementation steps of RedLock:

Get the current time using the method mentioned above to acquire the Redis locks of N nodes in turn. If the number of locks acquired is greater than (N _ lock validity time), and the acquisition time is less than the effective time of locks, it is considered that a valid lock has been acquired. The lock automatic release time is the time it takes to acquire the lock before the initial lock release time is subtracted. If the number of locks acquired is less than (N _ lock validity time), or if not enough is acquired within the lock's effective time (NAccord), it is considered to have failed to acquire locks. At this point, a message to release the lock needs to be sent to all nodes.

The implementation of releasing the lock is simple. All Redis nodes are expected to initiate a release operation, regardless of whether the lock was acquired successfully or not.

At the same time, you need to pay attention to a few details:

The interval between retries to acquire locks should be a random range rather than a fixed time. This prevents multiple clients from sending lock acquisition operations to the Redis cluster at the same time, avoiding simultaneous competition. Get the same number of locks at the same time. (although the probability is very low)

If a master node fails, the interval between replies should be greater than the effective time of the lock.

Suppose there are three Redis nodes of A _ Magi B ~ C. The client foo acquires both An and B locks. At this time, B is down and all the data in memory is lost. Node B replies. At this time, the client bar reacquires the lock and acquires the two nodes of BMagne C. At this point, two more clients have acquired the lock.

So if the recovery time will be longer than the effective time of the lock, the above situation can be avoided. At the same time, if the performance requirements are not high, you can even turn on the persistence option of Redis.

Summary

After understanding the implementation of Redis distribution, I think that the principle of most distributed systems is actually very simple, but in order to ensure the reliability of distributed systems, we need to pay attention to a lot of details, trivial anomalies.

The distributed lock realized by RedLock algorithm is simple and efficient, and the idea is quite ingenious.

But does RedLock have to be safe? I will also write an article to discuss this problem. Please look forward to it.

The above is the whole content of this article, I hope it will be helpful to your study, and I also hope that you will support it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.