Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

What matters should be paid attention to when implementing distributed locks with Redis

2025-01-24 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Database >

Share

Shulou(Shulou.com)05/31 Report--

In this article, the editor introduces in detail "what matters should be paid attention to in the implementation of distributed locks in Redis". The content is detailed, the steps are clear, and the details are handled properly. I hope that this article "what matters should be paid attention to in the implementation of distributed locks in Redis" can help you solve your doubts.

Implementation of distributed Lock with Redis

Recently, I saw a good article in the process of reading distributed locks, specially processing my own understanding:

Three core elements of Redis distributed lock implementation:

1. Add lock

The easiest way is to use the setnx command. Key is the unique identification of the lock, named according to the business, and value is the thread ID of the current thread.

For example, if you want to lock the flash sale activity of a commodity, you can name the key "lock_sale_ID". And what is value set to? We can set it to 1 for a while. The pseudo code for locking is as follows:

Setnx (key,1) when a thread executes setnx and returns 1, it means that key does not exist originally, and the thread successfully gets the lock. When another thread executes setnx, it returns 0, indicating that key already exists and the thread fails to grab the lock.

two。 Unlock

If you have a lock, you have to unlock it. When the thread that gets the lock finishes the task, the lock needs to be released so that other threads can enter. The easiest way to release the lock is to execute the del instruction, the pseudo code is as follows:

After del (key) releases the lock, other threads can continue to execute the setnx command to acquire the lock.

3. Lock timeout

What does lock timeout mean? If a thread that gets the lock dies during the execution of the task and has no time to explicitly release the lock, the resource will be locked forever and no other thread will ever come in.

Therefore, setnx's key must set a timeout to ensure that the lock is automatically released after a certain period of time, even if it is not explicitly released. Setnx does not support timeout parameters, so additional instructions are required. The pseudo code is as follows:

Combined with expire (key, 30), the first version of our distributed lock implementation's pseudo code is as follows:

If (setnx (key,1) = = 1) {expire (key,30) try {do something. } catch () {} finally {del (key)}}

Because in the pseudo code above, there are three fatal problems:

1. Non-atomicity of setnx and expire

Imagine an extreme scenario where when a thread executes setnx, it successfully gets the lock:

Setnx has just been executed successfully, and before it has time to execute the expire instruction, node 1 Duang hangs up.

If (setnx (key,1) = = 1) {/ / hung up here. Expire (key,30) try {do something. } catch () {} finally {del (key)}}

In this way, the lock does not set an expiration time, becomes "immortal", and other threads can no longer acquire the lock.

How to solve it? The setnx instruction itself does not support passing the timeout. Version 2.6.12 of Redis adds optional parameters to the set directive. The pseudo code is as follows: set (key,1,30,NX), which can replace the setnx instruction.

two。 Using del after timeout causes other threads' locks to be mistakenly deleted

Another extreme scenario, if a thread succeeds in getting the lock and sets a timeout of 30 seconds.

If thread An executes slowly for some reason and does not finish execution after 30 seconds, the lock expiration is automatically released and thread B gets the lock.

Thread A then finishes executing the task, and thread A then executes the del instruction to release the lock. But at this time, thread B has not finished execution, thread An actually removes the lock added by thread B.

How to avoid this situation? You can make a decision before del releases the lock to verify that the current lock is self-imposed.

As for the specific implementation, you can treat the current thread ID as a value when locking, and verify that the corresponding value of the key is the ID of your own thread before deleting it.

Lock: String threadId = Thread.currentThread (). GetId () set (key,threadId, 30dNX) doSomething. Unlock: if (threadId .equals (redisClient.get (key) {del (key)}

However, this implies a new problem: if judgment and release locks are two independent operations, not atomicity.

We are all extreme programmers, so this piece should be implemented in Lua scripts:

String luaScript ='if redis.call ('get', KEYS [1]) = = ARGV [1] then return redis.call (' del', KEYS [1]) else return 0 end'

RedisClient.eval (luaScript, Collections.singletonList (key), Collections.singletonList (threadId))

In this way, the validation and deletion process is an atomic operation.

3. The possibility of concurrency

It is still the scenario described in the second point just now. Although we avoid thread A mistakenly deleting key, it is still not perfect to have two threads, An and B, accessing the code block at the same time.

What should I do? We can ask the thread that acquired the lock to open a daemon thread to "renew" the lock that is about to expire.

When 29 seconds have elapsed and thread A has not finished executing, the daemon will execute the expire instruction to "renew" the lock for 20 seconds. The daemon thread starts execution in 29 seconds and executes every 20 seconds.

When thread A finishes the task, it explicitly shuts down the daemon thread.

On the other hand, if Node 1 suddenly loses power, the daemon thread will stop because thread An and the daemon thread are in the same process. When the lock is out of time, no one will renew its life, so it will be released automatically.

Implementation of distributed Lock with memcache

Home page top 10, loaded by the database into memcache cache for n minutes

The content cache of celebrities in Weibo, once it does not exist, a large number of requests will miss and load the database.

Data generated by multiple IO operations is stored in cache, such as querying db multiple times.

problem

In the case of large concurrency, when cache fails, a large number of concurrency can not get cache at the same time. Accessing db and setting cache at the same time may bring potential overload risk to the system. We have had a similar failure in the online system.

Solution method

If (memcache.get (key) = = null) {/ / 3 min timeout to avoid mutex holder crashif (memcache.add (key_mutex, 3 * 60 * 1000) = = true) {value = db.get (key); memcache.set (key, value); memcache.delete (key_mutex);} else {sleep (50); retry ();}}

Add a mutex key before load db, and then load db after mutex key add succeeds. If add fails, sleep retries to read the original cache data. To prevent deadlocks, mutex key also needs to set an expiration time. The pseudo code is as follows

Zookeeper implements distributed caching

The data storage structure of Zookeeper is like a tree, which is made up of nodes called Znode.

There are four types of Znode:

1. Persistent node (PERSISTENT)

The default node type. After the client that created the node is disconnected from the zookeeper, the node still exists.

two。 Persistent node sequential node (PERSISTENT_SEQUENTIAL)

A sequential node means that when a node is created, Zookeeper numbers the node name according to the chronological order in which it is created:

3. Temporary node (EPHEMERAL)

In contrast to persistent nodes, temporary nodes are deleted when the client that created the node is disconnected from zookeeper:

4. Temporary sequential node (EPHEMERAL_SEQUENTIAL)

As the name implies, temporary sequential nodes combine with temporary nodes and sequential nodes: when a node is created, Zookeeper numbers the node name according to the chronological order in which it is created; when the client that created the node is disconnected from zookeeper, the temporary node is deleted.

Zookeeper distributed locks just apply temporary sequential nodes. How can it be realized? Let's look at the detailed steps:

Acquire lock

First, create a persistent node ParentLock in Zookeeper. When the first client wants to acquire the lock, it needs to create a temporary sequential node Lock1 under the ParentLock node.

After that, Client1 looks for all the temporary order nodes under ParentLock and sorts them to determine whether the node Lock1 created by itself is the highest in order. If it is the first node, the lock is successfully obtained.

At this point, if another client Client2 comes to acquire the lock, create another temporary sequence node Lock2 in the ParentLock download.

Client2 looks for all the temporary order nodes under the ParentLock and sorts them to determine whether the node Lock2 created by itself is the highest in order. It turns out that the node Lock2 is not the smallest.

Therefore, Client2 registers Watcher with the node Lock1, which is sorted only before it, to listen for the existence of the Lock1 node. This means that the Client2 failed to grab the lock and entered the waiting state.

At this point, if another client Client3 comes to acquire the lock, create another temporary sequence node Lock3 in the ParentLock download.

Client3 looks for all the temporary order nodes under the ParentLock and sorts them to determine whether the node Lock3 created by itself is the highest in order. It is also found that the node Lock3 is not the smallest.

Therefore, Client3 registers Watcher with the node Lock2, which is sorted only before it, to listen for the existence of the Lock2 node. This means that Client3 also failed to grab the lock and entered the waiting state.

In this way, Client1 gets the lock, and Client2 listens to Lock1,Client3 and listens to Lock2. This creates a waiting queue, much like the AQS (AbstractQueuedSynchronizer) on which ReentrantLock in Java depends.

Release lock

There are two situations in which locks are released:

1. Task completed, client shows release

When the task is completed, the Client1 displays instructions to call the delete node Lock1.

two。 Client crashes during task execution

The Client1 that acquired the lock will break the link with the Zookeeper server if the Duang crashes during the execution of the task. According to the characteristics of the temporary node, the associated node Lock1 is automatically deleted.

Because Client2 has been listening for the existence of Lock1, when the Lock1 node is deleted, Client2 will be notified immediately. At this point, Client2 will query all the nodes under the ParentLock again to confirm whether the node Lock2 created by itself is the smallest node at present. If it is the smallest, then it makes sense for Client2 to acquire the lock.

Similarly, if Client2 also deletes node Lock2 because the task is completed or the node crashes, then Cient3 is notified.

In the end, Client3 succeeded in getting the lock.

Comparison of Zookeeper and Redis distributed Lock

The following table summarizes the advantages and disadvantages of Zookeeper and Redis distributed locks:

After reading this, the article "what to pay attention to when implementing distributed locks in Redis" has been introduced. If you want to master the knowledge points of this article, you still need to practice and use it yourself. If you want to know more about related articles, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 284

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Database

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report