Shulou(Shulou.com)06/02 Report--

What is the principle of distributed lock in Zookeeper? I believe many inexperienced people don't know what to do about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

ZNode node of Zookeeper

Before we can understand how Zookeeper implements distributed locks, we first need to know about the nodes in Zookeeper.

The nodes in Zookeeper can be divided into two categories, one is temporary node, the other is persistent node.

Temporary node, which means that after the node is created, if the session between the client that created the node and the Zookeeper server fails (such as disconnecting), then the node will be deleted.

Persistent node means that after a node is created, even if the session between the client that created the node and the Zookeeper server fails (for example, disconnection), the node will not be deleted. Only when the client initiates a request to delete the node will the node be deleted.

There is also a kind of node called ordered node, which is created with a sequence number, which is self-increasing. Ordered nodes can be either ordered temporary nodes or ordered persistence nodes.

All the data in Zookeeper is stored through nodes, and its directory structure is like a file tree, as shown in the following figure.

Zookeeper structure

The locks, register and data directories in the figure are customized and created to store data of different businesses. For example, locks is used to store distributed lock-related information, and register is used to store registry-related data.

Now we are going to get a distributed place, so suppose the K of this lock is called K1, then there is a client a, and then go to JK to create a distributed lock called K1, then he will create a folder called K1 under the nex directory, a file called K1. Now we are going to get a distributed place, so suppose the K of this lock is called K1, then there is a client a, and then go to JK to create a distributed lock called K1, then he will create a folder called K1 under the nex directory, a file called K1.

How to realize

Using Zookeeper to realize distributed lock, there are two schemes: 1. Based on temporary node implementation; 2. It is implemented based on temporary sequential nodes. The implementation principle of this scheme is also introduced below.

First, assume that all distributed locks are stored in the locks directory.

Option 1: implementation based on temporary nodes (not recommended)

Suppose clients A, B, and C all come to acquire the same distributed lock: Key1.

First, client An acquires the distributed lock Key1, then it tries to create a ZNode node called Key1 under the locks directory. If there is no Key1 ZNode node in the locks directory at this time, client A can successfully create the Key1 node, which means that client A has successfully acquired the Key1 lock.

Figure 1

At the same time, client B also comes to acquire the lock Key1. Client B also needs to go to the locks directory to create a Key1 ZNode node. At this time, the creation of client B will fail because the ZNode node Key1 already exists. If the creation fails, client B will fail to acquire the lock, so client B will register its own listener (Watcher) with Zookeeper to listen for changes in Key1, a ZNode node (Zookeeper will notify client B when the Key1 node changes).

If client An and client B request to Zookeeper at the same time, then Zookeeper has a mechanism that ensures that only one of the clients will successfully create the Key1 ZNode node.

Figure 2

By the same token, when client C comes to acquire the Key1 lock, it will not be able to obtain the lock, and it will register its own Watcher in ZK to listen for changes in the ZNode node Key1.

When client A has finished processing its business logic, it performs the operation of releasing the lock. When the lock is released, the client deletes the Key1 node. If the node is deleted successfully, the lock is released successfully. When the Key1 node is deleted, Zookeeper notifies all clients listening to the Key1 node, that is, clients B and C.

When client B and C are notified and know that the Key1 node has changed, they will re-request Zookeeper and try to create a Key1 node under the locks directory. At this time, only one client can successfully create the Key1 node. If client B is created successfully, it means that client B successfully acquired the lock. Client C fails to acquire the lock, so it continues to listen for changes in the Key1 node.

Figure 3

Why not recommend it?

The above is based on the temporary node scheme to implement Zookeeper distributed locking, but this scheme is usually not recommended. Why? This is because there is a big problem with using this scheme: the herding effect.

What do you mean?

As we can see from the above procedure, when client A releases the lock successfully, Zookeeper needs to notify all clients listening to the Key1 node. In our example above, there are only client B and client C, but there may be hundreds or more clients in the actual application. Zookeeper needs to send hundreds of requests at this moment, first of all, this efficiency is obviously not high, and when the competition for distributed locks is fierce, it is very likely that the network card of Zookeeper may be burst at this moment. And the system may not only store the lock Key1, there will be Key2, Key3, Key4..., locks will also be competitive, the pressure of Zookeeper will be greater.

In this process, we can obviously feel that this is unreasonable, because only one of the clients can obtain the distributed lock, so when the Key1 node is deleted, we need to notify other clients to acquire the lock, is it necessary for us to notify all clients?

Obviously it's not necessary, we just need to notify one of the clients. So option two emerged.

Scheme 2: implementation based on temporary sequential nodes (recommended)

When implementing distributed locks based on temporary sequential nodes, the temporary node Key1 is not created under the Linux directory. Instead, create a Key1 directory under the locks directory, and then create a temporary sequence node in the Key1 directory.

Suppose client a now comes to acquire the distributed lock Key1, then client A will create a temporary sequence node in the Key1 directory with the sequence number 001.

Then client A will determine whether the temporary sequence node 001 created by itself has the lowest sequence number in the Key1 directory. If it is the smallest, it means that client An acquired the lock successfully.

Then client B will also obtain the Key1 distributed lock, and it will also create a temporary sequence node under the Key1 directory. Because the sequence number has been changed to 002, because 001 has been created before, client B will create the temporary sequence node 002.

Figure 4

Similarly, client B will also determine whether the temporary sequence node 002 created by client B is the temporary node with the lowest sequence number in the current Key1 directory, obviously not, because there is a 001 temporary sequence node in front of it, so client B failed to acquire the lock at this time.

When client B fails to acquire the lock, it registers its listener with Zookeeper. It listens to the temporary sequential node in front of it, that is, 001.

Figure 5

At this time, if client C also comes to acquire the distributed lock Key1, it will create a temporary sequence node 003 in the Key directory. Similarly, 003 is not the temporary sequence node with the lowest sequence number, so client C also fails to acquire the lock, and then it will listen on the temporary sequence node 002.

When client A finishes processing the business logic, it releases the lock. The operation of releasing the lock is to delete the temporary sequence node created by client A under the Key1 directory, that is, to delete the temporary sequence node 001. When 001 is deleted, Zookeeper notifies all clients listening to 001, that is, client B. After client B receives the notification from Zookeeper, it determines whether the temporary sequence node 002 I currently created is the one with the lowest sequence number in the current Key1 directory. At this point, since 001 is no longer a sequential node, it is obvious that 002 is the smallest, so client B acquires the lock successfully.

Figure 6

Similarly, when client B releases the lock, 002 will be deleted, and after 002 is deleted, Zookeeper will notify client C that client C finds that the temporary sequence node 003 I currently created is the smallest sequence number in the Key1 directory, so client C acquired the lock successfully.

After reading the above, have you mastered the principle of distributed locks in Zookeeper? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.