Shulou(Shulou.com)06/03 Report--

This article mainly explains "what is the principle and implementation of distributed lock". The content of the explanation in this article is simple and clear, and it is easy to learn and understand. let's study and learn "what is the principle and implementation of distributed lock"?

Why is there a distributed lock?

As shown in the figure below, an application is deployed to multiple machines for load balancing. How can we solve this problem in order to ensure that a method or property can only be executed by the same thread at the same time in the case of high concurrency?

In the case of stand-alone deployment of traditional single applications, mutual exclusion control can be solved by using functions related to concurrent processing (such as API:ReentrantLcok or synchronized related to Java concurrent processing). However, with the development of business, the system architecture will be gradually optimized and upgraded, and the system originally deployed on a single machine has evolved into a distributed cluster system, because the distributed system is multi-threaded, multi-process and distributed on many different machines. this will make the concurrency control locking strategy of the original stand-alone deployment unable to meet, and can not provide the ability of distributed locking. In order to solve this problem, we need a cross-machine mutual exclusion mechanism to control the access to shared resources, which is the problem solved by distributed locks.

What are the application scenarios of distributed locks?

The application scenarios of distributed locks are deduced in reverse for the purpose of distributed locks, which mainly include two categories:

1. Improvement of processing efficiency: the application of distributed locks can reduce the execution of repetitive tasks and avoid the waste of resource processing efficiency.

2. data accuracy guarantee: the use of distributed locks can be placed on the concurrent access of data resources to avoid data inconsistencies and even data loss.

Implementation premise of distributed Lock

Distributed CAP theory:

No distributed system can satisfy consistency (Consistency), availability (Availability) and partition fault tolerance (Partition tolerance) at the same time.

Usually, everyone will sacrifice strong consistency for high availability of the system, so many of our scenarios only need to ensure the "ultimate consistency" of the data.

It is important to note that this final time needs to be within a range acceptable to the user.

In addition, to implement distributed locks, you need to have some conditions, including the following:

1. In a distributed system environment, a method can only be executed by one thread of a machine at a time.

2. High availability and high performance of acquisition and release locks

3. It has the feature of non-blocking lock. If you do not get the lock, you will directly return the failure to acquire the lock.

4. It has lock failure mechanism to prevent deadlock.

The above conditions mainly highlight the efficiency of the lock itself and the application characteristics of ensuring accuracy, while avoiding its own impact on resource access.

What are the ways to achieve it?

The implementation of distributed lock can be controlled in different links.

The common ones are mainly divided into the following categories:

1. Open source component lock control: ZooKeeper

ZooKeeper is an open source framework for distributed coordination services. It is mainly used to solve the problem of consistency of application systems in distributed clusters, such as how to avoid dirty reading caused by operating the same data at the same time. ZooKeeper is essentially a distributed small file storage system. Data storage based on a directory tree similar to a file system is provided, and the nodes of the tree species can be managed effectively.

So how to use ZooKeeper to implement distributed locks?

1) the client connects to zookeeper and creates temporary and ordered child nodes under / tmp. The first client corresponds to lock-0000, the second to lock-0001, and so on.

2) the client acquires the list of child nodes under / lock to determine whether the child node it creates is the child node with the lowest sequence number in the current child node list, and if so, it thinks that it has acquired the lock, otherwise it listens to the child node that happens to be in front of it to delete the message, and repeat this step until the lock is obtained after receiving the notification of the child node change.

For example, the list of child nodes under / tmp is: lock-0000, lock-0001, lock-0002. The client with sequence number 1 listens to the child node with sequence number 0 to delete the message, and the client with sequence number 2 listens to the child node with sequence number 1 to delete the message. (delete child nodes as soon as the business code is executed)

3) execute the business code process, delete the child node corresponding to the current client, and release the lock.

The performance of ZooKeeper distributed lock mode is worse than that of Redis mode. The main reason is that all write operations (acquiring lock release lock) need to be performed on Leader and then synchronized to follower.

2. Task processing lock control: Redis

Redis is completely open source and free, complies with the BSD protocol, and is a high-performance key-value database.

The main advantages include:

Extremely high performance-Redis can read at 11w + times / s and write at 8w + times / s

Rich data types-Redis mainly supports Strings, Lists, Hashes, Sets and Ordered Sets data types

Atomicity-all operations of Redis are atomic, while Redis also supports the atomicity of several merged operations

Rich features-Redis also supports publish/subscribe, notification, key expiration, and so on.

Redis implements a simple distributed locking process:

(1) acquire the lock: use setnx to add a lock, and use the expire command to add a timeout for the lock. After this time, the lock is automatically released. The value of the lock is a randomly generated UUID, which can be judged when the lock is released.

(2) acquisition lock: set a timeout for acquisition. If this time is exceeded, the acquisition lock is discarded.

(3) release lock: judge whether it is the lock by UUID, if it is the lock, execute delete to release the lock.

Using Redis to implement distributed locks, there are some possible disadvantages:

When executing delete to release the lock, if the operation to delete the lock fails, the Key-Value expiration time is difficult to control and may exist all the time, which may affect subsequent data verification.

3. Data write lock control: MySQL

At the database level, when the final data is written, the write control processing of the data is regarded as the final end of the distributed lock. It mainly includes the following three ways, which are introduced below.

Implementation method 1: unique index

UNIQUE KEY `uidx_ name` (`name`) USING BTREE

In the above case, we restrict the uniqueness of the index to the name field. When there are multiple new data requests submitted to the database at the same time, the database itself will use the unique index to ensure the uniqueness of the data.

Implementation method 2: exclusive lock

Perform the following SQL:

SELECT status FROM users WHERE id = 3 FOR UPDATE

Suppose, in another transaction, execute again:

SELECT status FROM users WHERE id = 3 FOR UPDATE

The second transaction waits for the commit of the previous transaction, and the second query is in a blocked state.

Application of exclusive lock:

During a transaction operation, through the "FOR UPDATE" statement, MySQL adds an exclusive lock to each row of data in the query result set, and other threads block update and delete operations on the record. Exclusive locks include row locks and table locks.

Implementation method 3: optimistic lock

Implementation logic: every time the optimistic lock performs a data modification operation, it will take a data version number. Once the version number and the data version number are the same, the modification operation can be performed and + 1 operation can be performed on the version number, otherwise the execution will fail. Because the version number increases with each operation, there is no ABA problem.

In addition to version, you can also use timestamps, because timestamps are naturally sequentially incremental.

The troublesome point is that you need to query the current version version before operating the business.

Possible disadvantages of database distributed lock implementation:

DB has poor operation performance and the risk of locking the table.

When a non-blocking operation fails, polling is required to consume cpu resources

Not commit for a long time or polling for a long time may take up more connection resources

Thank you for your reading. the above is the content of "what is the principle and implementation of distributed lock". After the study of this article, I believe you have a deeper understanding of the principle and implementation of distributed lock. the specific use of the situation also needs to be verified by practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.