Shulou(Shulou.com)06/01 Report--

How to implement database, cache and Zookeeper in distributed lock? in order to solve this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Several implementation methods of distributed Lock

At present, almost many large websites and applications are distributed, and the problem of data consistency in distributed scenarios has always been an important topic.

Distributed CAP theory tells us that no distributed system can satisfy Consistency, Availability and Partition tolerance at the same time, but only two at the same time.

Therefore, many systems have to make a choice between these three at the beginning of their design. In the vast majority of scenarios in the Internet field, strong consistency needs to be sacrificed for high availability of the system, and systems often only need to ensure "ultimate consistency", as long as the final time is within the range acceptable to the user.

In many scenarios, in order to ensure the ultimate consistency of data, we need a lot of technical solutions to support, such as distributed transactions, distributed locks and so on. Sometimes we need to ensure that a method can only be executed by the same thread at the same time. In a stand-alone environment, Java actually provides a lot of API related to concurrent processing, but these API are powerless in distributed scenarios.

In other words, Java Api alone does not provide the ability to distribute locks. Therefore, there are many schemes for the implementation of distributed locks.

At present, there are several commonly used schemes for the implementation of distributed locks:

1. Database implementation

two。 Implementation based on cache (redis,memcached, etc.)

Implementation of distributed Lock with 3.Zookeeper

Before analyzing these implementations, let's think about what kind of distributed locks we need. (here, take method lock as an example, resource lock is the same)

1) it can be guaranteed that in a distributed application cluster, the same method can only be executed by one thread on one machine at a time.

2) this lock is a reentrant lock (avoid deadlock)

3) this lock is preferably a blocking lock (consider this one according to business requirements)

4) highly available lock acquisition and release functions

5) the performance of acquiring and releasing locks is better.

Implementation of distributed Lock based on Database

1. Based on database table

Perhaps the easiest way to implement a distributed lock is to create a lock table directly and then do so by manipulating the data in the table.

When we want to lock a method or resource, we add a record to the table and delete it when we want to release the lock.

Create a database table like this:

When we want to lock a method, execute the following SQL:

Because we have made a uniqueness constraint on method_name, if there are multiple requests submitted to the database at the same time, the database will guarantee that only one operation can succeed, then we can assume that the successful thread has acquired the lock of the method and can execute the content of the method body.

After the method is executed, you need to execute the following Sql if you want to release the lock:

The above simple implementation has the following problems:

1. This lock strongly depends on the availability of the database, which is a single point. Once the database is down, it will cause the business system to be unavailable.

2. There is no expiration time for this lock. Once the unlock operation fails, the lock record will remain in the database all the time, and other threads will no longer be able to obtain the lock.

3. This lock can only be non-blocking, because the insert operation of the data will report an error directly if the insertion fails. The thread that did not acquire the lock will not enter the queue, and the lock acquisition operation will be triggered again in order to acquire the lock again.

4. The lock is non-reentrant, and the same thread cannot acquire the lock again until it is released. Because the data already exists.

Of course, there are other ways to solve the above problems.

Is the database a single point? Make two databases and synchronize the data in both directions before. Once you hang up, quickly switch to the standby library.

No expiration time? As long as you do a scheduled task, clean up the timeout data in the database at regular intervals.

Non-blocking? Create a while loop and return success until insert succeeds.

Non-reentrant? Add a field to the database table to record the host information and thread information of the machine that currently acquired the lock, then the next time you acquire the lock, query the database first. If the host information and thread information of the current machine can be found in the database, just assign the lock to him.

two。 Exclusive lock based on database

In addition to adding and deleting records in the data table, you can also use the locks in the data to achieve distributed locks.

We also use the database table we just created. Distributed locks can be implemented through exclusive locks in the database. The MySql-based InnoDB engine can use the following methods to implement locking operations:

Add for update after the query statement, and the database will add an exclusive lock to the database table during the query. (again, when adding locks, the InnoDB engine will use row-level locks only when retrieving through indexes, otherwise table-level locks will be used. If we want to use row-level locks here, we need to add an index to method_name. It is worth noting that this index must be created as a unique index, otherwise there will be problems that multiple overloaded methods cannot be accessed at the same time. If the method is overloaded, it is recommended to add the parameter type as well. ). When an exclusive lock is added to a record, other threads can no longer add an exclusive lock to that record.

We can assume that the thread that acquires the exclusive lock can acquire the distributed lock. After acquiring the lock, we can execute the business logic of the method, and then unlock it by the following methods:

The lock is released through the connection.commit () operation.

This method can effectively solve the problems mentioned above that can not release locks and block locks.

Blocking lock? The for update statement returns immediately after a successful execution and remains blocked when the execution fails until it succeeds.

After locking, the service goes down and cannot be released? In this way, the database releases the lock itself after the service goes down.

However, it is still impossible to solve the database single point and reentrant problems directly.

There may be another problem here, although we use a unique index on method_name and show the use of for update to use row-level locks. However, MySql optimizes the query, even if index fields are used in the condition, but it is up to MySQL to determine the cost of retrieving data by determining the cost of different execution plans. If MySQL thinks full table scanning is more efficient, such as for small tables, it will not use indexes, in which case InnoDB will use table locks instead of row locks. It would be a tragedy if this happened.

Another problem is that if we use exclusive locks for distributed locking lock, an exclusive lock will occupy the database connection if it is not committed for a long time. Once there are more similar connections, it is possible to burst the database connection pool.

3. Summary of distributed Lock based on Database

Summarize the ways of using the database to realize the distributed lock, both of which rely on a table of the database, one is to determine whether there is a lock by the existence of the records in the table, and the other is to realize the distributed lock through the exclusive lock of the database.

The advantages of distributed Lock based on Database

Directly with the help of the database, easy to understand.

The disadvantage of realizing distributed Lock in Database

There will be a variety of problems, which will make the whole solution more and more complex in the process of solving the problem.

Operating the database requires a certain amount of overhead, and performance issues need to be considered.

Using row-level locks on a database is not necessarily reliable, especially when our lock table is not large.

Implementation of distributed Lock based on Cache

Compared with the scheme of distributed lock based on database, the implementation based on cache will perform better in terms of performance. And many caches can be deployed in clusters to solve a single point of problem.

There are many mature cache products, including Redis,memcached and Tair within our company.

This paper takes Tair as an example to analyze the scheme of using cache to realize distributed lock. There are many articles about Redis and memcached on the network, and there are also some mature frameworks and algorithms that can be used directly.

The implementation of distributed lock based on Tair is actually similar to Redis, in which the main implementation method is to use TairManager.put method.

There are also several problems with the above implementation:

1. There is no expiration time for this lock. Once the unlocking operation fails, the lock record will always be in the tair, and other threads will no longer be able to obtain the lock.

2. This lock can only be non-blocking and will be returned directly regardless of success or failure.

3. The lock is non-reentrant. After a thread acquires the lock, it cannot acquire the lock again before releasing the lock, because the key used already exists in the tair. The put operation can no longer be performed.

Of course, there are also ways to solve it.

No expiration time? The put method of tair supports passing in the expiration time, and the data will be deleted automatically after the arrival time.

Non-blocking? While repeated execution.

Non-reentrant? After a thread acquires the lock, it saves the current host information and thread information, and checks whether it is the owner of the current lock before getting it next time.

However, how long should I set the expiration time? How to set the failure time is too short, the method is not finished, the lock is automatically released, then concurrency problems will arise. If you set it for too long, other threads that acquire the lock may have to wait a little longer. This problem also exists in using databases to implement distributed locks.

Cache implementation of distributed lock summary

Caching can be used instead of databases to implement distributed locks, which provides better performance, while many caching services are deployed in clusters to avoid a single point of problem. And many caching services provide methods that can be used to implement distributed locks, such as Tair's put method, redis's setnx method and so on. In addition, these caching services also provide support for automatic deletion of expired data, and you can directly set the timeout to control the release of locks.

Advantages of caching to implement distributed locks

The performance is good, and it is convenient to realize.

Disadvantages of caching distributed locks

It is not very reliable to control the failure time of the lock through the timeout.

Implementation of distributed Lock based on Zookeeper

Distributed locking based on zookeeper temporary ordered nodes.

The general idea is that when each client locks a method, a unique instantaneous ordered node is generated under the directory of the specified node corresponding to the method on the zookeeper. The way to determine whether or not to acquire a lock is simple, just to judge the one with the lowest sequence number in the ordered node. When the lock is released, you only need to delete the instantaneous node. At the same time, it can avoid the deadlock problem caused by the unreleased lock caused by service downtime.

Let's see if Zookeeper can solve the problem mentioned earlier.

The lock can't be released? Using Zookeeper can effectively solve the problem that the lock cannot be released, because when the lock is created, the client will create a temporary node in the ZK. Once the client suddenly dies after acquiring the lock (the Session connection is broken), then the temporary node will be deleted automatically. Other clients can acquire the lock again.

Non-blocking lock? The blocking lock can be implemented using Zookeeper. The client can create a sequential node in ZK and bind a listener on the node. If the node changes, Zookeeper will notify the client. The client can check whether the node created by itself has the lowest sequence number of all the nodes. If so, it can acquire the lock and execute the business logic.

Non-reentrant? Using Zookeeper can also effectively solve the problem of non-reentrant. When creating a node, the client writes the host information and thread information of the current client directly to the node, and the next time you want to acquire the lock, compare it with the data in the smallest node. If the information is the same as your own, then you get the lock directly, and if it is different, create a temporary sequential node to participate in the queue.

Single point problem? Using Zookeeper can effectively solve the single point problem. ZK is deployed in a cluster. As long as more than half of the machines in the cluster survive, it can provide services.

You can directly use the zookeeper third-party library Curator client, which encapsulates a reentrant lock service.

The InterProcessMutex provided by Curator is an implementation of distributed locks. The acquire method user acquires the lock, and the release method is used to release the lock.

Distributed locks implemented using ZK seem to meet all the expectations we had for a distributed lock at the beginning of this article. However, it is not. The drawback of distributed locks implemented by Zookeeper is that the performance may not be as high as that of caching services. Because every time in the process of creating and releasing locks, we have to dynamically create and destroy instantaneous nodes to achieve the lock function. The creation and deletion of nodes in ZK can only be performed through the Leader server, and then the data is not available on all Follower machines.

In fact, using Zookeeper can also cause concurrency problems, but it's not common. Considering this situation, due to network jitter, the session connection of the client to the ZK cluster is disconnected, so zk thinks that if the client is dead, the temporary node will be deleted, and other clients can acquire the distributed lock. Concurrency problems may arise. This problem is not common because zk has a retry mechanism. Once the zk cluster cannot detect the heartbeat of the client, it will retry. Curator clients support multiple retry strategies. The temporary node will only be deleted if it does not work after many retries. Therefore, it is also important to choose an appropriate retry strategy to find a balance between lock granularity and concurrency. )

Zookeeper implementation of distributed Lock Summary

Advantages of Zookeeper in implementing distributed Lock

Effectively solve the single point problem, non-reentrant problem, non-blocking problem and lock can not be released. It is relatively simple to implement.

Shortcomings of Zookeeper in implementing distributed Lock

In terms of performance, it is better to use caching to implement distributed locks. You need to understand the principles of ZK.

Comparison of three schemes

None of the above methods can be perfect. Like CAP, complexity, reliability, performance and other aspects can not be satisfied at the same time, so it is king to choose the most suitable for you according to different application scenarios.

1. From the point of view of the difficulty of understanding (from low to high)

Database > cache > Zookeeper

two。 From the perspective of implementation complexity (from low to high)

Zookeeper > = cache > database

3. From a performance perspective (from high to low)

Cache > Zookeeper > = database

4. From the perspective of reliability (from high to low)

Zookeeper > cache > database

The following is the advanced information on the architecture, which requires free learning permissions.

This is the answer to the question about the database, cache and Zookeeper implementation in the distributed lock. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.