Shulou(Shulou.com)06/02 Report--

This article mainly explains "the principle and implementation method of java distributed lock". The explanation content in this article is simple and clear, easy to learn and understand. Please follow the idea of Xiaobian and go deep into it slowly to study and learn "the principle and implementation method of java distributed lock" together!

Those who have learned Java multithreading should know what a lock is, and those who have not learned do not have to worry. The lock in Java can be simply understood as a thread synchronization mechanism for accessing critical resources in multithreaded situations.

In the process of learning or using Java, the process will encounter a variety of locking concepts: fair locks, unfair locks, spin locks, reentrant locks, biased locks, lightweight locks, heavyweight locks, read-write locks, mutex locks, etc. Java Multithreading Core Technology

Why use distributed locks?

When we are developing applications, if we need to access a shared variable multi-threaded synchronization, we can use the Java multi-threaded 18 martial arts we learned to handle, and can run perfectly, no bugs!

Note that this is a standalone application, that is, all requests will be allocated to the JVM of the current server, and then mapped to the threads of the operating system for processing! And this shared variable is just a piece of memory space inside this JVM!

Later, business development required clustering. An application needed to be deployed to several machines and then Load Balancer was performed, roughly as shown in the following figure:

As can be seen in the above figure, variable A exists in JVM1, JVM2 and JVM3 (this variable A is mainly embodied as a member variable in a class, which is an object with state, for example, a member variable of plastic type in UserController). If no control is added, variable A will allocate a piece of memory in JVM at the same time. Three requests are sent to operate on this variable at the same time. Obviously, the result is wrong! Even if they are not sent at the same time, the three requests operate on data from three different JVM memory regions respectively. There is no sharing between variables A, and there is no visibility. The processing result is also wrong!

If this scenario exists in our business, we need a way to solve it!

In order to ensure that a method or property can only be executed by the same thread at the same time under high concurrency conditions, in the case of traditional monolithic application stand-alone deployment, Java concurrency processing related APIs (such as ReentrantLock or Synchronized) can be used for mutual exclusion control. In a standalone environment, Java provides many concurrency related APIs. However, with the needs of business development, after the original single-machine deployment system has evolved into a distributed cluster system, because the distributed system is multi-threaded, multi-process and distributed on different machines, this will make the concurrency control locking strategy in the original single-machine deployment case invalid, and the simple Java API cannot provide the ability of distributed locking. To solve this problem, we need a mutual exclusion mechanism across JVMs to control access to shared resources. This is the problem that distributed locks solve!

2. What conditions should distributed locks have

Before examining the three ways distributed locks can be implemented, let's look at what distributed locks should have:

In a distributed system environment, a method can only be executed by one thread of one machine at a time.

2. Highly available acquisition lock and release lock;

3. High-performance acquisition lock and release lock;

4. It has reentrant characteristics;

5, with lock failure mechanism to prevent deadlock;

6, with non-blocking lock characteristics, that is, if you do not obtain the lock, you will directly return to obtain the lock failure.

Three ways to implement distributed locks

At present, almost many large-scale websites and applications are distributed deployment, and data consistency in distributed scenarios has always been an important topic. Distributed CAP theory tells us that "no distributed system can satisfy consistency, availability, and partition tolerance at the same time. "So many systems have to make trade-offs between these three at the beginning of their design. In the vast majority of scenarios in the Internet domain, it is necessary to sacrifice strong consistency in exchange for high availability of the system. The system often only needs to ensure "final consistency" as long as this final time is within the acceptable range of users.

In many scenarios, in order to ensure the final consistency of data, we need a lot of technical solutions to support, such as distributed transactions, distributed locks, etc. Sometimes we need to ensure that a method can only be executed by the same thread at the same time.

Distributed lock based on database;

Implement distributed locks based on cache (Redis, etc.);

Zookeeper Based Distributed Lock

Although there are these three schemes, different businesses should also be selected according to their own circumstances. There is no best choice between them.

IV. Implementation based on database

The core idea of the database-based implementation is to create a table in the database, which contains fields such as method name, and create a unique index on the method name field. If you want to execute a method, you use this method name to insert data into the table. If the insertion is successful, you will obtain the lock. After the execution is completed, delete the corresponding row data and release the lock.

1) Create a table:

DROP TABLE IF EXISTS `method_lock`;

CREATE TABLE `method_lock` (

`id` int(11) unsigned NOT NULL AUTO_INCREMENT COMMENT 'main key',

`method_name` varchar(64) NOT NULL COMMENT 'locked method name',

`desc` varchar(255) NOT NULL COMMENT 'comment information',

`update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,

PRIMARY KEY (`id`),

UNIQUE KEY `uidx_method_name` (`method_name`) USING BTREE

ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8 COMMENT='Method in lock';

(2) To execute a method, insert data into the table using the method name:

INSERT INTO method_lock (method_name, desc) VALUES ('methodName ', ' methodName');

1

Because we have a uniqueness constraint on method_name, where if multiple requests are submitted to the database at the same time, the database guarantees that only one operation will succeed, then we can assume that the thread that succeeds has obtained the lock of the method and can execute the method body content.

(3) If the insertion is successful, the lock will be acquired. After the execution is completed, the corresponding row data will be deleted to release the lock:

delete from method_lock where method_name ='methodName';

1

Note: This is only one way to use a database, there are many other ways to use a database to implement distributed locks!

Using this database-based implementation is simple, but it has a few problems to solve and optimize for what distributed locks should be:

Because it is based on database implementation, the availability and performance of the database will directly affect the availability and performance of distributed locks, so the database needs dual deployment, data synchronization, active and standby switching;

2. It does not have the feature of re-entrant, because the row data always exists before the same thread releases the lock, and it cannot insert data again successfully. Therefore, it is necessary to add a new column in the table to record the machine and thread information that currently obtains the lock. When obtaining the lock again, first query whether the machine and thread information in the table is the same as the current machine and thread. If they are the same, they will directly obtain the lock;

3. There is no lock failure mechanism, because it is possible that after successful insertion of data, the server is down, and the corresponding data is not deleted. When the service is restored, the lock has not been obtained. Therefore, a new column needs to be added to the table to record the failure time, and there needs to be a regular task to clear these invalid data.

4, do not have blocking lock characteristics, do not obtain the lock directly returned failure, so you need to optimize the acquisition logic, loop multiple times to obtain.

5, in the implementation process will encounter a variety of different problems, in order to solve these problems, the implementation will be more and more complex; dependence on the database requires a certain amount of resource overhead, performance issues need to be considered.

V. Redis-based implementation

1. Reasons for choosing Redis to implement distributed locks:

(1) Redis has high performance;

(2) The Redis command supports this well and is more convenient to implement.

2. Introduction to using commands:

（1）SETNX

SETNX val key: sets a string with val key if and only if key does not exist, returns 1; if key exists, does nothing, returns 0.

1

（2）expire

expire key timeout: Set a timeout period for the key, the unit is second, after which the lock will be automatically released to avoid deadlock.

1

（3）delete

delete key: delete key

1

When using Redis to implement distributed locks, these three commands are mainly used.

3. Realization of ideas:

(1) When acquiring the lock, use setnx to lock, and use the expire command to add a timeout time for the lock. If the time exceeds, the lock will be automatically released. The value of the lock is a randomly generated UUID, which is used to judge when the lock is released.

(2) When acquiring the lock, set a timeout time for acquisition. If this time is exceeded, the lock acquisition will be abandoned.

(3) When releasing the lock, judge whether it is the lock by UUID, if it is the lock, execute delete to release the lock.

4. Simple implementation code of distributed lock:

**

* Simple implementation code for distributed locks

* Created by liuyang on 2017/4/20.

*/

public class DistributedLock {

private final JedisPool jedisPool;

public DistributedLock(JedisPool jedisPool) {

this.jedisPool = jedisPool;

}

/**

* locking

* @param lockName lock key

* @param acquireTimeout Get timeout

* @param timeout The timeout of the lock

* @return Lock ID

*/

public String lockWithTimeout(String lockName, long acquireTimeout, long timeout) {

Jedis conn = null;

String retIdentifier = null;

try {

//Get connections

conn = jedisPool.getResource();

//generate a value randomly

String identifier = UUID.randomUUID().toString();

//lock name, i.e. key value

String lockKey = "lock:"> //timeout, after which the lock will be released automatically

int lockExpire = (int) (timeout / 1000);

//Time out for acquiring lock. If this time is exceeded, acquiring lock will be abandoned.

long end = System.currentTimeMillis() + acquireTimeout;

while (System.currentTimeMillis()

< end) { if (conn.setnx(lockKey, identifier) == 1) { conn.expire(lockKey, lockExpire); // 返回value值，用于释放锁时间确认 retIdentifier = identifier; return retIdentifier; } // 返回-1代表key没有设置超时时间，为key设置一个超时时间 if (conn.ttl(lockKey) == -1) { conn.expire(lockKey, lockExpire); } try { Thread.sleep(10); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } } catch (JedisException e) { e.printStackTrace(); } finally { if (conn != null) { conn.close(); } } return retIdentifier; } /** * 释放锁 * @param lockName 锁的key * @param identifier 释放锁的标识 * @return */ public boolean releaseLock(String lockName, String identifier) { Jedis conn = null; String lockKey = "lock:" + lockName; boolean retFlag = false; try { conn = jedisPool.getResource(); while (true) { // 监视lock，准备开始事务 conn.watch(lockKey); // 通过前面返回的value值判断是不是该锁，若是该锁，则删除，释放锁 if (identifier.equals(conn.get(lockKey))) { Transaction transaction = conn.multi(); transaction.del(lockKey); List results = transaction.exec(); if (results == null) { continue; } retFlag = true; } conn.unwatch(); break; } } catch (JedisException e) { e.printStackTrace(); } finally { if (conn != null) { conn.close(); } } return retFlag; } } 5、测试刚才实现的分布式锁 例子中使用50个线程模拟秒杀一个商品，使用-运算符来实现商品减少，从结果有序性就可以看出是否为加锁状态。 模拟秒杀服务，在其中配置了jedis线程池，在初始化的时候传给分布式锁，供其使用。 /** * Created by liuyang on 2017/4/20. */ public class Service { private static JedisPool pool = null; private DistributedLock lock = new DistributedLock(pool); int n = 500; static { JedisPoolConfig config = new JedisPoolConfig(); // 设置最大连接数 config.setMaxTotal(200); // 设置最大空闲数 config.setMaxIdle(8); // 设置最大等待时间 config.setMaxWaitMillis(1000 * 100); // 在borrow一个jedis实例时，是否需要验证，若为true，则所有jedis实例均是可用的 config.setTestOnBorrow(true); pool = new JedisPool(config, "127.0.0.1", 6379, 3000); } public void seckill() { // 返回锁的value值，供释放锁时候进行判断 String identifier = lock.lockWithTimeout("resource", 5000, 1000); System.out.println(Thread.currentThread().getName() + "获得了锁"); System.out.println(--n); lock.releaseLock("resource", identifier); } } 模拟线程进行秒杀服务： public class ThreadA extends Thread { private Service service; public ThreadA(Service service) { this.service = service; } @Override public void run() { service.seckill(); } } public class Test { public static void main(String[] args) { Service service = new Service(); for (int i = 0; i < 50; i++) { ThreadA threadA = new ThreadA(service); threadA.start(); } } } 结果如下，结果为有序的： 若注释掉使用锁的部分： public void seckill() { // 返回锁的value值，供释放锁时候进行判断 //String indentifier = lock.lockWithTimeout("resource", 5000, 1000); System.out.println(Thread.currentThread().getName() + "获得了锁"); System.out.println(--n); //lock.releaseLock("resource", indentifier); } 从结果可以看出，有一些是异步进行的：

5. Implementation based on Zookeeper

ZooKeeper is an open source component that provides consistency services for distributed applications. It is a hierarchical file system directory tree structure that stipulates that there can only be one unique file name under the same directory. The steps to implement distributed locks based on ZooKeeper are as follows:

(1) Create a directory mylock;

(2) Thread A creates temporary sequential nodes in the mylock directory if it wants to acquire locks;

(3) Get all the child nodes under the mylock directory, and then get the sibling node smaller than itself. If it does not exist, it means that the current thread sequence number is the smallest and the lock is obtained.

(4) Thread B acquires all nodes, judges that it is not the smallest node, and sets up to listen to nodes smaller than itself;

(5) Thread A finishes processing and deletes its own node. Thread B listens to the change event and judges whether it is the smallest node. If it is, it obtains the lock.

Curator, an Apache open source library, is a ZooKeeper client. InterProcessMutex provided by Curator is an implementation of distributed locks. The acquire method is used to acquire locks, and the release method is used to release locks.

Advantages: high availability, reentrant, blocking lock features, can solve the failure deadlock problem.

Disadvantages: Because of the frequent creation and deletion of nodes, performance is not as good as Redis.

6. Summary

The above three implementations are not perfect in all cases, so the most suitable implementation should be selected according to different application scenarios.

In a distributed environment, locking resources is sometimes important, such as snapping up a resource, when distributed locks can be used to control resources well.

Of course, in the specific use, there are many factors to consider, such as the selection of timeout time, the selection of lock acquisition time has a great impact on concurrency, the above implementation of distributed locks is only a simple implementation, mainly an idea, including the code in the text may not be applicable to formal production environments, only for introductory reference!

1、https://yq.aliyun.com/articles/60663

2、http://www.hollischuang.com/archives/1716

3、https://www.cnblogs.com/liuyang0/p/6744076.html

Thank you for reading, the above is the "java distributed lock principle and implementation method" content, after the study of this article, I believe we have a deeper understanding of java distributed lock principle and implementation method this problem, the specific use of the situation also needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.