Shulou(Shulou.com)05/31 Report--

In this article, the editor introduces in detail "how to achieve distributed locks based on curator". The content is detailed, the steps are clear, and the details are handled properly. I hope this article "how to achieve distributed locks based on curator" can help you solve your doubts.

First, what is it?

1. Application scenarios of locks:

In a single application, we will use ReentrantLock or Synchronized to deal with concurrent scenarios. For example, in the most common ticket selling scenario, if there are a total of 100 tickets, thread An and thread B operate at the same time, as shown below:

JMM memory model

At this time, there is a shared variable 100, and threads An and B copy 100 into their working memory. When thread A grabs the right to execute, the value in A's working memory is 100, and then it sells tickets and subtracts itself. Change the value in your working memory to 99. When A doesn't have time to brush 99 back into main memory, thread B comes in, and B gets the value of 100 main memory, and then sells tickets and subtracts itself, which is also 99. As a result, the same ticket was sold twice. So we will add locks and volatile to ensure atomicity and visibility.

2. What is a distributed lock?

In the above scenario, we can do this through ReentrantLock or Synchronized, because your project runs on only one server, there is only one JVM, and all the shared variables are loaded into the same main memory. In distributed applications, a project is deployed on multiple servers, and the basic architecture is shown below:

The simplest distributed architecture

For example, now server1, server2 and server3 all read 100 votes to the database. In each server, we can use the lock of JDK to ensure that there will be no problems when multiple users access my server at the same time. But the problem is, if client1 accesses server1, the number of votes is 100, and then buy tickets, it is not too late to change the number of votes in the database to 99MaginClient2 and also start to access the system to purchase tickets. If client2 visits server1, there will be no problem. If you visit server2, then server2 reads the number of votes in the database is still 100, then there will be a problem, and the same ticket has been sold twice. In distributed applications, the lock mechanism of JDK can not meet the requirements, so distributed locks appear.

3. The conditions that the distributed lock should meet:

Four ones: the same method can only be executed by one thread of one machine at a time. There are three features: reentrant feature; lock failure mechanism to prevent deadlock; non-blocking lock feature, that is, no lock acquisition, return lock acquisition failure, instead of waiting for two high performance acquisition and release locks, and high availability acquisition and release locks.

4. The implementation of distributed lock:

Database-based: implementation based on redis using database exclusive locks: using redis's set key value NX EX 30000; it can also be implemented using redis's third-party libraries such as Redisson based on zookeeper: using zookeeper's temporary sequential nodes; or using zookeeper's third-party libraries such as Curator II, based on database

1. Create a table:

CREATE TABLE `tb_distributed_ lock` (

`dl_ id`INT NOT NULL auto_increment COMMENT 'primary key, self-increment'

`dl_method_ name`VARCHAR (64) NOT NULL DEFAULT''COMMENT' method name'

`Info` VARCHAR NOT NULL DEFAULT''COMMENT' ip+ thread id'

`time of dl_operate_ time`TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT 'when the data is manipulated

PRIMARY KEY (`dl_ id`)

UNIQUE KEY `uq_method_ name` (`dl_method_ name`) USING BTREE

) ENGINE = INNODB DEFAULT charset = utf8 COMMENT = 'distributed lock table'

2. Train of thought:

When executing a method, we first try to insert a piece of data into the table. If the insertion is successful, the lock is successful, continue to execute, and delete the record. If the insertion fails, we query the current method name, the current machine ip+ thread id, and the operation time of the data within 5 minutes (5 minutes represents the time when the lock expires). If there is a record, it means that the thread of the machine has occupied the lock within 5 minutes, and the last record will be deleted directly. If there is no record, the lock will fail. A user is a thread, so we can combine machine ip and user id as dl_device_info.

3. Possession lock and release lock:

Possession lock: INSERT INTO tb_distributed_lock (

Dl_method_name

Dl_device_info

)

VALUES

('method name', 'ip& user id')

If insert fails, then:

SELECT

Count (*)

FROM

Tb_distributed_lock

WHERE

Dl_method_name = 'method name'

AND dl_device_info = 'ip& user id'

AND dl_operate_time

< SYSDATE() - 5; 释放锁：DELETE FROM tb_distributed_lock WHERE dl_method_name = '方法名' AND dl_device_info = 'ip&用户id'; 4、小总结： 以上表结构可能并不是很好，只是提供了这么一个思路。下面说它的优缺点： 优点：成本低，不需要引入其他的技术缺点：对数据库依赖性强，如果数据库挂了，那就凉凉了，所以数据库最好也是高可用的三、基于redis实现 1、原理： 基于redis的set key value nx ex 30，这条语句的意思就是如果key不存在就设置，并且过期时间为30s，如果key已经存在就会返回false。如果要以毫秒为单位，把ex换成px就好了。我们执行方法前，先将方法名当成key，执行这条语句，如果执行成功就是获取锁成功，执行失败就是获取锁失败。 2、代码实现： RedisUtil的部分代码：/** * key不存在时就设置，返回true，key已存在就返回false * @param key * @param value * @param timeout * @return */ public static boolean setIfAbsent(String key, String value, Long timeout) { return redisTemplate.opsForValue().setIfAbsent(key, value, timeout, TimeUnit.SECONDS); } /** * 获取key-value * @param key * @return */ public static String getString(String key) { return (String) redisTemplate.opsForValue().get(key); } /** * 删除key * @param key * @return */ public static boolean delKey(String key) { return redisTemplate.delete(key); } 业务方法中使用：public String hello() { // 方法名当作key String key = "hello"; String value = "hellolock"; if (RedisUtil.setIfAbsent(key, value, 60 * 2L)) { System.out.println("成功获取到锁，开始执行业务逻辑……"); // 假如执行业务逻辑需要1分钟 try {TimeUnit.MINUTES.sleep(1L); } catch (Exception e) { e.printStackTrace();}; // 释放锁先校验value，避免释放错 if (value.equals(RedisUtil.getString(key))) { RedisUtil.delKey(key); System.out.println("执行完业务逻辑，释放锁成功"); } return "success"; } else { System.out.println("锁被别的线程占有，获取锁失败"); return "acquire lock failed"; } } 3、小总结： 优点：简单易用，一条redis命令就搞定。可以设置过期时间，避免释放锁失败造成其他线程长时间无法获取锁的问题。 缺点：这种做法只适合redis是单机的时候，如果redis有集群，这样做就会出问题。假如一个线程在master上获取锁成功了，在master还没来得及将数据同步到slave上的时候，master挂了，slave升级为master。第二个线程进来尝试获取锁，因为新的master上并没有这个key，所以，也能成功获取到锁。 解决办法：针对上面的缺点，我们可以采用redis的RedLock算法。假如集群中有n个redis，我们先从这n个redis中尝试获取锁（锁的过期时间为x），并记录获取锁的消耗的总时间t，获取锁成功数量为s，当且仅当t < x 并且 s >

When = (nmax 2 + 1), the lock is considered to be acquired successfully.

4. Implementation based on Redisson

1. What is it?

Official website address: https://github.com/redisson/redisson/wiki/Table-of-Content Redisson is a very powerful redis client that encapsulates many distributed operations, such as distributed objects, distributed collections, distributed locks, and so on. It also has a lot of distributed locks, such as fair locks, reentrant locks, redlock, and so on. Let's take a look at how to use it in springboot projects.

2. Use redisson as a distributed lock:

Add dependencies:

Org.redisson

Redisson-spring-boot-starter

3.12.3

Io.netty

Netty-all

Application.yml:spring:

Application:

Name: distributed-lock

Redis:

# how to write stand-alone version of redis

Host: 192.168.2.43

Port: 6379

# how to write a cluster

# cluster:

# nodes:

#-192.168.0.106192.168.0.107

# how to write Sentinel

# sentinel:

# master: 192.168.0.106

# nodes:

#-192.168.0.107192.168.0.108

Usage: directly inject RedissonClient, and then use it to acquire the lock. After getting the lock, you can occupy the lock and release the lock. There are blocking locks and non-blocking locks. The specific usage is as follows: @ Autowired

Private RedissonClient redisson

/ * *

* if the expiration time is not set, it will always be blocked if you don't get it.

* @ return

, /

@ GetMapping ("/ testLock")

Public String testLock () {

Log.info ("enter the testLock method and start acquiring locks")

String key = "testLock"

RLock lock = redisson.getLock (key)

Lock.lock ()

Log.info ("acquire the lock successfully, start executing the business logic …")

Try {TimeUnit.SECONDS.sleep (10L);} catch (Exception e) {e.printStackTrace ();}

Log.info ("release the lock after executing the business logic")

Lock.unlock ()

Return "success"

}

/ * *

* if you try to acquire the lock, it will fail if you don't get it, and it won't block.

* @ return

, /

@ GetMapping ("/ testTryLock")

Public String testTryLock () {

Log.info ("enter the testTryLock method and start acquiring locks")

String key = "testTryLock"

RLock lock = redisson.getLock (key)

Boolean res = lock.tryLock ()

If (! res) {

Log.error ("attempt to acquire lock failed")

Return "fail"

} else {

Log.info ("acquire the lock successfully, start executing the business logic …")

Try {TimeUnit.SECONDS.sleep (30L);} catch (Exception e) {e.printStackTrace ();}

Log.info ("release the lock after executing the business logic")

Lock.unlock ()

Return "success"

}

}

/ * *

* Lock set expiration time. Even if the last unlock fails, the lock will be released automatically after 20 seconds.

* @ return

, /

@ GetMapping ("/ testLockTimeout")

Public String testLockTimeout () {

Log.info ("enter the testLockTimeout method and start acquiring locks")

String key = "testLockTimeout"

RLock lock = redisson.getLock (key)

/ / release the lock automatically after 20 seconds

Lock.lock (20, TimeUnit.SECONDS)

Log.info ("acquire the lock successfully, start executing the business logic …")

Try {TimeUnit.SECONDS.sleep (10L);} catch (Exception e) {e.printStackTrace ();}

Lock.unlock ()

Return "success"

}

/ * *

* an attempt to acquire a lock failed before it was acquired for 15 seconds. If acquired, the lock will be held for 20 seconds, and the lock will be released automatically after 20 seconds.

* @ return

, /

@ GetMapping ("/ testTryLockTimeout")

Public String testTryLockTimeout () {

Log.info ("enter the testTryLockTimeout method and start acquiring locks")

String key = "testTryLockTimeout"

RLock lock = redisson.getLock (key)

Boolean res = false

Try {

Res = lock.tryLock (15,20, TimeUnit.SECONDS)

} catch (InterruptedException E1) {

E1.printStackTrace ()

}

If (! res) {

Log.error ("attempt to acquire lock failed")

Return "fail"

} else {

Log.info ("acquire the lock successfully, start executing the business logic …")

Try {TimeUnit.SECONDS.sleep (10L);} catch (Exception e) {e.printStackTrace ();}

Log.info ("release the lock after executing the business logic")

Lock.unlock ()

Return "success"

}

}

3. Small summary:

The above is a simple demo that uses redisson to do distributed locks, which is very convenient to use. The above is integrated with the springboot project, so just use the starter of springboot provided by it. For more uses of it for distributed locks, please go to the official website: redisson distributed locks.

5. Implementation based on zookeeper

1. Review of zookeeper knowledge points:

Zookeeper has four types of nodes:

Persistent node: the default node type. After the client disconnects from zookeeper, the node still exists.

Persistent order nodes: the first is the persistent nodes, which means that zookeeper will number according to the order in which the nodes are created

Temporary node: the node no longer exists after the client is disconnected from zookeeper

Temporary sequential node: the node no longer exists after the client is disconnected from zookeeper, and the zookeeper will number according to the order in which the node is created

2. The principle of distributed lock based on zookeeper.

We use the temporary sequential node of zookeeper to implement distributed locking. First, let's create a persistent node called lock (node name is arbitrary). When thread 1 acquires the lock, it creates a temporary sequential node named lock1 under lock, and then finds all the nodes under lock to determine whether its lock1 is the first. If so, acquire the lock successfully, continue to execute the business logic, and delete the lock1 node after execution. If it is not the first one and the acquisition of the lock fails, watch the node in front of you. When the node in front of you is killed, check whether you are in the first place, and if so, acquire the lock successfully. The illustration process is as follows:

Principle of zookeeper distributed Lock

Thread 1 creates a lock1 and finds that the first node of the lock1 occupies the lock successfully; before thread 1 releases the lock, thread 2 comes and creates a lock2, and monitors the lock1 when it finds that lock2 is not the first, and thread 3 monitors the lock2 at this time. You don't occupy the lock successfully until you are the first node. It doesn't matter if the zookeeper collapses when a thread releases the lock, because it is a temporary node, the disconnected node is gone, and other threads can still acquire the lock normally, which is why temporary nodes are used.

To clarify the principle, it is not difficult to achieve with code, you can introduce the client side of zookeeper zkClient, write your own code implementation (lazy, do not write their own, interested can refer to my zookeeper article, certainly can write their own). However, there are excellent open source solutions such as curator, so let's take a look at how curator works.

6. Implementation based on curator

1. Springboot integrates curator:

Pom.xml:

Org.apache.zookeeper

Zookeeper

3.4.14

Org.apache.curator

Curator-framework

4.2.0

Org.apache.curator

Curator-recipes

4.2.0

Org.seleniumhq.selenium

Selenium-java

Application.yml: note that the following attributes spring of curator are not integrated, that is, there is no prompt for curator when writing:

RetryCount: the number of retries that failed in the 5 # connection

RetryTimeInterval: 5000 # retry every 5 seconds

Url: 192.168.2.43 2181 # zookeeper connection address

SessionTimeout: 60000 # session timeout 1 minute

ConnectionTimeout: 5000 # connection timeout 5 seconds

Configuration class: read the properties in application.yml and create an CuratorFramework instance @ Configuration

Public class CutatorConfig {

@ Value ("${curator.retryCount}")

Private Integer retryCount

@ Value ("${curator.retryTimeInterval}")

Private Integer retryTimeInterval

@ Value ("${curator.url}")

Private String url

@ Value ("${curator.sessionTimeout}")

Private Integer sessionTimeout

@ Value ("${curator.connectionTimeout}")

Private Integer connectionTimeout

@ Bean

Public CuratorFramework curatorFramework () {

Return CuratorFrameworkFactory.newClient (url, sessionTimeout, connectionTimeout

New RetryNTimes (retryCount, retryTimeInterval))

}

}

Test class: test whether the integration of the curator framework is successful @ SpringBootTest (classes = {DistributedLockApplication.class})

@ RunWith (SpringRunner.class)

Public class DistributedLockApplicationTests {

@ Autowired

Private CuratorFramework curatorFramework

@ Test

Public void contextLoads () {

CuratorFramework.start ()

Try {

CuratorFramework.create () .creatingParentContainersIfNeeded () .withMode (CreateMode.PERSISTENT) .forPath ("/ zhusl", "test" .getBytes ())

} catch (Exception e) {

E.printStackTrace ()

}

}

}

After ensuring that zookeeper starts successfully, run this unit test, and finally go back to linux and connect with zkCli.sh to see if the node has been created successfully.

2. Use Curator as a distributed lock:

Curator encapsulates many locks, such as reentrant shared lock, non-reentrant shared lock, reentrant read-write lock, interlock and so on. For more information, please refer to the official website: the usage of curator distributed locks.

ZookeeperUtil.java: tool class, encapsulating methods to acquire locks, release locks, etc. Here is a simple package of the four locks mentioned above, for reference only. @ Component

@ Slf4j

Public class ZookeeperUtil {

Private static CuratorFramework curatorFramework

Private static InterProcessLock lock

/ * * persistent node * /

Private final static String ROOT_PATH = "/ lock/"

/ * * shared locks can be reentered * /

Private static InterProcessMutex interProcessMutex

/ * do not reenter the shared lock * /

Private static InterProcessSemaphoreMutex interProcessSemaphoreMutex

/ * * read-write locks can be reentered * /

Private static InterProcessReadWriteLock interProcessReadWriteLock

/ * * multiple shared locks (use multiple locks as one) * /

Private static InterProcessMultiLock interProcessMultiLock

@ Autowired

Private void setCuratorFramework (CuratorFramework curatorFramework) {

ZookeeperUtil.curatorFramework = curatorFramework

ZookeeperUtil.curatorFramework.start ()

}

/ * *

* acquire reentrant exclusive locks

*

* @ param lockName

* @ return

, /

Public static boolean interProcessMutex (String lockName) {

InterProcessMutex = new InterProcessMutex (curatorFramework, ROOT_PATH + lockName)

Lock = interProcessMutex

Return acquireLock (lockName, lock)

}

/ * *

* acquire non-reentrant exclusive locks

*

* @ param lockName

* @ return

, /

Public static boolean interProcessSemaphoreMutex (String lockName) {

InterProcessSemaphoreMutex = new InterProcessSemaphoreMutex (curatorFramework, ROOT_PATH + lockName)

Lock = interProcessSemaphoreMutex

Return acquireLock (lockName, lock)

}

/ * *

* acquire a reentrant read lock

*

* @ param lockName

* @ return

, /

Public static boolean interProcessReadLock (String lockName) {

InterProcessReadWriteLock = new InterProcessReadWriteLock (curatorFramework, ROOT_PATH + lockName)

Lock = interProcessReadWriteLock.readLock ()

Return acquireLock (lockName, lock)

}

/ * *

* acquire a reentrant write lock

*

* @ param lockName

* @ return

, /

Public static boolean interProcessWriteLock (String lockName) {

InterProcessReadWriteLock = new InterProcessReadWriteLock (curatorFramework, ROOT_PATH + lockName)

Lock = interProcessReadWriteLock.writeLock ()

Return acquireLock (lockName, lock)

}

/ * *

* acquire the interlock (use the lock as one)

* @ param lockNames

* @ return

, /

Public static boolean interProcessMultiLock (List lockNames) {

If (lockNames = = null | | lockNames.isEmpty ()) {

Log.error ("no lockNames found")

Return false

}

InterProcessMultiLock = new InterProcessMultiLock (curatorFramework, lockNames)

Try {

If (! interProcessMultiLock.acquire (10, TimeUnit.SECONDS)) {

Log.info ("Thread:" + Thread.currentThread (). GetId () + "acquire distributed lock fail")

Return false

} else {

Log.info ("Thread:" + Thread.currentThread (). GetId () + "acquire distributed lock success")

Return true

}

} catch (Exception e) {

Log.info ("Thread:" + Thread.currentThread (). GetId () + "release lock occured an exception =" + e)

Return false

}

}

/ * *

* release the lock

*

* @ param lockName

, /

Public static void releaseLock (String lockName) {

Try {

If (lock! = null & & lock.isAcquiredInThisProcess ()) {

Lock.release ()

CuratorFramework.delete (). InBackground (). ForPath (ROOT_PATH + lockName)

Log.info ("Thread:" + Thread.currentThread (). GetId () + "release lock success")

}

} catch (Exception e) {

Log.info ("Thread:" + Thread.currentThread (). GetId () + "release lock occured an exception =" + e)

}

}

/ * *

* release the interlock

, /

Public static void releaseMultiLock (List lockNames) {

Try {

If (lockNames = = null | | lockNames.isEmpty ()) {

Log.error ("no no lockNames found to release")

Return

}

If (interProcessMultiLock! = null & & interProcessMultiLock.isAcquiredInThisProcess ()) {

InterProcessMultiLock.release ()

For (String lockName: lockNames) {

CuratorFramework.delete (). InBackground (). ForPath (ROOT_PATH + lockName)

}

Log.info ("Thread:" + Thread.currentThread (). GetId () + "release lock success")

}

} catch (Exception e) {

Log.info ("Thread:" + Thread.currentThread (). GetId () + "release lock occured an exception =" + e)

}

}

/ * *

* acquire the lock

*

* @ param lockName

* @ param interProcessLock

* @ return

, /

Private static boolean acquireLock (String lockName, InterProcessLock interProcessLock) {

Int flag = 0

Try {

While (! interProcessLock.acquire (2, TimeUnit.SECONDS)) {

Flag++

If (flag > 1) {

Break

}

}

} catch (Exception e) {

Log.error ("acquire lock occured an exception =" + e)

Return false

}

If (flag > 1) {

Log.info ("Thread:" + Thread.currentThread (). GetId () + "acquire distributed lock fail")

Return false

} else {

Log.info ("Thread:" + Thread.currentThread (). GetId () + "acquire distributed lock success")

Return true

}

}

}

ZookeeperLockController.java: write an interface, lock it with Curator, and then access @ RestController with a browser

@ RequestMapping ("/ zookeeper-lock")

Public class ZookeeperLockController {

@ GetMapping ("/ testLock")

Public String testLock () {

/ / acquire the lock

Boolean lockResult = ZookeeperUtil.interProcessMutex ("testLock")

If (lockResult) {

Try {

/ / simulate the execution of business logic

TimeUnit.MINUTES.sleep (1L)

} catch (InterruptedException e) {

E.printStackTrace ()

}

/ / release the lock

ZookeeperUtil.releaseLock ("testLock")

Return "success"

} else {

Return "fail"

}

}

}

Open a browser window to access, and print out the log of successful lock acquisition in the background. Within 1 minute, open another window to visit again and print out the log of failed lock acquisition, indicating that the distributed lock is in effect.

7. Compared with the schemes to realize the distributed lock, it is the simplest to implement based on the database, and there is no need to introduce third-party applications. However, because each lock and unlock has to be IO operation, the performance is not very good. Based on redis to achieve a more balanced, good performance, it is not very difficult, more reliable. It is difficult to implement based on zookeeper because you need to maintain a zookeeper cluster. If the project does not use zookeeper, it is better to use redis. After reading this, the article "how to implement distributed locks based on curator" has been introduced. If you want to master the knowledge points of this article, you still need to practice and use it yourself. If you want to know more about related articles, welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.