Shulou(Shulou.com)06/01 Report--

What is the idea of using ZooKeeper to realize the distributed lock optimization of Java cross-JVM? aiming at this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Problem: we all know that the mechanism for implementing locks within a single JVM is very convenient, and Java also provides a wealth of API implementations, such as Synchronized keywords, ReentrantLock, etc., but in a cluster environment, multiple JVM work together, so the distributed locks described above should be used when some global locks are needed. But the disadvantage of this kind of lock is that every time the client (the client here can be understood as a thread in a different JVM) needs to interact with the zook server to create a znode and wait for himself to acquire the lock, this kind of network communication will undoubtedly bring some pressure to the server, so is there any way to reduce this pressure?

Scenario: a very common scenario is to update the cache, so our general processing logic is as follows.

1. It is preferred to obtain resources according to key, and if resources exist, use them.

2. If it does not exist, apply for a lock (using a shared lock).

3. After acquiring the lock, again determine whether the resource exists (to prevent repeated updates). If it exists, someone has updated it, and the method exits, otherwise, update the cache.

4. release the lock.

Suppose there are 10 (1-10) threads executing the appeal logic at the same time, and if the resource does not exist, they will all perform step (2) to acquire the lock. At the same time, only one thread will acquire the lock, and the other nine threads will block. Waiting for the lock. Now let's assume that thread 1 acquires the lock and starts to perform step (3-4). In step (3), it determines again whether the resource exists (certainly not because it is the first to go in), so it is responsible for loading the resource into the cache, and then releasing the lock, and then other threads (2-10) acquire the lock in turn, and then perform the (3J4) action. Again, determine whether the resource exists (it already exists, because thread 1 is already in it), so they just exit and release the lock. So it makes sense that only thread 1 acquires the lock, but they all need to communicate with the zook on the network, which puts pressure on the network.

If we have two servers called A _ Magi B for clustering, and the requests for locks from these 10 threads come from these two servers respectively, for example, threads (1-5) come from server An and threads (6-10) come from server B, then they interact with zk 10 times and create 10 znode to apply for locks, but if we do some optimization, they only need to interact with zk twice. Let's modify the above logic.

1. It is preferred to obtain resources according to key, and if resources exist, use them.

2. If it does not exist, apply for a lock (a lock within the JVM process).

3. Get the lock (lock in the JVM process), determine whether the resource exists again, and exit if the resource exists, which is no good.

4. If the resource does not exist, apply for a lock (distributed lock).

5. Get the lock (distributed lock) again to determine whether the resource exists (to prevent repeated updates). If it exists, someone has updated it, the method exits, otherwise the cache is updated.

6. Release distributed locks and locks within the JVM process.

Code: I put the implementation logic together, it is convenient to demonstrate to you, if there are logic errors, you are welcome to comment and correct.

Package com.framework.code.demo.zook;import java.util.HashMap;import java.util.Map;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.TimeUnit;import java.util.concurrent.locks.ReentrantLock;import org.apache.curator.framework.CuratorFramework;import org.apache.curator.framework.CuratorFrameworkFactory;import org.apache.curator.framework.recipes.locks.InterProcessMutex;import org.apache.curator.retry.ExponentialBackoffRetry;import com.framework.code.demo.zook.lock.NoFairLockDriver Public class Main {/ / We use a static map to simulate a third-party independent cache public static Map redis = new HashMap (); public static final String key = "redisKey"; public static void main (String [] args) throws InterruptedException {/ / create two objects to simulate two processes RedisProcess processA = new RedisProcess () RedisProcess processB = new RedisProcess (); / / each process uses 50 threads to request concurrently ExecutorService service = Executors.newFixedThreadPool; for (int I = 0; I < 50; iSuppli +) {service.execute (processA); service.execute (processB) } service.shutdown (); service.awaitTermination (30, TimeUnit.SECONDS);} public static class RedisProcess implements Runnable {CuratorFramework client; / / ZK distributed lock InterProcessMutex distributedLock; / / JVM internal lock ReentrantLock jvmLock Public RedisProcess () {client = CuratorFrameworkFactory.newClient ("192.168.1.18purl 2181", new ExponentialBackoffRetry (1000Magne3)); client.start (); distributedLock = new InterProcessMutex (client, "/ mylock", new NoFairLockDriver ()) JvmLock = new ReentrantLock () } @ Override public void run () {/ / (1) first determine whether the resource in the cache exists if (redis.get (key) = = null) {try { / / the purpose of the delay of 1000 milliseconds here is to prevent threads from updating resources too quickly. Then the other threads return true at step (1). Thread.sleep (1000); / / acquire JVM lock (valid within the same process) jvmLock.lock () / / (2) again determine whether the resource already exists if (redis.get (key) = = null) {System.out.println ( "Thread:" + Thread.currentThread () + "gets the JVM lock Redis.get (key) is empty, ready to acquire ZK lock ") / / the purpose of the 500ms delay here is to prevent threads from updating resources too quickly, and other threads return true in step (2). Thread.sleep; try {/ / acquire zk distributed lock distributedLock.acquire () System.out.println ("Thread:" + Thread.currentThread () + "acquired the JVM lock, redis.get (key) is empty, acquired the ZK lock") / / again, if it is empty, you can update the resource if (redis.get (key) = = null) {redis.put (key) Thread.currentThread () + "updated cache") System.out.println ("Thread:" + Thread.currentThread () + "updated cache") } else {System.out.println ("Thread:" + Thread.currentThread () + "current resource already exists and does not need to be updated") }} catch (Exception e) {e.printStackTrace () } finally {/ / release ZK lock try {distributedLock.release () } catch (Exception e) {e.printStackTrace () } else {System.out.println ("Thread:" + Thread.currentThread () + "acquires the JVM lock Redis.get (key) is not empty, "+ redis.get (key)) }} catch (InterruptedException e) {e.printStackTrace () } finally {/ / release JVM lock jvmLock.unlock () }} else {System.out.println (redis.get (key));}}

Thread: thread [pool-5murthreadly2jin5main] acquires JVM lock, redis.get (key) is empty, ready to acquire ZK lock

Thread: thread [pool-5 ZK thread 3Jing 5main] acquires the JVM lock, and redis.get (key) is empty, ready to acquire the thread lock

Thread: thread [pool-5 ZK thread 3Jing 5 main] gets the JVM lock, redis.get (key) is empty, and gets the thread lock.

Thread: Thread [pool-5murthreadly3jin5main] updated the cache

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: thread [pool-5murthreadly19jin5main] got the JVM lock, redis.get (key) is not empty, threadpool-5Muthreadly3d5main] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: thread [pool-5murthreadly31jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by thread [pool-5murthreadly27d5main], redis.get (key) is not empty, thread [pool-5murthreadly3jin5main] updated the cache

Thread: threaded [pool-5murthreadly25d5main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3d5main] updated the cache.

Thread: thread [pool-5murthreadly3re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly37pj5remain] got the JVM lock, redis.get (key) is not empty, Thread.Thread [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: Thread [pool-5murthreadly2jin5main] gets the JVM lock, redis.get (key) is empty, and gets the ZK lock

Thread: Thread [pool-5murthreadly2jin5main] the current resource already exists and does not need to be updated

Thread: thread [pool-5murthreadly21jin5jimmain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly29jin5main] got the JVM lock, redis.get (key) is not empty, thread [pool-5Muthreadly3jin5main] updated the cache.

Thread: threaded [pool-5murthreadly55hero5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly5jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly41jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly67jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly39mem5main] got the JVM lock, redis.get (key) is not empty, Thread.Thread [pool-5murthreadly3re5dmain] updated the cache.

Thread: Thread [pool-5murthreadly43jin5remain] got the JVM lock, redis.get (key) is not empty, Thread.Thread [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly57jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly47d5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly51jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache

Thread: thread [pool-5murthreadly63re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: threaded [pool-5murthreadlight69jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: thread [pool-5murthreadly22pj5main] got the JVM lock, redis.get (key) is not empty, thread [pool-5murthreadly3jin5main] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by Thread [pool-5murthreadly45judicial 5main], redis.get (key) is not empty, threaded [pool-5Muthreadmere 3jin5main] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: Thread [pool-5murthreadly32re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly36pj5remain] got the JVM lock, redis.get (key) is not empty, Thread.Thread [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly49magentific main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3recover5main] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: JVM lock was acquired by thread [pool-5murthreadly12jue 5main]. Redis.get (key) is not empty. Thread [pool-5murthreadly3jin5main] updated cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: thread [pool-5murthreadly26jin5main] got the JVM lock, redis.get (key) is not empty, threadpool-5Muthreadly3jin5main] updated the cache

Thread: thread [pool-5murthreadly53jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: thread [pool-5murthreadly61jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: Thread [pool-5murthreadly65jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly34re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly97jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly40jin5main] got the JVM lock, redis.get (key) is not empty, threadpool-5Muthreadly3jin5main] updated the cache.

Thread: Thread [pool-5murthreadly91jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly64re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly42re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache

Thread: threaded [pool-5murthreadly46pj5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: Thread [pool-5murthreadly87rejin5remain] got the JVM lock, redis.get (key) is not empty, thread.Thread [pool-5Muthreadly3jin5main] updated the cache.

Thread: Thread [pool-5murthreadly85jin5jimmain] got the JVM lock, redis.get (key) is not empty, Thread.Thread [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly44re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly75jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: Thread [pool-5murthreadly71jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly77d5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly52jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: Thread [pool-5murthreadly93jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly56jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: threaded [pool-5murthreadmae95jing5jimmain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly89magentific main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3recover5main] updated the cache.

Thread: thread [pool-5murthreadly81jin5jimmain] got the JVM lock, redis.get (key) is not empty, Thread.Thread [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly73re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly68jp5main] got the JVM lock, redis.get (key) is not empty, thread [pool-5Muthreadmere 3jin5main] updated the cache.

Thread: Thread [pool-5murthreadly58jin5remain] got the JVM lock, redis.get (key) is not empty, Thread.Thread [pool-5murthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly62re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly66jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly38je main] got the JVM lock, redis.get (key) is not empty, thread [pool-5Muthreadmere 3jin5main] updated the cache

Thread: Thread [pool-5murthreadly54re5 main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: JVM (key) is not empty, thread (pool-5) has got the thread lock, and thread has updated the cache.

Thread: Thread [pool-5murthreadly83jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly96jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly79magentific main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3recover5main] updated the cache.

Thread: Thread [pool-5murthreadly92jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly90je main] got the JVM lock, redis.get (key) is not empty, thread [pool-5Muthreadmuri 3jin5main] updated the cache

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: Thread [pool-5murthreadly82jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly72jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly78rejin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: JVM lock was acquired by Thread, redis.get (key) is not empty, thread update cache is updated.

Thread: thread [pool-5murthreadly70jin5main] got the JVM lock, redis.get (key) is not empty, threadpool-5Muthreadly3jin5main] updated the cache.

Thread: threaded [pool-5murthreadmae885jue main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly84jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache

Thread: threaded [pool-5murthreadmae985jue main] got the JVM lock, redis.get (key) is not empty, threaded [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: Thread [pool-5murthreadly86jin5remain] got the JVM lock, redis.get (key) is not empty, Thread.Thread [pool-5Muthreadly3jin5dmain] updated the cache.

Thread: threaded [pool-5murthreadly76pj5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache.

Thread: thread [pool-5murthreadly74jin5remain] got the JVM lock, redis.get (key) is not empty, threaded [pool-5murthreadly3jin5dmain] updated the cache

By looking at the log, we found that there are only two times to acquire distributed locks, and the rest are blocked by JVM locks, which can greatly improve the performance of locks.

This is the answer to the question about how to use ZooKeeper to optimize the distributed lock of Java cross-JVM. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.