Shulou(Shulou.com)05/31 Report--

This article mainly introduces how to achieve the second kill function based on redis distributed lock, which has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, let the editor take you to understand it.

Business scenario

From a business point of view, the so-called second kill means that multiple users "scramble" for resources in a short period of time, and the resources here are commodities in most of the second kill scenarios; abstract the business, from a technical point of view, second kill means multiple threads operate on resources, so to achieve second kill, we must control the thread's scramble for resources, not only to ensure efficient concurrency, but also to ensure the correct operation.

Some possible implementations

As mentioned just now, the key point to achieve second kill is to control the thread's scramble for resources. According to the basic thread knowledge, you can think of the following methods without thinking:

1. The abstraction of second kill at the technical level should be a method, in which the possible operation is to inventory the goods-1, add the goods to the user's shopping cart, etc., and operate the database without considering the cache. Then the simplest and most direct implementation is to add the synchronized keyword to this method, which, in popular terms, is to lock the whole method.

2. Locking the whole method is a simple and convenient strategy, but it seems a little rough. You can optimize it a little bit to lock only the code blocks that kill seconds, such as writing to the database.

3. Since there is a concurrency problem, I will let him "not concurrency" and manage all threads in a queue, making it a serial operation, so that there will be no concurrency problem.

All the methods mentioned above are effective, but none of them are good. Why? The first and second methods are essentially "locking", but the locking granularity is still relatively high. What does it mean? Just imagine, if two threads execute the second kill method at the same time, the two threads operate on different goods and should be able to do so at the same time, but if the first or second method is adopted, the two threads will also compete for the same lock, which is actually unnecessary. The third method does not solve the problem mentioned above.

So how do you control the lock at a finer granularity? Consider setting a mutex lock for each item, with the string associated with the commodity ID as the unique identity, so that only threads competing for the same item are mutually exclusive, without causing all threads to be mutually exclusive. Distributed locks can just help us solve this problem.

What is a distributed lock

Distributed lock is a way to control synchronous access to shared resources between distributed systems. In distributed systems, it is often necessary to coordinate their actions. If one or a group of resources are shared between different systems or different hosts of the same system, mutual exclusion is often needed to prevent interference from each other to ensure consistency when accessing these resources. distributed locks are needed.

Let's assume the simplest second kill scenario: there is a table in the database. Column is the inventory corresponding to the commodity ID and the commodity ID. If the second kill succeeds, the inventory of this item will be-1. Now suppose there are 1000 threads to kill two items, 500 threads to kill the first item, and 500 threads to kill the second item. Let's explain distributed locks based on this simple business scenario.

Usually, the business systems with second-kill scenarios are more complex, the amount of business carried is very large, and the concurrency is also very high. Such systems often use distributed architecture to balance the load. Then these 1000 concurrency will come from different places. Commodity inventory is not only the shared resources, but also the 1000 concurrent resources. At this time, we need to manage the concurrency mutual exclusion. This is the application of distributed locks.

Key-value storage systems, such as redis, are important tools to implement distributed locks because of some of their features.

Concrete implementation

Let's first take a look at some basic redis commands:

SETNX key value

If key does not exist, set the key corresponding string value. In this case, the command is the same as SET. When key already exists, nothing is done. SETNX is "SET if Not eXists".

Expire KEY seconds

Sets the expiration time of the key. If the key has expired, it will be deleted automatically.

Del KEY

Delete key

As the author's implementation only uses these three commands, only introduce these three commands, more commands and the characteristics and use of redis, you can refer to the official website of redis.

Issues to be considered

1. How to operate redis? Fortunately, redis has provided a jedis client for java applications by calling jedis API directly.

2, how to achieve locking? "lock" is actually an abstract concept, which turns this abstract concept into a concrete thing, that is, a key-value pair stored in redis. Key is the unique identification of a string related to the ID of a commodity. Value is actually not important, because as long as this unique key-value exists, it means that the commodity has been locked.

3. How to release the lock? Since the existence of a key-value pair indicates a lock, releasing the lock naturally deletes the key-value pair in the redis.

4. Blocking or non-blocking? The author uses a blocking implementation, if the thread finds that it has been locked, it will poll the lock within a specific time.

5. How to deal with abnormal situations? For example, a thread locks an item, but due to various reasons, it does not complete the operation (in the above business scenario, it does not write inventory-1 to the database), and naturally does not release the lock. In this case, the author adds the lock timeout mechanism and uses redis's expire command to set the timeout for key. After the timeout, redis will automatically delete the key. That is, the lock is forcibly released (you can think of the timeout lock as an asynchronous operation, which is done by redis, and the application only needs to set the timeout according to the characteristics of the system).

Talk is cheap,show me the code

At the level of code implementation, annotations have concurrent methods and parameters, obtain annotation methods and parameters through dynamic agents, add locks in the agents, and release locks after executing the proxied methods.

Several annotation definitions:

Cachelock is a method-level annotation for annotations of methods that cause concurrency problems:

@ Target (ElementType.METHOD) @ Retention (RetentionPolicy.RUNTIME) @ Documentedpublic @ interface CacheLock {String lockedPrefix () default "; / / redis lock key prefix long timeOut () default 2000 / polling lock time int expireTime () default 1000 key existence time in redis, 1000s}

LockedObject is a parameter-level annotation used to annotate basic types of parameters such as commodity ID:

@ Target (ElementType.PARAMETER) @ Retention (RetentionPolicy.RUNTIME) @ Documentedpublic @ interface LockedObject {/ / No value required}

LockedComplexObject is also a parameter-level annotation used to annotate parameters of a custom type:

@ Target (ElementType.PARAMETER) @ Retention (RetentionPolicy.RUNTIME) @ Documentedpublic @ interface LockedComplexObject {String field () default ""; / / member variables that need to be locked in complex objects that contain member variables, such as the commodity ID of a commodity object.

CacheLockInterceptor implements the InvocationHandler interface, obtains the annotated method and parameters in the invoke method, adds a lock before executing the annotated method, and releases the lock after executing the annotated method:

Public class CacheLockInterceptor implements InvocationHandler {public static int ERROR_COUNT = 0; private Object proxied; public CacheLockInterceptor (Object proxied) {this.proxied = proxied;} @ Override public Object invoke (Object proxy, Method method, Object [] args) throws Throwable {CacheLock cacheLock = method.getAnnotation (CacheLock.class); / / without cacheLock comments, pass if (null = = cacheLock) {System.out.println ("no cacheLock annotation"); return method.invoke (proxied, args) } / / get the annotations of the parameters in the method Annotation [] [] annotations = method.getParameterAnnotations (); / / obtain the locked parameters Object lockedObject = getLockedObject (annotations,args); String objectValue = lockedObject.toString () based on the obtained parameter comments and parameter list; / / create a new lock RedisLock lock = new RedisLock (cacheLock.lockedPrefix (), objectValue); / / add a lock boolean result = lock.lock (cacheLock.timeOut (), cacheLock.expireTime ()) If (! result) {/ / failed to lock ERROR_COUNT + = 1; throw new CacheLockException ("get lock fail");} try {/ / lock successfully, execute method return method.invoke (proxied, args);} finally {lock.unlock () / / release the lock}} / * @ param annotations * @ param args * @ return * @ throws CacheLockException * / private Object getLockedObject (Annotation [] [] annotations,Object [] args) throws CacheLockException {if (null = = args | | args.length = = 0) {throw new CacheLockException ("method parameter is empty, no locked object");} if (null = = annotations | | annotations.length = = 0) {throw new CacheLockException ("No annotated parameter") } / / multiple parameter locking is not supported, only the parameter int index =-1 leading / marking parameter for (int I = 0 X I < annotations.length;i++) {for (int j = 0 X j < annotations [I] .length; jacks +) {if (annotations [I] [j] instanceof LockedComplexObject) {/ annotated as LockedComplexObject index = I) is supported. Try {return args [I] .getClass (). GetField (LockedComplexObject) annotations [I] [j]). Field ());} catch (NoSuchFieldException | SecurityException e) {throw new CacheLockException ("annotation object" + ((LockedComplexObject) annotations [I] [j]). Field ());} if (annotations [I] [j] instanceof LockedObject) {index = I; break }} / / break directly after the first one is found. Multi-parameter locking if (index! =-1) {break;}} if (index = =-1) {throw new CacheLockException ("please specify locked parameters");} return args [index];}} is not supported.

The most critical lock and unlock methods in the RedisLock class:

/ * * locking * is used as follows: * lock (); * try {* executeMethod (); *} finally {* unlock (); *} * @ param timeout timeout within the time range of polling lock * @ param expire set lock timeout * @ return success or failed * / public boolean lock (long timeout,int expire) {long nanoTime = System.nanoTime (); timeout * = MILLI_NANO_TIME Try {/ / continuously polling lock while (System.nanoTime ()-nanoTime < timeout) {/ / lock does not exist within the time range of timeout. Set the lock and set the lock expiration time, that is, lock if (this.redisClient.setnx (this.key, LOCKED) = = 1) {this.redisClient.expire (key, expire) / / the lock expiration time is set to disappear after the lock expires without releasing / / without causing permanent blocking this.lock = true; return this.lock;} System.out.println ("lock waiting"); / / brief hibernation to avoid possible livelock Thread.sleep (3, RANDOM.nextInt (30));} catch (Exception e) {throw new RuntimeException ("locking error", e);} return false } public void unlock () {try {if (this.lock) {redisClient.delKey (key); / / directly delete}} catch (Throwable e) {}}

The above code is framework code, now let's explain how to use the simple framework above to write a second kill function.

First, define an interface in which a second kill method is defined:

Public interface SeckillInterface {/ * currently only supports commenting on interface methods * / / cacheLock comments may cause concurrency @ CacheLock (lockedPrefix= "TEST_PREFIX") public void secKill (String userID,@LockedObject Long commidityID); / / the simplest second kill method with parameters of user ID and commodity ID. There may be multiple threads competing for a commodity, so commodity ID plus LockedObject comment}

The implementation class of the above SeckillInterface interface, that is, the specific implementation of the second kill:

Public class SecKillImpl implements SeckillInterface {static Map inventory; static {inventory = new HashMap (); inventory.put (10000001L, 10000l); inventory.put (10000002L, 10000l);} @ Override public void secKill (String arg1, Long arg2) {/ / the simplest second kill, here only as a demo example reduceInventory (arg2) } / / simulates the second kill operation, and thinks that a second kill means reducing the inventory by one. The actual situation is more complicated than public Long reduceInventory (Long commodityId) {inventory.put (commodityId,inventory.get (commodityId)-1); return inventory.get (commodityId);}}

Simulate the second kill scenario in which 1000 threads compete for two items:

@ Test public void testSecKill () {int threadCount = 1000; int splitPoint = 500; CountDownLatch endCount = new CountDownLatch (threadCount); CountDownLatch beginCount = new CountDownLatch (1); SecKillImpl testClass = new SecKillImpl (1); Thread [] threads = new Thread [threadCount]; / / from 500 threads, kill the first commodity for (int I = 0bot I < splitPoint) Threads +) {threads [I] = new Thread (new Runnable () {public void run () {try {/ / waiting on a semaphore, suspend beginCount.await (); / / call the secKill method SeckillInterface proxy = (SeckillInterface) Proxy.newProxyInstance (SeckillInterface.class.getClassLoader (), new Class [] {SeckillInterface.class}, new CacheLockInterceptor (testClass)); proxy.secKill ("test", commidityId1); endCount.countDown () } catch (InterruptedException e) {/ / TODO Auto-generated catch block e.printStackTrace ();}; thread [I] .start ();} / / another 500 threads, kill the second commodity for (int I = splitPoint;i < threadCount;i++) {threads [I] = new Thread (new Runnable () {public void run () {try {/ / wait on a semaphore and suspend beginCount.await () / call secKill method SeckillInterface proxy = (SeckillInterface) Proxy.newProxyInstance (SeckillInterface.class.getClassLoader (), new Class [] {SeckillInterface.class}, new CacheLockInterceptor (testClass)) by dynamic proxy; proxy.secKill ("test", commidityId2); / / testClass.testFunc ("test", 10000001L); endCount.countDown ();} catch (InterruptedException e) {/ / TODO Auto-generated catch block e.printStackTrace ();}) Thread [I] .start ();} long startTime = System.currentTimeMillis (); / / the main thread releases the start semaphore and waits for the end semaphore to ensure that 1000 threads execute at the same time, ensuring the correctness of the test beginCount.countDown (); try {/ / the main thread waits for the end semaphore endCount.await (); / / observe whether the second kill result is correct System.out.println (SecKillImpl.inventory.get (commidityId1)) System.out.println (SecKillImpl.inventory.get (commidityId2)); System.out.println ("error count" + CacheLockInterceptor.ERROR_COUNT); System.out.println ("total cost" + (System.currentTimeMillis ()-startTime));} catch (InterruptedException e) {/ / TODO Auto-generated catch block e.printStackTrace ();}}

Under the correct expectation, the inventory of each commodity should be reduced by 500. After many experiments, the actual situation is in line with expectations. If the locking mechanism is not used, the inventory will be reduced by 499498.

In this paper, the method of dynamic proxy is adopted, and the distributed lock ID is obtained by annotation and reflection mechanism, and the lock is added and released. Of course, these operations can also be done directly in the method, and the dynamic agent is also used to centralize the lock operation code in the agent for easy maintenance.

Usually, the second kill scenario occurs in a web project, so you can consider using the AOP feature of spring to put the lock operation code in the aspect. Of course, AOP is also a dynamic proxy in nature.

Thank you for reading this article carefully. I hope the article "how to realize the second kill function based on redis distributed lock" shared by the editor will be helpful to everyone. At the same time, I also hope that you will support and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.