Shulou(Shulou.com)06/02 Report--

This article focuses on "what are the characteristics of web distributed locks", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what are the characteristics of web distributed locks"?

Why use distributed locks

This question should be divided into the following two questions to answer.

Why does ① use locks?

It is guaranteed that shared resources can only be accessed by one client at the same time, which can be divided into the following two types according to the purpose of the lock:

Shared resources allow only one client operation

Shared resources allow multiple client operations

Only one client is allowed to access: the operation of sharing resources is not idempotent. It is common in the operation of modifying and deleting data.

In the above example:

Allow multiple client operations: the main application scenario is that the operation of sharing resources is idempotent, such as data query.

Now that they are idempotent, why do you need distributed locks, usually for efficiency or performance, to avoid repetitive operations (especially resource-consuming operations).

For example, our common caching scheme:

In the above example:

Because the resources here are idempotent, they are usually cached, which is the common lock + cache architecture.

It is often suitable for obtaining idempotent resources that consume more resources (time, memory, CPU, etc.), such as:

Query user information

Query historical orders

Of course, if the resource is idempotent only for a period of time, the architecture should be upgraded:

Lock + cache + cache invalidation / invalidation reacquisition / cache timing update.

Why do ② locks need to be distributed?

Take the caching scheme above as an example, with a slight change here:

In the above example:

What are the characteristics of distributed locks?

① repulsion

Only one client is allowed to acquire the lock at any one time.

PS: if multiple clients can acquire locks at the same time, locks are meaningless and the security of shared resources cannot be guaranteed.

Boss: when I receive client An in the conference room, other customers have to wait. You need to wait until I am free before you can bring the others to my office.

Ape: got it.

Receive customers (non-idempotent shared resources); wait until the boss is free (acquire locks).

② reentrant

Client An acquires the lock, and client A can continue to acquire the lock as long as the lock does not expire. Lock it with me, I will continue to use it, and no one else is allowed to rob it.

This feature can well support the lock renewal function. For example, client An acquires the lock, and the lock release time is 10s. When client An is about to reach 10s, client A does not complete the task and needs to apply for 5S again. If the lock is not reentrant, client A will not be able to renew the contract, causing the lock to be stolen by other clients.

Ape: yes, boss, you have another interview in 3 minutes.

Boss: little ape, it's rare for you to be so studious. I'm glad that our communication time will be postponed by 10 minutes, and other meetings will be postponed.

High performance of ③

The efficiency of acquiring locks should be high enough; you can't let the business block the acquisition of locks, right?

Ape: OK, I've been applying for a 10-minute extension of the meeting.

Boss: well, I have accepted the invitation to the meeting

Ape: boss, you are so efficient.

④ High availability

In the distributed and micro-service environment, it is necessary to ensure the high availability of services, otherwise it will affect other business modules or cause service avalanches.

Boss: my cell phone is turned on 24 hours a day. I can also contact my secretary when I can't get in touch with a meeting.

⑤ supports blocking and non-blocking locks

If you failed to acquire the lock, did you return the failure directly, or did you keep blocking until the acquisition was successful? Different business scenarios have different answers.

For example:

⑥ unlock permission

The client can only release (unlock) its own locks. A common solution is to add a random number (or ThreadID) to the lock.

Boss: little ape, I've told you so much, do you understand?

Parrot in the cage: got it, got it.

Boss: shut up. I'm asking about the little ape. Only the little ape is qualified to answer.

⑦ avoids deadlock

The locking party's abnormal termination can not release the lock actively; the general practice is to set the timeout when adding the lock, and if the lock is not released actively, the automatic expiration of Redis is used to passively release the lock.

The secretary broke in: boss, your 10-minute meeting is up, and Mr. Li next door can't wait.

Boss: I lost track of time when I was careless. I have to go to see Boss Li.

Ape: boss, we haven't finished talking yet.

⑧ exception handling

Common anomalies include Redis downtime, clock hopping, network failure and so on.

Little ape: no matter what happens, I will fail to acquire the lock. What should I do?

PS: this is complicated and needs to be analyzed according to specific business scenarios. For services that must be processed synchronously, there must be a failure alarm, and for services that allow delayed processing, consider recording failure information to be processed by other systems.

Distributed lock popular algorithm

Basic scheme SETNX

The SETNX instruction based on Redis completes the acquisition of locks.

① acquires lock SET lock:resource_name random_value NX PX 30000

Lock:resource_name: the resource name, the unique token of the locked object.

Random_value: usually stores the unique tag of the locking party, such as "UUID+ThreadID".

Set only if NX:Key does not exist, that is, the lock can only be locked if it is not locked by someone else.

PX: lock timeout.

Of course, this locking method does not support "lock reentrancy".

② release Lock (LUA script)

CheckValueThenDelete: check whether the unlocking party is the unlocking party. If so, unlocking is allowed, otherwise unlocking is not allowed.

The pseudo code is:

Public class RedisTool {/ / release lock success mark private static final Long RELEASE_LOCK_SUCCESS = 1L / * * release distributed lock * * @ param jedis Redis client * @ param lockKey lock tag * @ param lockValue lock party tag * @ return has been successfully released * / public static boolean releaseDistributedLock (Jedis jedis, String lockKey, String lockValue) {String script = "+" if redis.call ('get') KEYS [1]) = ARGV [1] then "+" return redis.call ('del', KEYS [1]) "+" else "+" return 0 "+" end " / / Collections.singletonList (): for scenarios with only one element, reduce memory allocation Object result = jedis.eval (script, Collections.singletonList (lockKey), Collections.singletonList (lockValue)); if (RELEASE_LOCK_SUCCESS.equals (result)) {return true;} return false;}}

Redlock algorithm

This algorithm is proposed by antirez, author of Redis, as a lock implementation scheme in distributed scenarios.

Redlock algorithm principle: [core] most nodes acquire the lock successfully and the lock is still valid.

Step 1: gets the current time (milliseconds).

Step 2: sequentially want N Redis nodes to acquire locks. Set random string random_value; to set lock expiration time:

Note 1: to acquire a lock, you need to set a timeout (to prevent a node from becoming unavailable), and the timeout should be much less than the lock validity time (tens of milliseconds).

Note 2: when a node fails to acquire a lock, it immediately acquires the lock from the next node (any type of failure, including that the lock on that node has been held by another client).

Step 3: calculates the total time-consuming totalTime to acquire the lock.

Step 4: lock acquired successfully.

Lock acquired successfully: the client successfully acquired the lock from most nodes (> = N _ totalTime), and the lock's validity period is not longer than that of the client.

Recalculate the lock validity time: the initial lock validity time minus 3.1 calculated time to acquire the lock.

Step 5: failed to acquire lock.

The release lock (Lua script) should be initiated to [all] clients immediately after the acquisition fails.

Step 6: release the lock.

The release lock (Lua script) should be initiated to [all] clients immediately after the completion of the business.

Advantages of Redlock algorithm:

High availability, most nodes are normal.

The problem of lock failure of single Redis node distributed lock in failover no longer exists.

Problem points of Redlock algorithm:

Redis node crash will affect lock security: the lock is not persisted before the node crash, and the lock will be lost after the node is restarted; Redis default AOF persistence is once per second (fsync), and in the worst case, data will be lost for 1 second.

Avoid jumping all the time: the administrator manually modifies the clock; using the ntpd (clock synchronization) program that does not jump to adjust the system clock, the clock modification is achieved through multiple fine-tuning.

Client blocking causes locks to expire, making shared resources insecure.

If it takes a long time to acquire the lock, resulting in a short effective time, should the lock be released immediately? Is it short if there are multiple segments?

Implementation with fencing token

Distributed system expert Martin Kleppmann discussed and pointed out that there are security problems in RedLock.

Battle of the Immortals: Martin Kleppmann believes that the RedLock algorithm proposed by antirez, the author of Redis, has security problems, and the two sides have had many rounds of discussion on the Internet.

Martin pointed out that the core problems of the RedLock algorithm are as follows:

Lock expiration or network delay will cause lock conflicts: client A process pause → lock expiration → client B holds lock → client A restores and initiates write requests to shared resources; network latency can have a similar effect.

RedLock security is strongly dependent on the system clock.

Principle of fencing token algorithm:

Fencing token is a monotonously increasing number that is returned to the client along with the lock when the client successfully acquires the lock.

The client accesses the shared resource with token.

The shared resource service checks the token and rejects delayed requests.

Problem points of fencing token algorithm:

Shared resource services need to be revamped.

If the resource service is also distributed, how to ensure that the token is incremented in multiple resource service nodes.

The order in which the two fencing token arrive at the resource service is reversed, and the service check will be abnormal.

[antirez] since there is a fencing mechanism to maintain mutually exclusive access to resources, why do you need distributed locking and strong security?

Other distributed locks

Database exclusive lock:

Acquire locks (select for update, pessimistic locks).

Deal with business logic.

Release the lock (connection.commit ()).

Note: when locking, the InnoDB engine will use row-level locks only when retrieving by index, otherwise table-level locks will be used. So must index lock_name.

ZooKeeper distributed Lock:

The client creates a znode node, and if it is successfully created, the lock is acquired successfully.

Delete the znode when the client that holds the lock accesses the shared resource.

Znode is created as ephemeral (znode feature) to ensure that the znode will be automatically deleted after the client that created the znode crashes.

[problem] Zookeeper relies on periodic heartbeat (heartbeat) maintenance based on the maintenance of Session,Session by the client and a server in Zookeeper.

If the Zookeeper fails to receive the client heartbeat for a long time, the task Session expires, and all ephemeral type znode nodes created by this Session will be deleted.

Chubby distributed locks for Google:

The sequencer mechanism (similar to fencing token) alleviates problems caused by delays.

The lock holder can request a sequencer at any time.

When the client operates on the resource, it passes the sequencer to the resource server.

The resource server checks the sequencer validity: ① calls Chubby's API (CheckSequencer) check. ② compares and checks the sequencer currently observed by the client and the resource server (similar to fencing token). ③ lock-delay: allows the client to specify a lock-delay delay time for holding the lock. When Chubby discovers that the client has lost contact, it organizes other clients to acquire the lock within lock-delay time.

At this point, I believe you have a deeper understanding of "what are the characteristics of web distributed locks?" you might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.