Shulou(Shulou.com)06/02 Report--

Most Internet systems are distributed deployment, distributed deployment can indeed improve performance and efficiency, but to this end, we need to solve the problem of data consistency in a distributed environment.

When a resource is shared among multiple systems, in order to ensure that everyone's access to the resource data is consistent, then it must be required that it can only be processed by one client at the same time and cannot be executed concurrently, otherwise there will be people writing and reading at the same time, and the data accessed by everyone will be inconsistent.

1. Why do we need distributed locks?

In the stand-alone era, although there is no need for distributed locks, we have faced similar problems, but in the case of a single machine, if multiple threads want to access a shared resource at the same time, we can adopt the mechanism of adding locks between threads. that is, when a thread acquires the resource, it immediately locks the resource, and when it is used, it is unlocked, and other threads can continue to use it. For example, in JAVA, there are even some API (synchronize/Lock, etc.) that specifically deal with locking mechanisms.

However, in the era of distributed systems, this inter-thread locking mechanism is useless, and the system may have multiple copies and be deployed on different machines, and these resources are no longer shared between threads. it belongs to resources shared between processes.

Therefore, in order to solve this problem, we must introduce "distributed locks".

Distributed locking means that in a distributed deployment environment, multiple clients can access shared resources with mutual exclusion through locking mechanism.

What are the requirements of distributed locks?

Exclusivity: only one client can acquire the lock at the same time, and other clients cannot acquire it at the same time.

Avoid deadlock: this lock must be released (normal or abnormal release) after a limited period of time.

High availability: the mechanism for acquiring or releasing locks must be highly available and have good performance

After talking about the background and theory, let's take a look at the specific classification and practical application of distributed locks.

Second, what are the ways to implement distributed locks?

At present, there are three mainstream ones. In terms of implementation complexity, the difficulty increases in turn from top to bottom:

Implementation based on database

Implementation based on Redis

Implementation based on ZooKeeper

No matter which way, in fact, it is not perfect, we still have to choose according to the actual scene of our business.

1. Based on database implementation:

There are usually two ways to do distributed locks based on a database:

Optimistic Lock based on Database

Pessimistic Lock based on Database

Let's first take a look at how to implement it based on "optimistic locks":

The optimistic locking mechanism is actually implemented by introducing a version field into the database table.

When we want to read the data from the database, we also read out the version field. If we want to update the read data and write it back to the database, we need to add 1 to the version and update the new data and the new version to the data table. At the same time, we must also check whether the version value in the current database is the previous version. If so, update normally. If not, the update fails, indicating that there are other processes to update the data in the process.

Here is an example of the picture:

(photo source network)

As shown in the figure, assuming that both user An and user B have to withdraw money from the same account, the original balance of the account is 2000, user A has to withdraw 1500, and user B has to withdraw 1000. If there is no locking mechanism, in the case of concurrency, the balance may be deducted 1500 and 1000 at the same time, resulting in incorrect or even negative final balance. But if the optimistic locking mechanism is used here, when two users read the balance in the database, they not only read more than 2000, but also read the current version number version=1. When user An or user B modifies the database balance, whoever operates first will add the version number to 1, that is, version=2, then another user will find that the version number is wrong and has become 2. If it is not the 1 it was originally read out, then this update fails and you have to re-read the latest database balance.

From the above example, we can see that using the "optimistic lock" mechanism, we must satisfy:

(1) the lock service must have an incremental version number of version

(2) every time you update the data, you must first determine whether the version number is correct, and then write a new version number.

Let's take a look at how to achieve this based on the "pessimistic lock":

Pessimistic locks, also known as exclusive locks, are implemented based on for update in Mysql, such as:

/ / Lock method-pseudo code public boolean lock () {connection.setAutoCommit (false) for () {result = select * from user where id = 100for update; if (result) {/ / the result is not empty, / / indicates that the lock was acquired return true;} / / did not acquire the lock, continue to acquire sleep (1000) } return false;} / / release the lock-pseudo code connection.commit ()

In the above example, id is the primary key in the user table, and through the for update operation, the database adds an exclusive lock to the record when querying.

(it should be noted that in InnoDB, only a field with an index will be a row-level lock, otherwise it is a table-level lock, so this id field should be indexed.)

When this record is added with an exclusive lock, no other thread can manipulate the record.

In that case, we can assume that the thread that acquired the exclusive lock has the distributed lock, and then we can execute the business logic we want to do, and then call the above statement to release the lock when the logic is complete.

two。 Implementation based on Redis

The locking mechanism implemented based on Redis mainly relies on the atomic operations of redis itself, such as:

SET user_key user_value NX PX 100

Starting with version 2.6.12 of redis, the SET command supports these parameters:

NX: set the key only when the key does not exist. The SET key value NX effect is equivalent to SETNX key value.

PX millisecond: set the expiration time of the key to millisecond milliseconds. When this time is exceeded, the set key will expire automatically.

The above code example refers to:

When the user_key key does not exist in the redis, a user_ key is set, the value of the key is set to user_value, and the survival time of the key is 100ms

Why can this command help us implement the locking mechanism?

Because this command is executed successfully only if a key does not exist. Then when multiple processes set the same key at the same time, only one process will always succeed.

When a process is set up successfully, you can execute the business logic, and then unlock it after the business logic has been executed.

Unlocking is very simple, you only need to delete the key, but before deleting it, you need to determine that the corresponding value of this key is the one you set up.

In addition, redis's Redlock mechanism can be used for distributed locks in redis cluster mode.

3. Implementation based on ZooKeeper

In fact, it is based on ZooKeeper, which is a distributed lock implemented by its temporary ordered nodes.

The principle is: when a client wants to add a lock logically, it generates a unique temporary ordered node under the directory of a specified node on the zookeeper, and then determines whether it is the one with the lowest sequence number among these ordered nodes. If so, it is considered to have acquired the lock. If not, the lock has not been acquired, then you need to find the node smaller than yourself in the sequence, and call the exist () method on it to listen for its registration event. When you hear that the node has been deleted, determine again whether the node you created has become the smallest node in the sequence. If yes, acquire the lock, if not, repeat the above steps.

When the lock is released, simply delete the temporary node.

(the picture is from the Internet)

As shown in the figure, locker is a persistent node, node_1/node_2/. / node_n is the temporary node mentioned above, which is created by the client client.

Client_1/client_2/... / clien_n are all clients that want to acquire locks. Take client_1 as an example. If it wants to obtain a distributed lock, it needs to run under locker to create a temporary node (if it is node_1). After it has been created, check whether its node sequence number is the smallest under locker. If so, it has acquired the lock. If not, go to find the node smaller than yourself (if it is node_2), and then listen to node_2 until node_2 is deleted, then start to determine again whether your node_1 is the smallest in the sequence, if so, acquire the lock, if not, continue to find a node.

This is the end of why we need distributed locking technology, as well as the three common mechanisms in distributed locking. Welcome to communicate with us.

Source: https://mp.weixin.qq.com/s/29FI3t7BYyRw3yjB_X1phA

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.