Shulou(Shulou.com)06/02 Report--

This article mainly introduces "how to achieve distributed locks through Redis". In daily operation, I believe many people have doubts about how to achieve distributed locks through Redis. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "how to achieve distributed locks through Redis". Next, please follow the editor to study!

The data of the commonly used SQL database is stored in disk, although the corresponding cache is also done at the bottom of the database to reduce the IO pressure of the database.

Because the cache of the database is generally aimed at the content of the query, and the granularity is relatively small, the cache of the database will have an effect only when the data in the table has not changed. However, this can not reduce the IO pressure of the business logic on the addition, deletion and modification of the database, so the cache technology arises at the historic moment, which realizes the cache of hot data and can greatly alleviate the pressure on the back-end database.

1. Mainstream application architecture:

When the client initiates a request to the database, it first goes to the cache layer to see if there is any data needed by the client, and if the cache layer has the data needed by the client, it returns directly from the cache layer, otherwise it makes a penetrating query and queries the database. If the data is queried in the database, the data is written back to the cache layer so that the next time the client queries again, it can get the data directly from the cache layer.

2. Why can Redis be so fast

Redis is very efficient, the official data is 100000+QPS, this is because: Redis is entirely based on memory, the vast majority of requests are purely memory operations, efficient execution. Redis uses a single-process and single-threaded database, which stores the data in memory, and its access is not limited by the hard disk IO, so its execution speed is extremely fast. In addition, single thread can also handle highly concurrent requests, avoid frequent context switching and lock competition, and start multiple instances if you want to run with multiple cores.

The data structure is simple and the data operation is simple. Redis does not use tables, does not force users to associate each relationship, and does not have complex relationship restrictions. Its storage structure is key-value pairs. The biggest advantage similar to HashMap,HashMap is that the time complexity of access is O (1).

Redis uses the multiplexing model of multi-channel Istroke O, which is a non-blocking IO. Note: epoll/kqueue/evport/select is the Istroke O multiplexing function adopted by Redis.

Choose the strategy:

According to the local conditions, we give priority to the Icano multiplexing function with time complexity O (1) as the underlying implementation. Because Select traverses every IO, its time complexity is O (n), which is usually used as a bottom-keeping scheme. Listens for Istroke O events based on the React design pattern.

3. Data type of Redis

String: the most basic data type whose value can be stored up to 512m, binary security (Redis's String can contain any binary data, including jpg objects, etc.). Note: if you repeatedly write the same key-value pair of key, the later will overwrite the previous write.

A dictionary composed of Hash:String elements, suitable for storing objects.

List: list, sorted by the order in which String elements are inserted. The order is last in, first out. Because of its stack feature, it is possible to implement functions such as the latest news rankings.

The unordered set composed of Set:String elements is implemented through a hash table (the time complexity of adding, deleting, changing and querying is O (1), and repetition is not allowed. In addition, when we use Smembers to traverse the elements in Set, the order is also uncertain, which is the result of the operation through Hash. Redis also provides operations such as intersection, union, difference and other operations for collections, which can achieve functions such as common concern, common friends and so on.

Sorted Set: sorts the members of the collection from small to large by scores.

More advanced Redis types: HyperLogLog for counting, Geo for storing geolocation information.

4. How to realize distributed lock through Redis

Distributed lock: a distributed lock is an implementation of a lock that controls common access to shared resources between distributed systems. If a system or different hosts of different systems share a certain resource, mutual exclusion is often needed to eliminate interference and satisfy data consistency.

The problems to be solved in distributed locks are as follows:

Mutually exclusive: only one client can acquire the lock at any time, and no two clients can acquire the lock at the same time.

Security: the lock can only be deleted by the client holding the lock, not by other clients.

Deadlock: the client that acquires the lock goes down for some reason and is unable to release the lock, and other clients can no longer acquire the lock, which leads to deadlock. At this time, a special mechanism is needed to avoid deadlock.

Fault tolerance: when each node, such as a Redis node, goes down, the client can still acquire or release the lock.

5. How to implement asynchronous queue

(1) use List in Redis as the queue

The List in the data structure of Redis mentioned above is used as a queue for Rpush production messages, and LPOP consumes messages. At this point, we can see that the queue uses the Rpush production queue and the LPOP consumption queue. In this producer-consumer queue, when there is no message in the LPOP, it is proved that there are no elements in the queue and that the producer has not had time to produce new data.

Cons: instead of waiting for a value in the queue before consuming, LPOP consumes directly.

Make up for it: you can call LPOP to retry by introducing a Sleep mechanism in the application layer.

(2) use BLPOP key [key …] Timeout

BLPOP key [key...] Timeout: block until the queue has a message or times out.

Disadvantages: according to this method, our post-production data can only be provided to individual consumers. Is it possible to make multiple consumers consume at once?

(3) Pub/Sub: topic subscriber model

The sender (Pub) sends the message and the subscriber (Sub) receives the message. Subscribers can subscribe to any number of channels. The disadvantage of Pub/Sub mode: the release of messages is stateless and cannot be guaranteed to be reachable. For publishers, messages are "lost or lost". At this time, if a consumer is offline when the producer publishes the message and cannot receive the message after it is re-online, a professional message queue, such as Kafka, is needed to solve this problem. I will not repeat it here.

6. Redis persistence

(1) what is persistence?

Persistence, that is, the persistent storage of data, without affecting the integrity of the data due to power outages or other complex external environments. Because Redis stores data in memory instead of disk, once the memory is powered off, the data stored in Redis will disappear, which is often not expected by users, so Redis has a persistence mechanism to ensure the security of the data.

(2) how to persist Redis

At present, there are two persistence methods for Redis, that is, RDB and AOF,RDB achieve data persistence by saving full data snapshots at a certain point in time. When restoring data, the data is recovered directly through the snapshot in the RDB file. How to quickly find what you need from huge amounts of data?

① fragmentation: data is divided according to certain rules and stored on multiple nodes. Reduce the pressure on a single Redis server by dividing the data among multiple Redis servers.

② consistent Hash algorithm: since the data is to be sliced, the usual way is to get the Hash value of the node, and then calculate the module according to the number of nodes. However, this method has obvious disadvantages, when the number of Redis nodes needs to increase or decrease dynamically, it will cause a large number of Key can not be hit. Therefore, the consistent Hash algorithm is introduced into Redis. In this algorithm, for 2 ^ 32, the Hash value space is formed into a virtual ring, and the whole ring is organized clockwise, and each node is 0, 1, 2 in turn. 2 ^ 32-1.

After that, the Hash operation of each server is carried out to determine the address of the server on the Hash ring. After determining the address of the server, the same Hash algorithm is used for the data to locate the data to a specific Redis server. If you navigate to a place where there is no Redis server instance, continue to search clockwise, and the first server found is the final server location of the data.

Data skew problem of ③ Hash Ring

When there are few server nodes in the Hash ring, it is easy to encounter the problem of uneven server nodes, which will cause data tilt, which means that most of the cached objects are concentrated on one or more servers of the Redis cluster. Most of the data after the consistent Hash algorithm is stored on the A node, while the B node only stores a small amount of data, over time A node will be burst. To solve this problem, virtual nodes can be introduced to solve the problem. To put it simply, multiple Hash is calculated for each server node, and one server node, called a virtual node, is placed at each result location, and a numbered implementation can be placed after the server IP or hostname.

At this point, the study on "how to implement distributed locks through Redis" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.