Shulou(Shulou.com)06/01 Report--

This article focuses on "what are the advantages and disadvantages of distributed redis". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let the editor take you to learn "what are the advantages and disadvantages of distributed redis"?

1. Why use redis

Analysis: bloggers feel that using redis in a project is mainly considered from two perspectives: performance and concurrency. Of course, redis can also do distributed locks and other functions, but if only for distributed locks and other functions, there are other middleware (such as zookpeer, etc.) to replace, it is not necessary to use redis. Therefore, this question is mainly answered from the perspectives of performance and concurrency.

Answer: as shown below, it is divided into two points

(1) performance

As shown in the following figure, when we encounter a SQL that takes a long time to execute and the results do not change frequently, it is particularly suitable to cache the run results. In this way, the subsequent request is read in the cache so that the request can respond quickly.

Digression: I suddenly want to talk about this standard of rapid response. In fact, there is no fixed standard for this response time depending on the effect of the interaction. However, someone once told me: "ideally, our page jump needs to be solved in an instant, and the operation on the page needs to be solved in an instant." in addition, time-consuming operations with more than one finger should be prompted and can be terminated or cancelled at any time, so as to give the user the best experience. "

So how much time is the moment, the moment, the flick of a finger?

According to the Maha Monk Law

An instant is a thought, twenty thoughts are an instant, twenty moments are a flick of a finger, twenty fingers are a stroke, twenty strokes are a moment, and there are thirty moments in a day and a night.

So, after careful calculation, it is 0.36 seconds in a moment and 0.018 seconds in a moment. A finger snap lasts 7.2 seconds.

(2) concurrency

As shown in the following figure, in the case of large concurrency, all requests directly access the database, and the database will have a connection exception. At this point, you need to use redis to do a buffering operation so that the request accesses the redis first instead of directly accessing the database.

2. What are the disadvantages of using redis

Analysis: we have used redis for so long, this problem must be understood, basically using redis will encounter some problems, common only a few.

Answer: there are four main questions

(1) consistency of double writes between cache and database

(2) the problem of cache avalanche

(3) the problem of cache breakdown

(4) concurrency competition of cache

These four problems, I personally think that in the project, more often encountered, specific solutions, given later.

3. Why single-threaded redis is so fast

Analysis: this problem is actually an investigation of the internal mechanism of redis. In fact, according to the interview experience of bloggers, many people do not know that redis is a single-threaded work model. Therefore, this question should be reviewed.

Answer: mainly the following three points

(1) Pure memory operation

(2) single-thread operation to avoid frequent context switching

(3) the non-blocking Iripple O multiplexing mechanism is adopted.

Digression: now we have to talk in detail about the Istroke O multiplexing mechanism, because this statement is so popular that most people don't understand what it means. The blogger takes an example: Xiaoqu has opened a delivery store in S City, which is responsible for the same-city express service. Xiaoqu hired a group of couriers because of financial constraints, and then Xiaoqu found that there was not enough money to buy a car for express delivery.

Mode one of operation

Every time a customer sends a delivery, Xiaoqu lets a courier keep an eye on it, and then the courier drives to deliver the delivery. Slowly Xiaoqu found the following problems in this mode of operation.

Dozens of couriers basically spend their time robbing cars, and most couriers are idle. Whoever gets the car can deliver it.

With the increase of express delivery, there are more and more couriers. Xiaoqu found that the express store is becoming more and more crowded and there is no way to hire new couriers.

Coordination between couriers takes a lot of time

Based on the above shortcomings, Xiaoqu learned from the bitter experience and put forward the following mode of operation.

Mode 2 of operation

Xiaoqu only employs one courier. Then, for the express delivery sent by the customer, Xiaoqu is marked according to the place of delivery, and then placed in one place in turn. Finally, the courier went to pick up the express delivery in turn, one at a time, then drove to deliver the express delivery, and then came back to get the next delivery.

Contrast

Comparing the above two modes of operation, do you obviously feel that the second mode of operation is more efficient and better? In the above analogy:

Per courier-> per thread

Every courier-> every socket (Icano stream)

Delivery place of express delivery-> different status of socket

Customer delivers express request-> request from client

The operation mode of Xiaoqu-> the code running on the server side

The audit of a car-- > CPU

So we have the following conclusion.

1. The first mode of operation is the traditional concurrency model, in which each Imax O stream (express) has a new thread (courier) management.

2. The second mode of operation is Ihammer O multiplexing. There is only a single thread (a courier) that manages multiple Ipicot O streams by tracking the status of each Ipicuro stream (the place of delivery of each delivery).

The following analogy to the real redi threading model is shown in the figure

According to the picture above, to put it simply, it is. When our redis-client operates, it produces socket with different event types. On the server side, there is a section of Iripple 0 multiplexing program, which is placed in the queue. Then, the file event dispatcher takes it from the queue in turn and forwards it to different event handlers.

It should be noted that the redis O multiplexing mechanism, redis also provides select, epoll, evport, kqueue and other multiplexing function libraries, you can understand.

4. The data types of redis and the usage scenarios of each data type

Analysis: do you think this problem is very basic? in fact, I also think so. However, according to the interview experience, at least 80% of people can not answer this question. It is suggested that after using it in the project, then compare memory, experience more deeply, do not memorize it. Basically, a qualified programmer will use all five types.

Answer: there are five kinds.

(1) String

There is nothing to say about this. For the most conventional set/get operation, value can be String or a number. Generally do some complex counting function of the cache.

(2) hash

Here value stores structured objects, and it is convenient to manipulate one of the fields. When bloggers do single sign-on, they use this data structure to store user information, use cookieId as key, and set 30 minutes as cache expiration time, which can well simulate the effect similar to session.

(3) list

Using the data structure of List, you can do a simple message queue function. Another is that you can use the lrange command to do redis-based paging with excellent performance and a good user experience.

(4) set

Because set stacks a collection of values that are not repeated. So you can do the global de-duplication function. Why not use the Set that comes with JVM to remove the weight? Because our systems are generally deployed in clusters, it is troublesome to use the Set that comes with JVM. Is it too troublesome to do a global de-duplication and start a public service?

In addition, the use of intersection, union, subtraction and other operations, you can calculate common preferences, all preferences, their own unique preferences and other functions.

(5) sorted set

Sorted set has an extra weight parameter score, and the elements in the collection can be arranged by score. Can do ranking application, take TOP N operation. In addition, referring to another article, "distributed delayed task scheme analysis", this paper points out that sorted set can be used to do delayed tasks. The last application is to do a range search.

5. Expiration policy and memory elimination mechanism of redis.

Analysis: this question is actually very important, in the end whether redis is used at home or not, this question can be seen. For example, you can only store 5 gigabytes of data in redis, but if you write 10 gigabytes, you will delete 5 gigabytes. How to delete, have you thought about this question? Also, your data has set the expiration time, but the time is up, the memory occupancy rate is still relatively high, have you thought about the reason?

Answer:

Redis uses a periodic delete + lazy delete strategy.

Why not delete policies regularly?

Delete regularly, use a timer to monitor the key, and delete automatically when it expires. Although memory is released in time, it consumes CPU resources. In the case of large concurrent requests, CPU uses time to process requests instead of deleting key, so it does not adopt this strategy.

How does regular deletion + lazy deletion work?

Delete regularly. By default, redis checks each 100ms to see if there is an expired key, and if there is an expired key, delete it. It is important to note that redis does not check all the key once for every 100ms, but is randomly selected for inspection (if every other 100ms, all key are checked, the redis will not be stuck). Therefore, if you only use the periodic deletion policy, it will result in a lot of key not being deleted by the time.

As a result, lazy deletion comes in handy. That is to say, when you get a key, redis will check whether the key has expired if the expiration time is set. If it expires, it will be deleted.

Is there no other problem with regular deletion + lazy deletion?

No, if you delete key periodically, you don't delete it. Then you didn't immediately request key, which means that lazy deletion didn't work either. In this way, the memory of redis will be higher and higher. Then the memory elimination mechanism should be adopted.

There is a line configuration in redis.conf

# maxmemory-policy volatile-lru

This configuration is equipped with a memory elimination strategy (what, you didn't match it? Reflect on yourself)

1) noeviction: when there is not enough memory to hold new write data, the new write operation will report an error. I don't think anyone uses it.

2) allkeys-lru: when there is not enough memory to hold newly written data, remove the least recently used key in the key space. Recommended, which is currently used in the project.

3) allkeys-random: when there is not enough memory to hold newly written data, a key is randomly removed from the key space. No one should use it, if you don't delete it, at least use Key to delete it randomly.

4) volatile-lru: when there is not enough memory to hold the newly written data, remove the least recently used key from the key space where the expiration time is set. In this case, redis is generally used as both caching and persistent storage. Not recommended

5) volatile-random: when there is not enough memory to hold newly written data, a key is randomly removed from the key space where the expiration time is set. Still not recommended.

6) volatile-ttl: when there is not enough memory to hold the newly written data, the key with earlier expiration time is removed first in the key space where the expiration time is set. Not recommended

Ps: if the key of expire is not set, the prerequisites (prerequisites) are not met; then the behavior of volatile-lru, volatile-random, and volatile-ttl policies is basically the same as that of noeviction (do not delete).

6. Double write consistency between redis and database

Analysis: consistency problem is a common distributed problem, which can be subdivided into final consistency and strong consistency. Database and cache double write, there are bound to be inconsistencies. To answer this question, first understand a premise. That is, if there are strong consistency requirements for data, you can't slow down the storage. What we do can only ensure the ultimate consistency. In addition, fundamentally speaking, the plan we have made can only be said to reduce the probability of inconsistency and cannot be completely avoided. Therefore, data with strong consistency requirements cannot be slowed down.

Answer: "distributed database and cache double write consistency scheme parsing" gives a detailed analysis, which is briefly described here. First of all, adopt the correct update strategy, update the database first, and then delete the cache. Second, because there may be a failure to delete the cache, you can provide a compensatory measure, such as using message queuing.

7. How to deal with cache penetration and cache avalanche

Analysis: these two problems, to tell you the truth, the general small and medium-sized traditional software enterprises, it is difficult to encounter this problem. If there are large concurrent projects, the traffic is about millions. These two issues must be considered deeply.

Answer: as follows

Cache traversal, that is, hackers deliberately request data that does not exist in the cache, resulting in all requests to the database, resulting in abnormal database connection.

Solution:

(1) using mutexes, when the cache expires, first get the lock, get the lock, and then request the database. If you don't get the lock, go dormant for a while and try again.

(2) the asynchronous update strategy is adopted, which is returned directly regardless of whether the key gets a value or not. A cache expiration time is maintained in the value value. If the cache expires, a thread is asynchronously set up to read the database and update the cache. Need to do cache warm-up (load the cache before the project starts).

(3) provide an interception mechanism that can quickly judge whether the request is valid, for example, using Bloom filter to maintain a series of legal and valid key internally. Quickly determine whether the Key carried by the request is legal or valid. If it is illegal, return directly.

Cache avalanche, that is, a large area of cache failure at the same time, when there is another wave of requests, resulting in requests to the database, resulting in abnormal database connection.

Solution:

(1) add a random value to the cache expiration time to avoid collective failure.

(2) mutexes are used, but the throughput of the scheme is significantly reduced.

(3) double cache. We have two caches, cache An and cache B. The expiration time of cache An is 20 minutes, and cache B has no expiration time. Do your own cache warm-up operation. Then subdivide the following dots

I read the database from cache A, and return it directly.

II A has no data, reads the data directly from B, returns directly, and starts an update thread asynchronously.

The III update thread updates both cache An and cache B.

8. How to solve the problem of concurrent competitive key of redis

Analysis: the problem is roughly that multiple subsystems go to set a key at the same time. What should we pay attention to at this time? Have you thought about it. Need to explain, the blogger ahead of Baidu, found that the answer is basically recommended to use the redis transaction mechanism. Bloggers do not recommend using redis's transaction mechanism. Because our production environment is basically a redis cluster environment, we have done data slicing operation. When you have multiple key operations involved in a transaction, the multiple key may not be stored on the same redis-server. Therefore, the transaction mechanism of redis is very chicken.

Answer: as follows

(1) if the key operation is performed, no order is required.

In this case, prepare a distributed lock, everyone to grab the lock, grab the lock to do set operation, it is relatively simple.

(2) if the key operation is required, order is required.

Suppose you have a key1, system A needs to set key1 to valueA, system B needs to set key1 to valueB, and system C needs to set key1 to valueC.

The value of key1 is expected to change in the order of valueA-- > valueB-- > valueC. In this case, we need to save a timestamp when the data is written to the database. Suppose the timestamp is as follows

System A key 1 {valueA 3:00} system B key 1 {valueB 3:05} system C key 1 {valueC 3:10}

So, suppose system B grabs the lock first and sets key1 to {valueB 3:05}. Then system A grabs the lock and finds that the timestamp of its valueA is earlier than the timestamp in the cache, so it does not do the set operation. and so on.

Other methods, such as using queues, can also turn the set method into serial access. In short, be flexible.

At this point, I believe you have a deeper understanding of "what are the advantages and disadvantages of distributed redis?" you might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.