Shulou(Shulou.com)06/01 Report--

The main content of this article is to explain "what is the cause of Redis Breakthrough Avalanche and how to solve it". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what is the cause of the Redis Breakthrough Avalanche and how to solve it?"

1. Preface

As we all know, one of the bottlenecks of the computer is IO. In order to solve the problem of mismatch between memory and disk speed, there is a cache, and some hot data are put in memory and can be fetched on demand, so as to reduce the request link to connect to the database and avoid database hanging. It is important to note that both the breakdown and the penetration and avalanche discussed later are under the premise of high concurrency, such as when a hot key in the cache fails.

2. the cause of the problem

There are two main reasons:

1. Key expires

2. Key is eliminated by page replacement.

The first reason is that in Redis, Key has expiration time. If the key expires at a certain time (if the mall does an activity, it starts at zero), then all the query requests for a commodity after zero will be pressed on the database, resulting in a database crash.

For the second reason, because memory is limited, it is necessary to cache new data and eliminate old data all the time, so in a certain page replacement strategy (common page replacement algorithm diagram), eliminate data. If some goods are not visited before doing activities, they are bound to be eliminated.

3. The way to deal with breakdown.

The normal processing request is shown in the figure:

Because the expiration of key is inevitable, when high traffic comes to Redis, according to the single-threaded characteristics of Redis, it can be considered that tasks are executed in turn in the queue. When the request arrives at Redis and finds that Key expires, perform an operation: set locks.

The process is roughly as follows:

The request arrives at Redis and finds that the Redis Key expires. Check if there is a lock. If there is no lock, go back to the queue and queue up.

Set the lock, note that this should be setnx (), not set (), because another thread may have already set the lock

Acquire the lock, get the lock, go to the database to get the data, and release the lock after the request is returned.

But it raises a new question: what if I get the request for the lock to get the data and then hang up? That is, the lock is not released, and other processes are waiting for the lock. The solution is:

Set an expiration time for the lock, and if the expiration time is not released, it will be released automatically. Here comes the problem again. It is easy to say that the lock is hung up, but what if the lock times out? That is, the data is not taken out in the set time, but the lock expires. The common idea is that the lock expiration time value increases progressively, but it is unreliable to think about it, because the first request may time out, what if the later one also times out. After a series of timeouts, the lock expiration time value is bound to be particularly large, which has too many disadvantages.

Another idea is to start another thread to monitor and delay the expiration of the lock appropriately if the thread fetching the data does not hang up.

4. Penetration

The main reason for penetration is that many requests are accessing data that does not exist in the database. For example, a shopping mall selling books has been asked to query tea products. Since Redis cache is mainly used to cache hot spot data, data that does not exist in the database cannot be cached, and this abnormal traffic will directly reach the database and return "no" query results.

In response to such requests, the access request is handled by adding a layer of filters, such as Bloom filters, enhanced Bloom filters, and cuckoo filters.

In addition to the Bloom filter, you can add some parameter verification. For example, the database data id is generally incremented. If you request a parameter such as id =-10, it is bound to bypass Redis. To avoid this situation, you can check the authenticity of users and other operations.

5. Avalanche

Avalanche is similar to breakdown, except that breakdown is a hot Key that fails at some point, while avalanche is a large number of hot Key that fails in an instant. Many blogs on the network emphasize that the strategy to solve avalanche is random expiration time, which is very inaccurate. For example, when banks do activities, the interest coefficient is 2% before, but the coefficient is changed to 3% after zero. In this case, can the corresponding key of the user be changed to a random expiration? If the past data is called dirty data.

Obviously not, also save f0c; you save until the end of the year interest 3 million, next door only 2 million, this must not fight ah, joke ~

The correct way of thinking is, first of all, to see if the Key expiration is related to time, if it has nothing to do with sex, it can be solved by random expiration time.

If it is time-related, for example, the bank just said to change a certain coefficient one day, then we should use the strong dependency breakdown scheme, and the strategy is to update all the key with the past thread first.

While updating the hotspot key in the background, the business layer delays the incoming request, such as a short sleep of a few milliseconds or seconds, to disperse the pressure on the later update hotspot key.

At this point, I believe that everyone has a deeper understanding of "what is the cause of Redis Breakthrough Avalanche and how to solve it". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.