Shulou(Shulou.com)06/03 Report--

This article mainly introduces the causes of Redis breakdown, penetration, avalanche, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

Breakdown

As we all know, one of the bottlenecks of the computer is IO. In order to solve the problem of mismatch between memory and disk speed, there is a cache, and some hot data are put in memory and can be fetched on demand, so as to reduce the request link to connect to the database and avoid database hanging. It should be noted that whether it is breakdown or the penetration and avalanche discussed later, under the premise of high concurrency, when a hot key in the cache fails.

Why is there a breakdown?

There are two main reasons:

Key expires

Key is eliminated by page replacement.

The first reason is that in Redis, Key has expiration time. If the key expires at a certain time (if the mall does an activity, it starts at zero), then all the query requests for a commodity after zero will be pressed on the database, resulting in a database crash.

For the second reason, because memory is limited, it is necessary to cache new data and eliminate old data all the time, so in a certain page replacement strategy (common page replacement algorithm diagram), eliminate data. If some goods are not visited before doing activities, they are bound to be eliminated.

Thoughts on dealing with Breakdown

The normal processing request is shown in the figure:

Because the expiration of key is inevitable, when high traffic comes to Redis, according to the single-threaded characteristics of Redis, it can be considered that the task is executed in turn in the queue. When the request arrives at Redis and finds that the Key expires, do an operation: set lock

The process is roughly as follows:

The request arrives at Redis and finds that the Redis Key expires. Check if there is a lock. If there is no lock, go back to the queue and queue up.

Set the lock, note that this should be setnx (), not set (), because another thread may have already set the lock

Acquire the lock, get the lock, go to the database to get the data, and release the lock after the request is returned.

But it raises a new question: what if I get the request for the lock to get the data and then hang up? That is, the lock is not released, and other processes are waiting for the lock. The solution is:

Set an expiration time for the lock, and if the expiration time is not released, it will be released automatically. Here comes the problem again. It is easy to say that the lock is hung up, but what if the lock times out? That is, the data is not taken out in the set time, but the lock expires. The common idea is that the lock expiration time value increases progressively, but it is unreliable to think about it, because the first request may time out, what if the later one also times out. After a series of timeouts, the lock expiration time value is bound to be particularly large, which has too many disadvantages.

Another idea is to start another thread to monitor and delay the expiration of the lock appropriately if the thread fetching the data does not hang up.

Piercing

The main reason for penetration is that many requests are accessing data that does not exist in the database. For example, a shopping mall selling books has been asked to query tea products. Since Redis cache is mainly used to cache hot spot data, data that does not exist in the database cannot be cached, and this abnormal traffic will directly reach the database and return "no" query results.

In response to such requests, a layer of filter is added to the access request, such as Redis filter, enhanced Bloom filter and cuckoo filter. For more information, please see: Bloom filter and cuckoo filter.

In addition to the Bloom filter, you can add some parameter verification. For example, the database data id is generally incremented. If you request a parameter such as id =-10, it is bound to bypass Redis. To avoid this situation, you can check the authenticity of users and other operations.

avalanche

Avalanche is similar to breakdown, except that breakdown is a hot Key that fails at some point, while avalanche is a large number of hot Key that fails in an instant. Many blogs on the network emphasize that the strategy to solve avalanche is random expiration time, which is very inaccurate. For example, when banks do activities, the interest coefficient is 2% before, but the coefficient is changed to 3% after zero. In this case, can the corresponding key of the user be changed to a random expiration? If the past data is called dirty data.

Obviously not, also save money, you save until the end of the year interest 3 million, next door only 2 million, this can not fight ah, joke ~

The correct way of thinking is, first of all, to see if the Key expiration is related to time, if it has nothing to do with sex, it can be solved by random expiration time.

If it is time-related, for example, the bank just said to change a certain coefficient one day, then we should use the strong dependency breakdown scheme. The strategy is to update all key from the past thread first.

While updating the hotspot key in the background, the business layer delays the incoming request, such as a short sleep of a few milliseconds or seconds, to disperse the pressure on the later update hotspot key.

Thank you for reading this article carefully. I hope the article "what are the causes of Redis breakdown, penetration and avalanche" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.