Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about big data's cache penetration, cache avalanche, cache breakdown and what the solution is. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

(1) Cache traversal: it means that the operation goes directly through the cache layer and directly to the db level, which is called cache penetration. This happens when you directly query a piece of data that does not exist (in fact, you still have to go through the cache layer, but if the cache layer does not have this data, it will initiate an operation at the db level).

Solution:

1. Bloom filter algorithm (Bloom filter? )

two。 Null cache: a relatively simple solution: after querying non-existent data for the first time, the key and the corresponding null values are also cached, but set to a shorter expiration time, such as a few minutes, to deal with a large number of key attacks in a short time, and set to a shorter expiration time because the value may have nothing to do with business and does not make much sense. And the query may not be initiated by the attacker, there is no need for too long storage, so it can be invalidated as soon as possible.

(2) cache avalanche: multiple cache data settings fail at the same time, and a large number of system requests change from request cache to request database, resulting in database crash.

Solution:

1. Staggered failure time (that is, to change the validity period of the data cache, it is recommended to set a random time to a fixed time, for example, any time between 3-5 seconds).

2. Redis High availability

The implication of this idea is that since it is possible for redis to fail, I will add a few more redis, so that the others can continue to work after one is dead, which is actually a cluster built.

3. Current limitation and degradation

The idea of this solution is to control the number of threads reading the database write cache by locking or queuing after the cache expires. For example, only one thread is allowed to query data and write cache for a key, while other threads wait.

4. Data preheating

Data heating means that before formal deployment, I access the possible data in advance, so that some of the data that may be accessed in large quantities will be loaded into the cache. Before large concurrent access is about to occur, manually trigger the loading of different caches of key, set different expiration times, and make the cache expiration time as uniform as possible.

(3) cache breakdown: cache breakdown is actually a special case of cache avalanche. The difference between breakdown and avalanche is that breakdown is for specific hot spot data, while avalanche is all data. (typical example: a hot topic on Weibo).

Solution: 1. Second-level cache (reference link) (Ali double eleven traffic cache breakdown solution) LRU-K algorithm

These are the cache penetration, cache avalanche, cache breakdown and solutions shared by the editor in big data. If you happen to have similar doubts, please refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.