Shulou(Shulou.com)06/02 Report--

This article mainly explains "what are the common problems in redis caching". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn what are the common problems in redis caching.

1. What is caching?

Caches, which are buffers for data exchange, can be built for different service objects (CPU, memory, disk, etc.).

The purpose is to save the data in the medium with slow reading and writing speed in the medium with fast reading and writing speed so as to improve the efficiency.

For example:

CPU cache

Memory cache

Keep the data commonly used on disk in memory.

In daily business, the database we use more is Mysql, and the cache is Redis. The performance of Redis is much faster than that of Mysql, so we cache the hot data in Mysql to Redis to improve read performance and reduce database pressure.

When reading the data, read it from redis, not from redis, and then read it from mysql.

When writing data, first write to redis, and then write back to redis asynchronously or synchronously.

Scene:

1. Forum posts are visited frequently and need to be updated in real time. Redis can be used to record the reading volume of posts to improve parallelism and performance.

2. The frequency of updating commodity information is not high, but the frequency of reading is relatively high, especially for hot items, so the hot product information is stored in Redis.

2. Common caching algorithms

LRU (least recently used) is the least recently used

LFU (least frequently used) is used least frequently

FIFO first in first out

3. Common caching tools and frameworks

This is an introduction to the Java environment

Local cache:

Guava LocalCache 、 Ehcache 、 Caffeine

Ehcache has more features, and Caffeine has better performance than Guava LocalCache.

Distributed caching:

Redis 、 MemCache 、 Tair

Redis is the most mainstream and commonly used.

4. Cache FAQ

Write question:

When will the cache be written? How to avoid rereading and writing in a multithreaded environment?

How does the cache fail?

How to ensure the consistency of cache and DB

Classic three consecutive questions:

How to avoid cache penetration?

How to avoid cache breakdown?

How to avoid cache avalanches?

5. How to avoid avalanches

In the actual business, if there is a problem with the cache system (downtime), the program should manually catch this exception, record the log, and then query the data from the database to the user, so that the business will not be unavailable. But one problem that comes with it is the cache avalanche.

Cache avalanche refers to the situation in which the cache cannot provide service for some reasons, and all requests arrive at DB, resulting in a great increase in DB load and eventually hang up.

How to solve:

Method (1) cache is highly available

By building the high availability of the cache, it can avoid the situation that the cache can not be provided due to the cache hanging, thus reducing the probability of cache avalanche.

Suppose we use Redis for caching, and we can use Redis Sentinel or Redis Cluster to achieve high availability.

Method (2) Local caching

Using the local cache, even if the distributed cache is down, the results of the DB query can be cached locally to prevent all subsequent requests from reaching the DB.

The local caching tools available in the java environment have been described above.

Method (3) request DB to limit current

Limiting the request tree per second of DB to avoid DB hanging has at least two benefits:

1. Some users may still be able to use it, and the system is not dead yet.

2. After the cache service is restored in the future, the system can recover immediately, and there is no need to deal with the situation where DB is also dead.

Of course, for requests that are restricted, it is best to deal with them accordingly, go to [Service downgrade], provide some default values, or friendly tips.

Guaua RateLimiter, Sentinel and Hystrix can be used in Java environment to realize current limiting function.

Method (4) drill in advance

6. How to avoid cache penetration

Cache traversal refers to the query cache and a certain non-existent data in the database. Because the query cannot be found, it will be searched in the database. If the same cannot be found, the cache will not be written. This is pointless and resource-consuming. DB may be dead when the traffic is heavy.

If someone takes advantage of the non-existent key to attack our application frequently, this is a vulnerability.

Two solutions:

Method (1) cache empty object

When the query data from the DB is empty, the empty result is still cached, using a special identity to distinguish it from the real data. In addition, you need to set a short expiration time, which is not more than five minutes.

Applicable scenarios: scenarios with low data hit rate and need to ensure consistency

Advantages: easy code maintenance

Disadvantages: need more memory, data inconsistency

Method (2) BloomFilter Bloom filter (commonly used)

On the basis of caching service, build a BloomFilter data structure to store the tag of whether the corresponding Key exists. If so, the corresponding value of the key is not empty. The whole logic is as follows:

Query the Bloom cache according to key. If it does not exist, return directly, and if it does, continue to execute down. [the follow-up process is the standard search and cache process]

Query the cache according to key. If it exists, return directly. If it does not exist, continue to execute down.

Query DB based on key and, if present, update to the cache and return the value.

Applicable scenarios: scenarios with low hit rate, relatively fixed data and low real-time requirements

Advantages: small cache footprint

Disadvantages: code maintenance is difficult

Scheme 2 is often used in the actual scene because of the provincial memory and the small load on the cache.

7. BloomFilter data structure is not supported in Redis, how to implement Bloom filter?

RedisBloom

Redis-Lua-scaling-bloom-filter, using lua script to realize the function of Bloom filter

Realizing the function of Bloom filter by Redisson BloomFilter,Java Redis Library

8. Why can it be determined that there is no corresponding data in the cache or database if key does not exist in BloomFilter?

9. What should be paid attention to when using Bloom filter

(1) misjudgment. What exists may not exist, and what does not exist must not exist. Due to the fact that the Bloom filter does not allow deletion, this will lead to the addition of a new data, and the Bloom filter will always rule that it does not exist.

When using a Bloom filter, you need to initialize the existing key to the Bloom cache in advance.

How can the key of the new data be added to the Bloom cache?

10. How to avoid breakdown?

Cache breakdown means that the data in a cache has expired at a time when there are a large number of requests to access the key. When these requests find that the cache has expired, they query the DB and write back to the cache. If the request is too large, it may overwhelm the DB instantly.

It is similar to avalanche and penetration.

Difference:

The difference between an avalanche and an avalanche: the former is for a certain KEY, while the latter is for many KEY. In fact, in my opinion, it makes no difference. It's all avalanches.

The difference between key and interpenetration: the corresponding data exists in the database.

Solution:

Scenario (1) use mutexes (distributed locks)

The purpose is to limit the number of DB queries, allowing only one thread to query DB

Scenario (2) manual expiration

11. How to ensure the consistency of cache and DB (usually the final consistency)?

There are two main situations that can lead to inconsistencies between cache and DB

1. In concurrent scenarios, read the old DB data and update it to the cache

Here, it mainly refers to the deletion of DB data before updating Cache data. There is no problem in low concurrency, but there is a problem in high concurrency. There happens to be a request between (deleting the Cache data and updating the DB data). If we use passive read, because the DB data is still old, the old data will be written to the Cache.

two。 Caching and DB operations are not in a single transaction. Only DB operations may succeed and cache operations fail, resulting in inconsistencies.

Of course, one thing we should pay attention to is the consistency of caching and DB, and we are talking more about the final consistency. As long as we use caching to improve the performance of read operations, the real business logic of writing operations is still based on the database.

Solution:

Plan (1) first eliminate the cache, and then write to the database

Implementation scheme: introducing distributed locks to change parallel writes into walkthrough writes

When writing a request, acquire the distributed lock before eliminating the cache.

When a read request is made and it is found that the cache does not exist, the distributed lock is acquired first.

Scenario (2) write to the database first, and then update the cache

Thank you for reading, these are the contents of "what are the common problems with redis caching?" after the study of this article, I believe you have a deeper understanding of the common problems with redis caching, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.