Shulou(Shulou.com)06/01 Report--

This article mainly explains "what are the common problems of caching in Java high concurrency scenarios". The explanation in this article is simple and clear, and is easy to learn and understand. Please follow the editor's train of thought to study and learn what are the common problems in caching in Java high concurrency scenarios.

1. Cache consistency

When the requirement of data timeliness is very high, it is necessary to ensure that the data in the cache is consistent with that in the database, and that the data in the cache node and copy are consistent, and there can be no differences.

This relies more on cache expiration and update strategies. Generally, when the data changes, the data in the cache will be actively updated or the corresponding cache will be removed.

Second, cache concurrency

After the cache expires, you will try to get data from the back-end database, which is a plausible process. However, in high concurrency scenarios, it is possible for multiple requests to obtain data from the database concurrently, which has a great impact on the back-end database, and even leads to the phenomenon of "avalanche".

In addition, when a cache key is updated, it may also be fetched by a large number of requests, which can also lead to consistency problems. So how to avoid similar problems?

We will think of a mechanism similar to "lock". In the case of cache update or expiration, we first try to acquire the lock, and then release the lock after the update or acquisition from the database is completed. Other requests only need to sacrifice a certain amount of waiting time. You can continue to get data from the cache directly.

III. Cache penetration

Cache penetration is also known as "breakdown" in some places. Many friends' understanding of cache traversal is that a large number of requests penetrate to the back-end database server due to cache failure or cache expiration, which has a great impact on the database.

This is actually a misunderstanding. The real cache traversal should look like this:

In a high concurrency scenario, if a key is accessed with high concurrency and missed, it will try to get it from the back-end database for the sake of fault tolerance, resulting in a large number of requests reaching the database. When the corresponding data of the key is empty, it will cause many unnecessary query operations to be executed concurrently in the database, resulting in great impact and pressure.

There are several common ways to avoid traditional caching problems:

1. Cache empty objects

Objects with empty query results are also cached. If it is a collection, an empty collection (not null) can be cached. If a single object is cached, it can be distinguished by field identification. This prevents requests from penetrating into the back-end database.

At the same time, it is also necessary to ensure the timeliness of cached data. This method has a lower cost and is more suitable for data that is not hit high, but may be updated frequently.

2. Separate filter processing

Store all the key that may be empty and intercept the request before the request, so as to prevent the request from penetrating to the back-end database. This method is relatively complex to implement and is more suitable for data that does not have a high hit but is not updated frequently.

IV. Buffer jolt problem

Cache jitter, which may be called "cache jitter" in some places, can be seen as a more minor failure than an avalanche, but it can also have an impact and performance impact on the system over a period of time. It is usually caused by cache node failure. The recommended practice in the industry is to solve the problem through consistent Hash algorithm.

5. Avalanche phenomenon of cache

Cache avalanche means that due to cache reasons, a large number of requests reach the back-end database, resulting in database crash, the whole system crash, disaster.

There are many reasons for this phenomenon. The problems mentioned above, such as "cache concurrency", "cache penetration", "cache bump" and so on, may actually lead to cache avalanche. These problems can also be exploited by malicious attackers.

In another case, for example, at a certain point in time, the cache preloaded by the system fails periodically, which may also lead to an avalanche. In order to avoid this periodic expiration, we can stagger the cache expiration by setting different expiration times, so as to avoid centralized cache invalidation.

From the perspective of application architecture, we can reduce the impact by means of current limitation, downgrade and circuit breaker, and we can also avoid this disaster through multi-level cache.

In addition, from the perspective of the whole R & D system process, we should strengthen stress testing, simulate the real scene as far as possible, and expose the problems as soon as possible to prevent them.

VI. The phenomenon of bottomless cache

The question was raised by facebook staff. Around 2010, facebook had 3000 memcached nodes and cached thousands of gigabytes of content.

They found a problem-memcached connection frequency, efficiency decreased, so add memcached node, after adding, found that the problem caused by connection frequency still exists, does not improve, called "bottomless pit phenomenon".

At present, the mainstream technology stacks such as database, cache, Nosql and search middleware all support "sharding" technology to meet the requirements of "high performance, high concurrency, high availability, scalability" and so on.

Some values are mapped to different instances through Hash modularization (or consistent Hash) on the client side, while others are mapped by range values on the client side. Of course, some are carried out on the server side.

However, each operation may require network communication with different nodes, and the more instance nodes, the greater the overhead and the greater the impact on performance.

It can be avoided and optimized from the following aspects:

1. Data distribution mode

Some business data may be suitable for Hash distribution, while some services may be suitable for range distribution, which can avoid the overhead of network IO to some extent.

2. IO optimization

We can make full use of connection pooling, NIO and other technologies to reduce connection overhead and enhance the ability of concurrent connections.

3. Data access mode

Getting a large dataset at one time is less expensive than the network IO that acquires a small dataset many times.

Of course, cache bottomless holes are not common. In most companies, you probably won't encounter it at all.

Thank you for your reading. The above is the content of "what are the common problems with caching in Java high concurrency scenarios". After the study of this article, I believe you have a deeper understanding of the common caching problems in Java high concurrency scenarios, and the specific usage needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.