Shulou(Shulou.com)06/03 Report--

This article mainly introduces "how to understand cache". In daily operation, I believe many people have doubts about how to understand cache. The editor consulted all kinds of data and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the questions of "how to understand cache"! Next, please follow the editor to study!

In the computer world, caching is the process of storing a subset of data in a highly accessible high-speed running layer, which is called cache.

This process is designed to quickly read highly used data and avoid additional computational burden when accessing previous data. The cache can only store data for a short period of time, which is the choice after weighing the capacity in exchange for higher running speed.

Hardware under the cache layer, such as random access memory (RAM) and memory storage engine, can be accessed quickly, and they are usually used to access data together with the software layer. There are basically two types of caches: local caching and remote caching. Local caches rely on the JVW (Java virtual machine) heap for storage, while remote (or clustered) caches use memory storage, such as Redis and Memcached.

What is on-heap local caching?

In-heap caching refers to storing data in the Java heap, where the data is automatically managed by the garbage collector (GC).

Advantages of in-heap caching:

GC automatically allocates and releases objects

Faster access to data

Shortcomings of in-heap caching:

Multiple pauses in GC

Because the data is stored in JVM memory, if the JVM crashes, the data will be lost. Therefore, it cannot be cached for a long time.

What is out-of-heap local caching?

Out-of-heap caching refers to storing data out of the heap. The garbage collector does not automatically process this data because it is stored outside the Java heap, so it is stored as a byte array, so there is an additional operational burden of serializing and deserializing the data.

Advantages of out-of-heap caching:

Allow large amounts of data to be cached without worrying about GC pause

JVM supports adding persistent layers to memory to recover data after a crash

Cached data can be shared between JVM

Shortcomings of out-of-heap caching:

Serialization and deserialization of data are the biggest inconveniences when using off-heap caching. This will put a computational burden on the lower-level programs. Because there is no common data structure, there is an additional cost to convert serialized data into separate objects.

Short-term data is more suitable for in-heap caching because it allows GC to run automatically. Therefore, identifying which data can be attributed to the in-heap cache results in additional computation.

Manual memory management (issues such as memory segmentation!)

In short, because out-of-heap caching can store large amounts of data for a long time, it is a better way to store data. Coupled with a large disk subsystem, you can increase the number of reads and writes per second (IOPS).

What is remote caching?

Remote caching is a buffer that stores data in the cloud. Because data can be retrieved in the cloud, this helps build a stronger and more powerful persistence layer. Redis and Memcached are two popular memory caching products.

Advantages of remote caching:

Remote cache clusters can be expanded according to demand

Remote caching is not only limited to a single data structure, but also supports multi-language programming, so it is easy to operate.

Performance is improved compared to low-speed disk access (because data is stored in memory and data is accessed faster)

How to determine that the system / service needs caching

Cache hit ratio: if the data provided by the service does not need to be refreshed frequently and belongs to frequently retrieved data, you should consider caching them.

Tolerance for ultimate consistency: carefully consider the rate of change of the source data and how often the cache is refreshed. You should also consider whether the service object values the reading of recent data.

Types of caching policies and corresponding challenges

(1) Local cache:

Using some storage method within a service, such as a hash table, will make it easier to execute the local cache, but it can also cause cache consistency problems. This means that the local cache varies from server to server, which will lead to data inconsistency.

For example, server S1 responds to request R1 with data D1 and stores it in the local cache. If the data in the database is updated to the D2 version, and then R1 sends the same request again, it is possible to return data D1 or D2, depending on which server the request arrives at.

Cold start is also an important issue of memory caching. Because each server starts without a cache, the number of requests for downstream dependencies increases even during deployment as the number of new servers increases. This problem can be solved by requesting a merge.

(2) external cache:

The above problem can be solved because external caches are stored separately, such as Redis and Memcached.

Provide more storage space and reduce cache obsolescence due to capacity.

Challenges include increasing the complexity of the overall system and adding more load to maintain additional cache servers.

Always add code to the service to resolve situations where the cache may not be available.

Downstream services can be called during this period. However, if the cache outage time is too long, it may lead to an increase in the load of downstream services.

Alternatively, external memory and internal memory can be used together to avoid falling back completely to downstream dependencies.

You can also consider adopting load offloading technology to avoid overloading downstream services by limiting the number of service requests.

(3) address the problems of cache expansion and elasticity:

If the cache reaches its maximum capacity, it needs to be expanded by adding more nodes to it. An in-depth understanding of the system and its reaction when the maximum capacity is reached (for example, the memory utilization of each container increases when the cache reaches its maximum capacity) helps to set accurate alerts.

These alerts can be used to extend the caching service. When extending the service, you need to keep two things in mind: whether the cache cluster supports adding nodes without downtime, or whether it supports a consistent hash algorithm to balance traffic allocation. Always be sure to test the extension strategy by simulating a failure.

(4) robustness of control data.

The cached data should be able to read the updated code format, and the updated code should be able to handle the older version of data provided by the cache.

Key points of cache implementation

(1) Cache size: the cache size can be determined according to the data type passed in the service in order to improve the cache hit ratio.

(2) cache elimination: this refers to the removal of data from the cache when the cache reaches its maximum capacity. The most common mode of cache obsolescence is LRU (the least recently used).

(3) Cache expiration: this is a strategy for determining how long data is retained in the cache, depending on how often the data is refreshed or the customer's ability to handle obsolete data.

(4) downstream services are not available

If the downstream service is unavailable for some reason, the cache service should not attack the downstream with a request to update the data, but should be able to protect the cache for a long time and wait for it to recover. According to the tradeoff and coordination with the customer, either report the outdated data in the cache to avoid the request to offline the downstream service, or use a mechanism to store the error response of the downstream service and explain it accordingly.

(5) Security

The security of sensitive data or customer data being cached is one of the concerns about caching clusters. Sensitive data should be encrypted before storage and secure during input and output of cached data. In addition, because the return speed of cached data is faster than the call obtained from the database, the attacker can identify the type of request made by the service according to the response time, which is called side channel timing attack.

(6) the problem of "shock group effect"

When a downstream service is unavailable, if multiple requests are made to obtain uncached data from the downstream service, multiple retries may be triggered, resulting in the service being offline. Multiple strategies can be combined to alleviate this situation, such as throttling by customer or request, request merging (that is, sending only one request for the same uncached data), and so on.

Caching can provide faster data access and improve the availability of downstream services, but it comes at the cost of more complex processing of cache nodes. By skillfully understanding the requirements of downstream services, a caching solution can be obtained. The performance of this solution should be carefully monitored to adjust the parameters in different scenarios, such as peak traffic, unavailable cache, offline downstream services, and so on.

At this point, the study on "how to understand caching" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.