Shulou(Shulou.com)06/03 Report--

This article focuses on "how to understand application-level caching". Interested friends may wish to take a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to understand application-level caching.

Hit rate of cache

The hit rate of the cache refers to the ratio of the number of times the data was obtained from the cache to the total number of reads. The higher the hit rate, the better the effect of the cache. This is a very important indicator, and we should monitor this indicator to determine whether our cache is set properly.

Cache reclamation strategy is based on time

Survival period: when setting the cache, set how long the cache can survive. No matter how many times it is accessed during the lifetime, the time will expire.

Idle period: refers to how long the cached data expires without being accessed

Based on space

Set the storage space of the cache, for example, set the cache space to 1G, and when it reaches 1G, some data will be removed according to a certain policy.

Based on the number of caches

Set the maximum number of entries in the cache. When the maximum number of entries is reached, the old data is removed according to a certain policy.

Based on Java object reference

Weak reference: when the garbage collector starts to reclaim memory, if a weak reference is found, it will be reclaimed immediately.

Soft reference: when the garbage collector finds that there is not enough memory, it will reclaim the soft reference object to make room to prevent memory overflow. Soft references are suitable for heap caching

Cache recovery algorithm

FIFO first-in-first-out algorithm

The least recently used algorithm for LRU

The least commonly used algorithm in LFU

Type heap cache of Java cache

Heap cache refers to caching data in JVM's heap memory. The advantage of using heap cache is that there are no serialization and deserialization operations, and it is the fastest cache. If the amount of data cached is large, in order to avoid causing soft references that OOM usually uses to store cached objects; the disadvantage of heap caching is that the cache space is limited and the garbage collector pauses longer.

Gauva Cache implements heap cache Cache cache = CacheBuilder.newBuilder () .build ()

Building cached objects through CacheBuilder

The main configuration and methods of Gauva Cache

Put: setting key-value to cache

V get (K key, Callable loader): get a cache value. If it is not in the cache, call loader to get one and put it in the cache.

ExpireAfterWrite: sets the lifetime of the cache, which expires after a specified time after data is written

ExpireAfterAccess: sets the idle period of the cache, which will be reclaimed if it is not accessed within a given period of time

MaximumSize: sets the maximum number of entries cached

WeakKeys/weakValues: setting weak reference cache

SoftValues: setting soft reference cache

Invalidate/invalidateAll: actively invalidate the cached data of the specified key

RecordStats: start recording statistics, and you can see the hit rate.

RemovalListener: this listener is called when the cache is deleted and can be used to see why the cache is deleted

Caffeine implements heap cache

Caffeine is a rewritten version of Guava caching using Java8, a high-performance Java local cache component, and an implementation of heap caching recommended by Spring. Integration with spring allows you to view the document https://docs.spring.io/spring-framework/docs/current/reference/html/integration.html#cache-store-configuration-caffeine.

Because it is an rewrite version of Guava cache, many configuration parameters are the same as Guava cache:

InitialCapacity: initial cache space size

MaximumSize: maximum number of cached entries

MaximumWeight: maximum weight of the cache

ExpireAfterAccess: expires after a fixed time after the last write or access

ExpireAfterWrite: expires after a fixed time after the last write

ExpireAfter: custom expiration policy

RefreshAfterWrite: refreshes the cache after a fixed interval after the cache is created or last updated

WeakKeys: open a weak reference to key

WeakValues: open a weak reference to value

SoftValues: open a soft reference to value

RecordStats: enable the statistics function

Official document of Caffeine: https://github.com/ben-manes/caffeine/wiki

Add dependencies to pom.xml

Com.github.ben-manes.caffeine caffeine 2.8.4

Caffeine Cache provides three cache filling strategies: manual, synchronous, and asynchronous loading.

Manual load: specify a synchronous function each time get key, and call this function to generate a value if key does not exist

Public Object manual (String key) {Cache cache = Caffeine.newBuilder () .idle reAfterAccess (1, TimeUnit.SECONDS) / / set the idle period length. MaximumSize (10) .build (); return cache.get (key, t-> setValue (key) .apply (key));} public Function setValue (String key) {return t-> "https://silently9527.cn";}"

Synchronous loading: when constructing Cache, the build method passes in a CacheLoader implementation class. Implement the load method to load value through key.

Public Object sync (String key) {LoadingCache cache = Caffeine.newBuilder () .maximumSize (100) .AfterWrite (1, TimeUnit.MINUTES) / / sets the survival time. Build (k-> setValue (key). Apply (key)); return cache.get (key);} public Function setValue (String key) {return t-> "https://silently9527.cn";}"

Asynchronous loading: AsyncLoadingCache inherits from the LoadingCache class. Asynchronous loading uses Executor to call the method and returns a CompletableFuture

Public CompletableFuture async (String key) {AsyncLoadingCache cache = Caffeine.newBuilder () .maximumSize (100) .construcreAfterWrite (1, TimeUnit.MINUTES) .buildAsync (k-> setAsyncValue (). Get ()); return cache.get (key);} public CompletableFuture setAsyncValue () {return CompletableFuture.supplyAsync (()-> "official account: beta JAVA");}

Listen for events when the cache is cleaned

Public void removeListener () {Cache cache = Caffeine.newBuilder () .removalListener ((String key, Object value, RemovalCause cause)-> {System.out.println ("remove lisitener"); System.out.println ("remove Key:" + key); System.out.println ("remove Value:" + value);}) .build () Cache.put ("name", "silently9527"); cache.invalidate ("name");}

Statistics

Public void recordStats () {Cache cache = Caffeine.newBuilder () .maximumSize (10000) .recordStats () .build (); cache.put ("official account", "beta JAVA"); cache.get ("official account", (t)-> "); cache.get (" name ", (t)->" silently9527 "); CacheStats stats = cache.stats () System.out.println (stats);}

Get the CacheStats through Cache.stats (). CacheStats provides the following statistical methods:

HitRate (): returns the cache hit ratio

EvictionCount (): number of cache reclaims

AverageLoadPenalty (): average time to load new values

EhCache implements heap cache

EhCache is an old Java open source cache framework, which appeared as early as 2003, has been very mature and stable now, is widely used in the field of Java applications, and can be well integrated with mainstream Java frameworks such as Srping. The features supported by Guava Cache,EnCache are richer, including out-of-heap cache and disk cache, and of course it is heavier to use. The Maven dependencies that use Ehcache are as follows:

Org.ehcache ehcache 3.6.3CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder (). Build (true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.heap (10); / / set the maximum number of cache entries CacheConfiguration cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder (String.class, String.class, resource) .withExpiry (ExpiryPolicyBuilder.timeToIdleExpiration (Duration.ofMinutes (10) .build (); Cache cache = cacheManager.createCache ("userInfo", cacheConfig)

ResourcePoolsBuilder.heap (10) sets the maximum number of entries in the cache, which is abbreviated to ResourcePoolsBuilder.newResourcePoolsBuilder () .heap (10, EntryUnit.ENTRIES)

ResourcePoolsBuilder.newResourcePoolsBuilder () .heap (10, MemoryUnit.MB) sets the maximum cache space 10MB

WithExpiry (ExpiryPolicyBuilder.timeToIdleExpiration (Duration.ofMinutes (10) sets cache idle time

WithExpiry (ExpiryPolicyBuilder.timeToLiveExpiration (Duration.ofMinutes (10) sets cache survival time

Remove/removeAll actively invalidates the cache. Similar to Guava Cache, it does not clear the collection immediately after calling the method. It only determines whether the cache expires when get or put is called.

WithSizeOfMaxObjectSize limits the size of a single cache object, and objects that exceed these two limits are not cached.

Off-heap cache

Out-of-heap cache means caching data in out-of-heap memory. The space size is only limited by local memory size and is not managed by GC. Using out-of-heap cache can reduce GC pause time, but objects in out-of-heap memory need to be serialized and deserialized. KEY and VALUE must implement Serializable interface, so the speed is slower than in-heap cache. In Java, you can set the upper limit of out-of-heap memory through the-XX:MaxDirectMemorySize parameter.

CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder () .build (true); / / out-of-heap memory cannot be limited by storage entries, but only by memory size. If the limit is exceeded, the cache ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder () .offheap (10, MemoryUnit.MB) is reclaimed. CacheConfiguration cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder (String.class, String.class, resource) .withDispatcherConcurrency (4) .withExpiry (ExpiryPolicyBuilder.timeToLiveExpiration (Duration.ofMinutes (10) .withSizeOfMaxObjectSize (10, MemoryUnit.KB) .build (); Cache cache = cacheManager.createCache ("userInfo2", cacheConfig); cache.put ("website", "https://silently9527.cn");System.out.println(cache.get("website")); disk cache")

When cached data is stored on disk, the cached data will not be affected when JVM is restarted, and both heap cache and out-of-heap cache will be lost; and disk cache has more storage space; but the data cached on disk also needs to be serialized and will be slower than memory. It is recommended to use faster disks to bring greater throughput, such as using flash memory instead of mechanical disks.

CacheManagerConfiguration persistentManagerConfig = CacheManagerBuilder .persistence (new File ("/ Users/huaan9527/Desktop", "ehcache-cache"); PersistentCacheManager persistentCacheManager = CacheManagerBuilder.newCacheManagerBuilder () .with (persistentManagerConfig) .build (true); / / disk the third parameter set to true means to persist the data to disk ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder (). Disk (100, MemoryUnit.MB, true); CacheConfiguration config = CacheConfigurationBuilder .newCacheConfigurationBuilder (String.class, String.class, resource). Build () Cache cache = persistentCacheManager.createCache ("userInfo", CacheConfigurationBuilder.newCacheConfigurationBuilder (config)); cache.put ("official account", "Beta Learning JAVA"); System.out.println (cache.get ("official account"); persistentCacheManager.close ()

When JVM stops, be sure to call persistentCacheManager.close () to ensure that the data in memory can be dump to disk.

This is a typical heap + offheap + disk structure diagram. The upper layer is faster than the lower layer, and the lower layer has more storage space than the upper layer. In ehcache, the space size is set to heap > offheap > disk, otherwise an error will be reported; ehcache will save the hottest data in a higher-level cache. The code for this structure is as follows:

CacheManagerConfiguration persistentManagerConfig = CacheManagerBuilder .persistence (new File ("/ Users/huaan9527/Desktop", "ehcache-cache")); PersistentCacheManager persistentCacheManager = CacheManagerBuilder.newCacheManagerBuilder () .with (persistentManagerConfig) .build (true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder () .heap (10, MemoryUnit.MB) .offheap (100, MemoryUnit.MB) / / the third parameter is set to true, which supports persistence. Disk (500, MemoryUnit.MB, true) CacheConfiguration config = CacheConfigurationBuilder.newCacheConfigurationBuilder (String.class, String.class, resource). Build (); Cache cache = persistentCacheManager.createCache ("userInfo", CacheConfigurationBuilder.newCacheConfigurationBuilder (config)); / / write cache cache.put ("name", "silently9527"); / / read cache System.out.println (cache.get ("name")); / / need to manually release resources persistentCacheManager.close () before closing the program; distributed centralized cache

The in-heap cache and out-of-heap cache mentioned earlier will have two problems in the case of multiple JVM instances: 1. After all, the capacity of a single machine is limited; 2. The data cached by multiple JVM instances may be inconsistent; 3. If the cached data fails at the same time, the requests will be called to the database, and the pressure on the database will increase. At this time, we need to introduce distributed cache to solve the problem. Now the most frequently used distributed cache is redis.

When the distributed cache is introduced, the architecture of the application cache can be adjusted to the above structure.

Practice of caching usage patterns

The patterns used in caching are roughly divided into two categories: Cache-Aside and Cache-As-SoR (SoR represents the system that actually stores the data, that is, the data source)

Cache-Aside

The business code is written around the cache, usually to get the data from the cache. If the cache does not hit, look it up from the database, and then put the data into the cache after the query. When the data is updated, you also need to update the data in the cache accordingly. This model is also the one we usually use the most.

Read the scene

Value = cache.get (key); / / read data if from cache (value = = null) {value = loadFromDatabase (key); / / query cache.put (key, value) from database; / / put into cache}

Write the scene

WirteToDatabase (key, value); / / write to the database cache.put (key, value); / / put it into the cache or delete the cache cache.remove (key), and check it again when reading

The Cache extension of Spring is the Cache-Aside mode used. In order to separate the business code from the cached read and update, Spring encapsulates the Cache-Aside mode with AOP and provides multiple annotations to implement the read and write scenario. Official reference documentation:

@ Cacheable: it is usually placed on the query method, which implements the scenario of Cache-Aside reading. Check the cache first, and if it does not exist in the query database, finally put the query results into the cache.

@ CachePut: it is usually used to save the update method, which implements the scenario written by Cache-Aside. After the database is updated, the data is put into the cache.

@ CacheEvict: deletes the cache of the specified key from the cache

> for some basic data that allows a little update delay, you can consider using canal to subscribe to binlog logs to complete incremental updates to the cache. > > there is another problem with Cache-Aside. If the hotspot data cache fails at some point, many requests will be called to the back-end database at the same time, and the pressure on the database will increase instantly.

Cache-As-SoR

The Cache-As-SoR schema also treats Cache as a data source, all operations are for caching, and Cache is delegating to the real SoR to read or write. You will only see the operation of Cache in the business code, and this pattern is divided into three types

Read Through

The application always requests data from the cache, and if there is no data in the cache, it is responsible for retrieving data from the database using the provided data loader, and after retrieving the data, the cache updates itself and returns the data to the calling application. Gauva Cache, Caffeine, and EhCache all support this mode.

Caffeine implementation Read Through since the Gauva Cache implementation is similar to the Caffeine implementation, only the implementation of Caffeine is shown here. The following code comes from the official Caffeine documentation.

LoadingCache cache = Caffeine.newBuilder () .maximumSize (10000) .AfterWrite (10, TimeUnit.MINUTES) .build (key-> createExpensiveGraph (key)); / / Lookup and compute an entry if absent, or null if not computableGraph graph = cache.get (key); / / Lookup and compute entries that are absentMap graphs = cache.getAll (keys)

Specify a CacheLoader when build Cache

[1] call cache.get (key) directly in the application

[2] first query the cache and return the data directly if the cache exists

[3] if it does not exist, it will be delegated to CacheLoader to query the data in the data source, and then put it in the cache and return it to the application

> CacheLoader should not return null directly. It is recommended to encapsulate it into a self-defined Null object, which can prevent cache breakdown when it is put into the cache.

In order to prevent the pressure on the back-end database from increasing due to the invalidation of some hot data, I can use the lock restriction in CacheLoader to allow only one request to query the database, and all other requests will be fetched from the cache after the first request query is completed. In the last article, "long text chat Cache (I)", we talked about similar configuration parameters for Nginx.

Value = loadFromCache (key); if (value! = null) {return value;} synchronized (lock) {value = loadFromCache (key); if (value! = null) {return value;} return loadFromDatabase (key);}

Implementing Read Through with EhCache

CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder () .build (true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder () .heap (10, MemoryUnit.MB) / / set the maximum number of cache entries CacheConfiguration cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder (String.class, String.class, resource) .withExpiry (ExpiryPolicyBuilder.timeToLiveExpiration (Duration.ofMinutes (10) .withLoaderWriter (new CacheLoaderWriter () {@ Override public String load (String key) throws Exception {/ / load from database return "silently9527" } @ Override public void write (String key, String value) throws Exception {} @ Override public void delete (String key) throws Exception {}}) .build (); Cache cache = cacheManager.createCache ("userInfo", cacheConfig); System.out.println (cache.get ("name"))

In EhCache, CacheLoaderWriter is used to load data from the database; solving the problem of increasing pressure on the back-end database due to the failure of some hot data can also be implemented in load as in the above way.

Write Through

Similar to the Read Through pattern, when the data is updated, the SoR is updated first, and the cache is updated after success.

Implementing Write Through with Caffeine

Cache cache = Caffeine.newBuilder () .maximumSize (100) .writer (new CacheWriter () {@ Override public void write (@ NonNull String key, @ NonNull String value) {/ / write data to database System.out.println (key); System.out.println (value) } @ Override public void delete (@ NonNull String key, @ Nullable String value, @ NonNull RemovalCause removalCause) {/ / delete from database}}) .build (); cache.put ("name", "silently9527")

Caffeine uses CacheWriter to implement that Write Through,CacheWriter can synchronously listen to cache creation, change and delete operations, and update the cache only if the write is successful.

Implementing Write Through with EhCache

CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder (). Build (true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder (). Heap (10, MemoryUnit.MB); / / set the maximum number of cache entries CacheConfiguration cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder (String.class, String.class, resource) .withExpiry (ExpiryPolicyBuilder.timeToLiveExpiration (Duration.ofMinutes (10) .withLoaderWriter (new CacheLoaderWriter () {@ Override public String load (String key) throws Exception {return "silently9527") } @ Override public void write (String key, String value) throws Exception {/ / write data to database System.out.println (key); System.out.println (value) } @ Override public void delete (String key) throws Exception {/ / delete from database}) .build (); Cache cache = cacheManager.createCache ("userInfo", cacheConfig); System.out.println (cache.get ("name")); cache.put ("website", "https://silently9527.cn");")

EhCache is also implemented through CacheLoaderWriter. When we call cache.put ("xxx", "xxx") to write cache, EhCache will first delegate the write operation to CacheLoaderWriter, and CacheLoaderWriter.write will be responsible for writing the data source.

Write Behind

This mode usually writes the data to the cache first and then asynchronously to the database for data synchronization. This design can not only reduce the direct access to the database and reduce the pressure, but also merge the operation of multiple modifications to the database, which greatly improves the bearing capacity of the system. However, this model also carries risks, such as the possibility of data loss when the cache machine goes down.

If Caffeine wants to implement Write Behind, it can send data to MQ in the CacheLoaderWriter.write method to achieve asynchronous consumption, which can ensure the security of data, but in order to implement merge operation, it needs to extend the more powerful CacheLoaderWriter.

Implementing Write Behind with EhCache

/ / 1 define thread pool PooledExecutionServiceConfiguration testWriteBehind = PooledExecutionServiceConfigurationBuilder .newPooledExecutionServiceConfigurationBuilder () .pool ("testWriteBehind", 5,10) .build (); CacheManager cacheManager = CacheManagerBuilder.newCacheManagerBuilder () .using (testWriteBehind) .build (true); ResourcePoolsBuilder resource = ResourcePoolsBuilder.newResourcePoolsBuilder () .build (10, MemoryUnit.MB) / / set the maximum number of cache entries / / 2 set the writeback mode configuration WriteBehindConfiguration testWriteBehindConfig = WriteBehindConfigurationBuilder .newUnBatchedWriteBehindConfiguration () .queueSize (10) .concurrencyLevel (2) .useThreadPool ("testWriteBehind") .build () CacheConfiguration cacheConfig = CacheConfigurationBuilder .newCacheConfigurationBuilder (String.class, String.class, resource) .withLoaderWriter (new CacheLoaderWriter () {@ Override public String load (String key) throws Exception {return "silently9527") } @ Override public void write (String key, String value) throws Exception {/ / write data to database} @ Override public void delete (String key) throws Exception {}) .add (testWriteBehindConfig) .build (); Cache cache = cacheManager.createCache ("userInfo", cacheConfig)

First define the thread pool configuration using PooledExecutionServiceConfigurationBuilder; then use WriteBehindConfigurationBuilder to set the write mode configuration, where newUnBatchedWriteBehindConfiguration means no batch write operation, because it is asynchronous write, so the write operation needs to be put into the queue first, the queue size is set through queueSize, and useThreadPool specifies which thread pool to use; concurrencyLevel sets how many concurrent threads and queues are used for WriteBehind

It is also easy for EhCache to implement batch write operation.

First, replace newUnBatchedWriteBehindConfiguration () with newBatchedWriteBehindConfiguration (10, TimeUnit.SECONDS, 20). Here, batch processing is performed if the number reaches 20, and if not within 10 seconds, it will be processed.

Secondly, wirteAll and deleteAll are implemented in CacheLoaderWriter for batch processing.

> if you need to merge operations on the same key to record only the last data, you can enable merging through enableCoalescing ()

At this point, I believe you have a deeper understanding of "how to understand application-level caching". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.