Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about how to understand the Redis memory recovery strategy, which may not be well understood by many people. in order to make you understand better, the editor has summarized the following for you. I hope you can get something according to this article.

Overview

Redis can also cause errors due to insufficient memory, or may lead to long-term system standstill due to long-term recycling, so it is necessary to master the implementation of recycling strategy. In the Redis configuration file, when the memory of the Redis reaches the specified maximum, it is allowed to configure one of the six strategies to eliminate key values and reclaim some key-value pairs.

Maxmemory-policy parameter # Set a memory usage limit to the specified amount of bytes.# When the memory limit is reached Redis will try to remove keys# according to the eviction policy selected (see maxmemory-policy). # # If Redis can't remove keys according to the policy, or if the policy is# set to 'noeviction', Redis will start to reply with errors to commands# that would use more memory, like SET, LPUSH, and so on, and will continue# to reply to read-only commands like GET.## This option is usually useful when using Redis as an LRU or LFU cache Or to# set a hard memory limit for an instance (using the 'noeviction' policy). # # WARNING: If you have slaves attached to an instance with maxmemory on,# the size of the output buffers needed to feed the slaves are subtracted# from the used memory count, so that network problems / resyncs will# not trigger a loop where keys are evicted, and in turn the output# buffer of slaves is full with DELs of keys evicted triggering the deletion# of more keys, and so forth until the database is completely emptied.## In short... If you have slaves attached it is suggested that you set a lower# limit for maxmemory so that there is some free RAM on the system for slave# output buffers (but this is not needed if the policy is' noeviction'). # # maxmemory# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory# is reached. You can select among five behaviors:## volatile-lru-> Evict using approximated LRU among the keys with an expire set.# allkeys-lru-> Evict any key using approximated LRU.# volatile-lfu-> Evict using approximated LFU among the keys with an expire set.# allkeys-lfu-> Evict any key using approximated LFU.# volatile-random-> Remove a random key among the ones with an expire set.# allkeys-random-> Remove a random key, any key.# volatile-ttl-> Remove the key with the nearest expire time (minor TTL) # noeviction-> Don't evict anything Just return an error on write operations.## LRU means Least Recently Used# LFU means Least Frequently Used## Both LRU, LFU and volatile-ttl are implemented using approximated# randomized algorithms.## Note: with any of the above policies, Redis will return an error on write# operations When there are no suitable keys for eviction.## At the date of writing these commands are: set setnx setex append# incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd# sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby# zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby# getset mset msetnx exec sort## The default is:## maxmemory-policy noeviction proactive cleanup strategy

Proactive cleanup strategy has implemented a total of 6 memory elimination strategies before Redis 4.0.After that, two more strategies have been added, a total of 8:

[deal with key with expiration time set]

Volatile-ttl: when filtering, key-value pairs with expiration time set will be deleted according to the sequence of expiration time. The earlier the expiration time, the earlier it will be deleted.

Volatile-random: just like its name, delete randomly from the key-value pair with the expiration time set.

Volatile-lru: the LRU algorithm is used to filter the deletion of key-value pairs that have the expiration time set.

Volatile-lfu: the LFU algorithm is used to filter the deletion of key-value pairs with expiration time set.

[do processing for all key]

Allkeys-random: randomly selects and deletes data from all key-value pairs.

Allkeys-lru: use the LRU algorithm to filter and delete all data.

Allkeys-lfu: use the LFU algorithm to filter and delete all data.

[do not process (default)]

Noeviction: no data is stripped out, all writes are rejected and the client error message "(error) OOM command not allowed when used memory" is returned, where Redis only responds to read operations.

Redis uses the noeviction policy by default. In other words, if the memory is full, no write operation is provided, only a read operation is provided. Obviously, this often does not meet our requirements, because for Internet systems, millions or more users are often involved, so it is often necessary to set up a recycling strategy.

Strategy selection

LRU algorithm (Least Recently Used, least recently used): eliminates data that has not been accessed for a long time, using the last access time as a reference

LFU algorithm (Least Frequently Used, least frequently used): eliminates the data that has been accessed the least in the most recent period of time, using the number of times as a reference

It should be pointed out that LRU algorithm or TTL algorithm is not a very accurate algorithm, but an approximate algorithm. Redis does not determine the most accurate time value by comparing all key-value pairs to determine which key-value pair to delete, as this will take too much time and cause garbage collection to take too long, resulting in service standstill.

When there is hot data, the efficiency of LRU is very good, but sporadic and periodic batch operations will lead to a sharp decline in LRU hit rate and serious cache pollution. It may be better to use LFU at this time.

Configure maxmemory-policy (default is noeviction) according to your business type, and it is recommended to use volatile-lru.

Maxmemory-sample

In the default configuration file of Redis, there is the parameter maxmemory-sample

# LRU, LFU and minimal TTL algorithms are not precise algorithms but approximated# algorithms (in order to save memory), so you can tune it for speed or# accuracy. For default Redis will check five keys and pick the one that was# used less recently, you can change the sample size using the following# configuration directive.## The default of 5 produces good enough results. 10 Approximates very closely# true LRU but costs more CPU. 3 is faster but not very accurate.## maxmemory-samples 5

The larger the setting of maxmemory-samples, the more accurate the Redis deletes, but at the same time, the downside is that Redis takes more time to calculate the more accurate values that match.

The disadvantage of the recycling timeout strategy is that the key-value pair of the timeout must be specified, which will bring some code that sets the timeout to the program development, which undoubtedly increases the workload of the developer.

By recycling all the key-value pairs, it is possible to delete the key-value pairs that are in use, increasing the instability of storage.

For garbage collection strategies, it is also important to pay attention to the time of collection, because the system is slow during garbage collection by Redis.

Therefore, it is good to control the recovery time, but the time should not be too short or too long. Too short will lead to too frequent recycling times, and too long will lead to a single garbage collection pause time is too long, which is not conducive to the stability of the system, which requires designers to think in the actual work.

If the maximum memory is not set, when the Redis memory exceeds the physical memory limit, the data in the memory will begin to swap with the disk frequently, resulting in a sharp decline in the performance of the Redis.

After reading the above, do you have any further understanding of how to understand the Redis memory recovery strategy? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.