Shulou(Shulou.com)05/31 Report--

Redis how to use memory elimination strategy, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

Redis configuration memory

1. Configure the memory size through the configuration file by adding the following configuration to the redis.conf configuration file under the Redis installation directory

/ / set the maximum memory occupied by Redis to 100m maxmemory 100mb

The configuration file of redis does not necessarily use the redis.conf file under the installation directory. When starting the redis service, you can pass a parameter to specify the configuration file of redis.

2. Modify Redis by command to support dynamic modification of memory size at run time through command

/ / set the maximum memory occupied by Redis to 100m 127.0.0.1 Redis 6379 > config set maxmemory 100mb / / get the maximum memory size that can be used by set Redis 127.0.0.1 Redis 6379 > config get maxmemory

If you do not set the maximum memory size or set the maximum memory size to 0, the memory size is not limited in 64-bit operating systems, and the maximum use of 3GB memory in 32-bit operating systems

Memory obsolescence of Redis

Now that you can set the maximum memory occupied by Redis, there is a time when the configured memory runs out. Then when the memory is used up, will there be no memory available if you continue to add data to the Redis? In fact, Redis defines several strategies to deal with this situation:

1.noeviction (default policy): no service is provided for write requests, and errors are returned directly (except for DEL requests and some special requests)

2.allkeys-lru: phase out using the LRU algorithm from all key

3.volatile-lru: use LRU algorithm to eliminate from key with expiration time set

4.allkeys-random: randomly eliminates data from all key

5.volatile-random: randomly eliminated from key with expiration time set

6.volatile-ttl: in a key with an expiration time set, it is eliminated according to the expiration time of the key. The earlier it expires, the more priority it will be eliminated.

When using volatile-lru, volatile-random, and volatile-ttl strategies, if no key can be eliminated, an error will be returned just like noeviction

How to get and set the memory elimination policy to obtain the current memory elimination policy:

127.0.0.1 purl 6379 > config get maxmemory-policy

Set the phase-out policy through the configuration file (modify the redis.conf file):

Maxmemory-policy allkeys-lru

Modify the phase-out policy by command:

127.0.0.1 purl 6379 > config set maxmemory-policy allkeys-lru

LRU algorithm

1. What is LRU? It is mentioned above that the maximum memory available for Redis has been used up, and the LRU algorithm can be used for memory elimination, so what is the LRU algorithm?

LRU (Least Recently Used), which is the least recently used, is a cache replacement algorithm. When using memory as a cache, the size of the cache is generally fixed. When the cache is full and you continue to add data to the cache, you need to eliminate some of the old data and free up memory space to store the new data. At this point, you can use the LRU algorithm. The core idea is that if a data is not used recently, it is unlikely to be used in the future, so it can be eliminated.

Using java to implement a simple LRU algorithm

Public class LRUCache {/ / capacity private int capacity; / / current statistics of how many nodes are private int count; / / cache node private Map nodeMap; private Node head; private Node tail; public LRUCache (int capacity) {if (capacity)

< 1) { throw new IllegalArgumentException(String.valueOf(capacity)); } this.capacity = capacity; this.nodeMap = new HashMap(); //初始化头节点和尾节点，利用哨兵模式减少判断头结点和尾节点为空的代码 Node headNode = new Node(null, null); Node tailNode = new Node(null, null); headNode.next = tailNode; tailNode.pre = headNode; this.head = headNode; this.tail = tailNode; } public void put(k key, v value) { Node node = nodeMap.get(key); if (node == null) { if (count >

= capacity) {/ / remove a node removeNode () first;} node = new Node (key, value); / / add node addNode (node);} else {/ / move node to header node moveNodeToHead (node) }} public Node get (k key) {Node node = nodeMap.get (key); if (node! = null) {moveNodeToHead (node);} return node;} private void removeNode () {Node node = tail.pre; / / remove removeFromList (node) from the linked list NodeMap.remove (node.key); count--;} private void removeFromList (Node node) {Node pre = node.pre; Node next = node.next; pre.next = next; next.pre = pre; node.next = null; node.pre = null } private void addNode (Node node) {/ / add nodes to header addToHead (node); nodeMap.put (node.key, node); count++;} private void addToHead (Node node) {Node next = head.next; next.pre = node; node.next = next; node.pre = head Head.next = node;} public void moveNodeToHead (Node node) {/ / remove removeFromList (node) from the linked list; / / add nodes to the header addToHead (node);} class Node {k key; v value; Node pre; Node next Public Node (k key, v value) {this.key = key; this.value = value;}

The above code implements a simple LUR algorithm, the code is very simple, but also annotated, it is easy to read carefully.

Implementation of LRU in Redis

1. Approximate LRU algorithm Redis uses the approximate LRU algorithm, which is not quite the same as the conventional LRU algorithm. The approximate LRU algorithm eliminates the data by random sampling, randomly producing 5 (default) key at a time, eliminating the least recently used key from it.

You can modify the number of samples through the maxmemory-samples parameter: for example, the larger the configuration of maxmemory-samples 10 maxmenory-samples, the closer the elimination result is to the strict LRU algorithm.

In order to implement the approximate LRU algorithm, Redis adds an additional 24bit field to each key to store the time when the key was last accessed.

2.Redis3.0 optimizes the approximate LRU Redis3.0 optimizes the approximate LRU algorithm. The new algorithm maintains a candidate pool (size 16). The data in the pool is sorted according to the access time, and the first randomly selected key will be put into the pool, and then each randomly selected key will be put into the pool only when the access time is less than the minimum time in the pool, until the candidate pool is full. When full, if there is a new key to be put in, remove the last access time (most recently accessed) from the pool.

When it needs to be eliminated, just select the key that has the least recent access time (the longest has not been accessed) from the pool to be eliminated.

3.LRU algorithm comparison we can compare the accuracy of each LRU algorithm through an experiment, first add a certain amount of data n to Redis, so that Redis available memory is used up, and then add new data to Redis, at this time we need to eliminate part of the data. If we follow the strict LRU algorithm, we should eliminate the data that was added first. Generate a comparison diagram of the following LRU algorithms

You can see that there are three different colored dots in the picture:

1. Light gray is the eliminated data.

two。 Gray is the old data that has not been eliminated.

3. Green is the new data.

We can see that the graph generated by a Redis3.0 sample number of 10 is closest to the strict LRU. Using the same five samples, Redis3.0 is also better than Redis2.8.

LFU algorithm

LFU algorithm is a new elimination strategy added in Redis4.0. Its full name is Least Frequently Used, and its core idea is to eliminate the key according to the frequency of recent visits, the priority of those who are rarely visited is eliminated, and those who are visited more are left behind. The LFU algorithm can better express the popularity of a key being visited. If you use the LRU algorithm, a key has not been accessed for a long time, only occasionally, then it is considered to be hot data, will not be eliminated, and some key is likely to be accessed in the future will be eliminated. This will not happen if you use the LFU algorithm, because using it once does not make a key hot data. LFU has two strategies:

Volatile-lfu: use LFU algorithm to phase out key in key with expiration time set

Allkeys-lfu: use the LFU algorithm to eliminate data in all key

Set to use these two elimination strategies as mentioned above, but it should be noted that the policy can only be set in Redis4.0 and above in these two weeks. If you set it below Redis4.0, an error will be reported.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.