Shulou(Shulou.com)06/01 Report--

Editor to share with you why the data written in Redis is gone, I believe most people do not understand, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

1. Why is the data I wrote in Redis missing?

Students who use Redis, you should understand one thing: why do you use Redis? What is the function of using redis? What are the advantages of using redis? Think more about why and the reasons behind everything.

Not long ago, a friend told me why the Redis of their production environment often lost some data. It's written in, and it may be gone after a while. Oh, my God, if you ask this question, it means that Redis is useless. Redis is a cache. You pawned it and used it, didn't you?

First of all, you need to understand what is caching? Why use caching?

Redis is cached in memory. Is the memory unlimited? On the contrary, memory is precious and limited, and disks are cheap and abundant. A machine may have dozens of gigabytes of memory, but it can have several T of hard disk space. Redis is mainly based on memory for high-performance, high-concurrency read and write operations.

Well, since the memory is limited, for example, Redis can only use 10 gigabytes, you keep writing data in it, keep writing and writing the last 10 gigabytes, you can also write, what do you think will happen? Of course, it will kill some of the data, and then keep 10 gigabytes of data. Do you think it will cause data loss?

What kind of data will Redis kill? What kind of data do you keep? Of course, it is to get rid of the less commonly used data and keep the commonly used data.

So this is one of the most basic concepts of caching: data is out of date. Either you set an expiration time or Redis kills it yourself.

So if you use your Redis improperly, store your production data in it, and don't persist it to mysql, you may lose it.

2. My data is obviously out of date, why is it still occupying memory?

Another is that if you set an expiration time for key, you know that if you check the key at a certain time, there will be no more, but do you know how redis makes it expired for you? When will it be deleted?

If you don't know, in the actual use process, you may find such a question: why a lot of data should be out of date, only to find that the redis memory footprint is still very high? That's because you don't know how Redis deletes expired key.

For example, Redis has a total of 10 gigabytes of memory. You now write 5 gigabytes of data into it, and then you set all the data to expire after 10 minutes. As a result, 10 minutes later, why is the memory usage of Redis still 50%? Five gigabytes of data are out of date, I look up from redis, is not found, the result of expired data why still occupy the memory of Redis?

If you don't even know this question, you will be confused and unable to answer it. I suggest you do more homework before using Redis, otherwise when you write the code, you will take it for granted that the data written into Redis will exist, which will lead to various vulnerabilities and bug in the system, which will be difficult to do.

3. Problem analysis (1) set expiration time

The set key value expiration time (1 hour) means that the key,1 hours entered by set will be gone and will be invalid.

When we set key, we can all give an expire time, that is, the expiration time, and specify that this key can only survive for 1 hour, for example? Ten minutes? This is very useful and we can specify that the cache expires.

If you set a batch of key to survive for only 1 hour, how does redis delete the batch of key after the next hour?

The answer is: regular deletion + lazy deletion

The so-called periodic deletion means that by default, Redis randomly selects some key with expiration time set every 100ms to check whether they have expired, and delete them if they expire.

Why is it selected at random?

Suppose you put 100000 key in the Redis and set the expiration time, and you check 100000 key every few hundred milliseconds, then the redis is basically dead, because the cpu load will be very high, all spent on your check expired key.

So it's not every 100ms that traverses all the key,Redis that sets the expiration time. It would be a performance disaster if it was set to check all Key. So in fact, redis randomly selects some key every other 100ms to check and delete.

But the problem is, random sampling to see if key has in the past led to regular deletion of policies may result in a lot of expired key not being deleted by the time, so what? So another strategy for Redis is lazy deletion.

Lazy deletion means that when you get a key, Redis will check whether the key has expired if the expiration time is set. If it expires, it will be deleted and nothing will be returned to you.

So it's not that the key will be deleted when the time comes, but when you query the key, Redis is lazy to check it again.

Through the above two means, it is guaranteed that expired key will be killed.

Then the problem just now is not difficult to understand, that is to say, your expired key is not deleted by regular deletion, but still stays in memory and occupies your memory. Unless your system checks that key, it will be deleted by redis. If all are out of date, regularly deleted only a little bit, and you did not check, did not trigger lazy delete, then your redis memory usage will not come down in a short period of time.

But in fact, this is still a problem. What happens if you regularly delete and miss a lot of expired key, and then you don't check it in time, so you don't delete it lazily? What if a large number of expired key accumulates in memory, resulting in the depletion of redis memory blocks?

Don't worry, Redis has a plan: memory obsolescence.

(2) memory elimination strategy

If Redis takes up too much memory, memory obsolescence occurs at this time. Redis provides the following rich optional strategies:

1) noeviction: when there is not enough memory to hold new write data, the new write operation will report an error.

(this is usually not used by anyone, it's really disgusting)

2) allkeys-lru: when there is not enough memory to hold newly written data, remove the least recently used key from all key spaces

(this is the most commonly used)

3) allkeys-random: when there is not enough memory to hold newly written data, a key is randomly removed from all key spaces.

(this is usually not used by anyone, why do you have to kill my important key at random? it must be the least used one recently.)

4) volatile-lru: when there is not enough memory to hold the newly written data, remove the least recently used key from the key space where the expiration time is set.

(this is generally not appropriate.)

5) volatile-random: when there is not enough memory to hold newly written data, a key is randomly removed from the key space where the expiration time is set.

6) volatile-ttl: when there is not enough memory to hold the newly written data, the key with earlier expiration time is removed first in the key space where the expiration time is set.

For example, there are 10 key in Redis. Now the memory is full, and the elimination strategy is allkeys-lru. At this time, Redis needs to delete some key to ensure that you can continue to write. Of the 10 key, 1 key has been queried 100 times in the last minute, 1 key, 50 times in the last 10 minutes, 1 key, and once in the last hour. I'm sure the ones that have been least used recently have been killed.

Why is the data stored in redis sometimes lost?

Quite simply, you write too much data, the memory is full, or some conditions are triggered, such as redis uses the allkeys-lru memory elimination strategy to automatically clean up some of the data you have rarely used recently.

The above is all the contents of the article "Why are the data written in Redis gone?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.