Shulou(Shulou.com)05/31 Report--

This article mainly introduces what is the defect of RDB and AOF persistence mode in Redis, it has certain reference value, interested friends can refer to it, I hope you can learn a lot after reading this article, let's take a look at it.

1. Defects of RDB persistence mode

1. Problem description:

After 4 hours, a large number of API calls began to fail. The error message is as follows:

{"data": {"sendResult": null}, "base": {"returncode": "99999", "returndesc": "system exception: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error."}, "qrybase": {"total": 0, "count": 0, "start": 0}}

2. Cause analysis:

Interpret the error message and think that it is caused by insufficient disk, but it is found that the disk still has 42% left, as shown below:

So the Redis log is opened according to the error message, and the stress test continues. The interface still reports an error, but from the Redis log information

Can't save in background: fork: Cannot allocate memory

The improper use of memory by the process is related to the fact that the main process occupies memory as follows: it takes up nearly 55% of the 4G memory.

Specific reason: Redis to save data to the hard disk in order to avoid the death of the main process, need to Fork a main process, and then in the Fork process to save the data to the hard disk operation, if the main process uses the memory of 2.2GB, form child process when the need for additional 2.2GB, then the memory is not enough, Fork failed, and then the data to save the hard disk also failed.

3. Mitigation plan (which can not fundamentally solve the problem):

3.1 modify the configuration item stop-writes-on-bgsave-error no in the redis.conf file (default is yes), that is, stop writing data to disk when an error occurs in the bgsave snapshot operation, so that subsequent write errors will fail. In order not to affect the subsequent write operation, you need to change the value of this item to no

3.2 modify kernel parameters (in the following 3 ways), but root permission is required:

(1) Edit / etc/sysctl.conf, change vm.overcommit_memory=1, and then sysctl-p to make the configuration file effective (2) sysctl vm.overcommit_memory=1 (3) echo 1 > / proc/sys/vm/overcommit_memory

II. Defects of AOF persistence mode

1. Question 1 description:

All the master and slave nodes of Redis enable AOF mode, and the simulation keeps writing Redis. After 15 minutes, a large number of API calls begin to fail, and the Linux virtual server where Redis resides hangs.

The error message for the API is as follows:

{"data": null, "base": {"returndesc": "system exception", "returncode": "999999"}, "qrybase": null} Biz (dubbo) API error is as follows: 2015-06-05 11returncode 11returncode: error while validate jedisdisredis.clients.jedisConnectionException: java.net.SocketTimeoutException: Read timed out

Cause analysis:

From the error message of the dubbo interface, it is caused by the Redis timeout of the interface API operation. From the system log and IO monitoring, it is indicated that the above problems are caused by the IO bottleneck (the system IO is too busy), as follows:

It can also be seen from the system log that the IO blocking time is more than 120 seconds, and the machine hangs due to the system security mechanism.

Summary

The test results show that the AOF pattern has the most obvious defect, that is, when the access pressure is high, IO will become a performance bottleneck, which will lead to the unavailability of services.

3. Mitigation plan (can not solve the problem fundamentally)

Edit / etc/sysctl.conf and add the following configuration:

Vm.dirty_background_ratio = 5vm.dirty_ratio = 10

Then sysctl-p makes the configuration file effective.

Question 2 description:

No matter using AOF mode or RDB (snapshot mode), when the size of two files (.aof or .rdb) exceeds 80% of the system memory, the dint Redis process will be dropped by the system Kill, causing the service to be unavailable.

The above problems show that we need to plan the capacity of the system memory in advance when using Redis, because once the Redis goes down, it will cause a lot of data loss and is unrecoverable.

Thank you for reading this article carefully. I hope the article "what are the defects of RDB and AOF persistence mode in Redis" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.