Shulou(Shulou.com)06/02 Report--

This article will explain in detail about the adjustment of MONGODB large memory parameters and the relationship between checkpoint and performance. The content of the article is of high quality, so the editor will share it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

Each kind of database has its own characteristics, and the business it faces is different, which will lead to each kind of database need to be adjusted to meet the needs of a certain business.

In most scenarios, the configuration of MONGODB is relatively rough, which is, of course, relative to PG, MYSQL, a variety of minor adjustments, pondering some parameters. Well, in fact, the parameters of MONGODB itself do not need to be pondered, he can adapt to various situations, quantitative change to qualitative change, we all understand.

Let's review the contents of the document.

1 by default, MONGODB 3.4 uses 50% memory in the database with 1G memory or more, and if 256MB is used in the case of less than 1G memory, it provides memory services for MONGODB.

The question is why it is 50% instead of 60%-80% like MYSQL, that is, the bigger the better. Here, if you have used POSTGRESQL, you may understand why MONGODB should be 50% instead of larger, because they all have to carry out related data processing based on LINUX's buffering mechanism.

PG does not repeat, and the main reason why MONGODB mainly uses LINUX buffering technology is data compression. People who have used MONGODB all know that the ratio between the compressed data and the actual data size of MONGODB is relatively large, which will save data storage space and related data processing costs.

However, any data needs to be decompressed before it is processed, and if the decompression is from disk to memory, the speed and related performance consumption will not be too low, so MONGODB chooses LINUX's buffer cache as an environment for decompression and compression.

Question 1: how often does MONGODB checkpoint? let's make a test to answer this question.

From the picture you can clearly see this problem, the time is 1 minute, 1 minute for checkpoint operation.

Here, through the program, the MONGODB is pressure tested, resulting in 3000 connections, each connection writing 200000 rows of data.

There will be a contradiction here, if I have a large memory, such as 512GB, and use half of the memory 256GB, then refresh the dirty pages and brush the data into the disk every 60 seconds.

So we will have a few questions to consider, a lot of data is written, do we have time to flush these memory data into disk in 1 minute, and what happens if we can't finish it? Whether the pressure on the disk will be under great pressure at the moment. Is this a matter that needs to be considered when it is subject to a large number of writes in some MONGODB databases?

The important eviction_trigger is to ensure that when the percentage of memory is used, the data from memory will be brushed to disk, while eviction_target will always brush data into disk as long as the share of memory is greater than the set value of 80% by default.

Eviction_targetcontinue evicting until the cache has less total memory than the value, as a percentage of the total cache size. Must be less than eviction_trigger.an integer between 10 and 99; default 80.eviction_triggertrigger eviction when the cache is using this much memory, as a percentage of the total cache size.

In the case of high concurrent writes and insufficient memory, the main library crashed. Here are the related pre-crash logs.

Imagine that if you have a lot of memory, use default parameters, and continue to write a lot, your disk performance is still average, hehe.

So the purpose of this text is how to adjust the parameters in the above case to optimize the current problem.

Of course, if your disk is SSD and the number of writes is small, you can ignore everything discussed here.

Db.adminCommand ({"setParameter": 1, "wiredTigerEngineRuntimeConfig": "eviction= (threads_min=3,threads_max=6), checkpoint= (wait=120), eviction_trigger=80,eviction_target=50"})

After the adjustment, continue with 3000 concurrent data of 200000 each. You can compare the first graph in the article from the following figure, and you can see that the dirty is compared to the first graph.

So for MONGODB with large memory, in the case of high concurrent and high writes, proper adjustment of eviction_trigger and eviction_target may improve the performance of the system.

In addition, the above adjusted parameters need to be adjusted according to their own system, not according to the parameters in the article.

On the adjustment of MONGODB large memory parameters and how the relationship between checkpoint and performance is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.