Shulou(Shulou.com)05/31 Report--

This article will explain in detail how MongoDB uses memory in Linux. The editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

How Linux manages memory

In Linux (similar to other systems), there is physical memory and virtual memory in memory. There is no need to explain what physical memory is. Virtual memory is actually an abstraction of physical memory. In most cases, for convenience reasons, the program accesses the virtual memory address, and then the operating system translates it into physical memory address.

Many people will confuse virtual memory with Swap. In fact, Swap is just a technology derived from virtual memory: once the operating system runs out of physical memory, in order to free up memory space for new content, it will put the contents of the current physical memory into the swap partition and retrieve it later. It should be noted that the use of Swap may bring performance problems, so occasionally there is no need to be nervous. Unfortunately, data exchanges between physical memory and swap partitions occur frequently, which is called Swap bumps. Once this happens, it is necessary to know what causes it. If it is out of memory, it can be solved by adding memory. However, sometimes this problem may occur even if there is enough memory. For example, this may happen in MySQL. The solution is to limit the use of Swap:

Shell > sysctl-wvm.swappiness=0

The most common way to check memory is the free command:

Shell > free-m totalusedfreesharedbufferscached Mem:32101293772723023925880-/ + buffers/cache:325828842 Swap:204702047

Beginners see that the used column value is too large and the free column value is too small, so they tend to think that memory is running out. In fact, this is not the case, because whenever we operate files, Linux will cache the files in memory as much as possible, so that the next time we visit, we can get the results directly from memory, so the value in the cached column is very large, but don't worry, this part of memory is recyclable, the operating system will eliminate cold data according to LRU algorithm. In addition to cached, there is a buffers, which is similar to cached and is recyclable, but its focus is on alleviating blockages caused by inconsistent operating speeds of different devices, which will not be explained here.

Knowing the principle, we can infer that the memory available to the system is free+buffers+cached:

Shell > echo "2723" 239 "25880" | bc-l

28842

As for the memory actually used by the system is used-buffers-cached:

Shell > echo "29377-239-25880" | bc-l

3258

In addition to the free command, you can also use the sar command:

Shell > sar-r kbmemfreekbmemused%memusedkbbufferskbcached 32243922964773290.1924611626070160 31163242975580090.5299226157372 29595202991260491.0024555626316396 27923007987691.54568026485672 27182603015386491.73245684265640 shell > sar-W pswpin/spswpout/s 0.000.00 0.000.00 0.000.00

I hope you are not frightened by% memused. If you are unfortunately right, please refer to the explanation of the free command.

How to use memory in MongoDB in Linux

Currently, MongoDB uses a memory-mapped storage engine, which converts disk IO operations into memory operations. If it is a read operation, the data in memory acts as a cache. If it is a write operation, memory can also convert random write operations into sequential write operations, which can greatly improve performance. MongoDB does not interfere with memory management, but leaves these tasks to the virtual cache manager of the operating system. The advantage is that it simplifies the work of MongoDB, but the disadvantage is that you have no way to easily control how much memory MongoDB takes up. In fact, MongoDB will take up all available memory, so it is best not to put other services and MongoDB together.

Sometimes, even if MongoDB uses a 64-bit operating system, it may encounter the infamous OOM problem, which is mostly due to the limit on the size of virtual memory. You can view the current value like this:

Shell > ulimit-a | grep'virtual'

Most operating systems set it to unlimited by default, and if your operating system is not, you can modify it like this:

Shell > ulimit-vunlimited

Note, however, that the use of ulimit is contextual and is best placed in the startup script of MongoDB.

Sometimes, for some reason, you may want to release the memory occupied by MongoDB, but as mentioned earlier, memory management is controlled by the virtual memory manager, so usually you can only release memory by restarting the service. You must be disdainful of this method, but fortunately you can use MongoDB's built-in closeAllDatabases command to achieve this goal:

Mongo > useadmin

Mongo > db.runCommand ({closeAllDatabases:1})

In addition, the cache can also be freed by adjusting the kernel parameter drop_caches:

Shell > sysctl-wvm.drop_caches=1

Normally, you can monitor the memory usage of MongoDB through the mongo command line, as follows:

Mongo > db.serverStatus () .mem: {"resident": 22346, "virtual": 1938524, "mapped": 962283}

You can also monitor the memory usage of MongoDB through the mongostat command, as shown below:

Shell > mongostat mappedvsizeresfaults 940g1893g21.9g0 940g1893g21.9g0 940g1893g21.9g0 940g1893g21.9g0 940g1893g21.9g0

The meaning of memory-related fields is:

Mapped: data size mapped to memory

Visze: amount of virtual memory consumed

Res: the amount of memory actually used

Note: if the operation cannot be completed in memory, the value of the faults column will not be 0, and there may be performance problems depending on the size.

In the above results, vsize is twice the size of mapped, and mapped is equal to the size of the data file, so vsize is twice the size of the data file. This is because in this case, MongoDB turns on journal and needs to map the data file in memory one more time. If you turn off journal, vsize and mapped are roughly the same.

If you want to verify this, you can observe the file mapping through the pmap command after turning journal on or off:

Shell > pmap$ (pidofmongod)

How much memory is appropriate for MongoDB? Generally speaking, the more the better. To be exact, it depends on your data and the size of the index. It is best to have all the data and indexes in memory, but in many cases, the data will be larger than memory, such as the MongoDB instance mentioned in this article:

Mongo > db.stats () {"dataSize": 1004862191980, "indexSize": 1335929664}

In this example, the index is only more than 1G, and the memory can be installed completely, while the data file reaches 1T. It is estimated that it is difficult to find such a large memory. At this time, it is guaranteed that the memory can hold hot data. As for the number of hot data, this is a question of proportion, depending on the specific application. This makes the memory size clear: memory > index + hot data.

This is the end of this article on "how MongoDB uses memory in Linux". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.