Shulou(Shulou.com)06/01 Report--

Goal: to solve the problem that a large number of Log writes take up a lot of File Cache, and inadequate content utilization leads to swap

Basic principle: try to use memory, reduce swap, at the same time, flush to external memory as soon as possible, and release memory for writing cache as early as possible. -this optimization is very effective, especially in continuous write operations.

Tuning measures:

Change vm.swapiness: 60 to 10vm.dirty_ratio:90, change 10vm.dirty_background_ratio:60 to 5vm.dirty_expire_centisecs:3000, change 500vm.vfs_cache_pressure:100 to 500

The following highlights the meaning of each configuration:

First, vm.swappiness optimization:

The size of the value of swappiness has a lot to do with how swap partitions are used. Swappiness=0 means to maximize the use of physical memory, followed by swap space, and swappiness=100 means to actively use swap partitions and move the data in memory to swap space in time. The basic default setting for linux is 60, as shown below:

Cat / proc/sys/vm/swappiness60

In other words, the use of swap partitions begins to appear when you use 100-60% of your memory. As we all know, the speed of memory will be much faster than that of disk, which will increase the io of the system. At the same time, a large number of pages will be swapped in and out, seriously affecting the performance of the system. So at the operating system level, we should use memory as much as possible and adjust this parameter.

The method of temporary adjustment is as follows, we adjust it to 10:

Sysctl vm.swappiness=10vm.swappiness = 10cat / proc/sys/vm/swappiness10

This is only a temporary adjustment. After rebooting, you will return to the default setting.

To make a permanent adjustment, you need to modify it in / etc/sysctl.conf, plus:

Cat / etc/sysctl.confvm.swappiness=10

Second, vm.dirty_ratio: brushing dirty pages synchronously will block the application.

This parameter controls the size of the synchronous write buffer of the file system, in percentage, indicating that when the write buffer uses how much system memory is used (that is, when the number of dirty pages cached by the file system reaches how many percent of the system memory (for example, 10%), start writing data to disk, that is, the system has to start processing cache dirty pages (because there are already a large number of dirty pages at this time In order to avoid data loss, some dirty pages need to be brushed into external memory), in this process, many application processes may be blocked because the system turns to processing the file IO.

As a result, more system memory is used for disk write buffering, which can also greatly improve the write performance of the system. However, when you need a continuous, constant write situation, you should lower its value, which usually defaults to 10 on startup.

Third, vm.dirty_background_ratio: brush dirty pages asynchronously and will not block the application

This parameter controls the background process of the file system and when to refresh the disk. The unit is the percentage, which represents the percentage of system memory, which means that when the write buffer uses the amount of system memory, it will trigger background write-back processes such as pdflush/flush/kdmflush to run, flushing certain cached dirty pages into external memory asynchronously. As a result, more system memory is used for disk write buffering, which can also greatly improve the write performance of the system. However, when you need a continuous, constant write situation, you should reduce its value, which is usually 5 by default on startup.

Note: if the dirty_ratio setting is larger than dirty_background_ratio, you may think that the trigger condition of dirty_ratio cannot be achieved, because the condition of vm.dirty_background_ratio must be reached each time. However, it is true that the condition of vm.dirty_background_ratio is reached first and then the flush process is triggered to perform asynchronous write-back operations, but the application process can still perform write operations during this process. If more than one application process writes more than the flush process brushes out, it will naturally reach the threshold set by the parameter vm.dirty_ratio. At this time, the operating system will switch to the process of dealing with dirty pages synchronously, blocking the application process.

Fourth, vm.dirty_expire_centisecs:

After this parameter declares that the data in the Linux kernel write buffer is "old", the pdflush process starts to consider writing to disk. The unit is one hundred seconds. The default is 3000, that is, 30 seconds of data will be refreshed even if it is old. For particularly overloaded write operations, it is good to shrink this value appropriately, but not too much, because too much reduction will also cause the IO to increase too quickly. It is recommended to set it to 1500, which means 15 seconds is old. Of course, if your system has a lot of memory, and the write mode is intermittent, and the amount of data written each time is small (for example, tens of megabytes), then this value is better.

Fifth, Vm.dirty_writeback_centisecs:

This parameter controls the running interval of the kernel's dirty data refresh process pdflush. The unit is one hundred seconds. The default value is 500, which is 5 seconds. If your system is writing continuously, it's actually better to lower this value so that you can flatten the spike of writes into multiple writes. The setting method is as follows:

Echo 2000s > / proc/sys/vm/dirty_writeback_centisecs

If your system is a short-term peak write operation, and the write data is small (tens of megabytes per time), and the memory is more rich, then you should increase this value:

6. Vm.vfs_cache_pressure:

Increasing this parameter sets the tendency of virtual memory to reclaim directory and inode buffers, the higher the value. The easier it is to recycle

This file indicates the kernel's tendency to recycle memory for directory and inode cache; the default value of 100 indicates that the kernel will keep directory and inode cache at a reasonable percentage based on pagecache and swapcache; lowering this value below 100 will cause the kernel to tend to retain directory and inode cache; by more than 100, which will cause the kernel to tend to recycle directory and inode cache.

This variable controls the tendency of the kernel to reclaim thememory which is used for caching of VFS caches, versus pagecache and swap.Increasing this value increases the rate at which VFS caches are reclaimed.Itis difficult to know when this should be changed, other than byexperimentation. The slabtop command (part of the package procps) shows topmemory objects used by the kernel. The vfs caches are the "dentry" and the "* _ inode_cache" objects. If these are consuming a largeamount of memory in relation to pagecache, it may be worth trying to increasepressure. Could also help to reduce swapping. The default value is 100.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.