Shulou(Shulou.com)05/31 Report--

In this issue, the editor will bring you about how to optimize Linux performance through NoSQL memory database. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

Not only do we have to choose NoSQL databases that suit our tastes, such as memory-based Memcache or Redis, but also Cassandra or HBase that are more suitable for enterprise architecture, but optimization for Linux servers is also essential. The following editor will explain how to optimize Linux performance based on NoSQL in-memory database.

Linux performance optimization is a big topic, involving the hierarchical architecture of the application stack, not an exact science. This article, the first in a series of Linux performance tuning articles, focuses on Linux server performance optimization for NoSQL in-memory databases. To do this, we will ReFactor the kernel of the Linux server.

You may have downloaded and compiled the Linux kernel yourself, so don't forget the risks before you ReFactor the Linux kernel.

The workload challenges we mentioned can be addressed by optimizing the following options in vanilla 3.x kernel:

CONFIG_TASK_IO_ACCOUNTING: very important, allowing you to monitor the disk activity status of the server process.

CONFIG_SLUB:Chris Lameter's kernel object caching system. Compared to SLAB, it is more efficient in managing kernel memory allocation.

CONFIG_JUMP_LABEL: kernel branch optimization to make the kernel faster.

CONFIG_NUMA and friends: suitable for multi-core processors, enable NUMA for the kernel, and improve consistent support for cache and memory.

CONFIG_SPARSEMEM_VMEMMAP: sparse memory optimization options for pfn_to_page and page_to_pfn functions.

CONFIG_TRANSPARENT_HUGEPAGE, CONFIG_COMPACTION, CONFIG_MIGRATION, CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS: these parameters can be used to allocate memory pages larger than 4KB to speed up the allocation of memory for memory hunger processes. In addition, they can compress and migrate memory pages to securely query large pages and further reduce memory fragmentation.

How to optimize Linux performance based on NoSQL main memory database

CONFIG_KSM: untagging memory pages in MADV_MERGEABLE is a very important mechanism that can save memory in the ultimate way.

CONFIG_ZRAM: provides a memory-based block device. Data written in this block device is compressed and stored in memory. This is very useful for temporary storage space (mounted in the / tmp directory), and since this feature is used for temporary areas, please build it as a module and use it carefully.

CONFIG_ZCACHE CONFIG_ZSMALLOC & CONFIG_CLEANCACHE: an in-memory page compression framework that transparently compresses clean and swapped pages providing significant performance improvements for memory-based workloads. CleanCache uses the ZCache framework to compress in-memory pages, which can effectively reduce the number of requests in a high-memory environment. Because this feature is used in temporary areas, use it with caution.

Of course, our optimization is based on the x86x64 architecture. In addition, other options involve performance optimization for workloads:

CONFIG_PERF_EVENTS: kernel performance counter, used with tools such as perf. Especially suitable for in-depth performance monitoring.

CONFIG_PROFILING: performance analysis counter, used with tools such as OProfile. As important as PERF_EVENTS.

HAVE_BPF_JIT: a time-based compiler for PCAP (packet capture library)-based user tools that use Berkeley packet filters (Berkeley Packet Filter, such as tcpdump) to greatly improve the processing performance of complex rules. Enable it through / proc/sys/net/core/bpf_jit_enable.

We can choose the option that works best for us, save, compile and run it, and see if the final result can give you a sense of achievement!

There may be many other technologies that can help us build an efficient infrastructure and optimize memory-based workloads (for example, RAMSter can provide swap clusters). In any case, we need to test the results of our kernel optimization to build a stable production environment.

The above is the editor for you to share how to optimize the performance of Linux through the NoSQL memory database, if you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.