Shulou(Shulou.com)05/31 Report--

This article mainly explains "what are the tuning skills of MongoDB WiredTiger engine in Mongo". The explanation content in this article is simple and clear, easy to learn and understand. Please follow the ideas of Xiaobian and go deep into it slowly to study and learn "what are the tuning skills of MongoDB WiredTiger engine in Mongo" together!

MongoDB introduced the concept of pluggable storage engines in 3.0. Currently, there are quite a few storage engines to choose from: MMAPV1, WiredTiger, MongoRocks, TokuSE, etc. Each storage engine has its own advantages, and you need to choose the most suitable one based on performance requirements and application characteristics.

Starting with 3.2.x, WiredTiger became the default storage engine. As MongoDB's most popular storage engine today, WiredTiger has the following advantages over the original MMAPV1:

Performance & concurrency: WiredTiger has significantly higher performance than MMAPV1 under most workloads. The WiredTiger engine is tailor-made for modern multicore systems to better utilize the processing power of multicore systems. The MMAPV1 engine uses table-level locking, so throughput is limited when there are concurrent operations on a single table. WiredTiger uses document-level locking, resulting in increased concurrency and throughput. For typical applications, switching to the WiredTiger engine can result in a 5- 10x performance boost.

Compression & Encryption: The MMAPV1 engine requires data to be in consistent form in memory and on disk (map disk memory map). Therefore, it does not support compression and encryption. WiredTiger does not have this layer limitation and can be better supported.

Index Prefix Compression: Wired Tiger stores indexes using prefix compression--the same prefix is stored only once. The result: smaller indexes and less use of physical memory.

Next, I'll show you a few key parameters used to tune Wired Tiger engine performance.

Tuning Cache Size

WiredTiger's most important tuning parameter is cache size. By default, MongoDB reserves 50% of available physical memory as a data cache starting with 3.x (60% in 3.2). Although the default settings will work for most applications, it is worth tweaking to find the best configuration for a particular application. The cache size must be large enough to hold the entire working set of the application.

In addition to this cache, MongoDB requires additional memory for operations such as aggregation, sorting, join management, etc. Therefore, it is important to ensure that sufficient memory is available, otherwise the MongoDB process risks being killed by an OOM killer.

To adjust this parameter, first understand how cache is used in the default configuration. Run the following command to get cache statistics:

db.serverStatus().wiredTiger.cache

An example of command output is as follows:

{ "tracked dirty bytes in the cache" : 409861, "tracked bytes belonging to internal pages in the cache" : 738956332, "bytes currently in the cache" : 25769360777, "tracked bytes belonging to leaf pages in the cache" : 31473298388, "maximum bytes configured" : 32212254720, "tracked bytes belonging to overflow pages in the cache" : 0, "bytes read into cache" : 29628550664, "bytes written from cache" : 34634778285, "pages evicted by application threads" : 0, "checkpoint blocked page eviction" : 102, "unmodified pages evicted" : 333277, "page split during eviction deepened the tree" : 0, "modified pages evicted" : 437117, "pages selected for eviction unable to be evicted" : 44825, "pages evicted because they exceeded the in-memory maximum" : 74, "pages evicted because they had chains of deleted items" : 33725, "failed eviction of pages that exceeded the in-memory maximum" : 1518, "hazard pointer blocked page eviction" : 34814, "internal pages evicted" : 21623, "maximum page size at eviction" : 10486876, "eviction server candidate queue empty when topping up" : 8235, "eviction server candidate queue not empty when topping up" : 3020, "eviction server evicting pages" : 191708, "eviction server populating queue, but not evicting pages" : 2996, "eviction server unable to reach eviction goal" : 0, "pages split during eviction" : 8821, "pages walked for eviction" : 157970002, "eviction worker thread evicting pages" : 563015, "in-memory page splits" : 52, "percentage overhead" : 8, "tracked dirty pages in the cache" : 9, "pages currently held in the cache" : 1499798, "pages read into cache" : 2260232, "pages written from cache" : 3018846}

The first value to look at is the percentage of dirty data in cache. If this percentage is high, then increasing the cache size is likely to improve performance. If the application is reread, you can pay attention to the bytes read into cache metric. If this metric is high, then increasing cache size is likely to improve read performance.

Adjusting cache size does not have to restart the service, we can adjust it dynamically:

db.adminCommand( { "setParameter": 1, "wiredTigerEngineRuntimeConfig": "cache_size=xxG"})

If you want the tweaks to work after a reboot, you need to tweak the configuration file accordingly.

Control Read/Write Tickets

WiredTiger uses tickets to control the number of read/write operands that can be processed simultaneously by the storage engine. The default value is 128, which works well in most cases. If this value drops to 0 frequently, all subsequent operations will be queued. For example, if a drop in read tickets is observed, the system may have a large number of time-consuming operations (unindexed operations). If you want to find out what is slow, you can use some third-party tools. You can adjust tickets up or down depending on system needs and performance impacts.

Run the following command to confirm the use of tickets:

db.serverStatus().wiredTiger.concurrentTransactions

Here is an example of output:

{ "write" : { "out" : 0, "available" : 128, "totalTickets" : 128 }, "read" : { "out" : 3, "available" : 128, "totalTickets" : 128 }}

Similarly, tickets can be dynamically adjusted:

db.adminCommand( { setParameter: 1, wiredTigerConcurrentReadTransactions: xx } )db.adminCommand( { setParameter: 1, wiredTigerConcurrentWriteTransactions: xx } )

Once adjustments are made, watch the performance of the system to ensure that the impact is as expected.

Thank you for reading, the above is "MongoDB WiredTiger engine tuning skills in Mongo" content, after learning this article, I believe you have a deeper understanding of MongoDB WiredTiger engine tuning skills in Mongo, the specific use needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.