Shulou(Shulou.com)06/01 Report--

MongoDB version: 3.2.6

CacheSizeGB: 32

Engine: WiredTiger

Schemas: replica set

The business request MongoDB timed out.

Check the MongoDB log, there are a large number of slow update statements, check the server monitoring through netdata, eliminate io problems, and find that 100% of single-core CPU continues to appear.

View the CPU usage of MongoDB processes through pidstat

[root@127-0-0-1 ~] # pidstat-t-p ALL | grep mongod

It is suspected that mongodb itself scrubbed the dirty page. Check the mongod process through perf.

[root@127-0-0-1] # perf top-p pid

Then select Annotate to find the hot spot (which can be accurate to the CPU instruction): (forget to take a screenshot, just take a screenshot)

In order to ensure the availability and stability of the online business, first switch primary to provide normal services for the business.

Solution:

1. Upgrade the version to 3.2.10, and officially fix the problem.

2. Eviction parameter tuning: lower eviction_target or eviction_dirty_target, so that evict can swipe data from wiredtiger's cache to operating system's page cache as soon as possible, so as to brush disk early.

Db.runCommand ({setParameter: 1, wiredTigerEngineRuntimeConfig: "eviction_dirty_target=5,eviction_target=80"})

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.