Shulou(Shulou.com)06/01 Report--

Journal Principle:

WiredTiger uses checkpoints to provide a consistent view of data on disk and allows MongoDB to recover from the last checkpoint. However, if MongoDB unexpectedly exits between checkpoints, logging needs to be used to recover information that occurred after the last checkpoint.

Through logging, the recovery process is as follows:

View the data file to find the identifier of the last checkpoint.

Searches the log file for records that match the identifier of the previous checkpoint.

Apply these actions in the log file since the last checkpoint.

MongoDB WiredTiger first uses memory buffers to store log records and does not write to disk until it exceeds 128 kB. WiredTiger synchronizes buffered journal records to disk based on the following time intervals or conditions:

3.2 New features: every 50 milliseconds.

In version 3.6: MongoDB set checkpoints at 60-second intervals.

WiredTiger forces WiredTiger log file synchronization if the write operation contains a write command with j: true.

Because MongoDB uses a log file size limit of 100 MB, WiredTiger creates a new log file approximately every 100 MB. When WiredTiger creates a new log file, WiredTiger synchronizes the previous log file.

Note: While log records remain in the WiredTiger buffer between write operations, updates may be lost when mongod is forced to close.

Log mode:

Do not open journal, write wiredtiger data, and will not immediately persist storage; instead, it will make a full checkpoint every minute (storage.syncPeriodSecs configuration item, default is 1 minute) to persist all data. When journal is turned on, an operation log is recorded for each write (journal can reconstruct the written data). In this way, even if there is an outage, Wiredtiger will restore the data to the point of the most recent checkpoint at startup, and then replay the subsequent journal operation log to recover the data.

The journal behavior in MongoDB is mainly controlled by two parameters, storage.journal.enabled determines whether to enable journal, storage.journal.commitInternalMs determines the interval between journal swipes (the maximum interval between journal operations). It can be between 2-300ms, with low values helping persistence but adding an extra burden to the disk.

If journal and data files are on the same disk, the default is 100 ms. 30ms if on a different disk.

If you force mongod to submit log files, you can specify j:true, after specifying, the time becomes one-third of the original), the default is 100ms, users can also specify writeConcern as {j: true} when writing to ensure that the journal swipes every time.

Log Switch:

use admin

db.runCommand( { logRotate : 1 } )

The latter:

1 3 * * * killall -SIGUSR1 mongod

3 3 * * * killall -SIGUSR1 mongos

15 3 * * * /usr/bin/find /data/mongodb/data/prod_shard2_1/log/ -name "mongodb.log.* " -mtime +30 -exec rm -fr {} \;

19 3 * * * /usr/bin/find /data/mongodb/data/prod_configdb2/log/ -name "mongodb.log.* " -mtime +30 -exec rm -fr {} \;

21 3 * * * /usr/bin/find /data/mongodb/data/prod_mongos/log/ -name "mongodb.log.* " -mtime +30 -exec rm -fr {} \;

Log level added to profile:

use admin

db.runCommand( {

setParameter: 1,

logComponentVerbosity: {

verbosity: 1,

query: {

verbosity: 2

},

storage: {

verbosity: 2,

journal: {

verbosity: 1

}

}

}

} )

Or:

verbose&&quiet

By default, mongodb log files are very large, recording a large number of debugging information such as connections every second, which has a great impact on our management logs and positioning errors. We need to modify its default log level.

The log level of mongodb is determined by the verbose and quiet parameters in the configuration file. verbose indicates that debug information will be printed. The example configuration is as follows

verbose=true

vv=true

Note: vv here indicates debug level, there are vv-vv, the more v, the more detailed log information recorded.

If the quiet=true parameter is set, it means that the output will be quiet, that is, there will be no debug information, and only some key information will be printed in the log, such as automatic failover, system errors and other information, equivalent to error log. You need to comment out the verbose parameter. Example configurations are as follows:

#verbose=true

quiet=true

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.