Shulou(Shulou.com)06/01 Report--

Oplog plays a key role in replica set synchronization of MongoDB. Can Oplog Size be changed at will?

Https://docs.mongodb.com/manual/reference/configuration-options/#replication.oplogSizeMB

Official configuration documentation description

Replication.oplogSizeMB Type: integer The maximum size in megabytes for the replication operation log (i.e., the oplog). Starting in MongoDB 4.0, the oplog can grow past its configured size limit to avoid deleting the majority commit point. By default, the mongod process creates an oplog based on the maximum amount of space available. For 64-bit systems, the oplog is typically 5% of available disk space. Once the mongod has created the oplog for the first time, changing the replication.oplogSizeMB option will not affect the size of the oplog. To change the oplog size of a running replica set member, use the replSetResizeOplog administrative command. ReplSetResizeOplog enables you to resize the oplog dynamically without restarting the mongod process. See Oplog Size for more information. The replication.oplogSizeMB setting is available only for mongod.

If no configuration specifies the default oplog size, it defaults to 5% of the current disk space on 64-bit operating systems.

One generation of oplog space is valid for life, and subsequent changes to the mongod.conf configuration file and restart of the service will not take effect.

Added instructions in MongoDB 3.6 + that can dynamically adjust the replica set Oplog Size without the need to restart the mongod service.

View the Oplog Size size allocated in the current mongod service

There are several similar instructions to view the currently allocated oplog size size

Sh2:PRIMARY > rs.printReplicationInfo () configured oplog size: 12958.975341796875MBlog length start to end: 938secs (0.26hrs) oplog first event time: Thu Apr 11 2019 15:03:05 GMT+0800 (CST) oplog last event time: Thu Apr 11 2019 15:18:43 GMT+0800 (CST) now: Thu Apr 11 2019 15:18:43 GMT+0800 (CST) sh3:PRIMARY > rs.printReplicationInfo () sh3:PRIMARY > db.printReplicationInfo () sh3:PRIMARY > db.getReplicationInfo () {"logSizeMB": 102 400 "usedMB": 905.03, "timeDiff": 4704, "timeDiffHours": 1.31, "tFirst": "Thu Apr 11 2019 13:54:30 GMT+0800 (CST)", "tLast": "Thu Apr 11 2019 15:12:54 GMT+0800 (CST)", "now": "Thu Apr 11 2019 15:12:54 GMT+0800 (CST)"}

LogSizeMB / configured oplog size: the amount of space currently configured, in MB.

UsedMB: currently used space size, in MB.

TFirst / oplog first event time: the current recording time of the beginning of the oplog.

TLast / oplog last event time: the time when the latest changes to the current oplog were recorded.

TimeDiff / log length start to end: the current change recording time in the current mongod-the last recording time, in seconds.

TimeDiffHours / log length start to end: the current change recording time in the current mongod-the last recording time, in hours.

Plan the oplog size size that needs to be changed

Obtain the number of mongod instance writes during peak hours according to different business scenarios

Sh2:PRIMARY > rs.printReplicationInfo () configured oplog size: 12958.975341796875MBlog length start to end: 938secs (0.26hrs) oplog first event time: Thu Apr 11 2019 15:03:05 GMT+0800 (CST) oplog last event time: Thu Apr 11 2019 15:18:43 GMT+0800 (CST) now: Thu Apr 11 2019 15:18:43 GMT+0800 (CST)

As shown above, most of the projects are insert, update and other update operations, so the 12G oplog space can only store the last 900s change operation log at the peak of the business.

If you need to keep the oplog for a longer cycle, you need to reasonably plan for a larger oplog.rs space.

Overview of changing oplog size size

Oplog exists internally as an upper limit set, so you cannot modify its size during normal operation.

In most cases, the default oplog size is an acceptable size

However, in some cases, you may need a larger or smaller oplog.

For example, if your application performs a large number of updates or deletions in a short period of time, you may need to change the oplog size.

In order to modify the oplog size, we need to do maintenance manual operations for each node in the replication set. The process requires stopping the mongod process, starting it as a non-single node, modifying the oplog size, and then restarting the node.

[important]

Please make sure that we start with the slave node (Secondary) of the replica set and end up maintaining the master node (Primary).

Process flow

Restart the node in single-node mode

You can use the rs.stepDown () directive on the Primary node to force a manual switch to the Secondary node.

And keep the entry of the oplog as the query condition, empty the oplog.rs set and re-establish the oplog.rs collection of the specified size.

Start the mongod instance in replica set mode.

1. First close the mongod instance on the secondary node. For example, turn it off with the db.shutdownServer () command:

> db.shutdownServer ()

Restart the mongod instance in single-node mode (excluding the-- replSet parameter) on the other ports. The command is as follows:

# sudo-u mongod mongod-- port 37017-- dbpath / var/lib/mongo

two。 Copy an existing Oplog (optional)

We can choose to back up the oplog of the existing node just in case, as follows:

Mongodump-port 37017-db local-collection 'oplog.rs'

3. Rebuild oplog with new size and Seed Entry

Save the latest entries in oplog. For example, connect to mongo Shell and enter the local database with the following command:

> use local

In the mongo Shell window, we can also set up db using the following command:

> db = db.getSiblingDB ('local')

Make sure the local.temp collection is empty:

> db.temp.drop ()

Use natural order sorting to find the last piece of data in oplog and insert it into the local.temp collection with the db.collection.save () command:

> db.temp.save (db.oplog.rs.find ({}, {ts: 1, h: 1}) .sort ({$natural:-1}) .limit (1) .next ()

Verify the last piece of oplog data saved in the local.temp collection:

> db.temp.find ()

4. Delete existing old Oplog collections

Delete the old oplog.rs collection from the local library with the following command:

> db = db.getSiblingDB ('local') > db.oplog.rs.drop ()

The result will return true

5. Create a new Oplog collection size

Create a new oplog (new size) through the create command. Specify size (unit is bytes). The following command creates an oplog collection with a size of 20 * 1024 * 1024 * 1024, that is, 20g:

> db.runCommand ({create: "oplog.rs", capped: true, size: (20 * 1024 * 1024 * 1024)})

When the command is executed successfully, the following is returned:

{"ok": 1}

6. Insert the last old piece of oplog data previously saved in the local.temp collection into the new oplog collection. For example:

> db.oplog.rs.save (db.temp.findOne ())

Confirm with the following command:

> db.oplog.rs.find ()

7. Finally, restart the mongod instance in replica set mode.

> db = db.getSiblingDB ('admin') > db.shutdownServer () # service mongod restart

The replication set will be restored and the "catch up" data will be restored before it becomes the primary node.

8. Repeat the above on all machines where we want to change the oplog size. Finally, the operation is done on the primary node.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.