Shulou(Shulou.com)06/01 Report--

[problem description]

Add secondary:10.9.197.6:27017 to the production environment with 140G of data, but it hasn't caught up with the data after one day of synchronization. Check the synchronization situation as follows:

Looking at the master-slave replication status command, the results are consistent in the following two ways:

Method 1:

Use admin

Db.runCommand ({replSetGetStatus: 1})

The specified value does not affect the output of the command. The data provided by this command is derived from the data contained in the heartbeat sent to the current instance by other members of the replica set. Due to the frequency of the heartbeat, these data may be out of date in a few seconds. For more information, please refer to the official file: https://docs.mongodb.com/manual/reference/command/replSetGetStatus/

Method 2:

Rs.status ()

Check the replication status and find that the status is "stateStr": "RECOVERING". The message is "infoMessage": "could not find member to sync from", and using rs.syncFrom ("10.9.161.130 rs.syncFrom 27017") will not allow it to continue to synchronize normally. The specific information is as follows:

Method 1: stop the application, so that the main library will not have an operation, so that the oplog window will not be smaller than the operation of the main library. Method 2: adjust the size of oplog

If the library cannot be stopped, it is obvious that method one is not appropriate, and method two should be chosen: adjust the oplog size.

Modify oplog size

Method 1: under the condition of constant service

Reference file: https://docs.mongodb.com/v3.2/tutorial/change-oplog-size/

1 Restart a Secondary in Standalone Mode on a Different Port

1) db.shutdownServer ()

2) restart the instance in stand-alone mode with other ports without using the-- replSet parameter

Method 1: start mongo according to the settings of the production environment parameters file, that is, specify the non-default parameters

The following parameters are set according to the production parameter file, and the situation varies:

/ data/servers/app/mongodb-3.2.8/bin/mongod-port 37017-dbpath / data/servers/data/mg27017/data/-directoryperdb-wiredTigerDirectoryForIndexes-nojournal &

This method is more troublesome, it is recommended to choose the following method 2:

Method 2: modify the parameter file: annotate the replSet section, modify the port to 37017, and then start mongo with the modified control file

/ data/servers/app/mongodb-3.2.8/bin/mongod-f / data/servers/data/mg27017/mongod.conf

The screenshot below shows that as long as the part is changed, the port number can be changed to any one that is not occupied, and here it will be changed to 37017

Netstat-anp | grep $port to check whether the port number has been started

2 Create a Backup of the Oplog (Optional)

In stand-alone mode (non-replSet mode), the collection corresponding to the existing oplog,oplog of port 37017 is the oplog.rs under the local database. Here are the specific commands:

/ data/servers/app/mongodb-3.2.8/bin/mongodump-- db local-- collection 'oplog.rs'-- port 37017-- host=127.0.0.1-uroot-p111111111-- authenticationDatabase=admin-o / data/servers/data/mg27017/dump

[command description]

-o (--out) is the output directory. This directory requires the user who performs the backup to have the appropriate permissions and does not need to be created in advance.

-- authenticationDatabase is the authentication database corresponding to user name and password. If the environment does not require password authentication, then-u,-p,-- authenticationDatabase do not need to be specified.

3 Recreate the Oplog with a New Size and a Seed Entry

Save the last entry in oplog

Log in to local database

Use local

Define object: db

Db = db.getSiblingDB ('local')

Use the temp collection to save the last entry, which ensures that there is no data in it: db.temp.drop (), make sure that the data can be deleted before deletion, and if not, it is the same with another collection. Temp has no data here

Using the db.collection.save () method: find the last entry in reverse order in natural order and save it to a temporary collection

Db.temp.save (db.oplog.rs.find ({}, {ts: 1, h: 1}) .sort ({$natural:-1}) .limit (1) .next ()

The result after insertion is

4 Remove the Existing Oplog Collection

Delete the oplog.rs collection under local, and the result is returned as true

Db = db.getSiblingDB ('local')

Db.oplog.rs.drop ()

5 Create a New Oplog

Create an oplog.rs fixed collection and set the size to 4G, which is based on the actual situation

Db.runCommand ({create: "oplog.rs", capped: true, size: (4: 1024 * 1024: 1024)})

6 Insert the Last Entry of the Old Oplog into the New Oplog

Insert the last entry of the previously saved oplog into the new oplog

Db.oplog.rs.save (db.temp.findOne ())

The comparison is consistent with the results of temp.

7 Restart the Member

Shut down the stand-alone instance. You need to use admin to close it.

Use admin

Db.shutdownServer ()

Restore the previously changed operation and start mongo

/ data/servers/app/mongodb-3.2.8/bin/mongod-f / data/servers/data/mg27017/mongod.conf

Check the master-slave replication status to make sure the status is normal

Db.runCommand ({replSetGetStatus: 1}) or rs.status ()

8 Repeat Process for all Members that may become Primary

Repeat this procedure for all oplog members who want to change the secondary size.

9 Change the Size of the Oplog on the Primary

For the master library, you need to first cut the master library into slave libraries, and then repeat the above oplog adjustment process

Method 1:

Rs.stepDown ()

Method 2:

Config=rs.conf ()

Config.members [2] .priority = 6

Rs.reconfig (config)

The number 2 here is the order in which the secondary in rs.conf () to become the main library is located, starting with 0 and has nothing to do with id. The maximum number of priority becomes the main library. After the old main library is adjusted, remember to change the priority to 1.

Method 2: when the service is out of service

This method is the easiest to operate, but needs to be out of service. The specific steps are

1 close the mongod instance (all nodes)

Use admin

Db.shutdownServer ()

2 Delete all files under the local database (PRIMARY node)

Rm-rf / data/servers/data/mg27017/local/*

3 delete mongo data directory (secondary)

If you are not sure who is the main library, put the data directory under mv.

Rm-rf / data/servers/data/mg27017/data/*

4 modify all node configuration files (oplogsize)

OplogSizeMB: 4096

5 restart all nodes mongod

/ data/servers/app/mongodb-3.2.8/bin/mongod-f / data/servers/data/mg27017/mongod.conf

This method will cause if the master library is abnormal, no slave library can be switched, and this method is not recommended.

[section]

Which oplog is appropriate to set? you can estimate an appropriate size based on the current data size, io and approximate oplog window time.

Rs.printReplicationInfo ()

Log length start to end: when oplog is full, it can be understood as a time window.

Oplog last event time: the time when the last operation occurred

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.