Shulou(Shulou.com)05/31 Report--

This article shows you what the role of oplog in MongoDB is, the content is concise and easy to understand, it can definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

Introduction to 1:oplog

Oplog is a fixed collection under the local library, and Secondary copies it by looking at the oplog collection of Primary. Each node has an oplog that records the information copied from the primary node so that each member can serve as a source of synchronization to other nodes.

Oplog can be said to be the link of Mongodb Replication.

2: the process of data synchronization in replica set

The detailed process of data synchronization in the replica set: the Primary node writes the data, the Secondary gets the replication information by reading the oplog of the Primary, starts to replicate the data and writes the replication information to its own oplog. If an operation fails (only if the data of the synchronous source is corrupted or inconsistent with the primary node), the backup node stops copying data from the current data source. If a backup node dies for some reason, when it is restarted, it will automatically start synchronization from the last operation of oplog. After the synchronization is completed, the information will be written to its own oplog. Since the replication operation is to copy the data first and then write to oplog, it is possible that two copies of the same operation will be synchronized. However, MongoDB considered this problem at the beginning of its design and performed the same operation of oplog many times. The effect is the same as that of one execution.

Function:

When Primary writes, these write records are written to the Oplog of Primary, and then Secondary copies the Oplog to the local machine and applies these operations, thus realizing the function of Replication.

At the same time, because it records write operations on the Primary, it can also be used for data recovery.

You can simply think of it as a binlog in Mysql.

The growth rate of 3:oplog

Oplog is a fixed size, and it can only save a certain number of operation logs. Usually, the growth rate of oplog usage space is about the same as the speed at which the system processes write requests. If the 1KB write data is processed every minute on the master node, then oplog also writes 1KB data every minute. If multiple documents are affected by a single operation (such as deleting multiple documents or updating multiple documents), oplog may have multiple action logs. If db.testcoll.remove () deletes 1000000 documents, there will be 1000000 action logs in oplog. If there are a large number of operations, it is possible that oplog will soon be full.

Size:

Oplog is a capped collection.

On 64-bit Linux, Solaris, FreeBSD, and Windows systems, Mongodb sets its size to 5% of the available disk space by default (the default minimum is 1G and the maximum is 50G), or you can set the oplogSize in the mongo.conf to the desired value before initializing the mongodb replication set instance.

Local.oplog.rs a capped collection collection. You can use the-- oplogSize option to set the size of the collection from the command line.

However, because of Oplog, it ensures the normal operation of replication, as well as the security and disaster tolerance of data.

4:oplog considerations:

A special collection of local.oplog.rs. Used to record the operations of the Primary node.

The size of the oplog

Capped collection is a fixed-size collection in MongoDB that provides high-performance insert, read, and delete operations. When the collection is filled, the new inserted document overwrites the old document.

Therefore, it makes sense for oplog tables to use capped collection because it is not possible to grow oplog indefinitely. MongoDB has a default oplog size when initializing the replica set:

On 64-bit Linux,Solaris,FreeBSD and Windows systems, MongoDB allocates 5% of the remaining disk space as the size of oplog, and if this part is less than 1GB, allocate 1GB space

183MB is assigned on 64 OS X systems

On 32-bit systems, only 48MB is assigned.

The size setting of oplog is a problem worth considering. If the oplog size is too large, the storage space will be wasted; if the oplog size is too small, the old oplog records will be overwritten quickly, so it is easy for the down node to fail to synchronize data.

For example, based on the above example, we stop a backup node (port=33333), then insert the following record through the primary node, and then look at the oplog and find that the previous oplog has been overwritten.

Connect to this node through MongoDB shell and you will find that the node has been in the RECOVERING state.

Solution:

Data synchronization

In a replica set, there are two ways to synchronize data:

Initial sync (initialization): this process occurs when a new database is created in the replica set or one of the nodes has just recovered from the downtime, or when a new member is added to the replica set, by default, the node in the replica set replicates oplog from the nearest node to synchronize data, which can be primary or the secondary node with the latest oplog replica.

This operation generally reinitializes the backup node, which is expensive.

Replication (replication): this operation continues after initialization to maintain data synchronization between secondary nodes.

Initial sync

When you encounter a problem that cannot be synchronized in the above example, you can only use the following two ways to initial sync

The first way is to stop the node, then delete the files in the directory, and restart the node. In this way, the node will execute the initial sync

Note: in this way, the sync time is based on the amount of data. If the amount of data is too large, the sync time will be very long.

At the same time, there will be a lot of network transmission, which may affect the work of other nodes.

The second way is to stop the node, then delete the files in the directory, find a relatively new node, and then copy the files in the node directory to the node directory where you want to sync

In either of the above ways, the node of "port=33333" can be restored. Change the error that has been in the RECOVERING state.

View the information of oplog

You can view the information of oplog through the "db.printReplicationInfo ()" command

Field description:

Configured oplog size: oplog file size

Log length start to end: the enabling period of the oplog log

Oplog first event time: the time when the first transaction log was generated

Oplog last event time: the time when the last transaction log was generated

Now: the time now

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.