Shulou(Shulou.com)06/02 Report--

MongoDB how to back up and export imported data, I believe that many inexperienced people do not know what to do, so this article summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

On Tuesday, a classmate asked how to back up MONGODB and how to migrate data. Recently, we are going to do data migration for a project, including MONGODB. It happens to look at MongoDb's view of data migration and backup from the point of view of a project. If there are any omissions or incorrect views, please correct them.

To do this, in fact, there must be a test link, or production environment, here with two stand-alone MOGNODB4.0 to operate. MONGODB itself supports the migration and import of two kinds of data, of course, there will also be data migration between two replication collections.

General small systems use mongodump and mongorestore for system backup and recovery mongodump can be used to dump entire databases, collections, or query results. Mongodump can dump the oplog to generate a consistent snapshot of the data. The mongorestore utility restores data to a new or existing database. Mongorestore imports content from the BSON database dump generated by mongodump and replays oplog.

Mongodump captures only documents in the database. The resulting backup is space-efficient, but mongorestore or mongod must rebuild the index after the data is restored.

But in fact, many online MONGODB systems may not be backed up and restored in this way.

But let's first take a look at the relevant commands, which are still used for small systems to back up.

Backup (J is the thread used for the operation,-- gzip is compressed)

Mongodump-host=192.168.198.180-- port=27027-- username=admin-- password=1234.com-- authenticationDatabase=admin-- gzip-- db test-v-j 8-o / mongodata/backup

Data recovery

Mongorestore-host=192.168.198.181-- port=27027-- username=admin-- password=1234.com-- authenticationDatabase=admin-- db test-v-- drop-J8-- gzip-- dir='/mongodata/backup/test'

For a stand-alone, the above backup and recovery methods are basically satisfied. Of course, you can write a script, and then rsync to the corresponding backup storage place.

But in fact, a lot of MONGODB uses replication sets, so exactly how the replication replica set is backed up.

Mongodump-- uri= "mongodb://backup:1234.com@10.5.1.114:27027,10.5.1.115:27027,10.5.1.116:27027/DB_AAP_LOG?replicaSet=repl&readPreference=secondary"-- gzip-v-j 8-o / wu

The above command is to connect to MONGODB by introducing uri from MONGODB3.46 to back up a large number of data by backing up a large number of data for replication collections. Because there is a conflict with the-- db command, you can directly back up the specified authentication database that is the data block to be backed up. (please point out if you have any questions)

In fact, there is a point here, that is, whether MONGODB also needs locks similar to traditional databases to limit the consistency of backed up data. In fact, there is a choice here if you add it when you choose mongodb backup-- oplog will add relevant oplog logs from the backup, which can be used during recovery. Here, I personally think that the type is similar to single-transaction in mysqldump. But what is needed is that it can only be applied when the database is backed up by FULL, but not the individual libraries in the database. It is only natural that after data recovery, it is necessary to confirm whether the index is needed.

The following script can be run directly in MONGODB and get the index information of the current database

Var collectionList = db.getCollectionNames ()

For (var index in collectionList) {

Var collection = collectionList [index]

Var cur = db.getCollection (collection). GetIndexes ()

If (cur.length = = 1) {

Continue

}

For (var index1 in cur) {

Var next = cur [index1]

If (next ["key"] ["_ id"] = ='1') {

Continue

}

Print (

"try {db.getCollection (\"+ collection+"\ "). EnsureIndex (" + JSON.stringify (next.key) + ", {background:1, unique:" + (next.unique | | false) + "+ (next.expireAfterSeconds > 0?", expireAfterSeconds: "+ next.expireAfterSeconds:") + "})} catch (e) {print (e)}")}}

There is another problem, that is, if I do not back up the entire MONGODB, I will only back up one library, and then what do I want to do incrementally? I may need your collection to have the relevant time field, and then export import will do it according to time, which may be a little complicated.

For example, in the following data, we just want to export some data according to the date boundary.

Mongoexport-uXXX-pXXXXX-- host 192.168.198.180 test-c testData-Q'{date: {$lte: {"$date": "2020-03-30T02:23:25Z"}'--out / mongodata/backup/testData.json

So we export the data you need.

In fact, if you use the backup way, the data can be compressed relatively small, but the flexibility is relatively poor, import and export data is characterized by flexibility, but the time and space will be large.

Mongoimport-uXXX-pXXXXX-- host 192.168.198.181 test-c testData-- type json-- file='/mongodata/backup/testData.json'

We use the above command to import the corresponding data

It is estimated that many students will have questions about the flexibility of mongodump mongorestore and mongoimport mongexport compared with the latter.

For example, when data goes from one table to another, and some fields have trade-offs, and because the data may be repetitive, that is, _ id may be repeated, the probability that you can choose is mongoimport mongoexport. Through the options in the following figure, you can make those repeated data that may have conflicts "quietly" follow your command, at least you can have a choice. If you use dump resotre, then.

Before going back to backup, if your database is relatively large, back up in this way, then. Relatively slow, is there a better way to back up the database? because MONGODB itself does not have a strong transactional nature, such a command is often used in MONGODB backup. Volume backup has been supported since MONGODB 3.2.

Is the following command, in db.fsyncLock () to lock your MONGODB write, and then you can start to copy your files, copy your files away, of course, you can LVM and other ways, specifically according to your wishes

After the copy is complete, directly, type db.fsyncUnlock () to unlock, and the data can continue to be written.

In fact, most MONOGDB databases are backed up in this way, especially in the cluster way, you will lock from the database, then copy the data from the database to the backup location, and then unlock it, but you need to pay attention to the time in the process of operation, and do not forget that your database is locked, as well as alarm and other problems, you need to deal with.

In fact, there is another idea for the backup of MONGODB, that is, to set up a backup node directly on the basis of the replication set. I know that DB who do traditional databases may not agree with this idea. What if other people's data is to be rolled back and what if the data is to be archived? Of course, there can also be a delay library (friends who have used MYSQL should be familiar with this concept)

At this point, it's still my mantra. Everything depends on your business logic. If your MONGODB is just a current account and some logs, do you have to back up these things? what's the meaning of backup? for example, how to back up if your database is cassandra with hundreds of nodes? So different databases may subvert some of your long-standing methods and attitudes towards database operations. The world is changing fast, follow it. (if you back up at production time, it is strongly recommended that you back up all on the slave library of the replication collection)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.