Shulou(Shulou.com)06/01 Report--

I. scene description

An engineer in the company executes db.giveget_card.drop () and deletes the online table by mistake.

Fortunately, backups are made every day, which reflects the importance of backup at this time.

Second, simulate the fault process

Backup data size:

Rs_test01:PRIMARY > use ycsbswitched to db ycsbrs_test01:PRIMARY > db.giveget_card.count (); 3173391

This table has been updated before it is deleted.

Rs_test01:PRIMARY > db.giveget_card.insert ({id:1}); WriteResult ({"nInserted": 1}) rs_test01:PRIMARY > db.giveget_card.insert ({id:2}); WriteResult ({"nInserted": 1}) rs_test01:PRIMARY > db.giveget_card.insert ({id:3}); WriteResult ({"nInserted": 1}) rs_test01:PRIMARY > db.giveget_card.insert ({id:4}); WriteResult ({"nInserted": 1})

Other tables also have update operations.

Rs_test01:PRIMARY > db.tab.find (); {"_ id": ObjectId ("59354ba202d9a99ab2f879c6"), "name": "a"} {"_ id": ObjectId ("59354ba602d9a99ab2f879c7"), "name": "b"} {"_ id": ObjectId ("59354ba802d9a99ab2f879c8"), "name": "c"} {"_ id": ObjectId ("59354baa02d9a99ab2f879c9"), "name": "d"}

After the delete operation, this and other tables are updated.

Rs_test01:PRIMARY > db.giveget_card.find (); {"_ id": ObjectId ("59354c28d905432aeaccd53c"), "id": 5} {"_ id": ObjectId ("59354c2bd905432aeaccd53d"), "id": 6} rs_test01:PRIMARY > db.tab.find () {"_ id": ObjectId ("59354ba202d9a99ab2f879c6"), "name": "a"} {"_ id": ObjectId ("59354ba602d9a99ab2f879c7"), "name": "b"} {"_ id": ObjectId ("59354ba802d9a99ab2f879c8"), "name": "c"} {"_ id": ObjectId ("59354baa02d9a99ab2f879c9"), "name": "d"} {"_ id": ObjectId ("59354ccfd905432aeaccd542") "name": "e"} {"_ id": ObjectId ("59354cd2d905432aeaccd543"), "name": "f"}

III. Recovery steps

1. Copy the giveget_card.bson and giveget_card.metadata.json files of the tab table in the backup to the / tmp/restore/ycsb directory (self-built directory). Ycsb is the library name.

# cp / data/backup/rs07/ycsb/giveget_card.* / tmp/restore/ycsb

2. Export the oplog after the backup time and before the erroneous deletion operation to restore the table

# mongodump-- port 2203-d local-c oplog.rs-Q'{"ts": {$gte: Timestamp (1496664480, 10430), $lte: Timestamp (1496665113, 10430)}'- o / tmp/oplog

-- the timestamp is the result of conversion using the conversion tool.

3. Use bsondump to view the oplog log and find the timestamp 1496665069 of the drop operation.

# bsondump / tmp/oplog/local/oplog.rs.bson {"ts": {"$timestamp": {"t": 1496664760, "I": 1}}, "t": {"$numberLong": "12"}, "h": {"$numberLong": "7079172056815894727"}, "v": 2, "op": "I", "ns": "ycsb.giveget_card", "o": {"_ id": {"$oid": "59354ab8c5308d8c7a9da8b5"} "id": 1.0} {"ts": {"$timestamp": {"t": 1496664762, "I": 1}}, "t": {"$numberLong": "12"}, "h": {"$numberLong": "- 1797107728294067016"}, "v": 2, "op": "I", "ns": "ycsb.giveget_card", "o": {"_ id": {"$oid": "59354abac5308d8c7a9da8b6"} "id": 2.0} {"ts": {"$timestamp": {"t": 1496664765, "I": 1}}, "t": {"$numberLong": "12"}, "h": {"$numberLong": "86046791509150392"}, "v": 2, "op": "I", "ns": "ycsb.giveget_card", "o": {"_ id": {"$oid": "59354abdc5308d8c7a9da8b7"} "id": 3} {"ts": {"$timestamp": {"t": 1496664768, "I": 1}}, "t": {"$numberLong": "12"}, "h": {"$numberLong": "9018614066505371436"}, "v": 2, "op": "I", "ns": "ycsb.giveget_card", "o": {"_ id": {"$oid": "59354ac0c5308d8c7a9da8b8"} "id": 4} {"ts": {"$timestamp": {"t": 1496664994, "I": 1}}, "t": {"$numberLong": "12"}, "h": {"$numberLong": "- 4471524661347063602"}, "v": 2, "op": "c", "ns": "ycsb.$cmd", "o": {"create": "tab"} {"ts": {"$timestamp": {"t": 1496664994, "I": 2}} "t": {"$numberLong": "12"}, "h": {"$numberLong": "4215905958456607246"}, "v": 2, "op": "I", "ns": "ycsb.tab", "o": {"_ id": {"$oid": "59354ba202d9a99ab2f879c6"}, "name": "a"} {"ts": {"$timestamp": {"t": 1496664998, "I": 1}}, "t": {"$numberLong": "12"} "h": {"$numberLong": "6170506962401844481"}, "v": 2, "op": "I", "ns": "ycsb.tab", "o": {"_ id": {"$oid": "59354ba602d9a99ab2f879c7"}, "name": "b"} {"ts": {"$timestamp": {"t": 1496665000, "I": 1}, "t": {"$numberLong": "12"}, "h": {"$numberLong": "- 8071456063660489895"}, "v": 2 "op": "I", "ns": "ycsb.tab", "o": {"_ id": {"$oid": "59354ba802d9a99ab2f879c8"}, "name": "c"} {"ts": {"$timestamp": {"t": 1496665002, "I": 1}}, "t": {"$numberLong": "12"}, "h": {"$numberLong": "43878848668659146"}, "v": 2, "op": "I", "ns": "ycsb.tab" "o": {"_ id": {"$oid": "59354baa02d9a99ab2f879c9"}, "name": "d"} {"ts": {"$timestamp": {"t": 1496665069, "I": 1}}, "t": {"$numberLong": "12"}, "h": {"$numberLong": "- 6913449254950935781"}, "v": 2, "op": "c", "ns": "ycsb.$cmd" "o": {"drop": "giveget_card"} 2017-06-05T20:27:25.552+0800 10 objects found

4. Copy the bson file of oplog to the corresponding directory

# cp / tmp/oplog/local/oplog.rs.bson / tmp/restore/oplog.bson

The directory structure restored at this time:

# pwd/tmp/restore# lsoplog.bson ycsb

5. At this point, all the preparatory operations have been done and the data has been restored.

[root@ops-db-test02 restore] # mongorestore-port 2203-oplogReplay-oplogLimit=1496665069:1 / tmp/restore2017-06-05T20:36:45.361+0800 building a list of dbs and collections to restore from / tmp/restore dir2017-06-05T20:36:45.364+0800 reading metadata for ycsb.giveget_card from / tmp/restore/ycsb/giveget_card.metadata.json2017-06-05T20:36:45.364+0800 restoring ycsb.giveget_card from / tmp/restore/ycsb/giveget_card.bson2017-06 -05T20:36:48.362+0800 [....] Ycsb.giveget_card 15.4MB/475MB (3.2%) 2017-06-05T20:36:51.362+0800 [#...] Ycsb.giveget_card 31.1MB/475MB (6.6%) 2017-06-05T20:36:54.362+0800 [# #..] Ycsb.giveget_card 46.6MB/475MB (9.8%) 2017-06-05T20:36:57.362+0800 [# #.] Ycsb.giveget_card 62.1MB/475MB (13.1%) 2017-06-05T20:37:00.362+0800 [# #.] Ycsb.giveget_card 76.4MB/475MB (16.1%) 2017-06-05T20:37:03.362+0800 [# #.] Ycsb.giveget_card 90.7MB/475MB (19.1%) 2017-06-05T20:37:06.362+0800 [#.] Ycsb.giveget_card 105MB/475MB (22.0%) 2017-06-05T20:37:09.362+0800 [#.] Ycsb.giveget_card 120MB/475MB (25.2%) 2017-06-05T20:37:12.362+0800 [#.] Ycsb.giveget_card 133MB/475MB (28.0%) 2017-06-05T20:37:15.362+0800 [#.] Ycsb.giveget_card 146MB/475MB (30.8%) 2017-06-05T20:37:18.363+0800 [#.] Ycsb.giveget_card 163MB/475MB (34.3%) 2017-06-05T20:37:21.362+0800 [#.] Ycsb.giveget_card 178MB/475MB (37.4%) 2017-06-05T20:37:24.362+0800 [#.] Ycsb.giveget_card 196MB/475MB (41.3%) 2017-06-05T20:37:27.362+0800 [#.] Ycsb.giveget_card 214MB/475MB (45.0%) 2017-06-05T20:37:30.362+0800 [#.] Ycsb.giveget_card 231MB/475MB (48.6%) 2017-06-05T20:37:33.362+0800 [#.] Ycsb.giveget_card 245MB/475MB (51.5%) 2017-06-05T20:37:36.362+0800 [#.] Ycsb.giveget_card 261MB/475MB (54.8%) 2017-06-05T20:37:39.362+0800 [#.] Ycsb.giveget_card 279MB/475MB (58.7%) 2017-06-05T20:37:42.362+0800 [#.] Ycsb.giveget_card 297MB/475MB (62.5%) 2017-06-05T20:37:45.362+0800 [#.] Ycsb.giveget_card 312MB/475MB (65.8%) 2017-06-05T20:37:48.362+0800 [#.] Ycsb.giveget_card 328MB/475MB (69.0%) 2017-06-05T20:37:51.362+0800 [#.] Ycsb.giveget_card 341MB/475MB (71.8%) 2017-06-05T20:37:54.362+0800 [#.] Ycsb.giveget_card 356MB/475MB (74.9%) 2017-06-05T20:37:57.362+0800 [#.] Ycsb.giveget_card 373MB/475MB (78.5%) 2017-06-05T20:38:00.362+0800 [#.] Ycsb.giveget_card 388MB/475MB (81.7%) 2017-06-05T20:38:03.362+0800 [#....] Ycsb.giveget_card 405MB/475MB (85.2%) 2017-06-05T20:38:06.362+0800 [#...] Ycsb.giveget_card 419MB/475MB (88.2%) 2017-06-05T20:38:09.362+0800 [#...] Ycsb.giveget_card 434MB/475MB (91.4%) 2017-06-05T20:38:12.362+0800 [# #..] Ycsb.giveget_card 442MB/475MB (93.1%) 2017-06-05T20:38:15.362+0800 [#.] Ycsb.giveget_card 459MB/475MB (96.6%) 2017-06-05T20:38:18.362+0800 [#.] Ycsb.giveget_card 475MB/475MB (99.9%) 2017-06-05T20:38:18.427+0800 [#] ycsb.giveget_card 475MB/475MB (100.0%) 2017-06-05T20:38:18.427+0800 restoring indexes for collection ycsb.giveget_card from metadata2017-06-05T20:38:44.680+0800 finished restoring ycsb.giveget_card (3173391 documents) 2017-06-05T20 : 38 replaying oplog2017 44.680 0800 replaying oplog2017-06-05T20:38:44.739+0800 done

6. View the results of the recovery

Rs_test01:PRIMARY > db.giveget_card.find ({id: {$gte: 1}}) {"_ id": ObjectId ("59354cb9d905432aeaccd540"), "id": 5} {"_ id": ObjectId ("59354cc0d905432aeaccd541"), "id": 6} {"_ id": ObjectId ("59354ab8c5308d8c7a9da8b5"), "id": 1} {"_ id": ObjectId ("59354abac5308d8c7a9da8b6"), "id": 2} {"_ id": ObjectId ("59354abdc5308d8c7a9da8b7"), "id": 3} {"_ id": ObjectId ("59354ac0c5308d8c7a9da8b8") "id": 4}

The data content is the same, but the storage order is different from the previous data storage order.

Rs_test01:PRIMARY > db.giveget_card.count (); 3173397

Results the updated data 6 after the count= backup table data 3173391 +.

7. Because the oplog from dump also contains the operations of other tables. Check to see if other tables are affected during the recovery process.

Rs_test01:PRIMARY > db.tab.find () {"_ id": ObjectId ("59354ba202d9a99ab2f879c6"), "name": "a"} {"_ id": ObjectId ("59354ba602d9a99ab2f879c7"), "name": "b"} {"_ id": ObjectId ("59354ba802d9a99ab2f879c8"), "name": "c"} {"_ id": ObjectId ("59354baa02d9a99ab2f879c9"), "name": "d"} {"_ id": ObjectId ("59354ccfd905432aeaccd542") "name": "e"} {"_ id": ObjectId ("59354cd2d905432aeaccd543"), "name": "f"}

-- check that the data of the tab table is the same as that of the original table, without any impact, indicating that the logs of other tables are running empty.

This is the recovery operation of backup combined with oplog.

Backup is very important! Backup is very important! Backup is very important! Tell the important things three times.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.