Shulou(Shulou.com)06/03 Report--

Data backup of HBase 1.1 backs up tables based on classes provided by HBase

Use the classes provided by HBase to export the data from a table in HBase to HDFS and then to the test hbase table.

(1) = = Export from hbase to HDFS==

[hadoop@node01 shells] $hbase org.apache.hadoop.hbase.mapreduce.Export myuser / hbase_data/myuser_bak

(2) = file import hbase table =

Create backup target table in hbase shell

Create 'myuser_bak','f1','f2'

Import data from HDFS into the backup destination table

Hbase org.apache.hadoop.hbase.mapreduce.Driver import myuser_bak/ hbase_data/myuser_bak/*

Supplementary explanation

All of the above are full backups of the data, and the incremental data backup of the table can also be realized later. The incremental backup is similar to the full backup operation, but with a timestamp on the back.

For example:

Export HBase data to HDFS

Hbase org.apache.hadoop.hbase.mapreduce.Export test / hbase_data/test_bak_increment start timestamp end timestamp 1.2 back up tables based on snapshot snapshots

Realize the migration and copy of HBase data through snapshot snapshots. This method is more commonly used and efficient, and it is also the most recommended way of data migration.

HBase's snapshot is actually a set of = = metadata== information (file list). Through the collection of metadata information, you can roll back the data of the table to the data at that time in snapshot.

First of all, we need to understand the system structure of the so-called HBase's LSM type. we know that in HBase, the data is first written to Memstore, and when the data in Memstore reaches certain conditions, it will be flush to HDFS to form HFile, and then it is not allowed to modify or delete it in place. If you want to update or delete, you can only append write to the new file. Since the data will not be modified or deleted in place after the data is written, this is where snapshot makes a story. When doing snapshot, you only need to create pointers (metadata sets) for all the files corresponding to the snapshot table, and when restoring, you only need to find the corresponding files according to these pointers to recover Ok. This is the simplest description of the principle, and the following figure is a simple process for describing a snapshot:

1.3Snapshot practice 1. Snapshotsnapshot 'tableName',' snapshotName' for creating tables

2. View snapshot

List_snapshots

​ finds snapshot that starts with test

List_snapshots' test.*'3, restore snapshot

​ ps: here you need to disable the table. First set the table to unavailable state, and then perform the restore_snapshot operation.

Disable 'tableName'restore_snapshot' snapshotName'enable 'tableName'

4. Delete snapshot

Delete_snapshot 'snapshotName'

5. Migrate snapshot

Hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot\-snapshot snapshotName\-copy-from hdfs://src-hbase-root-dir/hbase\-copy-to hdfs://dst-hbase-root-dir/hbase\-mappers 1\-bandwidth 1024 for example: hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot\-snapshot test\-copy-from hdfs://node01:8020/hbase\-copy-to hdfs://node01:8020/hbase1\-mappers 1\-bandwidth 1024

​ Note: this method is used when migrating snapshot tables to another cluster. MR is used to copy data, which is very fast. Remember to set the bandwidth parameter when you use it to avoid online business failures caused by full network.

6. Import snapshot using bulkload

Hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles\ hdfs://dst-hbase-root-dir/hbase/archive/datapath/tablename/filename\ tablename for example: create a new table create 'newTest','f1','f2'hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles hdfs://node1:9000/hbase1/archive/data/default/test/6325fabb429bf45c5dcbbe672225f1fb newTest

DIncrementalHFiles hdfs://node1:9000/hbase1/archive/data/default/test/6325fabb429bf45c5dcbbe672225f1fb newTest

# #

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.