Shulou(Shulou.com)06/01 Report--

If the hadoop mr output needs to be imported into hbase, it is best to export it to HFile format first, and then import it to HBase. Because HFile is the internal storage format of HBase, the import efficiency is very high. Here is an example.

1. Create HBase table T1

Hbase (main): 157row 0 * create 't1century row (s) in 1.3280 secondshbase (main): 158row (s) in 1.2770 seconds

two。 Write MR homework

HBaseHFileMapper.java

Package com.test.hfile;import java.io.IOException;import org.apache.hadoop.hbase.io.ImmutableBytesWritable;import org.apache.hadoop.hbase.util.Bytes;import org.apache.hadoop.io.LongWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Mapper;public class HBaseHFileMapper extends Mapper {private ImmutableBytesWritable immutableBytesWritable = new ImmutableBytesWritable () @ Overrideprotected void map (LongWritable key, Text value,org.apache.hadoop.mapreduce.Mapper.Context context) throws IOException, InterruptedException {immutableBytesWritable.set (Bytes.toBytes (key.get ()); context.write (immutableBytesWritable, value);}}

HBaseHFileReducer.java

Package com.test.hfile;import java.io.IOException;import org.apache.hadoop.hbase.KeyValue;import org.apache.hadoop.hbase.io.ImmutableBytesWritable;import org.apache.hadoop.hbase.util.Bytes;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Reducer;public class HBaseHFileReducer extends Reducer {protected void reduce (ImmutableBytesWritable key, Iterable values,Context context) throws IOException, InterruptedException {String value= ""; while (values.iterator () .hasNext ()) {value= values.iterator () .next () .toString () If (value! = null & &! ".equals (value)) {KeyValue kv = createKeyValue (value.toString ()); if (kvault null) context.write (key, kv);}

/ / str format is row:family:qualifier:value simple simulation private KeyValue createKeyValue (String str) {String [] strstrs = str.split (":"); if (strs.length scan 't1'ROW COLUMN+CELLr1 column=f1:c1, timestamp=1314591150788, value=value1 R2 column=f1:c2, timestamp=1314591150814, value=value2 R3 column=f1:c3, timestamp=1314591150815, value=value3 3 row (s) in 0.0210 seconds

The data has been imported!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.