Shulou(Shulou.com)06/01 Report--

1.HBase returns empty if the column is qualified, if the column does not exist. Look at the following code:

Get get = new Get (Bytes.toBytes ("100s"); get.addColumn (Bytes.toBytes ("info"), Bytes.toBytes ("name"))

Column qualification is added here, that is, only the name field under the column family info is returned. But if the name field doesn't exist, the returned Result returns true when result.isEmpty () is called, which means that even if other fields exist, nothing is returned, including rowkey. Of course, if you limit multiple columns, you can return normally as long as one column exists. So we need to pay attention.

2.HBase cannot add "-" to the StartRow specified when scan. See the following code:

Scan scan = new Scan (); scan.setStartRow (Bytes.toBytes ("3136947 -")); scan.setStopRow (Bytes.toBytes ("3136947 -" + 1))

My intention was to query the lines where rowkey starts with 3136947 -, but because I have a-("bar") in it, nothing is returned, and it is normal after removing it. This means that -,-can't be used here, and it's not an escape character, and scan can't come out after escape. I don't know if other characters won't work, there's no test. So we need to pay attention.

3.HBase filters out records that do not exist in the specified column during scan

If you want to return a row that must exist in a field, there is no record for that field to filter out and do not return, as follows:

Scan scan = new Scan (); scan.setStartRow (Bytes.toBytes ("3136947")); scan.setStopRow (Bytes.toBytes ("3136947" + 1)); scan.addColumn (Bytes.toBytes ("info"), Bytes.toBytes ("name")); SingleColumnValueFilter filter = new SingleColumnValueFilter (Bytes.toBytes ("info"), Bytes.toBytes ("name"), CompareFilter.CompareOp.NOT_EQUAL, Bytes.toBytes ("0"); filter.setFilterIfMissing (true) Scan.setFilter (filter)

Note: if you are judging whether a column exists, you must add the column to the addColumn, that is, the field that must be returned must contain the column, otherwise it will not be returned, because the filter will not be called until addColumn is called during processing.

The filter here specifies that the field value of the column must not be equal to 0 (of course, if you have 0 in your name, of course, you can't use 0), and set setFilterIfMissing to true, that is, to filter out the data if the column does not exist, and the default is false.

4. Using MapReduce to export hbase data

If hbase is the output of the data, the job settings are as follows:

Configuration conf = HBaseConfiguration.create (); Scan scan = new Scan (); scan.setStartRow (Bytes.toBytes ("3136947")); scan.setStopRow (Bytes.toBytes ("3136947" + 1)); scan.addColumn (Bytes.toBytes ("info"), Bytes.toBytes ("name")); scan.addFamily (UserStoreHelper.FAMILY_INFO); scan.addColumn (UserStoreHelper.FAMILY_INFO, UserStoreHelper.USER_ID); scan.addColumn (UserStoreHelper.FAMILY_INFO, UserStoreHelper.FRIENDS); scan.addColumn (UserStoreHelper.FAMILY_INFO, UserStoreHelper.LEVEL_CODE) Final Job job = new Job (conf, "exportHBaseUser"); job.setJarByClass (TestJobCreator.class); job.setOutputFormatClass (TextOutputFormat.class); FileOutputFormat.setOutputPath (job, new Path ("test1")); TableMapReduceUtil.initTableMapperJob (Bytes.toBytes ("usertable"), scan, TestMapper.class, Text.class, NullWritable.class, job)

The map set in initTableMapperJob must inherit org.apache.hadoop.hbase.mapreduce.TableMapper, and the last two parameters set are the key and value types of the output of the self-defined map.

5. Insert data into HBase using mapReduce

If hbase is used as data input. The code is as follows:

Final Configuration conf = HBaseConfiguration.create (); final Job job = new Job (conf, "Sync-To-HBase"); job.setJarByClass (PostStoreExportHBaseJobCreator.class); / / I use mongodb as input job.setInputFormatClass (MongoInputFormat.class); TableMapReduceUtil.initTableReducerJob ("usertable", null, job); / / mapjob.setMapperClass (TestMapper.class) that converts data to hbase table format; / / reduce job.setNumReduceTasks (0) is not required for direct entry into hbase library

Where the output of map must be key, ImmutableBytesWritable,value and Put

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.