Shulou(Shulou.com)06/01 Report--

This article mainly shows you "hbase how to write mapreduce", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "hbase how to write mapreduce" this article.

Package com.hbase.test;import java.io.IOException;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.hbase.HBaseConfiguration;import org.apache.hadoop.hbase.client.Mutation;import org.apache.hadoop.hbase.client.Put;import org.apache.hadoop.hbase.client.Result;import org.apache.hadoop.hbase.client.Scan;import org.apache.hadoop.hbase.io.ImmutableBytesWritable;import org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil;import org.apache.hadoop.hbase.mapreduce.TableMapper Import org.apache.hadoop.hbase.mapreduce.TableReducer;import org.apache.hadoop.hbase.util.Bytes;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Job;import org.apache.hadoop.mapreduce.Mapper;import org.apache.hadoop.mapreduce.Reducer;public class HbaseMrTest {public static void main (String [] args) throws IOException, ClassNotFoundException, InterruptedException {Configuration conf = HBaseConfiguration.create () / / configure conf conf.set ("hbase.zookeeper.quorum", "bigdata01,bigdata02,bigdata03"); conf.set ("hbase.zookeeper.property.clientPort", "2181"); Job job = Job.getInstance (conf, "word-count") / / specify the main class job.setJarByClass (HbaseMrTest.class) that executes job; Scan scan = new Scan (); / / define the columns scan.addColumn (Bytes.toBytes ("content"), Bytes.toBytes ("words") to be scanned by mapper) / / configure mapper TableMapReduceUtil.initTableMapperJob ("wordcount", scan,HMapper.class, Text.class, IntWritable.class, job); / / configure recuder TableMapReduceUtil.initTableReducerJob ("result", HReducer.class, job); / / submit job System.exit (job.waitForCompletion (true)? 0:1) }} / / Text, IntWritable is the output type class HMapper extends TableMapper {Text out = new Text (); IntWritable iw = new IntWritable (1) @ Override protected void map (ImmutableBytesWritable key, Result value, Mapper.Context context) throws IOException, InterruptedException {/ / get the value of content:words directly through result byte [] bytes = value.getValue (Bytes.toBytes ("content"), Bytes.toBytes ("words")) If (byteswords are null) {String words = Bytes.toString (bytes); / / split a line of words String [] ws = words.split ("") For (String wd: ws) {out.set (wd); / / write a value, such as: you 1 context.write (out, iw) }} / / Text, IntWritable is the output type of mapper class HReducer extends TableReducer {@ Override protected void reduce (Text text, Iterable iter, Reducer.Context context) throws IOException, InterruptedException {int sum = 0 / / A pair of iter traverses for (IntWritable intw: iter) {sum+= intw.get ();} / new the value in a put constructor is row key Put put = new Put (Bytes.toBytes (text.toString () / put add columnfamily and column put.addColumn (Bytes.toBytes ("info"), Bytes.toBytes ("wordcnt"), Bytes.toBytes (String.valueOf (sum)); / / write out each word as row key, and put is the sum of context.write (new ImmutableBytesWritable (Bytes.toBytes (text.toString (), put) }} finally, put the java file export as RaunableJar to linux java-jar hbase.jar com.hbase.test.HbaseMrTest to run

Raw data:

Running result:

The above is all the content of the article "how hbase writes mapreduce". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.