Get the App
SLTechnology News&Howtos  ›  Servers  › 

How to implement counter in Hadoop

Shulou Source: shulou.com Published: 2022-05-31 21:31:05 09月12日 Update

This article will explain in detail how to implement counters in Hadoop. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Package mapreduce;import java.io.IOException;import org.apache.hadoop.io.LongWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Counter;import org.apache.hadoop.mapreduce.Mapper / * KEYIN indicates the starting position of each line on K1 business Also known as offset * VALUEIN, that is, the text content of each line in the v1 business * KEYOUT, that is, each word in the K2 business represents each line * VALUEOUT, that is, the number of occurrence of each word in the v2 business. The constant 1 * @ author Xr * * / public class MyMapper extends Mapper {/ * parses the text of each line. Parse into each word, count the number of times each word appears * / @ Override protected void map (LongWritable key, Text value,Context context) throws IOException, InterruptedException {/ / counter Counter counter = context.getCounter ("Words", "hello") / / counter.setValue (""); / / set the initial value / / the text content of each line String string = value.toString (); if (string.contains ("hello")) {int times = string.split ("hello") .length-1; counter.increment (times) / / automatic accumulation} / / an array of words per line String [] split = string.split (""); for (String word: split) {context.write (new Text (word), new LongWritable (1)) This is the end of the article on "how to implement counters in Hadoop". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Tags: Line word table above business counter content text article more number of times good practical location also known as constant array article knowledge offset reference Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno Microsoft Apple MariaDB NVidia OPPO Reno