Shulou(Shulou.com)05/31 Report--

Editor to share with you the example analysis of storm cluster WordCount. I hope you will get something after reading this article. Let's discuss it together.

Storm cluster instance is running

To run storm locally, you only need storm's jar package. The result can be seen directly in the console. The storm cluster runs, and the result should be viewed in the log log or stored. Moreover, when the cluster runs, the output in the execute method can be seen, but the output in cleanup cannot be seen, because the cleanup is executed only after the end of the topology, while the storm runs continuously in real time, so the output is put in the execute or saved to view.

Wordcount instance code

The code has been written in the previous blog, just to make some changes to WordCounter.

Package com.storm.stormDemo; import java.io.BufferedWriter;import java.io.FileWriter;import java.io.IOException;import java.util.HashMap; import java.util.Iterator;import java.util.Map; import org.apache.log4j.Logger;import com.storm.stormTest.MergeObjects;import backtype.storm.task.OutputCollector; import backtype.storm.task.TopologyContext; import backtype.storm.topology.IRichBolt; import backtype.storm.topology.OutputFieldsDeclarer; import backtype.storm.tuple.Fields;import backtype.storm.tuple.Tuple Public class WordCounter implements IRichBolt {public static Logger LOG = Logger.getLogger (WordCounter.class); Integer id; String name; Map counters; private OutputCollector collector; BufferedWriter output; public void prepare (Map stormConf, TopologyContext context, OutputCollector collector) {this.counters = new HashMap (); this.collector = collector; this.name = context.getThisComponentId (); this.id = context.getThisTaskId () Try {output = new BufferedWriter (new FileWriter ("/ home/zhanghuan/Downloads/wordcount.txt", true));} catch (IOException e) {/ / TODO Auto-generated catch block try {output.close ();} catch (IOException E1) {/ / TODO Auto-generated catch block e1.printStackTrace ();} e.printStackTrace () }} public void execute (Tuple input) {String str = input.getString (0); if (! counters.containsKey (str)) {counters.put (str, 1);} else {Integer c = counters.get (str) + 1; counters.put (str, c) } Iterator iterator = counters.keySet (). Iterator (); while (iterator.hasNext ()) {String next = iterator.next (); try {System.out.print (next + ":" + counters.get (next) + "); output.write (next +": "+ counters.get (next) +"); output.flush () } catch (IOException e) {e.printStackTrace (); try {output.close ();} catch (IOException E1) {e1.printStackTrace ();} / / confirm that a tuple collector.ack (input) has been processed successfully The cleanup work performed by Topology, such as closing the connection and releasing resources, is written here * because this is only a Demo, we use it to print our counter * * / public void cleanup () {LOG.info ("- Word Counter [" + name + "-" + id + "] -"). For (Map.Entry entry: counters.entrySet ()) {LOG.info (entry.getKey () + ":" + entry.getValue ());} counters.clear ();} public void declareOutputFields (OutputFieldsDeclarer declarer) {/ / TODO Auto-generated method stub / / declarer.declare (new Fields ("word", "number")) } public Map getComponentConfiguration () {/ / TODO Auto-generated method stub return null;}}

Cluster operation

Storm jar StormDemo.jar com.storm.stormDemo.WordCountTopologyMain StormDemo / home/zhanghuan/Downloads/test.txt

Note: the path of the main function should be fully written.

If the cluster reports an error as follows:

Then open the third-party jar package folder you typed, find storm-core-0.10.0.jar in it, delete the default.yarml file in the jar package, or delete the storm jar package you typed.

After the topology is submitted, the corresponding number of worker and logwriter processes will be started, and the operation of the topology can also be seen on the ui interface.

At this point, you can view the log log file or the storage location to see the results.

When there is no data input, the log, like the bottom, keeps communicating.

Stop topology running

The name of storm kill topology

After reading this article, I believe you have some understanding of "sample Analysis of storm Cluster WordCount". If you want to know more about it, you are welcome to follow the industry information channel. Thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.