Shulou(Shulou.com)05/31 Report--

This article introduces the knowledge of "how to use storm real-time sorting TopN". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Reading background: 1 you need to understand the scenarios used by TOP

2 you need to understand the difference between current TOPN processing and timing interval processing

Talk according to the code.

Package com.cc.storm;import com.cc.storm.bolt.MergeBolt;import com.cc.storm.bolt.RankBolt;import com.cc.storm.bolt.RollingAllCountBolt;import com.cc.storm.bolt.RollingCountBolt;import com.cc.storm.spout.RandomEmitSpout;import com.cc.storm.spout.RedisPubSubSpout;import backtype.storm.Config;import backtype.storm.LocalCluster;import backtype.storm.StormSubmitter;import backtype.storm.generated.AlreadyAliveException;import backtype.storm.generated.InvalidTopologyException;import backtype.storm.topology.TopologyBuilder;import backtype.storm.tuple.Fields / * ToPN is a common mode that calculates "Streaming topN" of streaming data: * for example, it is a hot topic in the recent period of time, hot click pictures, popular product browsing, hot item purchase * * since you dare to deal with it in real time. [] [] * @ author Yin Shuai * / public class TOP10 {public static void main (String [] args) throws AlreadyAliveException, InvalidTopologyException, InterruptedException {final int TOP_N = 10 Final int time = 1; TopologyBuilder builder = new TopologyBuilder (); builder.setSpout ("$datasource$", new RandomEmitSpout (), 1); builder.setBolt ("$count$", new RollingCountBolt (3, time), 1) .fieldsGrouping ("$datasource$", new Fields ("merchandiseIDS")) Builder.setBolt ("$rank$", new RankBolt (TOP_N), 2). FieldsGrouping ("$count$", new Fields ("merchandiseID")); builder.setBolt ("$merge$", new MergeBolt (TOP_N)). GlobalGrouping ("$rank$"); Config conf = new Config () Conf.setDebug (false); conf.setNumWorkers (2); conf.setMaxSpoutPending (5000); LocalCluster cluster = new LocalCluster (); cluster.submitTopology ("Getting-Started-Toplogie", conf, builder.createTopology ()); Thread.sleep (5000);}}

The whole process is shown in the figure:

This is the end of the content of "storm real-time sorting TopN". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.