Shulou(Shulou.com)06/02 Report--

SPARK uses scala to realize packet fetching topN

Original file:

Class1 33

Class2 56

Class1 87

Class2 77

Class1 76

Class2 88

Class1 95

Class1 74

Class2 85

Class2 67

Class2 77

Class1 99

Class1 59

Class2 60

Import org.apache.spark.SparkConf

Import org.apache.spark.SparkContext

Import org.apache.commons.collections.map.Flat3Map

Object GroupTopN {

Def main (args: Array [String]): Unit = {

Val conf = new SparkConf ()

.setAppName ("TopN")

.setMaster ("local")

Val sc = new SparkContext (conf)

Val lines = sc.textFile ("C://Users//Administrator//Desktop//spark//groupTopN.txt", 1)

Val pairs = lines.map {line = > line.split ("")}

Val xx = pairs.map {pair = > (pair (0), pair (1) .toInt)}

Val sort = xx.groupByKey ()

Val result = sort.map (s = > (s. Class1, camera. 2.toList.sorted (Ordering.Int). Take (4)

Result.foreach (r = > println (r.room1 + "" + r.room2))

}

}

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.