Shulou(Shulou.com)05/31 Report--

Spark API programming in the Spark 1.2 version of how to sort the Job output, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain in detail for you, people with this need can come to learn, I hope you can get something.

From the output of wordcount, we can see that the result is unsorted. How to sort the output of spark?

First, the reduceByKey results are replaced by key,value positions (numbers, characters), then the numbers are sorted, and then the key,value positions are replaced. Finally, the results are stored in HDFS.

You can find that we have successfully sorted the output!

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.