Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what parameters are used in MapReduce's secondary sorting". In daily operation, I believe many people have doubts about what parameters are used in MapReduce's secondary sorting. I have consulted all kinds of data and sorted out simple and easy operation methods. I hope to help you answer the doubts about "what parameters are used in MapReduce's secondary sorting"! Next, please follow the small series to learn together!

There are a few things about quadratic sorting:

used before 0.20.0 is

setPartitionerClass

setOutputkeyComparatorClass

setOutputValueGroupingComparator

Use Yes after 0.20.0

job.setPartitionerClass(Partitioner p);

job.setSortComparatorClass(RawComparator c);

job.setGroupingComparatorClass(RawComparator c);

In the final stage of map, first use setPartitionerClass to correspond to reducer partition, and each partition is sorted by setSortComparatorClass.

After the reducer receives all map outputs, it uses setSortComparatorClass to do the total sort, and then uses setGroupingComparatorClass to construct iterators for values on data packets.

At this point, the study of "what parameters are used in the quadratic sorting of MapReduce" is over, hoping to solve everyone's doubts. Theory and practice can better match to help everyone learn, go and try it! If you want to continue learning more relevant knowledge, please continue to pay attention to the website, Xiaobian will continue to strive to bring more practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.