Shulou(Shulou.com)06/01 Report--

This article mainly explains the "MapReduce default partition Hash Partitioner how to use", the article explains the content is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in depth, together to study and learn "MapReduce default partition Hash Partitioner how to use" it!

1 the default partitioning method for projection MapReduce

The source code of the default partition

Public int getPartition (K2 key, V2 value,int numReduceTasks) {return (key.hashCode () & Integer.MAX_VALUE)% numReduceTasks;}

Key.hashCode (), which takes the hashCode value of the key output from map

& is the and operator in java, that is, in binary terms, it is equal to 1 and 1, otherwise it is 0.

Demo:

Public class test {public static void main (String [] args) {int a = 129 int b = 128 System.out.println (aqb); / / 128}}

/ * parsing: "a" is 129. the conversion to binary is: 10000001 "b" is 128. the conversion to binary is: 10000000 according to the characteristics of & and: 1 and 1 are 11 and 0 are 0so, and the result is 10000000, that is, 128 /.

Therefore, the purpose of key.hashCode () & Integer.MAX_VALUE is to ensure that the partition determined by any key and numReduceTashs output on the map side is a positive integer.

2, hashCode value of common data

1Numeric value of type 1 minint

Public void test () {for (int I =-3; I 98. X-> 120y-> 121z-> 122cm /

Thus it can be seen that the hashCode value of English a-> z or A-> Z is its corresponding integer value.

Thank you for your reading, the above is the "MapReduce default partition Hash Partitioner how to use" the content, after the study of this article, I believe you on the MapReduce default partition Hash Partitioner how to use this problem has a deeper understanding, the specific use also needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.