Shulou(Shulou.com)06/03 Report--

Sort is often used, but how can it be implemented in the form of Map, Reduce in big data?

First of all, you have to define the target object, who to sort, if it is a custom object, you need to implement its CompareTo method, because this is the method of comparing sizes between objects.

In addition, you need to design sorting strategies, special cases, normal cases, programming, you should write special cases first, and then write normal cases, so that the logic is clearer.

How to design it?

First write Map and Reduce program according to normal logic, then output, finally get some text,

Write a Map, Reduce program is writing, sort. The idea is also the same, offset is key,(when I first learned, I didn't know what offset is, but later I understood that it is the offset of the field, for example, the first line, the starting offset is 1, the second line, the offset is 11, why, because, the first line has 10 characters, occupying 10 bits)

In all mapreduce programs k1,v1,k2,v2,k3,v3 k1 is not used at all, as for the other k2 and v2, they can design their own, these are just output, can be objects, can also be values, including v3, can also be NULLwriteable and so on.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.