Shulou(Shulou.com)05/31 Report--

This article mainly introduces the difference between mapred package and mapreduce package in Hadoop, which has certain reference value. Interested friends can refer to it. I hope you will gain a lot after reading this article. Let Xiaobian take you to understand it together.

#Difference

mapred stands for hadoop's old API, mapreduce stands for hadoop's new API

New APIs tend to use abstract classes rather than interfaces, which are easier to extend. For example, we can add a method (with the default implementation) to an abstract class without modifying the previous implementation method of the class. Therefore, in the new API, Mapper and Reducer are abstract classes.

The new API makes extensive use of context objects and allows user code to communicate with MapReduce systems. For example, in the new API, MapContext basically plays the role of OutputCollector and Reporter for JobConf.

The new API unifies configurations. The old API had a special JobConf object for job configuration, which was an extension of the usual Hadoop Configuration object. In the new API, this distinction is lost, so job configuration is done through Configuration. Job control execution is handled by the Job class, not JobClient, and JobConf and JobClient are missing from the new API. This is why Jobconf is only available in mapred, as mentioned above.

The new API supports both push and pull iterations. In both new and old APIs, key/value record pairs are pushed into the mapper, but in addition, the new API allows records to be pulled out of the map() method, which also applies to the reducer. Batch processing records is an example of applying the pull pattern.

Thank you for reading this article carefully. I hope that Xiaobian's article "What is the difference between mapred package and mapreduce package in Hadoop" will be helpful to everyone. At the same time, I hope that everyone will support you a lot and pay attention to the industry information channel. More relevant knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.