Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

How to set the number of Map in Mapreduce

2025-03-31 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >

Share

Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to set the number of Map in Mapreduce. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

The description is as follows:

Setting the number of Map is not supported in the new version of MapReduce, namely MR on yarn.

The number of Map is determined by MinInputSplitSize, the formula:

Number of Map = TotalSize / MinInputSplitSize

To control the number of Map, you can control the number of Map by controlling the size of the MinInputSplitSize.

If the MinInputSplitSize is greater than BlockSize,MinInputSplitSize, the value is set; otherwise, the value of MinInputSplitSize less than BlockSize,MinInputSplitSize is set to BlockSize.

If the default setting is maintained, MinInputSplitSize is BlockSize

On how to set the number of Map in Mapreduce to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Servers

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report