Shulou(Shulou.com)06/03 Report--

1.MapReduce, tell me about your understanding.

Perform engine calculation

What is 2.Map?

Map: mapping function

What is 3.Reduce?

Reduce: specification function

4.shuffle, tell me about your understanding.

It is equivalent to shuffling cards and grouping according to key

Do you remember the picture of 5.wordcount?

Not too deep.

6. What does input slicing usually have to do with? You can have as many map task as you can with as many pieces as you want. Is that right?

Small files will be merged if the block size is incorrect.

Which abstract concept does the application AM and task run in 7.mapreduce architecture design?

Container

8. If the hdfs files are all small files, then there will be multiple map task. When we produce the general design, we need to merge or design our file storage to keep a file in a block size as far as possible (understand the problem)

Right

What are the memory parameters of the 9.NM node and the container?

Yarn.nodemanager.resource.memory-mb

Yarn.scheduler.minimum-allocation-mb

Yarn.scheduler.maximum-allocation-mb

Yarn.scheduler.minimum-allocation-vcores

Yarn.scheduler.maximum-allocation-vcores

10. Does the container use physical core or virtual core

Virtual core

11. Summarize the concept of mapreduce architecture design wordcount blog container, memory and vcore parameter settings

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.