Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Configuration and Optimization method of hadoop2.6

2025-04-04 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >

Share

Shulou(Shulou.com)06/01 Report--

This article mainly introduces "the configuration and optimization methods of hadoop2.6". In the daily operation, I believe many people have doubts about the configuration and optimization methods of hadoop2.6. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "configuration and optimization methods of hadoop2.6". Next, please follow the editor to study!

1.vi / opt/hadoop-2.6.0/etc/hadoop/hadoop-env.sh

Export JAVA_HOME=/opt/jdk1.7.0_75

2.vi / opt/hadoop-2.6.0/etc/hadoop/core-site.xml

Fs.default.name

Hdfs://spore:9000

Note: spore is the hostname of the machine.

Hadoop.native.lib

True

Dfs.permissions

False

Hadoop.tmp.dir

/ opt/hadoop-2.6.0/tmp

3.vi / opt/hadoop-2.6.0/etc/hadoop/hdfs-site.xml

Dfs.replication

one

4.vi / opt/hadoop-2.6.0/etc/hadoop/mapred-site.xml

Mapreduce.framework.name

Yarn

5.vi / opt/hadoop-2.6.0/etc/hadoop/slaves

Add hostname for slave

Optimization:

Try to use combiner to reduce the number of key-value pairs, merge key-value pairs locally, reduce network transmission, and the optimization effect is obvious.

Increase the memory of the mapreduce intermediate result cache

Skillfully use compound keys to allow the system to complete sorting, so it is not necessary to sort by yourself.

At this point, the study of "hadoop2.6 configuration and optimization method" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Servers

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report