Shulou(Shulou.com)05/31 Report--

This article will explain in detail what you should pay attention to in hadoop cluster deployment. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

1. Namenode or datanode is forced to upgrade.

1) downgrade namenode, disadvantages: need to stop service, too many uncertain factors (downgrade may lead to loss of all data), long and complex testing cycle; advantages: just operate one node

2) upgrade all datanode, disadvantages: need to operate all machines in the cluster (or only a single machine can be upgraded); advantages: simple test and low risk

Obviously, the second option is the preferred one, and the testing process is as follows:

1) find a collection server, which has deployed a hadoop client. Start datanode successfully. Log information is shown in version 203.1.

2) Delete the build directory and start datanode again. Failed

3) restore the build directory and start datanode again.

4) copy the build directory to the error datanode, modify the hdfs-site.xml configuration, change the data directory to a new directory, and start datanode successfully.

5) modify the data directory to the original configuration and start datanode successfully

The second scheme is tested successfully. Run the fsck command on namenode and output normally.

This is the end of this article on "what are the points for attention in hadoop cluster deployment?". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.