Shulou(Shulou.com)05/31 Report--

This article focuses on "how to test the Hadoop distributed cluster environment", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to test Hadoop distributed cluster environment.

Test Hadoop distributed cluster environment

First, format the cluster's file system through the Master node:

Enter "Y" to complete the formatting:

After the formatting is complete, we start the hadoop cluster

We are trying to stop the Hadoop cluster:

The "no datanode to stop" error occurs for the following reasons:

Every time you use the "hadoop namenode-format" command to format a file system, a new namenodeId appears. When we built the pseudo-distributed version of Hadoop stand-alone, we distributed the data to the tmp directory we created. Now we need to empty the contents of the tmp and its subdirectories under "/ usr/local/hadoop/hadoop-1.2.1/" on each machine. At the same time, empty all the hadoop-related contents in the "/ tmp" directory, and finally empty the contents of the data and name folders in our custom hdfs folder:

Delete the same content from Slave1 and Slave2.

Reformat and restart the cluster, and then go to the Web console of Master:

At this point, you can see that there are only three Live Nodes, which is exactly what we expected, because our Master, Slave1, Slave2 are all set to DataNode, and of course Master itself is also NameNode.

At this point, let's look at the process information on the three machines with the JPS command:

It is found that the various services of the Hadoop cluster are started normally.

At this point, the Hadoop cluster has been built.

At this point, I believe you have a deeper understanding of "how to test the Hadoop distributed cluster environment". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.