Shulou(Shulou.com)05/31 Report--

This article introduces the knowledge of "how to configure Hadoop pseudo-distribution mode and run Wordcount". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Step 4: configure Hadoop pseudo-distribution mode and run the Wordcount example

The pseudo-distribution mode mainly involves the following configuration information:

Modify the core configuration file core-site.xml of Hadoop, mainly to configure the address and port number of HDFS

Modify the configuration file hdfs-site.xml of HDFS in Hadoop, mainly to configure replication

Modify the configuration file mapred-site.xml of Hadoop's MapReduce, mainly to configure the address and port of JobTracker

Before the specific operation, let's create several folders under the Hadoop directory:

Let's start building a specific pseudo-distributed process and testing it:

First configure the core-site.xml file:

Enter the core-site.xml file:

The contents of the configured file are as follows:

Use the ": wq" command to save and exit.

Next, configure hdfs-site.xml to open the file:

Opened file:

Configured files:

Enter ": wq" to save the modification information and exit.

Next, modify the mapred-site.xml configuration file:

Enter the configuration file:

The contents of the modified mapred-site.xml configuration file are:

Use the ": wq" command to save and exit.

With the above configuration, we have completed the simplest pseudo-distributed configuration.

Next, do the namenode format of hadoop:

Enter "Y" to complete the formatting process:

Next, start Hadoop!

Start Hadoop as follows:

Use the jps command that comes with java to query all daemons:

Start the Hadooptics engine!

Next, use the Web page in Hadoop to monitor the cluster status to view the health status of Hadoop. The specific pages are as follows:

Http://localhost:50030/jobtracker.jsp

Http://localhost:50060/tasttracker.jsp

Http://localhost:50070/dfshealth.jsp

The above Hadoop running status monitoring page shows that our pseudo-distributed development environment is completely successful!

Next, we use the new pseudo-distributed platform to run the wordcount program:

First create the input directory in dfs:

Since the file created at this time does not specify a specific directory for hdfs, it will create a "input" directory under the current user "rocky" and view the Web console:

Perform file copy operation

This is the end of "how to configure Hadoop pseudo-distribution mode and run Wordcount". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.