Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "how to install Hadoop under Windows". Many people will encounter such a dilemma in the operation of actual cases, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

There are usually two ways for Windows downlink Hadoop: one is to install a Linux operating system with VM, which can basically realize the Hadoop operation of the full Linux environment; the other is to simulate the Linux environment through Cygwin. The advantage of the latter is that it is easy to use and the installation process is simple. Here let's take a look at the second option: how to quickly install a Hadoop environment under Windows, and combine with the Eclipse development environment to study and adjust the Hadoop code.

The entire installation process consists of the following three steps:

Install and configure Cygwin (http://cygwin.com/install.html)

Install and configure Hadoop-1.2.1 (http://hadoop.apache.org/docs/stable/cluster_setup.html)

Install and configure the Eclipse development environment

1 install and configure Cygwin

It is a simple and convenient way to simulate the Linux environment by installing Cygwin under Windows, and then install Hadoop. The installation process for Hadoop to simulate the Linux environment is as follows:

1.1 download the installation file

Download the corresponding installation files for different system types at http://cygwin.com/install.html.

My system here is window 7, so I downloaded setup-x86.exe.

1.2 install Cygwin

The file you just downloaded is a software package download and management tool that simulates the Linux system. In the future, you need to install or update software in the simulated Linux environment through this tool. Let's run this tool as follows:

Left-click the setup-x86.exe file to run the installation wizard:

Cygwin installation

Click the "next" button to enter the program boot installation page. Here are three options. Select the first network installation:

Network installation: download and install software packages over the network

Download but not install: download software packages over the network

Local installation: is installed with a local package

Cygwin installation

Click "next" to enter the wizard page for selecting the root directory and user of the simulated Linux system. There is only one root directory in the file system of Linux. Select the directory here is the root directory in Linux. Choose the default: C:\ cygwin; users choose the first item: all valid users of this system.

Cygwin installation

Click [next] to select the local package directory, and the tool will automatically remember and place all future downloaded packages in the directory specified here. I choose here: C:\ Users\ Administrator\ Desktop\ 1. If you choose that no directory exists, you will be prompted whether to create a directory or not. Choose Yes on OK.

Cygwin installation

Click [next] to select your network connection. I am using a proxy server to surf the Internet, so I choose the second item: use IE browser proxy settings. After testing, the third item is selected to enter the address and port of the proxy server, but the network cannot be accessed normally. The reason is unknown.

Cygwin installation

Click "next" and wait for the list of mirror sites to download. After the download is complete, the site where you choose to download the package appears. As shown in the figure:

Cygwin installation

Choose the appropriate download address according to your own situation. I have selected 163 domestic sites here, and click "next". This tool will automatically download the package information list and enter the installation package selection page, as shown below:

Cygwin installation

This step is important to ensure that the following packages are installed:

Cygwin installation

Note: this package list includes: classification, current installation version, latest version, installation executable file? , install source code files? , size, package name and description.

Basic software package: Base and all the software packages below it, how to do it: click the Default after Base for Install.

SSH-related software packages: OpenSSL and OpenSSH under Net are used for SSH access required by Hadoop. How to do this: click [+] to expand the Net node, and click the Keep in the latest version number column in front of each package for the selected installation.

Other software packages can be installed according to their own needs. I also choose common tools such as Emacs, VIM, Perl, Python, Ruby, Science, subversion and so on.

After selecting the software package, click "next" to enter the automatic download and installation, as shown below:

Cygwin installation

Click "next" to enter the end page of the wizard, check the create desktop shortcut and click "finish", as shown in the figure:

Cygwin installation

At this point, you have completed the installation of the simulation Linux environment, left-click the icon on the desktop to open the simulation Linux terminal window to enter several common Linux commands to experience the simulation of the Linux system, in addition to the implementation of common Linux commands, you can also execute Windows commands such as: net start service_name and so on. Continue with the following configuration work after the experience is complete.

1.3Configuring SSH services for Cygwin

After the Cygwin installation is completed, you need to configure the SSH service to meet the SSH password-less login proposed by Hadoop. The process is as follows:

Open the analog Linux terminal and enter the Linux environment

Execute command: ssh-host-config as shown in the figure:

Hadoop installation

Ask "Should privilege separation be used? (yes/no)" for the first time, enter no to enter enter.

The second time: "Do you want to install sshd a service?", enter yes to enter.

The third prompt: "Enter the value of CYGWIN for the demon: []", enter directly.

The fourth query: "Do you want to use a different name? (yes/no)", enter no enter.

The fifth prompt: "Please enter the password for user 'cyg_server':", enter your password and enter your password here.

Finally, it is prompted that the configuration is complete.

1.4 start the SSH service

Execute net start sshd or command cygrunsrv-S sshd from the Linux terminal or the Windows command line to start the SSH service.

Test ssh login to this machine:

Execute the command on the terminal: ssh localhost

Prompt for password: enter password to enter, as shown below:

Hadoop installation

1.5 configure SSH password-free login

Execute the command at the terminal: ssh-keygen-t dsa-P "- f ~ /. Ssh/id_dsa to generate the secret key.

Execute the command: cat ~ / .ssh/id_dsa.pub > > ~ / .ssh/authorized_keys to generate the authentication file.

Execute command: ssh localhost tests whether you can log in without entering a password.

This is the end of the content of "how to install Hadoop under Windows". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.