Shulou(Shulou.com)06/01 Report--

This article mainly explains "zookeeper cluster building and hadoop cluster deployment method introduction", the content of the article is simple and clear, easy to learn and understand, now please follow the editor's train of thought slowly in depth, together to study and learn "zookeeper cluster building and hadoop cluster deployment method introduction"!

1: first of all, the installation of 6 64-bit virtual machines on VM mainly encountered the problems of installation package and environment deployment.

The following is the download address of the ISO file used for your own installation:

Http://mirrors.sohu.com/centos/6.5/isos/x86_64/CentOS-6.5-x86_64-bin-DVD1.iso

In fact, I downloaded all the installation packages on this page before, tried to install them one by one, and then finally chose the non-desktop version. I can choose the settings when I can install them again, which is a bit time-consuming. Then I deployed jdk,maven,git,mysql,openssh-clients on each host, and deployed related projects, the song process is also a bit time-consuming, and then I want to try to write a ssh script to automate the installation and test.

After configuring the environment variables, we began to build the zookeeper cluster, which is mainly used in conjunction with Ali's open source project dubbo (git download: https://github.com/alibaba/dubbo)). I used five virtual machines to deploy the zookeeper cluster as the registration center of dubbo. This process is also relatively simple, because there are a lot of online materials about the deployment of dubbo before the deployment of the zookeeper cluster. Of course, because the script deployment method is not used when deploying the zookeeper cluster, this process is also time-consuming. Next, we will try the script deployment method. After deployment, the demo that ran dubbo checked whether it was OK or not. Of course, due to the limited conditions, we did not go any further.

The compilation of 2:hadoop is rather tedious, so I also revise and sort it out one step at a time. After that, I found that the steps and errors recorded in an article on the Internet were not much different from those recorded by me during the operation, probably the following steps.

The first thing you need to install is:

Maven:maven has the problem of version conflicts, it is true, my machine is installed with the latest version, but will report an error, and then I downloaded the 3.0.5 version of this version, the error 9 disappeared. Wget http://mirrors.cnnic.cn/apache/maven/maven-3/3.0.5/binaries/apache-maven-3.0.5-bin.tar.gz

Yum install gcc-c++

Wget http://prdownloads.sourceforge.net/findbugs/findbugs-2.0.3.tar.gz?download

Findbugs needs to configure environment variables

Wget https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz

Wget http://www.cmake.org/files/v2.8/cmake-2.8.12.2.tar.gz

Yum install pkgconfig

Yum install openssl

Yum install openssl-devel

Vi hadoop-common-project/hadoop-auth/pom.xml (plus sign)

Org.mortbay.jetty

+ jetty-util

+ test

+

+

+ org.mortbay.jetty

Jetty

Test

Finally, execute the command: mvn package-DskipTests-Pdist,native-Dtar

After performing the above steps, there is a long wait. Anyway, I waited for at least 30 minutes, but luckily I finally compiled it successfully. The next thing to do is to transfer files repeatedly using the scp command, and there are a lot of things about building hadoop+zookeeper clusters on the web.

Thank you for your reading. The above is the content of "introduction to zookeeper cluster building and hadoop cluster deployment method". After the study of this article, I believe you have a deeper understanding of zookeeper cluster building and hadoop cluster deployment method introduction, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.