Shulou(Shulou.com)06/02 Report--

[1] Build HA High Availability Hadoop-2.3 (Planning + Environment Preparation)

[2] Build HA High Availability Hadoop-2.3 (install zookeeper)

[3] Build HA high availability hadoop-2.3 (deployment configuration hadoop--cdh6.1.0)

[4] Build HA high availability hadoop-2.3 (deployment configuration HBase)

Build HA High Availability Hadoop-2.3

Foreword:

The first step in installing Hadoop is to choose the appropriate version of Hadoop for the situation, this time using CDH5.1.0, based on Hadoop version 2.3. So what is CDH? Let's take a look at the science below.

Cloudera is a company that provides Hadoop support, consulting and management tools. It has a pivotal position in the Hadoop ecosystem. Its flagship product is Cloudera's Distribution for Hadoop, or CDH for short. The software, like Apache Hadoop, is completely open source, based on the Apache software license, free for personal and commercial use. Coudera started with a stable Apache Hadoop release, continued to release new and patched versions, provided installation files for various production environments, and had many Apache Hadoop contributors on the Cloudera team, so Cloudera's strength was beyond doubt.

When a typical user installs Hadoop, not only HDFS and MapReduce are installed, but Hive, HBase, Spark, etc. are also installed as needed. Cloudera integrates these related projects into a CDH version. Currently CDH includes Hadoop, HBase, Hive, Pig, Sqood, Zooksspe, Spark, Flume, Oozie, Mahout, etc., almost covering the Hadoop ecosystem. The advantage of doing so is to ensure compatibility between components, because there are completely independent versions between each project, and there will inevitably be compatibility problems between each version and Hadoop. If CDH is selected, then there will be no compatibility problems between each component in the same CDH version. So CDH is a good choice for beginners to build Hadoop.

Planning:

Machine: 5 OS: centos6.5

machine planning

hostname

IPHDFSYarnHBasemaster110.64.8.1Namenode1ResourceManagerHMastermaster110.64.8.2Namenode2

slave110.64.8.3Datanode1NodeManagerHRegionServerslave210.64.8.4Datanode2NodeManagerHRegionServerslave310.64.8.5Datanode3NodeManagerHRegionServer

Preparations:

(1) Turn off selinux and firewall

master1、master2、slave1、slave2、slave3

#setenforce 0&& service iptables stop &&chkconfig iptables off

(2) Modify the host name

master1

#hostname master1 && echo master1 >/etc/hostname

master2

#hostname master1 && echo master1 >/etc/hostname

slave1

#hostname slave1 && echo slave1 >/etc/hostname

slave2

#hostname slave2 && echo slave2 >/etc/hostname

slave3

#hostname slave3 && echo slave3 >/etc/hostname

master1、master2、slave1、slave2、slave3

#cat >> /etc/hosts >/etc/rc.local

(6) System parameter optimization

master1、master2、slave1、slave2、slave3

cat >>/etc/sysctl.conf /etc/security/limits.conf

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.