Shulou(Shulou.com)06/02 Report--

Ambari+HDP installation on Yun2 cluster

1. Installation environment description:

System version: CentOS7

Installation version:

Ambari version: 2.2.2.0

HDP version: 2.4.2.0

Cluster machines:

Ambari Server: yun2

Ambari Agent:yun3, yun4,yun6,yun18,yun19,yun20,yun21

two。 Operating system environment preparation

2.1 configure SSH password-free login

Log in to the root user in the main node to perform the following steps:

Ssh-keygen-t rsa

Cd / root/.ssh

Cat id_rsa.pub > > authorized_keys

Distribute the configured authorized_keys in the master node to each slave node:

Scp-r / root/.ssh/authorized_keys yun3,: / root/.ssh/authorized_keys

Scp-r / root/.ssh/authorized_keys yun4,: / root/.ssh/authorized_keys

Scp-r / root/.ssh/authorized_keys yun6,: / root/.ssh/authorized_keys

Scp-r / root/.ssh/authorized_keys yun18,: / root/.ssh/authorized_keys

Scp-r / root/.ssh/authorized_keys yun19,: / root/.ssh/authorized_keys

Scp-r / root/.ssh/authorized_keys yun20,: / root/.ssh/authorized_keys

Scp-r / root/.ssh/authorized_keys yun21,: / root/.ssh/authorized_keys

2.2 create ambari system users and user groups

Operate only on the primary node

Add ambari installation, run users and user groups, or use root or other accounts of the system without creating new users

Useradd admin

Passwd admin

2.3 enable NTP service

All nodes on the cluster need to operate.

Yum install ntp

Systemctl is-enabled ntpd

Systemctl enable ntpd

Systemctl start ntpd

2.4 turn off the firewall

All nodes should be set up

Systemctl disable firewalld

Systemctl stop firewalld

2.5 turn off SELinux

All nodes should be set up

View SELinux status:

Sestatus

It is enabled if the parameter SELinux status is enabled.

SELinux status: enabled

Temporarily shut down, no need to restart the machine:

Setenforce 0

Restarting the machine is required to modify the configuration file:

Vi / etc/sysconfig/selinux SELINUX=disabled

3. Make a local source

Making a local source only needs to be done on the primary node.

3.1 related preparatory work

3.1.1 install Apache HTTP server

Install a HTTP server to allow http services to pass through the firewall (permanent)

Yum install httpd

Firewall-cmd-add-service=http

Firewall-cmd-permanent-add-service=http

Add Apache services to the system layer to start automatically with the system

Systemctl start httpd.service

Systemctl enable httpd.service

3.1.2 install tools related to local source production

Download the installation resources of Ambari 2.2.2 and HDP 2.4.2. This installation is on Centos 7. Only centos7 resources are listed. For other systems, please use the system resources now. For different versions, please find them on http://docs.hortonworks.com/index.html.

Ambari 2.2.2 download Resources

OS Format URL

CentOS 7 Base URL http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.2.2.0

CentOS 7 Repo File http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.2.2.0/ambari.repo

CentOS 7 Tarball md5 asc http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.2.2.0/ambari-2.2.2.0-centos7.tar.gz

HDP 2.4.2 download Resources

OS Repository Name Format URL

CentOS 7 HDP Base URL http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.4.2.0

CentOS 7 HDP Repo File http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.4.2.0/hdp.repo

CentOS 7 HDP Tarball md5 asc http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.4.2.0/HDP-2.4.2.0-centos7-rpm.tar.gz

CentOS 7 HDP-UTILS Base URL http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7

CentOS 7 HDP-UTILS Repo File http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7/HDP-UTILS-1.1.0.20-centos7.tar.gz

The compressed packages you need to download are as follows:

Ambari 2.2.2 http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.4.0.0/HDP-2.4.0.0-centos7-rpm.tar.gz

HDP 2.4.2 http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.2.2.0/ambari-2.2.2.0-centos7.tar.gz

HDP-UTILS 1.1.0 http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7/HDP-UTILS-1.1.0.20-centos7.tar.gz

In the root directory of the httpd website, the default is / var/www/html/

Cd / var/www/html/

Create a directory based on the version: Ambari2.2.2.0-HDP2.4.2.0

Mkdir Ambari2.2.2.0-HDP2.4.2.0

Enter / var/www/html/ambari/

Cd / var/www/html/ambari/

Extract the downloaded package to the / var/www/html/ Ambari2.2.2.0-HDP2.4.2.0 directory

Tar-zxvf ambari-2.2.2.0-centos7.tar.gz

Tar-zxvf HDP-2.4.2.0-centos7-rpm.tar.gz

Tar-zxvf HDP-UTILS-1.1.0.20-centos7.tar.gz

Verify that the httd website is available, and you can access the following address directly using a browser:

HTTP://192.168.1.202/Ambari2.2.2.0-HDP2.4.2.0

Configure local sources for ambari, HDP, and HDP-UTILS

First, download the corresponding repo file in the resource list above, and change the URL to the local address. The relevant configuration is as follows:

Ambari.repo:

# VERSION_NUMBER=2.2.2.2.0

[ambari-2.2.2.0]

Name=ambari Version-ambari-2.2.2.0

Baseurl= http://192.168.1.202/Ambari2.2.2.0-HDP2.4.2.0/AMBARI-2.2.2.0/centos7/2.2.2.0-460

Gpgcheck=1

Gpgkey= http://192.168.1.205/Ambari2.2.2.0-HDP2.4.2.0/AMBARI-2.2.2.0/centos7/2.2.2.0-460/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

Enabled=1

Priority=1

Hdp.repo:

# VERSION_NUMBER=2.4.2.0-258

[HDP-2.4.2.0]

Name=HDP Version-HDP-2.4.2.0

Baseurl= http://192.168.1.202/Ambari2.2.2.0-HDP2.4.2.0/HDP/centos7/2.x/updates/2.4.2.0

Gpgcheck=1

Gpgkey= http://192.168.1.202/Ambari2.2.2.0-HDP2.4.2.0/HDP/centos7/2.x/updates/2.4.2.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

Enabled=1

Priority=1

[HDP-UTILS-1.1.0.20]

Name=HDP Utils Version-HDP-UTILS-1.1.0.20

Baseurl= http://192.168.1.202/Ambari2.2.2.0-HDP2.4.2.0/HDP-UTILS-1.1.0.20/repos/centos7

Gpgcheck=1

Gpgkey= http://192.168.1.202/Ambari2.2.2.0-HDP2.4.2.0/HDP-UTILS-1.1.0.20/repos/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

Enabled=1

Priority=1

Put the above modified source under / etc/yum.repos.d/

Yum clean all

Yum list update

Yum makecache

Yum repolist

4. To install Ambari

4.1 install Ambari2.2.2

4.1.1 install Ambari

Yum install ambari-server

If you report an error, prompt:

Error: Package: ambari-server-2.2.2.0-460.x86_64 (ambari-2.2.2.0)

Requires: postgresql-server > = 8.1

The reason is that postgresql is not installed or the version is too low, copy the Postgresql-related RPM installation package into the warehouse, and then install

Yum install postgresql-devel-9.2.7-1.el7.x86_64.rpm

Yum install postgresql-server-9.2.7-1.el7.x86_64.rpm

Yum install postgresql-9.2.7-1.el7.x86_64.rpm

Install Ambari after that

Yum install postgresql-9.2.7-1.el7.x86_64.rpm

4.1.2 configuring Ambari

Ambari-server setup

The following is the configuration execution process, following the prompts

1. Check to see if SELinux is closed, enter y:

Using python / usr/bin/python

Setup ambari-server

Checking SELinux...

SELinux status is' disabled'

WARNING: SELinux is set to 'permissive' mode and temporarily disabled.

OK to continue [YBO] (y)? Y

two。 Prompts whether to customize the settings. Enter: y

Customize user account for ambari-server daemon [YBO] (n)? Y

3. Ambari-server account. Enter directly

Enter user account for ambari-server daemon (root):

Adjusting ambari-server permissions and ownership...

4. Check the firewall and do nothing if it is turned off

Checking firewall status...

Redirecting to / bin/systemctl status iptables.service

5. Set up the JDK. Enter: 3, Custom JDK

Checking JDK...

Checking JDK...

[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8

[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7

[3] Custom JDK

=

Enter choice (1): 3

6. If you choose 3 custom JDK above, you need to set up JAVA_HOME. Enter: / usr/local/jdk

WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.

WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.

Path to JAVA_HOME: / usr/local/jdk Validating JDK on Ambari Server...done. Completing setup...

7. Database configuration. Select: y

Configuring database... Enter advanced database configuration [YBO] (n)? Y

8. Select the database type. Input: 1, Ambari built-in PostgreSQL database

Configuring database... =

Choose one of the following options:

[1]-PostgreSQL (Embedded)

[2]-Oracle

[3]-MySQL

[4]-PostgreSQL

[5]-Microsoft SQL Server (Tech Preview)

[6]-SQL Anywhere

=

Enter choice (1): 1

9. Set the specific configuration information of the database and enter it according to the actual situation. If it is the same as in parentheses, you can enter directly.

Hostname (localhost):

Port ():

Database name (ambari):

Username (ambari): ambari

Enter Database Password (bigdata): ambari

4.1.3 start Amabri

Execute the startup command to start the Ambari service

Ambari-server start

Enter the Ambari address in the browser after successful startup:

Http://192.168.1.202:8080/

Login screen appears, default administrator account login, account: admin password: admin

After a successful login, the following interface appears, and Ambari has been successfully installed.

4.2 install HDP 2.4.2 configure the cluster

Click the Launch Install Wizard button on the login success page above to configure the cluster

4.2.1 set the cluster name

4.2.2 setting up the HDP installation source

Select HDP2.4, and set the information of Advanced Repository Options, this time use the local source, so modify the installation source of the peer system to the local source address.

4.2.3 set up cluster machines

4.2.4 Host confirmation

Make sure that all the machines in the hosts list in the previously configured cluster are available, or you can remove the related machines. After the Success of the machines in the cluster, proceed to the next step.

4.2.5 Select the service to install

4.2.6 Master configuration for each service

4.2.6 configuration of Slaves and Clients sections for services

4.2.7 customized configuration of services

The Grafana Admin Password in Ambari Metrics must be set at initialization: password: grafana

The DATABASE Password of HIVE's METASTORE DATAABSE must also be set to: hive

4.2.8 display configuration information

4.2.9 start installation

Install various services, and the relevant services will be started after the installation is completed. The installation process is relatively long. If an error occurs midway, please follow the specific prompts or log to operate.

The interface for all successful installations is as follows

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.