Shulou(Shulou.com)06/03 Report--

I. Overview

Cloudera version (Cloudera's Distribution Including Apache Hadoop, referred to as "CDH"), based on Web user interface, supports most Hadoop components, including HDFS, MapReduce, Hive, Pig, Hbase, Zookeeper, Sqoop, which simplifies the difficulty of installation and use of big data platform.

Installation and deployment sequence number IP address hostname system version 1172.20.2.222cm-servercentos7.32172.20.2.203hadoop-1centos7.33172.20.2.204hadoop-2centos7.34172.20.2.205hadoop-3centos7.32.2.1 basic environment deployment

a. Modify hostname configuration hosts

Systemctl stop firewalldhostnamectl set-hostname cm-server # change the hostname sed-I's sys/kernel/mm/transparent_hugepage/defragecho never for sys/kernel/mm/transparent_hugepage/enabled2.2.2 Cloudera Manager installation

a. Download the decompression related software package

Mkdir / software & & cd / softwarewget-c https://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.14.1_x86_64.tar.gz wget-c http://archive.cloudera.com/cdh6/parcels/5.14.2/CDH-5.14.2-1.cdh6.14.2.p0.3-el7.parcelwget-c http://archive.cloudera.com/cdh6/parcels/5.14.2/CDH-5.14.2 -1.cdh6.14.2.p0.3-el7.parcel.sha1-O CDH-5.14.2-1.cdh6.14.2.p0.3-el7.parcel.shawget-c http://archive.cloudera.com/cdh6/parcels/5.14.2/manifest.jsonwget-c https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.ziptar-zxvf cloudera-manager-centos7-cm5.14.1_x86_64 .tar.gz-C / opt/ # extract the cm package unzip mysql-connector-java-5.1.46.zip # extract the java-mysql connection jar package cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar / opt/cm-5.14.1/share/cmf/lib/ # copy the jar package to the lib directory of cm cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar

b. Create users and initialize databases

Useradd-- system-- home=/opt/cm-5.14.1/run/cloudera-scm-server/-- no-create-home-- shell=/bin/false-- comment "Cloudera SCM User" cloudera-scm # creates a user vim / opt/cm-5.14.1/etc/cloudera-scm-agent/config.ini on each node and points the server_host=cm-server # to cm-serverusage: / opt/cm-5.14.1/share/ Cmf/schema/scm_prepare_database.sh [options] (postgresql | mysql | oracle) database username [password] # use the option / opt/cm-5.14.1/share/cmf/schema/scm_prepare_database.sh mysql cmdb-h "cm-server"-uroot-pmysqladmin-- scm-host cm-server scm scm scm

c. Distribute files modified by cm-server to other nodes

For i in `seq 1 3`; do scp-r / opt/cm-5.14.1 hadoop-$i:/opt/;done

d. Create a local source

Mv CDH-5.14.2-1.cdh6.14.2.p0.3Murel 7.parcel* manifest.json / opt/cloudera/parcel-repo/

e. Start the service

Start server and agent services on cm-server and agent services on other nodes

Web interface configuration of / opt/cm-5.14.1/etc/init.d/cloudera-scm-server start/opt/cm-5.14.1/etc/init.d/cloudera-scm-agent start2.2.2 Cloudera Manager

After all the servers are started, the browser can access port 7180 of cm-server with a user name / password of admin/admin

Accept the agreement to continue

You can choose to apply 60 days.

Prompt for some information related to license

Check the managed host to continue the operation.

Select CDH-5.14 version

Parcel installation

Host correctness check

Cluster settings (select installed services)

Customize the role assignment and select which node to install on

Database Settin

You need to create the database in advance and authorize other nodes to connect normally.

Review changes

Cluster installation

Complete the installation

Services can be added later

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.