Shulou(Shulou.com)06/03 Report--

Modify ip and restart / etc/init.d/netwrok restart

PROXY_METHOD=noneBROWSER_ONLY=noBOOTPROTO=staticDEFROUTE=yesIPV4_FAILURE_FATAL=noIPV6INIT=noIPV6_AUTOCONF=noIPV6_DEFROUTE=noIPV6_FAILURE_FATAL=noIPV6_ADDR_GEN_MODE=stable-privacyNAME=ens32UUID=a0ad438d-a586-4c30-b624-601b342a39c4DEVICE=ens32ONBOOT=yesNETMASK=255.255.255.0IPADDR=192.168.52.130GATEWAY=192.168.52.2DNS1=223.5.5.5

Modify hostname

Hostnamectl set-hostname bigdata-cdh01.kfk.com

Modify native DNS / etc/host

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4#::1 localhost localhost.localdomain localhost6 localhost6.localdomain6192.168.52.130 bigdata-cdh01.kfk.com bigdata-cdh01192.168.52.131 bigdata-cdh02.kfk.com bigdata-cdj02192.168.52.132 bigdata-cdh03.kfk.com bigdata-cdh03

Turn off the firewall

Systemctl stop firewalld.servicesystemctl disable firewalld.service (boot does not start) systemctl status firewalld.service

Disable selinux cat / etc/sysconfig/selinux

SELINUX=disabled

Temporarily shut down selinux

Getenforcesetenforce 0getenforce

Disable ipv6

Echo "> > / etc/modprobe.d/dist.confecho" alias net-pf-10 off "> > / etc/modprobe.d/dist.confecho" alias ipv6 off "> > / etc/modprobe.d/dist.conf

Uninstall JDK

Rpm-qa | grep javarpm-e-- nodeps xxx

Set the number of file openings and the maximum number of processes for users

Number of file openings

Ulimit-a

Maximum number of processes for users

Ulimit-u

Setup / etc/security/limits.conf

* soft nofile 32768 * hard nofile 1048576 * soft nproc 65536 * hard nporc unlimited* soft memlock unlimited* hard memlock unlimited

Ssh mutual trust

Ssh-keygen-t rsatouch authorized_keyschmod 600authorized_keyscat id_rsa.pub > authorized_keysssh-copy-id-I id_rsa.pub root@192.168.52.130

Using ntp for time synchronization

Primary node

Driftfile / var/lib/ntp/driftrestrict 192.168.52.130 nomodify notrap nopeer noquery restrict 127.0.0.1restrict:: 1restrict 192.168.52.2 mask 255.255.255.0 nomodify notrapserver 127.127.1.0Fudge 127.127.1.0 stratum 10includefile / etc/ntp/crypto/pwkeys / etc/ntp/keysdisable monitor

Slave nodes, other nodes are the same

Driftfile / var/lib/ntp/driftrestrict 192.168.52.131 nomodify notrap nopeer noquery restrict 127.0.0.1restrict:: 1restrict 192.168.52.2 mask 255.255.255.0 nomodify notrapserver 192.168.52.130Fudge 192.168.52.130 stratum 10includefile / etc/ntp/crypto/pwkeys / etc/ntp/keysdisable monitor

Shut down ntpd first

Ntpdate 192.168.52.130service ntpd restartsystemctl enable ntpd

Ntpq-p checks whether it is synchronized. A * indicates that it has been synchronized.

Turn off transparent large pages and swap memory

Echo never > / sys/kernel/mm/transparent_hugepage/defrag echo never > / sys/kernel/mm/transparent_hugepage/enabled echo "echo never > / sys/kernel/mm/transparent_hugepage/enabled" > > / etc/rc.local echo "echo never > / sys/kernel/mm/transparent_hugepage/defrag" > > / etc/rc.local chmod + x / etc/rc.d/rc.local echo "vm.swappiness=1" > > / etc/sysctl.conf sysctl-p / / to make the configuration effective

Build a yum source

Rpm-ivh httpd-2.4.6-80.el7.centos.x86_64.rpm httpd-tools-2.4.6-80.el7.centos.x86_64.rpmmount / dev/sr0 / mntln-s / mnt centos7/etc/ yum.repos.d [CentOS-7] name=myrepobaseurl= http://bigdata-cdh01/centos7/enabled=1gpgcheck=0yum clean allyum makecacheyum install-y createrepocd / var/www/html/cm-5.16createrepo .cat myrepo.repo [myrepo] name=myrepobaseurl= http://bigdata-cdh01/cm-5.16/enabled=1gpgcheck=0

Install the download package for mysql5.7 database official website (mysql-5.7.13-linux-glibc2.5-x86_64.tar.gz)

Groupadd mysqluseradd mysql- s / sbin/nologin-M-g mysqltar-zxvf mysql-5.7.13-linux-glibc2.5-x86_64.tar.gz mv mysql-5.7.13-linux-glibc2.5-x86_64 / usr/local/mysqlchown mysql:mysql-R mysql/mkdir datachown-R mysql:mysql data/yum install-y perl-Data-Dumper.x86_64yum install libaio*./bin/mysqld-- initialize-- user=mysql-- basedir=/usr/local/mysql-- datadir=/usr/local / mysql/data/scp-r support-files/mysql.server / etc/init.d/mysqldvim / etc/init.d/mysqld basedir=/usr/local/mysqldatadir=/usr/local/mysql/dat Editing / etc/ my.cnf [mysqld] user=mysqldatadir=/usr/local/mysql/datasocket=/tmp/mysql.socklog-error=/usr/local/mysql/data/error.logpid-file=/usr/local/mysql/data/mysql/mysql.pidtmpdir=/tmp Settings mysql boot chkconfig mysqld on change password alter user user () identified by "123456" Flush privileges; opens remote login GRANT ALL PRIVILEGES ON *. * TO 'root'@'%' IDENTIFIED BY' 123456' WITH GRANT OPTION;flush privileges; to view the user name and password of mysql select host,user,authentication_string from mysql.user

Create a database

Create database metastore default character set utf8;CREATE USER 'hive'@'%' IDENTIFIED BY' 123456 has made Grant ALL PRIVILEGES ON metastore. * TO 'hive'@'%';FLUSH PRIVILEGES;create database hive default character set utf8;CREATE USER' hive'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON hive. * TO 'hive'@'%';FLUSH PRIVILEGES;create database cm default character set utf8;CREATE USER' cm'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON cm. * TO 'cm'@'%';FLUSH PRIVILEGES;create database am default character set utf8;CREATE USER' am'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON am. * TO 'am'@'%';FLUSH PRIVILEGES;create database rm default character set utf8;CREATE USER' rm'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON rm. * TO 'rm'@'%';FLUSH PRIVILEGES;create database hue default character set utf8;CREATE USER' hue'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON hue. * TO 'hue'@'%';FLUSH PRIVILEGES;create database oozie default character set utf8;CREATE USER' oozie'@'%' IDENTIFIED BY '123456 Taiwan Grant ALL PRIVILEGES ON oozie. * TO 'oozie'@'%';FLUSH PRIVILEGES

Copy the package of jdbc, all nodes need to be installed

Cd / usr/share/java/ln-s mysql-connector-java-5.1.38-bin.jar mysql-connector-java.jar

Install Metabase

/ usr/share/cmf/schema/scm_prepare_database.sh mysql cm cm 123456

Install jdk from the command line

Yum list | grep oracleyum install-y oracle-j2sdk1.7.x86_64export JAVA_HOME=/usr/java/jdk1.7.0_67-clouderaexport JRE_HOME=$JAVA_HOME/jreexport CLASSPATH=$JAVA_HOME/libexport PATH=$PATH:$JAVA_HOME/bin

Install on the primary node

Yum install-y cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent

Install from nod

Yum install-y cloudera-manager-daemons cloudera-manager-agent

Upload CDH files

CDH-5.16.2-1.cdh6.16.2.p0.8-el7.parcel CDH-5.16.2-1.cdh6.16.2.p0.8-el7.parcel.sha1 manifest.jsonmv CDH-5.16.2-1.cdh6.16.2.p0.8-el7.parcel.sha1 CDH-5.16.2-1.cdh6.16.2.p0.8-el7.parcel.sha

Start

Systemctl start cloudera-scm-server

Do not choose

Add Servic

Kafka installation

Upload http://archive.cloudera.com/kafka/parcels/3.1.1/KAFKA-3.1.1-1.3.1.1.p0.2-el7.parcelKAFKA-3.1.1-1.3.1.1.p0.2-el7.parcel.shamanifest.json to / opt/cloudera/parcel-repo upload KAFKA-1.2.0.jar to / opt/cloudera/csd

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.