Shulou(Shulou.com)06/03 Report--

1. Software Environment RHEL6 Role jdkhadoop-2.8.1.tar.gzapache-maven-3.3.9mysql-5.1xx.xx.xx ip address NNhadoop01xx.xx.xx ip address DNhadoop02xx.xx.xx ip address DNhadoop03xx.xx.xx ip address DNhadoop04xx.xx ip address DNhadoop05

This involves pseudo-distributed deployment only to host hadoop01, software installation reference pseudo-distributed deployment final chapter

Before compiling hive, we need to read readme.txt of compiled hive so that we can download the corresponding software version

Requirements

============

- Java 1.6, 1.7 ---jdk can only use 1.6 or 1.7, not 1.8

- Hadoop 1.x, 2.x

2. Install jdk

mkdir /usr/java && cd /usr/java/

tar -zxvf /tmp/server-jre-7u80-linux-x64.tar.gz

chown -R root:root /usr/java/jdk1.7.0_80/

echo 'export JAVA_HOME=/usr/java/jdk1.7.0_80'>>/etc/profile

source /etc/profile

3. Install maven

cd /usr/local/

unzip /tmp/apache-maven-3.3.9-bin.zip

chown root: /usr/local/apache-maven-3.3.9 -R

echo 'export MAVEN_HOME=/usr/local/apache-maven-3.3.9'>>/etc/profile

echo 'export MAVEN_OPTS="-Xms256m -Xmx512m"'>>/etc/profile

echo 'export PATH=$MAVEN_HOME/bin:$JAVA_HOME/bin:$PATH'>>/etc/profile

source /etc/profile

--JDK and maven deployment can refer to---Big Data---Deployment Installation Compilation Packaging Hadoop Ultimate

4. Install mysql

yum -y install mysql-server mysql

/etc/init.d/mysqld start

chkconfig mysqld on

mysqladmin -u root password 123456

mysql -uroot -p123456

use mysql;

GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'v123456' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO 'root'@'127.0.0.1' IDENTIFIED BY '123456' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;

update user set password=password('123456') where user='root';

delete from user where not (user='root') ;

delete from user where user='root' and password='';

drop database test;

DROP USER ''@'%';

flush privileges;

5. Download hive source package:

# http://archive.cloudera.com/cdh6/cdh/5/

#Select Hive package according to CDH version:

# hive-1.1.0-cdh6.7.1-src.tar.gz

#After decompression, use maven command to compile into installation package

6. Compile:

cd /tmp/

tar -xf hive-1.1.0-cdh6.7.1-src.tar.gz

cd /tmp/hive-1.1.0-cdh6.7.1

mvn clean package -DskipTests -Phadoop-2 -Pdist

#Compile generated packages in the following locations:

# packaging/target/apache-hive-1.1.0-cdh6.7.1-bin.tar.gz

7. Install the compiled Hive package and test it.

cd /usr/local/

tar -xf /tmp/apache-hive-1.1.0-cdh6.7.1-bin.tar.gz

ln -s apache-hive-1.1.0-cdh6.7.1-bin hive

chown -R hadoop:hadoop apache-hive-1.1.0-cdh6.7.1-bin

chown -R hadoop:hadoop hive

echo 'export HIVE_HOME=/usr/local/hive'>>/etc/profile

echo 'export PATH=$HIVE_HOME/bin:$PATH'>>/etc/profile

8. Change environment variables

su - hadoop

cd /usr/local/hive

cd conf

1、hive-env.sh

cp hive-env.sh.template hive-env.sh&&vi hive-env.sh

HADOOP_HOME=/usr/local/hadoop

2、hive-site.xml

vi hive-site.xml

javax.jdo.option.ConnectionURL

jdbc:mysql://localhost:3306/vincent_hive? createDatabaseIfNotExist=true

javax.jdo.option.ConnectionDriverName

com.mysql.jdbc.Driver

javax.jdo.option.ConnectionUserName

root

javax.jdo.option.ConnectionPassword

vincent

9, Copy mysql driver package to $HIVE_HOME/lib

#hive-site.xml above uses java mysql driver package

#Need to upload this package to hive under lib directory

#Unzip the file corresponding to mysql-connector-java-5.1.45.zip to the directory

cd /tmp

unzip mysql-connector-java-5.1.45.zip

cd mysql-connector-java-5.1.45

cp mysql-connector-java-5.1.45-bin.jar /usr/local/hive/lib/

No copy has related errors:

The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH.

Please check your CLASSPATH specification,

and the name of the driver.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.