Shulou(Shulou.com)06/01 Report--

Editor to share with you how to install Hive, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's learn about it!

1. Hive installation 1.1, hive installation address

1) Hive official website address http://hive.apache.org/

2) document view address https://cwiki.apache.org/confluence/display/Hive/GettingStarted

3) download address http://archive.apache.org/dist/hive/

4) github address https://github.com/apache/hive

1.2. Mysql installation 1.2.1, check whether mysql#rpm-qa is currently installed query |-ivh installation |-e-- nodeps uninstall (do not check dependencies) rpm-qa | grep mariadb-- > mariadb-libs-5.5.56-2.el7.x86_64-- if there is, uninstall sudo rpm-e-- nodeps mariadb-libs1.2.2 with the following command. Unzip and install mysql# must be installed in the following order: # MySQl runtime environment sudo rpm-ivh mysql-community-common-5.7.28-1.el7.x86_64.rpm#MySQL run jar package sudo rpm-ivh mysql-community-libs-5.7.28-1.el7.x86_64.rpm#MySQL extension jar package sudo rpm-ivh mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm#MySQL client sudo Rpm-ivh mysql-community-client-5.7.28-1.el7.x86_64.rpm#MySQL server sudo rpm-ivh mysql-community-server-5.7.28-1.el7.x86_64.rpm1.2.3, Delete the contents in the datadir directory # View the datadir path cat / etc/my.cnf-- > datadir=/var/lib/mysql# clear the contents in the path directory sudo rm-rf / var/lib/mysql1.2.4, initialize the database

Sudo mysqld-initialize-user=mysql

1.2.5. Start the database and modify the relevant configuration # start the mysql service sudo systemctl start mysqldsudo systemctl status mysqld# to view the temporary password sudo cat / var/log/mysqld.log# login database mysql-uroot-p (enter the temporary password in the line break) # change the password policy set global validate_password_length=4;set global validate_password_policy=0 # modify the login password set password = password ('123456') # modify the root user in the user table under the mysql library to allow any ip connection updata mysql.user set host= "%" where user= "root" # refresh the database flush privileges1.3, Hive installation 1.3.1, Installation and deployment 1) upload apache-hive-3.1.2-bin.tar.gz to linux's / opt/software directory 2) extract apache-hive-3.1.2-bin.tar.gz to / opt/module/ directory-- tar-zxvf / opt/software/apache-hive-3.1.2-bin.tar.gz-C / opt/module/3) modify the name of apache-hive-3.1.2-bin.tar.gz to Hive--mv / opt/module/apache-hive-3.1.2-bin/ / opt/module/hive4) modify / etc/profile.d/my_env.sh Add environment variable-sudo vim / etc/profile.d/my_env.sh5) add content # HIVE_HOME export HIVE_HOME=/opt/module/hive export PATH=$PATH:$HIVE_HOME/bin6) resolve log Jar package conflicts-mv $HIVE_HOME/lib/log4j2-slf4j-impl-2.10.0.jar $HIVE_HOME/lib/log4j2-slf4j-impl-2.10.0.bak1.3.2, hive metadata configuration to Mysql#1. Place the JDBC driver of MySQL in the lib directory of Hive cp / opt/software/mysql-connector-java-5.1.37.jar $HIVE_HOME/lib#2. Configure Metastore to MySQLvim / opt/module/hive/conf/hive-site.xml javax.jdo.option.ConnectionURL jdbc:mysql://hadoop102:3306/metastore?useSSL=false javax.jdo.option.ConnectionDriverName com.mysql.jdbc.Driver javax.jdo.option.ConnectionUserName root javax.jdo.option.ConnectionPassword 123456 Hive.metastore.warehouse.dir / user/hive/warehouse hive.server2.thrift.port 10000 hive.server2.thrift.bind.host hadoop102 hive.metastore.uris thrift://hadoop102:9083 hive.metastore.event.db.notification.api.auth false Hive.metastore.schema.verification false hive.server2.active.passive.ha.enable true 1.4 、 Start Hive1.4.1, element database initialization # login MySQL create Metabase create database metastore# exit MySQL initialization metadata schematool-initSchema-dbType mysql-verbose1.4.2, hive startup service # start metadata service (foreground service Connect mysql to get metadata) hive-service metastore# starts the hiveserver2 service (foreground service, jdbc connection service) hive-service hiveserver2#--0 standard input-1 standard output-2 error output-> & writes the output of one handle to the input of another handle. #-nohup: the beginning of the command Indicates that the terminal process is not suspended, that is, the terminal process is still running at the end of the &: command, indicating that nohup hive-- service metastore > log.txt 2 > & 1 & nohup hive-- service hiveserver2 > log2.txt 2 > & 1 & 1.4.3, hive startup service script #! / bin/bashHIVE_LOG_DIR=$HIVE_HOME/logsif [!-d $HIVE_LOG_DIR] then mkdir-p $HIVE_LOG_DIRfi# checks whether the process is running properly. Parameter 1 is the process name Parameter 2 is the process port function check_process () {pid=$ (ps-ef 2 > / dev/null | grep-v grep | grep-I $1 | awk'{print $2}') ppid=$ (netstat-nltp 2 > / dev/null | grep $2 | awk'{print $7}'| cut-d'/'- f 1) echo $pid [["$pid" = ~ "$ppid"] & & ["$ppid"] & return 0 | return 1} Function hive_start () {metapid=$ (check_process HiveMetastore 9083) cmd= "nohup hive-- service metastore > $HIVE_LOG_DIR/metastore.log 2 > & 1 &" cmd=$cmd "sleep 4 Hdfs dfsadmin-safemode wait > / dev/null 2 > & 1 "[- z" $metapid "] & & eval $cmd | | echo" Metastroe service started "server2pid=$ (check_process HiveServer2 10000) cmd=" nohup hive-- service hiveserver2 > $HIVE_LOG_DIR/hiveServer2.log 2 > & 1 & "[- z" $server2pid "] & & eval $cmd | | echo" HiveServer2 service started "function hive_stop () {metapid=$ (check_process HiveMetastore 9083) ["$metapid"] & & kill $metapid | | echo "Metastore service not started" server2pid=$ (check_process HiveServer2 10000) ["$server2pid"] & & kill $server2pid | | echo "HiveServer2 service not started"} case $1 in "start") hive_start ; "stop") hive_stop;; "restart") hive_stop sleep 2 hive_start;; "status") check_process HiveMetastore 9083 > / dev/null & & echo "Metastore service is running normally" | | echo "Metastore service operation exception" check_process HiveServer2 10000 > / dev/null & & echo "HiveServer2 service operation is normal" | echo "HiveServer2 service operation exception";; *) echo Invalid Args! Echo 'Usage:' $(basename $0) 'start | stop | restart | status';; esac1.4.4, Hive two startup methods # 1.Hive start bin/hive# to print the current library and header need to be configured with hive-site.xml hive.cli.print.header true Whether to print the names of the columns in query output. Hive.cli.print.current.db true Whether to include the current database in the Hive prompt.#2.JDBC starts beeline-u jdbc:hive2://hadoop102:10000-n luck1.5 hive interactive command # View all commands bin/hive-help#1. Hive-e does not enter the hive window to execute the sql statement bin/hive-e "select id from student" "# 2. View hdfs file system hive > dfs-ls#4.hive input history / home/luck/. in echo" select * from student "> > sql.txtbin/hive-f / opt/module/hive/datas/sql.txt > / opt/module/datas/hive_result.txt#3.hive in hive-f execution script Hivehistory / home/luck/.beeline/history1.6 attribute configuration description # 1.hive log storage location # modify hive-log4j.properties.template to hive-log4j.properties property.hive.log.dir=/opt/module/hive/logs # 2.JVM heap memory setting # modify hive-env.sh.template to hive-env.sh export HADOOP_HEAPSIZE=1024#3. Priority of parameter configuration # 3.1Profiles (permanent) # Hadoop Profiles-> default profile: hive-default.xml-- > user defined Profiles: hive-site.xml # 3.2Command Line parameters (valid only for this hive before hive startup) beeline-u jdbc:hive2://hadoop102:10000-n luck-hiveconf mapreduce.job.reduces=10 # 3.3 Parameter statement (valid for this hive only after hive startup) hive (default) > set mapreduce.job.reduces=10; # priority: parameter declaration > command line parameters > configuration file above are all the contents of the article "how to install Hive". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.