Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to install hive1.1 + mysql in Ubuntu, which is very detailed and has certain reference value. Friends who are interested must finish it!

One, download

1, apache-hive-1.1.0-bin.tar.gz download

Download address: http://apache.dataguru.cn/hive/hive-1.1.0/apache-hive-1.1.0-bin.tar.gz

2meme MySQL music connectorjava color 5.0.8.tar.gz download

Second, installation

1Die MySQL installation:

Sudo apt-get install mysql-server

Set the root password in the middle, please write it carefully and remember it.

Start MySQL:

The user who created the hive connection, and the database where the hive stores the metadata, as follows:

Mysql > GRANT ALL PRIVILEGES ON *. * TO 'hive'@'%' IDENTIFIED BY' hive'

Query OK, 0 rows affected (0.09 sec)

The mysql > create database hive default character set latin1; / / character set must be latian1, and if it is utf8, many hive metadata tables cannot be created.

Query OK, 1 row affected (0.01sec)

Specific startup commands:

Start metastore: (remote mysql needs to be started)

Hive-service metastore-hiveconf hbase.zookeeper.quorum=db96,db98,db99-hiveconf hbase.zookeeper.property.clientPort=2181

Start hiveservice:

Hive-- service hiveserver-hiveconf hbase.zookeeper.quorum=db96,db98,db99

-hiveconf hbase.zookeeper.property.clientPort=2181 / / (start the service so that the jdbc:hive can be connected. The default port is 10000. You must bring the following part, otherwise you can't connect with eclipse)

2Phinehive installation: enter the hive download directory and extract the apache-hive-1.1.0-bin.tar.gz to the / usr/local directory

Sudo tar-zxvf apache-hive-1.1.0-bin.tar.gz-C / usr/local

Mv apche-hive-1.1.0-bin hive

Sudo chown-R hadoop:hadoop hive

Third, environment configuration

1,/etc/profile:

Exprot HIVE_HOME=/usr/local/hive

Export PATH=$HIVE_HOME/bin:$PATH

2 cp hive-default.xml.template hive-site.xml and configure hive.site.xml

Hive.metastore.warehouse.dir

Hdfs://node0:9000/usr/hive/warehouse

(this corresponds to the previous hdfs dfs-mkdir-p / usr/hive/warehouse)

Node0 refers to the hostname of the author's NameNode.

Hive.exec.scratchdir

Hdfs://node0:9000/usr/hive/warehouse

/ / this is the same as when derby is configured.

Hive.querylog.location

/ usr/hive/log

-

Javax.jdo.option.ConnectionURL

Jdbc:mysql://localhost:3306/hive?createDatabaseIfNoExist=true

Javax.jdo.option.ConnectionURL

This parameter is used to set the metadata connection string

-

Javax.jdo.option.ConnectionDriverName

Com.mysql.jdbc.Driver

Javax.jdo.option.ConnectionDriverName

About using java to develop interaction with mysql in hive

Need to use a connector about mysql

This can describe the operation on database described by java language.

Into statements that mysql can understand.

Connector is a jar file described in the Java language.

This connector can be downloaded from the official website.

It has been confirmed that even if the version numbers of connector and mysql are not the same

The corresponding functions can also be realized.

Connector is to copy to the / usr/hive/lib directory.

Cp mysql-connector-java-5.1.1.18-bin / usr/hive/lib

-

Javax.jdo.option.ConnectorUserName

Hive

This javax.jdo.option.ConnectionUserName.

Is used to set the user name of the database (in this case, the mysql database) in which hive holds the metadata.

And this' hive' can be set according to the user's own needs.

-

Javax.jdo.option.ConnectionPassword

Hive

This javax.jdo.option.ConnetionPassword is used to set the

The password that the user needs to enter when logging in to the database.

This' hive' can also be set up according to the user's own needs.

Hive.aux.jars.path

File:///usr/local/hive/lib/hive-hbase-handler-0.13.1.jar,file:///usr/local/hive/lib/protobuf-java- 2.5.0.jar, file:///usr/local/hive/lib/hbase-client-0.96.0-hadoop2.jar,file:///usr/local/hive/lib/hbase-common-0.96.0-hadoop2.jar,file:///usr/local/hive/lib/zookeeper-3.4.5.jar, File:///usr/local/hive/lib/guava-11.0.2.jar

Hive.metastore.uris

Thrift://192.168.8.96:9083

The jar packages used in the above configuration are all under hbase/lib.

[root@db96 lib] # pwd / / found that the file does not exist in the lib directory of hive. Go to hbase's lib to find the following four file copies

/ / go to the lib directory of hive.

/ usr/local/hive/lib

[root@db96 lib] # ll protobuf-java-2.5.0.jar hbase-client-0.96.2-hadoop2.jar hbase-common-0.96.2-hadoop2.jar guava-12.0.1.jar

Ls: cannot access protobuf-java-2.5.0.jar: there is no such file or directory

Ls: cannot access hbase-client-0.96.2-hadoop2.jar: there is no such file or directory

Ls: cannot access hbase-common-0.96.2-hadoop2.jar: there is no such file or directory

Ls: cannot access guava-12.0.1.jar: there is no such file or directory

[root@db96 lib] # pwd

/ usr/local/hbase/lib

[root@db96 lib] # ll protobuf-java-2.5.0.jar hbase-client-0.96.2-hadoop2.jar hbase-common-0.96.2-hadoop2.jar guava-12.0.1.jar

-rw-r--r-- 1 root root 1795932 December 11 2013 guava-12.0.1.jar

-rw-r--r-- 1 root root 826678 March 25 07:03 hbase-client-0.96.2-hadoop2.jar

-rw-r--r-- 1 root root 371546 March 25 07:03 hbase-common-0.96.2-hadoop2.jar

-rw-r--r-- 1 root root 533455 December 11 2013 protobuf-java-2.5.0.jar

[root@db96 lib] # cp protobuf-java-2.5.0.jar hbase-client-0.96.2-hadoop2.jar hbase-common-0.96.2-hadoop2.jar guava-12.0.1.jar / usr/local/hive/lib

3,cp hive-log4j.properties.template hive-log4j.proprties

Vi hive-log4j.properties

Hive.log.dir=

This is where the corresponding log documents are stored when hive is running

(mine:hive.log.dir=/usr/hive/log/$ {user.name})

Hive.log.file=hive.log

This is the name of the hive log file.

The default is fine, as long as you can recognize the log.

There is only one important one that needs to be revised, otherwise an error will be reported.

Log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter

If there is no modification, it will appear:

WARNING: org.apache.hadoop.metrics.EventCounter is deprecated.

Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.

(just follow the warning prompt to modify it).

4. Finally, in the hive/lib directory, there will be a jline-2.12.jar to copy to hadoop/share/hadoop/yarn/lib to replace jline-0.19.jar.

The above is all the contents of the article "how to install hive1.1 + mysql in Ubuntu". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.