Shulou(Shulou.com)06/02 Report--

How to carry out hue installation and hive hdfs oozie integration, I believe that many inexperienced people do not know what to do, so this article summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

[toc]

Environmental requirements jdk

Maven

Git

Third party relies on yum install-y gcc-c++ libxml2-devel.x86_64 libxslt-devel.x86_64 python-devel openldap-devel asciidoc cyrus-sasl-gssapi openssl-devel mysql-devel sqlite-devel gmp-devel libffi-devel npm. What else is missing to install on your own?

Create hue users and user groups

Groupadd hue

Useradd-g hue hue

Switch users and enter the hue root directory

Su hue

Cd ~

Pull hue source code from git

Git clone https://github.com/cloudera/hue.git

Go to the hue-master directory and compile

Cd hue-master

Make apps needs to wait for a long time. If there is a lack of dependencies in the middle, you can install it yourself.

Modify hue configuration file

Vi / home/hue/hue-master/desktop/conf/pseudo-distributed.ini

Time zone

[desktop]

Time_zone=Asia/Shanghai

User role

[desktop]

Server_user=hue

Server_group=hue

Default_user=hue

Hdfs configuration

[hadoop]

[[hdfs_clusters]]

Fs_defaultfs=hdfs:// cluster name

Webhdfs_url= http://hdfsMaster1:14000/webhdfs/v1

Yarn configuration

[hadoop]

[[yarn_clusters]]

[default]

Resourcemanager_host=resourcemanager

Resourcemanager_api_url= http://resourcemanager:8088

Submit_to=True

Proxy_api_url= http://172.19.131.104:8088

History_server_api_url= http://172.19.131.104:19888

Database configuration

(hue database is created first and permissions are granted to hue users) [database]

Engine=mysql

Host=mysqlhost

Port=3306

User=hue

Password=123456

Name=hue

Oozie configuration (default but not matched)

[oozie]

Remote_data_dir=/user/hue/oozie/workspaces

Enable_cron_scheduling=true

Enable_document_action=true

Liboozie configuration

[liboozie]

Oozie_url= http://master1:11000/oozie

Beeswax configuration (hive)

[beeswax]

Hive_server_host=172.19.131.107

Hive_server_port=10000

Database initialization

/ home/hue/hue-master/build/env/bin/hue syncdb

/ home/hue/hue-master/build/env/bin/hue migrate

Running

/ home/hue/hue-master/build/env/bin/supervisor

Problem solving if the hdfs path is inaccessible

Configure hadoop.proxyuser.hue.groups=* under the core-site configuration file of hadoop

Hadoop.proxyuser.hue.hosts=*

If the hive database cannot be accessed

You can authorize the database of hive at the rights management place (I use ranger).

If oozie submits the task and fails to execute it

You can view the log of MapReduce. It may be that the current user cannot submit task configuration information to the directory under yarn.

After reading the above, have you mastered how to install hue and how to integrate hive hdfs oozie? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.