Shulou(Shulou.com)06/01 Report--

This article mainly introduces how CDH5.14 integrates ranger0.5.3. It is very detailed and has certain reference value. Friends who are interested must finish it!

The hive version of CDH5.14 is 1.1.0. After a week of trying, I started to choose the latest version 1.2.0 of ranger. Finally, when I connected to jdbc, I reported an error because of the mismatch between the hive-jdbc version and the hive version, so I chose the ranger0.5.3 version to do the integration test. There are still some problems in the process. I feel that hontonworks and cloudera are not compatible. It's not perfect for the time being, but it's finally done.

Install ranger-admin first.

1.cd / home/ranger

Download the tar package for ranger:

Wget https://archive.apache.org/dist/incubator/ranger/0.5.3-incubating/apache-ranger-incubating-0.5.3.tar.gz

Extract the tar package:

2.tar-xzvf apache-ranger-incubating-0.5.3.tar.gz

Then enter the decompressed directory:

3./home/ranger/apache-ranger-incubating-0.5.3.tar.gz

It takes a long time to compile the whole project. It is recommended that nohup be hung in the background:

4.nohup mvn clean compile package assembly:assembly install-DskipTests=true-Drat.skip=true-Dmaven.test.skip=true > maven.log &

Solve a problem before compilation, otherwise the compilation will not pass:

The 1.python version requires python2.7, and there will be problems with 2.6or 3.

Enter the directory after compilation

/ home/ranger/apache-ranger-1.2.0/target

All the files we need have been built.

Let's install ranger-admin first, cp to / usr/local

Cp / home/ranger/apache-ranger-incubating-0.5.3/target/ranger-0.5.3-admin.tar.gz / usr/local

Decompress:

Tar-xzvf / usr/local/ranger-0.5.3-admin.tar.gz

Enter / usr/local/ranger-0.5.3-admin

Modify the configuration items in install.properties, and then

Using root users to execute

Bash / usr/local/ranger-0.5.3-admin/setup.sh

It will be installed.

Then execute ranger-admin start to start ranger-admin

The browser http://hostname:6080 has the following login interface, and admin/admin can log in.

User name: admin

Password: admin

Here we focus on the configuration and installation of ranger-hive-plugin.

After the ranger source code has been compiled, there is a package for the hive plug-in in the ranger directory:

/ home/ranger/apache-ranger-incubating-0.5.3/target/ranger-0.5.3-hive-plugin.tar.gz

Cp the package to the node of hiveserver and put it in the same level path of the hive installation directory, such as the cdh we use. Generally, the hive installation directory is in / opt/cloudera/parcels/CDH/lib/hive/, so we put ranger-hive-plugin in the / opt/cloudera/parcels/CDH/lib directory because

Later, when ranger-hive-plugin is integrated into hive, it defaults to its relative path to find dependent jar packages, which can be found directly here without having to build a soft connection as it is said on the Internet.

Modify / opt/cloudera/parcels/CDH/lib/ranger-0.5.3-hive-plugin/install.properties later

After the change, run the enable-hive-plugin.sh script and restart the configuration of hiveserver2,ranger. However, starting hiveserver2 from cdh will not take effect, because it will not read the config directory under the hive installation path, but will dynamically generate a config directory, which can only be started manually.

The command is hive-- service hiveserver2

Let's see if we can solve this problem.

If you enter ranger admin, add hive rules, and test jdbc connectivity will report an error, because our next version of ranger comes with the version of hive1.2.0, and the version of hive-jdbc we use is different from the version of hive, so we need to replace the relevant jar packages under admin.

Under the / usr/local/ranger-0.5.3-admin/ews/webapp/WEB-INF/classes/ranger-plugins/hive directory, all hive1.2-related packages are replaced with our version, and all related packages can be found in the cdh installation path (/ opt/cloudera/parcels/CDH-5.14.0-1.cdh6.14.0.p0.24/lib/hive/lib/)

After that, it can be used normally. Testing ranger with beeline client is normal, but an error will be reported when using hue. The error message shows that java.lang.IllegalArgumentException:Cannot modify hive.server2.proxy.user at runtime., is probably the agent user queried by hive that does not have the right to query, so it is necessary to add some system users to the whitelist of hive.

Add to the hive-site.xml:

Hive.security.authorization.sqlstd.confwhitelist.append

Mapred. | hive. | mapreduce. | spark.

Hive.security.authorization.sqlstd.confwhitelist

Mapred. | hive. | mapreduce. | spark.

Can

The above is all the content of the article "how CDH5.14 integrates ranger0.5.3". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.