Shulou(Shulou.com)06/03 Report--

Warm Tip: to see the high-definition no-code picture, please open it with your mobile phone and click the picture to enlarge.

1. Purpose of document writing

This document focuses on how to use Sentry to manage Hive/Impala external table permissions.

Content Overview

1. Create a test library and external tables

two。 Create roles and authorize

3. Authorization test

4. Test summary

Test environment

1. The operating system is CentOS6.5

Version 5.12.1 for 2.CM and CDH

3. Adopt root user operation

Precondition

1. The cluster is running normally.

two。 Kerberos is enabled in the cluster and is in normal use

3.HDFS/Hive/Impala/Hue service has been integrated with Sentry

4.Hive users are super users

Friendly Tip: summary is the essence.

two。 Create a test library and external tables

Log in to Kerberos using the hive user and log in to HiveServer2 through beeline

Create a fayson database

0: jdbc:hive2://localhost:10000/ > create database fayson;INFO: Compiling command (queryId=hive_20170916155353_12e7c551-6a72-4ff3-b581-353c4dbd0fb0): create database faysonINFO: Semantic Analysis Completed... INFO: OKNo rows affected (0.232 seconds) 0: jdbc:hive2://localhost:10000/ >

two。 Create an external table student_hive under the fayson library with the following statement

Create external table if not exists student_hive (name string, age int, addr string) ROW FORMAT DELIMITED FIELDS TERMINATED BY', 'LOCATION' / extwarehouse/student_hive'

Put data to / extwarehouse/student_hive table

[root@ip-172-31-6-148i] # hadoop fs-put student.txt / extwarehouse/student_ hive [root @ ip-172-31-6-148i] # hadoop fs-ls / extwarehouse/student_hiveFound 1 items-rw-r--r-- 3 hive supergroup 59 2017-09-16 16:05 / extwarehouse/student_hive/ student.txt [root @ ip-172-31-6-148i] #

The / extwarehouse/student_hive data catalog is not saved, is automatically generated when the external table is created, and the data directory owner is hive.

0: jdbc:hive2://localhost:10000/ > select * from student_hive ... INFO: OK+--+ | student_hive.name | student_hive.age | student_hive.addr | +-- -+ | fayson | 23 | guangdong | | zhangsan | 24 | shenzhen | | lisi | 55 | guangzhou | +- -+-+ 3 rows selected (0.216 seconds) 0: jdbc:hive2://localhost:10000/ >

3. Create roles and authorize

Create faysonall roles and authorize fayson user groups

Authorized fayson user group has all permissions of fayson library

Create role faysonall;grant all on database fayson to role faysonall;grant role faysonall to group fayson

4. Authorization test

Log in to Kerberos using the fayosn user and connect to HiveServer2 [fayson @ ip-172-31-6-148148.fayson.com@FAYSON.COMscan complete in 2msConnecting to jdbc:hive2://localhost:10000/] $beeline Beeline version 1.1.0-cdh6.12.1 by Apache Hivebeeline >! connect jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-6-148.fayson.com@FAYSON.COMscan complete in 2msConnecting to jdbc:hive2://localhost:10000/ via beeline Principal=hive/ip-172-31-6-148.fayson.com@FAYSON.COMConnected to: Apache Hive (version 1.1.0-cdh6.12.1) Driver: Hive JDBC (version 1.1.0-cdh6.12.1) Transaction isolation: TRANSACTION_REPEATABLE_READ0: jdbc:hive2://localhost:10000/ >

two。 Switch to fayson database for student_hive table operation

You can insert data into a table

0: jdbc:hive2://localhost:10000/ > insert into student_hive values ('lisi', 22,' beijing');... INFO: OKNo rows affected (22.501 seconds) 0: jdbc:hive2://localhost:10000/ >

Table data can be queried

0: jdbc:hive2://localhost:10000/ > select * from student_hive ... INFO: OK+--+ | student_hive.name | student_hive.age | student_hive.addr | +-- -+ | lisi | 22 | beijing | | fayson | 23 | guangdong | | zhangsan | 24 | shenzhen | | lisi | 55 | | guangzhou | +-+ 4 rows selected (0.215 seconds) 0: jdbc:hive2://localhost:10000/ > |

3.HDFS verification

Fayson users can browse the data directory of student_hive and view the contents of files under the data directory, but they do not have permission for delete and put files.

[fayson@ip-172-31-6-14814] $hadoop fs-ls / extwarehouse/student_hiveFound 2 items-rwxr-xr-x 3 hive supergroup 16 2017-09-16 16:16 / extwarehouse/student_hive/000000_0-rw-r--r-- 3 hive supergroup 59 2017-09-16 16:05 / extwarehouse/student_hive/ student.txt [fayson @ ip-172-31-6-148t] $hadoop fs-rmr / extwarehouse/student_ Hive/student.txtrmr: DEPRECATED: Please use'rm-r 'instead.rmr: Failed to move to trash: hdfs://ip-172-31-6-148.fayson.com:8020/extwarehouse/student_hive/student.txt: Permission denied: user=fayson Access=WRITE, inode= "/ extwarehouse/student_hive": hive:supergroup:drwxr-xr-x [fayson@ip-172-31-6-14814] $hadoop fs-put student1.txt / extwarehouse/student_hive/put: Permission denied: user=fayson, access=WRITE, inode= "/ extwarehouse/student_hive": hive:supergroup:drwxr-xr-x [fayson@ip-172-31-6-14814] $

Test summary:

An external table created by hive, which can be queried and inserted by the fayson user group using beeline and Hue after being authorized by Sentry. However, you cannot add or delete data directories on HDFS and Hue FileBrowser because fayson users do not have permission to manipulate data directories.

5. Test summary

If the directory of this external table is not configured in cm to be managed by sentry, ACL synchronization cannot be done after being empowered by Sentry, and this is not recommended in production systems. If you need to manage external tables, then you need to follow the previous standard documentation. How to use Sentry to manage Hive external table permissions

Drunken whips are famous horses, and teenagers are so pompous! Lingnan Huan Xisha, under the vomiting liquor store! The best friend refuses to let go, the flower of data play!

Warm Tip: to see the high-definition no-code picture, please open it with your mobile phone and click the picture to enlarge.

It is recommended to follow Hadoop practice, the first time, share more Hadoop practical information, welcome to forward and share.

Original article, welcome to reprint, reprint please indicate: reproduced from the official account of Wechat Hadoop

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.