Shulou(Shulou.com)06/03 Report--

How to use sentry to manage the rights of hive? I believe that many novice rookies can do nothing about this. Through the summary of this article, I hope you can find a solution.

Background:

1. Apache Sentry is an open source Hadoop component released by Cloudera, which provides fine-grained, role-based authorization and multi-tenant management model.

2. Sentry can be integrated with Hive/Hcatalog, Apache Solr and Cloudera Impala to provide rights management services for these components.

3. Role-based management (role-based acess control) by creating a role, granting permissions for each component to this role, and then adding this role to the user (group), the user will have the right to access the component by this role.

4. When using sentry to manage the rights of hive, the component here can be the entire server, a single db, or a single table.

The tests are as follows:

1.1 View all databases

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "show databases;"

My attempt to create a library first is as follows:

[hadoop@uhadoop-4wvgxxla-master2] $beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "create database test;"

Transaction isolation: TRANSACTION_REPEATABLE_READ

Error: Error while compiling statement: FAILED: SemanticException No valid privileges

User hive does not have privileges for CREATEDATABASE user configuration unit does not have the privilege of CREATEDATABASE

The required privileges: Server=uhadoop-4wvgxxla-master1- > action=create- > grantOption=false; (state=42000,code=40000)

Closing: 0: jdbc:hive2://uhadoop-4wvgxxla-master2:10000

1.2 View all roles

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "show roles;"

Transaction isolation: TRANSACTION_REPEATABLE_READ

+-+

| | role |

+-+

+-+

No rows selected (1.151 seconds)

Beeline version 2.3.3 by Apache Hive

Closing: 0: jdbc:hive2://uhadoop-4wvgxxla-master2:10000

/ / user role is empty

1.3 View current role

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "show current roles;"

Driver: Hive JDBC (version 2.3.3)

Transaction isolation: TRANSACTION_REPEATABLE_READ

+-+

| | role |

+-+

+-+

No rows selected (0.446 seconds)

Beeline version 2.3.3 by Apache Hive

Closing: 0: jdbc:hive2://uhadoop-4wvgxxla-master2:10000

/ / shows that there are currently no roles

1.4 View current user

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "select current_user ();"

Driver: Hive JDBC (version 2.3.3)

Transaction isolation: TRANSACTION_REPEATABLE_READ

+-+

| | _ c0 |

+-+

| | hive |

+-+

1 row selected (1.124 seconds)

Beeline version 2.3.3 by Apache Hive

Closing: 0: jdbc:hive2://uhadoop-4wvgxxla-master2:10000

/ / the user currently operating hiveserver2 is hive.

The hive user grants administrator privileges

2.1Create the administrator role admin

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "CREATE ROLE admin;"

2.2 Grant full server permissions to the admin role

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive / / after entering the interior of hiveserver2, execute as follows:

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > grant all on server uhadoop-4wvgxxla-master1 to role admin

No rows affected (0.491 seconds)

2.3Grant hive users the admin role

/ / after this step, the hive user can perform all data and permissions operations as an administrator user.

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "GRANT ROLE admin TO GROUP hive;"

Create table student (

Sno char (9) COMMENT 'user ID'

Sname char (20)

Ssex char (2)

Sage int

Sdept char (20)

);

Insert into student values (200215121,'Li Yong', 'male', 20 'female CS')

There is no problem of garbled code in Chinese, please test

The verification method requires:

Show create table xxx

Desc xxx

Desc formatted xxx

Check whether there is no problem of Chinese garbled in all three ways.

Create a test database (using hive users)

3.1 create a test db1,db2

/ / use the administrator user to log in to create two db1,db2 databases.

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "create database db1;create database db2;"

/ / create a test table and insert data

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "create table db1.t1 (id string);"

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "insert into db1.t1 values ('t 1x 001'), ('t 1x 002');"

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "create table db2.t2 (id string);"

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "insert into db2.t2 values ('t2y001'), (' t2you002');"

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > use db1

No rows affected (0.173 seconds)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show tables

+-+

| | tab_name |

+-+

| | T1 |

+-+

1 row selected (0.208 seconds)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > select * from T1

+-+

| | t1.id |

+-+

| | T1room001 | |

| | t1room002 | |

+-+

2 rows selected (0.294 seconds)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > select * from db2.t2

+-+

| | t2.id |

+-+

| | t2room001 |

| | t2room002 |

+-+

2 rows selected (0.304 seconds)

3.2 create linux test user user1, user2 on the master1,master2 node

Useradd-M-s / sbin/nologin user1

Useradd-M-s / sbin/nologin user2

Cat / etc/passwd

User1:x:1004:1005::/home/user1:/sbin/nologin

User2:x:1005:1006::/home/user2:/sbin/nologin

Create two roles in hive to grant different role permissions

/ / create the role role1 and grant it administrative rights to db1

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "CREATE ROLE role1;"

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "grant all on database db1 to role role1 with grant option;"

/ / create the role role2 and grant it administrative rights to db2

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "CREATE ROLE role2;"

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "grant all on database db2 to role role2 with grant option;"

/ / show grant role role1; (view the permission list of the role1 role)

/ / show grant role role2; (view the permission list of the role2 role)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show grant role role1

+- -+

| | database | table | partition | column | principal_name | principal_type | privilege | grant_option | grant_time | grantor |

+- -+

| | db1 | role1 | ROLE | | true | 1583739035000 |-|

+- -+

1 row selected (0.215 seconds)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show grant role role2

+- -+

| | database | table | partition | column | principal_name | principal_type | privilege | grant_option | grant_time | grantor |

+- -+

| | db2 | role2 | ROLE | | true | 1583739057000 |-|

+- -+

1 row selected (0.119 seconds)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show grant role admin

+- -+

| | database | table | partition | column | principal_name | principal_type | privilege | grant_option | grant_time | grantor |

+- -+

| | admin | ROLE | | false | 1583737318000 |-|

+- -+

1 row selected (0.131 seconds)

3.4.Admin users log in to hive and assign different roles to the two users

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "GRANT ROLE role1 TO GROUP user1;"

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n hive-e "GRANT ROLE role2 TO GROUP user2;"

/ / show role grant group user1 (view the list of roles in user1)

/ / show role grant group user2 (view the list of roles in user2)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show role grant group user1

+-+

| | role | grant_option | grant_time | grantor | |

+-+

| | role1 | false | 0 |-| |

+-+

1 row selected (0.144 seconds)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show role grant group user2

+-+

| | role | grant_option | grant_time | grantor | |

+-+

| | role2 | false | 0 |-| |

+-+

1 row selected (0.125 seconds)

4 use user1, user2 users to log in and verify permission isolation

/ / user1 login, you can only see the db1 database

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n user1-e "show databases;"

/ / user2 users log in and can only see the db2 database

Beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n user2-e "show databases;"

Other usage tests

5.1 remove role from the character

REVOKE ROLE role1 FROM GROUP user1

Delete role

/ / check the role list first

Show roles

/ / Delete roles

Drop role role2

Role permission revocation

/ / check the current authorization information of the role first

Show grant role role1

/ / revoke the operation permission of db1 from role1

Revoke all on database db1 from role role1

Authorization statement description:

Role authorization and revocation

GRANT ROLE role_name [, role_name] TO GROUP [, GROUP]

REVOKE ROLE role_name [, role_name] FROM GROUP [, GROUP]

Grant and revocation of authority

GRANT [,] ON TO ROLE [, ROLE]

REVOKE [,] ON FROM ROLE [, ROLE]

View role / group permissions

SHOW ROLES

SHOW CURRENT ROLES

SHOW ROLE GRANT GROUP

SHOW GRANT ROLE

SHOW GRANT ROLE on OBJECT

View all roles

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show roles

+-+

| | role |

+-+

| | admin |

| | role1 |

| | role2 |

+-+

3 rows selected (0.12 seconds)

# Grant a database read permission to a role

GRANT SELECT ON DATABASE db_name TO ROLE role_name

# Grant read access to the S1 column of the test table to role_name (TABLE may not be written)

GRANT SELECT (S1) ON TABLE test TO ROLE role_name

# select permission of test table is given to role_name role

GRANT SELECT ON TABLE test TO ROLE role_name

Example:

There are currently 2 users

User1 / / has all the permissions of the T1 table under db1

User2 / / has all the permissions of the T2 table under db2

There are roles at present.

+-+

| | role |

+-+

| | admin | / / the highest permission all of all libraries |

| | role1 | / / only all permissions of db1 library |

| | role2 | / / only all permissions under the db2 library |

+-+

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show databases

+-- +

| | database_name |

+-- +

| | db1 |

| | db2 |

| | default |

| | temp |

| | test_hive_ucloud10086 |

+-- +

[hadoop@uhadoop-4wvgxxla-master1] $beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n user2-e "select * from db2.t2;"

Transaction isolation: TRANSACTION_REPEATABLE_READ

+-+

| | t2.id |

+-+

| | t2room001 |

| | t2room002 |

+-+

2 rows selected (0.631 seconds)

Beeline version 2.3.3 by Apache Hive

[hadoop@uhadoop-4wvgxxla-master1] $beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n user2-e "insert into db2.t2 values ('t2003'), (' t2004');"

Connected to: Apache Hive (version 2.3.3)

Driver: Hive JDBC (version 2.3.3)

Transaction isolation: TRANSACTION_REPEATABLE_READ

No rows affected (25.708 seconds)

Query again, ok inserted successfully

Transaction isolation: TRANSACTION_REPEATABLE_READ

+-+

| | t2.id |

+-+

| | t2room001 |

| | t2room002 |

| | t2room003 | |

| | t2room004 |

+-+

4 rows selected (0.605 seconds)

Beeline version 2.3.3 by Apache Hive

Closing: 0: jdbc:hive2://uhadoop-4wvgxxla-master2:10000

Next, I have a requirement. I want to open the student table under temp to user2.

First of all, user2 must have the select permission of the temp library for the role2 role.

Then give the select permission of the student table under the temp library to the role of role2

Then user2 belongs to the role2 role and naturally has the select permission of the student table under temp.

GRANT SELECT ON TABLE temp.student TO ROLE role2

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > GRANT SELECT ON TABLE temp.student TO ROLE role2

No rows affected (0.145 seconds)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show role grant group user2

+-+

| | role | grant_option | grant_time | grantor | |

+-+

| | role2 | false | 0 |-| |

+-+

1 row selected (0.129 seconds)

0: jdbc:hive2://uhadoop-4wvgxxla-master2:1000 > show grant role role2

+- -+

| | database | table | partition | column | principal_name | principal_type | privilege | grant_option | grant_time | grantor |

+- -+

| | db2 | role2 | ROLE | * | true | 1583739057000 |-|

| | temp | student | role2 | ROLE | SELECT | false | 1583740481000 |-|

+- -+

2 rows selected (0.125 seconds)

Verify as follows:

[hadoop@uhadoop-4wvgxxla-master1] $beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n user2-e "show databases;"

Transaction isolation: TRANSACTION_REPEATABLE_READ

+-+

| | database_name |

+-+

| | db2 |

| | default |

| | temp |

+-+

3 rows selected (0.614 seconds)

[hadoop@uhadoop-4wvgxxla-master1] $beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n user2-e "use temp;show tables;"

No rows affected (0.476 seconds)

+-+

| | tab_name |

+-+

| | student |

+-+

1 row selected (0.282 seconds)

Beeline version 2.3.3 by Apache Hive

[hadoop@uhadoop-4wvgxxla-master1] $beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n user2-e "select * from temp.student;"

Transaction isolation: TRANSACTION_REPEATABLE_READ

+-+-+

| | student.sno | student.sname | student.ssex | student.sage | student.sdept | |

+-+-+

| | 200215121 | Li Yong | male | 20 | CS |

+-+-+

1 row selected (0.667 seconds)

Beeline version 2.3.3 by Apache Hive

[hadoop@uhadoop-4wvgxxla-master1] $beeline-u "jdbc:hive2://uhadoop-4wvgxxla-master2:10000"-n user2-e "insert into temp.student values (100215122, 'Liu Chen', 'female', 19 user2');"

Transaction isolation: TRANSACTION_REPEATABLE_READ

Error: Error while compiling statement: FAILED: SemanticException No valid privileges

User user2 does not have privileges for QUERY / / user user2 does not have the privilege of QUERY

The required privileges: Server=uhadoop-4wvgxxla-master1- > Db=temp- > Table=student- > action=insert- > grantOption=false; (state=42000,code=40000)

Closing: 0: jdbc:hive2://uhadoop-4wvgxxla-master2:10000

After reading the above, do you know how to use sentry to manage the rights of hive? Is there a way to do it? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.