Shulou(Shulou.com)06/03 Report--

Warm Tip: to see the high-definition no-code picture, please open it with your mobile phone and click the picture to enlarge.

5.Sentry column rights management

1. Add fayson_r users to all nodes in the cluster

[root@ip-172-31-6-148cdh-shell-bak] # useradd fayson_ r [root @ ip-172-31-6-148cdh-shell-bak] # id fayson_ruid=504 (fayson_r) gid=504 (fayson_r) groups=504 (fayson_r) [root@ip-172-31-6-148cdh-shell-bak] #

two。 Create Kerberos user fayson_r

[root@ip-172-31-6-14814] # kadmin.localAuthenticating as principal hive/admin@FAYSON.COM with password.kadmin.local: addprinc fayson_r@FAYSON.COMWARNING: no policy specified for fayson_r@FAYSON.COM; defaulting to no policyEnter password for principal "fayson_r@FAYSON.COM": Re-enter password for principal "fayson_r@FAYSON.COM": Principal "fayson_r@FAYSON.COM" created.kadmin.local:

3. Log in to Kerberos using the hive user

Connect to HiveServer2 using beeline, create the columnread role and authorize read access to the column of test table S1, and authorize the columnread role to the fayson_r user group

[root@ip-172-31-6-14814] # klistTicket cache: FILE:/tmp/krb5cc_0Default principal: hive/ip-172-31-6-148.fayson.com@FAYSON.COMValid starting Expires Service principal09/07/17 15:27:58 08 beeline Beeline version 17 15:27:58 krbtgt/FAYSON.COM@FAYSON.COM renew until 09 peg 12 17 15:27: 58 [root @ ip-172-31-6-14814] # beeline Beeline version 1 .1.0-cdh6.12.1 by Apache Hivebeeline > create role columnread No current connectionbeeline >! connect jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-6-148.fayson.com@FAYSON.COM...0: jdbc:hive2://localhost:10000/ > create role columnread;...INFO: OKNo rows affected (0.183 seconds) 0: jdbc:hive2://localhost:10000/ > grant select (S1) on table test to role columnread;...INFO: OKNo rows affected (0.105 seconds) 0: jdbc:hive2://localhost:10000/ > grant role columnread to group fayson_r ... INFO: OKNo rows affected (0.105 seconds) 0: jdbc:hive2://localhost:10000/ >

4.fayson_r user testing

Log in to Kerberos using fayson_r and connect to HiveServer2 through beeline

[root@ip-172-31-6-14814] # kinit fayson_rPassword for fayson_r@FAYSON.COM: [root @ ip-172-31-6-14814] # klistTicket cache: FILE:/tmp/krb5cc_0Default principal: fayson_r@FAYSON.COMValid starting Expires Service principal09/08/17 03:16:47 09 krbtgt/FAYSON.COM@FAYSON.COM 17 03:16:47 krbtgt/FAYSON.COM@FAYSON.COM Renew until 09 take 15 Chark 17 03:16: 47 [root @ ip-172-31-6-14814] # beelineBeeline version 1.1.0-cdh6.12.1 by Apache Hivebeeline >! connect jdbc:hive2://localhost:10000/ Principal=hive/ip-172-31-6-148.fayson.com@FAYSON.COMscan complete in 2msConnecting to jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-6-148.fayson.com@FAYSON.COMConnected to: Apache Hive (version 1.1.0-cdh6.12.1) Driver: Hive JDBC (version 1.1.0-cdh6.12.1) Transaction isolation: TRANSACTION_REPEATABLE_READ0: jdbc:hive2://localhost:10000/ > show databases ... INFO: OK+-+--+ | database_name | +-+-+ | default | +-+-- + 1 row selected (0.336 seconds) 0: jdbc:hive2://localhost:10000/ > show tables INFO: OK+-+--+ | tab_name | +-+-+ | test | +-+-- + 1 row selected (0.202 seconds) 0: jdbc:hive2://localhost:10000/ > select * from test;Error: Error while compiling statement: FAILED: SemanticException No valid privileges User fayson_r does not have privileges for QUERY The required privileges: Server=server1- > Db=default- > Table=test- > Column=s2- > action=select (state=42000,code=40000) 0: jdbc:hive2://localhost:10000/ > select S1 from test | INFO: OK+-+--+ | S1 | +-+-+ | a | | 1 | 1 | 2 | testaa | 1 | | 2 | | 3 | | 0.433 rows selected (0.433 seconds) 0: jdbc:hive2://localhost:10000/ > select count (*) from test Error: Error while compiling statement: FAILED: SemanticException No valid privileges User fayson_r does not have privileges for QUERY The required privileges: Server=server1- > Db=default- > Table=test- > action=select; (state=42000,code=40000) 0: jdbc:hive2://localhost:10000/ > select count (S1) from test;...INFO: OK+-+--+ | _ c0 | +-+-+ | 11 | +-+-+ 1 row selected (33.012 seconds) 0: jdbc:hive2://localhost:10000/ >

5. Browse the HDFS directory

# klistTicket cache: FILE:/tmp/krb5cc_0Default principal: fayson_r@FAYSON.COMValid starting Expires Service principal09/08/17 03:16:47 09 krbtgt/FAYSON.COM@FAYSON.COM renew until 17 03:16:47 krbtgt/FAYSON.COM@FAYSON.COM renew until 09 15 03:16: 47 [root@ip-172 @ ip-172-31-6-14814] # hadoop fs-ls / user/hive/warehousels: Permission denied: user=fayson_r, access=READ_EXECUTE Inode= "/ user/hive/warehouse": hive:hive:drwxrwx-- x [root @ ip-172-31-6-148x] # hadoop fs-ls / user/hive/warehouse/testls: Permission denied: user=fayson_r, access=READ_EXECUTE, inode= "/ user/hive/warehouse/test": hive:hive:drwxrwx-- x [root @ ip-172-31-6-148x] #

6. Use the admin user to log in to Hue and create the fayson_r user

Log in using the fayson_r user

Cannot query all columns of test table

You can query the S1 column of the test table

Cannot browse the data directory / user/hive/warehouse/test of test table through File Brwoser

Test summary:

The user group to which the fayson_r user belongs is fayson_r, which only has read access to the S1 column of the test table, so when select and count, the user can only browse all directories under / user/hive/warehouse for select and count,fayson_r columns; using hue can only perform select and count operations on the S1 column of the test table, but not browse the / user/hive/warehouse directory and all subdirectories under the directory.

Note: Sentry only supports column authorization for SELECT and cannot be used for column authorization for INSERT and ALL.

6. Remarks

After the Sentry service is enabled in the cluster, it is recommended that Hive CLI be disabled because Sentry does not support Hive CLI rights management.

How to restrict users from using Hive CLI operations

Enter the Hive service and modify the hadoop.proxyuser.hive.group configuration. This configuration overrides the hive proxy user group configuration in the HDFS service. If the default is empty, the hive proxy user configuration in the HDFS service is inherited.

The above configuration means that hue, hive and impala user groups can use Hive CLI to restart Hive and its related services.

Note: if the configuration is empty, all user groups are prohibited. It should be noted that if the configuration is empty, Hue will not be available and hive users will not be able to access Hive through beeline and Hive CLI.

two。 Test whether the configuration is in effect

Log in to Kerberos using the hive user, and then use the Hive CLI operation

[root@ip-172-31-6-148251-hive-HIVEMETASTORE] # kinit-kt hive.keytab hive/ip-172-31-6-148.fayson.com @ FAYSON.com [root @ ip-172-31-6-148fayson.com] # klistTicket cache: FILE:/tmp/krb5cc_0Default principal: hive/ip-172-31-6-148.fayson.com@FAYSON.COMValid starting Expires Service principal09/07/17 13:33:21 09G0817 13:33:21 krbtgt/FAYSON.COM@FAYSON.COM renew until 09Compact 12 Accord 17 13:33: 21 [root @ ip-172-31-6-148 251-hive-HIVEMETASTORE] #

Log in to Hive CLI and perform SQL operation

[root@ip-172-31-6-148 251-hive-HIVEMETASTORE] # hive...hive > show databases;OKdefaultTime taken: 1.881 seconds, Fetched: 1 row (s) hive > show tables;OKtesttest_hive_delimitertest_tableTime taken: 0.034 seconds, Fetched: 3 row (s) hive > select * from test OKa b1 2111 222a b1 22 333testaa testbbb1 test2 fayson3 zhangsan222 2323Time taken: 0.477 seconds, Fetched: 11 row (s) hive > select count (*) from test;...OK11Time taken: 31.143 seconds, Fetched: 1 row (s) hive >

Use hue users to log in to Kerberos to test [root@ip-172-31-6-148259-hue-HUE_SERVER] # kinit-kt hue.keytab hue/ip-172- 31-6-148.fayson.com@FAYSON.COM [root@ip-172-31-6-148259-hue-HUE_SERVER] # klistTicket cache: FILE:/tmp/krb5cc_0Default principal: hue/ip-172- 31-6-148.fayson.com@FAYSON.COMValid starting Expires Service principal09/07/17 13:37:22 08 259-hue-HUE_SERVER 17 13:37:22 krbtgt/FAYSON.COM@FAYSON.COM renew until 09 259-hue-HUE_SERVER 12 13:37: 22 [root @ ip-172-31-6-148] #

Operate through Hive CLI

[root@ip-172-31-6-148 259-hue-HUE_SERVER] # hive...hive > show databases;OKdefaultTime taken: 1.892 seconds, Fetched: 1 row (s) hive > show tables;OKtesttest_hive_delimitertest_tableTime taken: 0.036 seconds, Fetched: 3 row (s) hive > select * from test FAILED: SemanticException Unable to determine if hdfs://ip-172-31-6-148.fayson.com:8020/user/hive/warehouse/test is encrypted: org.apache.hadoop.security.AccessControlException: Permission denied: user=hue, access=READ Inode= "/ user/hive/warehouse/test": hive:hive:drwxrwx--x at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkAccessAcl (DefaultAuthorizationProvider.java:363) at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check (DefaultAuthorizationProvider.java:256) at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission (DefaultAuthorizationProvider.java:168) at org.apache.sentry.hdfs.SentryAuthorizationProvider.checkPermission (SentryAuthorizationProvider.java At org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission (FSPermissionChecker.java:152) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission (FSDirectory.java:3530) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission (FSDirectory.java:3513) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPathAccess (FSDirectory.java:3484) at org.apache. Hadoop.hdfs.server.namenode.FSNamesystem.checkPathAccess (FSNamesystem.java:6624) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getEZForPath (FSNamesystem.java:9267) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getEZForPath (NameNodeRpcServer.java:1637) at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getEZForPath (AuthorizationProviderProxyClientProtocol.java:928) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getEZForPath (ClientNamenodeProtocolServerSideTranslatorPB. Java:1360) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod (ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call (ProtobufRpcEngine.java:617) at org.apache.hadoop.ipc.RPC$Server.call (RPC.java:1073) at org.apache.hadoop.ipc.Server$Handler$1.run (Server.java:2217) at org.apache.hadoop.ipc.Server$Handler$1.run (Server.java:2213 ) at java.security.AccessController.doPrivileged (Native Method) at javax.security.auth.Subject.doAs (Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs (UserGroupInformation.java:1917) at org.apache.hadoop.ipc.Server$Handler.run (Server.java:2211) hive >

Log in to Kerberos using the impala user to test [root@ip-172-31-6-148253-impala-STATESTORE] # kinit-kt impala.keytab impala/ip-172-31-6-148.fayson.com@FA YSON.com [root @ ip-172-31-6-148253-impala-STATESTORE] # klistTicket cache: FILE:/tmp/krb5cc_0Default principal: impala/ip-172-31-6-148.fayson. Com@FAYSON.COMValid starting Expires Service principal09/07/17 13:41:25 09 08 253-impala-STATESTORE 17 13:41:25 krbtgt/FAYSON.COM@FAYSON.COM renew until 09 253-impala-STATESTORE 12 13:41: 25 [root @ ip-172-31-6-148] #

Use Hive CLI operation

[root@ip-172-31-6-148253-impala-STATESTORE] # hive...hive > show databases;OKdefaultTime taken: 1.941 seconds, Fetched: 1 row (s) hive > show tables;OKtesttest_hive_delimitertest_tableTime taken: 0.037 seconds, Fetched: 3 row (s) hive > select * from test;OKa b1 2111 222a b1 22 333testaa testbbb1 test2 fayson3 zhangsan222 2323Time taken: 0.523 seconds, Fetched: 11 row (s) hive >

Using the fayson user test, this user is prohibited from using HiveCLI

Log in to the Kerberos test using the fayson user

[root@ip-172-31-6-14814] # kinit faysonPassword for fayson@FAYSON.COM: [root@ip-172-31-6-14814] # klistTicket cache: FILE:/tmp/krb5cc_0Default principal: fayson@FAYSON.COMValid starting Expires Service principal09/07/17 13:44:29 09 krbtgt/FAYSON.COM@FAYSON.COM renew until krbtgt/FAYSON.COM@FAYSON.COM renew until 09 / 14 to 17 13:44: 29 [root @ ip-172-31-6-14814] #

Use Hive CLI operation

# hive...hive > show databases;FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection resethive > show tables;FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection resethive >

Test summary:

Through Hive configuration (hadoop.proxyuser.hive.groups), you can restrict user groups from using HiveCLI to access Hive, and unconfigured user groups are not accessible through HiveCLI (such as fayson users). In the test, it is found that impala and hive users can access the hive table through Hive CLI without permission control, while hue users can only show databases and show tables can not select table. An exception was reported for HDFS access, because both hive,hive and impala users in the hive table belong to the hive group, and hue users do not belong to the hive group.

Note: hadoop.proxyuser.hive.groups is for user group restrictions. If the hive user group is configured to access Hive through Hive CLI, all users belonging to the hive group can access the Hive table through Hive CLI and are not controlled by Sentry permissions.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.