Shulou(Shulou.com)06/03 Report--

Warm Tip: to see the high-definition no-code picture, please open it with your mobile phone and click the picture to enlarge.

1. Overview

This document describes the process of installing Kerberos Client under Windows Server2008 R2 (the kernel version of windows is 6.1, the same as windows 7) and HTTP accessing Web UI of HDFS, Yarn, Hive, HBase and other Hadoop services under FireFox (such as the 8088 of Namenode 50070 Yarn). The installation documentation is mainly divided into the following steps:

1. Install Kerberos Client on Windows Server2008 R2 64-bit.

two。 Use kinit testing under Windows.

3. Kinit operation is performed under Windows through keytab.

4. Configure FireFox.

5. Verify that the service is accessed properly.

This document focuses on the installation and use of Kerberos under Winodws Server R2, based on the following assumptions:

The 1.CDH environment has been set up and running normally

2.HDFS, Yarn, HBase, Hive and other components have been installed and deployed

3. The cluster has completed the configuration of Kerberos and is in normal use.

The following are the hard limits for this test environment, but not for this operation manual:

1. Operating system: Redhat7.2

2.CM version: CM5.11.1

3.CDH version: CDH5.11.1

4. Deploy the cluster with ec2-user

2. Kerberos installation

2.1 Kerberos installation package preparation

Download the Kerberos installation package (http://web.mit.edu/kerberos/dist/) on the Kerberos official website, and take the following screenshot:

Download different Client,32 bits or 64 bits according to your Windows operating system.

2.2 install Kerberos

1. Double-click the downloaded client to open it.

two。 Click next, as shown in the following figure

3. Select "agree" and click next.

4. Select "Typical" to proceed to the next step

5. Click "install" to install

Click "finish" to complete the installation

6. Configure the C:\ ProgramData\ MIT\ Kerberos5\ krb5.ini file

Copy part of the / ect/krb5.conf file on the KDC Server server to the krb5.ini file

Note: copy the annotated part to the krb5.ini file here, otherwise the MIT Kerberos will not start normally. If you directly rename the krb5.conf file to the ini file and replace the krb5.ini file, there will be file format problems and the MIT Kerberos will not start normally.

7. After the configuration is complete, start the MIT Kerberos client, as shown in the following figure:

3 use kinit to test under Window

3.1 Log in to Kerberos with a password

Create a user through admin on 1.KDC Server

Ec2-user@ip-172-31-18-97 keytab$ sudo kadmin.local Authenticating as principal root/admin@CLOUDERA.COM with password. Kadmin.local: addprinc test@CLOUDERA.COM WARNING: no policy specified for test@CLOUDERA.COM; defaulting to no policy Enter password for principal "test@CLOUDERA.COM": Re-enter password for principal "test@CLOUDERA.COM" Principal "test@CLOUDERA.COM" created.

The test is normal

two。 On the Winodw side, through the MIT Kerberos client Get Ticket

You can see the obtained Ticket in the following list

3. Destroy the acquired Ticket

Select the Ticket to be destroyed in the list, and click Destroy Ticket

4. Initialize under the command line

You can see the Ticket initialized successfully on the client side

5. Kdestroy under the command line

Client data has been destroy

3.2 Log in to Kerberos using the Keytab file

1. Create a ketab file in KDC Server, using the test@CLOUDERA.COM you created in the previous step

Ec2-user@ip-172-31-18-97 keytab$ sudo kadmin.localAuthenticating as principal root/admin@CLOUDERA.COM with password.kadmin.local: xst-norandkey-k test.keytab test@CLOUDERA.COMkadmin.local: Principal test.keytab does not exist.Entry for principal test@CLOUDERA.COM with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:-k.Entry for principal test@CLOUDERA.COM with kvno 1 Encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:-k.Entry for principal test@CLOUDERA.COM with kvno 1, encryption type des3-cbc-sha1 added to keytab WRFILE:-k.Entry for principal test@CLOUDERA.COM with kvno 1, encryption type arcfour-hmac added to keytab WRFILE:-k.Entry for principal test@CLOUDERA.COM with kvno 1, encryption type camellia256-cts-cmac added to keytab WRFILE:-k.Entry for principal test@CLOUDERA.COM with kvno 1 Encryption type camellia128-cts-cmac added to keytab WRFILE:-k.Entry for principal test@CLOUDERA.COM with kvno 1, encryption type des-hmac-sha1 added to keytab WRFILE:-k.Entry for principal test@CLOUDERA.COM with kvno 1, encryption type des-cbc-md5 added to keytab WRFILE:-k.

Note: you need to add the parameter "- norandkey" when generating keytab files, otherwise it will lead to a password error when initializing using kinit test@CLOUDERA.com directly.

Generated successfully

Test the test.keytab file

Note: under non-root users, the permissions of the generated keytab files need to be set to more than 644, otherwise initialization will fail or initialize with sudo permissions.

two。 Copy the generated test.keytab file to Windows Server and initialize it on the CMD command line

Initialization successful, displayed on MIT Kerberos client

4 FireFox browser access

4.1 FireFox browser configuration

1. Open a FireFox browser and enter about:config in the address bar

two。 Modify configuration parameters

Network.negotiate-auth.trusted-uris = ip-172-31-18-97.aplym Southeastly1.compute.PUBG recorder ipPRAQ 172-31-19-209.ap-southeast-1.compute.internalnetwork.auth.use-sspi = false

Note: network.negotiate-auth.trusted-uris this parameter mainly configures the hostname that needs to access the service, and is divided by "," if you need to access multiple host.

4.2 Test access to the HDFS service

1. Access port 50070 of HDFS

The ticket of hdfs cannot be accessed normally because it is not initialized at this time. Prompt for principal and password

two。 Select the hdfs.keytab file in the latest xxx-hdfs-NAMENODE directory under the / var/run/cloudera-scm-agent directory of the CM node

3. Test whether the hdfs.keytab is normal

List the identities of all hdfs through the kadmin.loca command

Root@ip-172-31-18-97 577-hdfs-NAMENODE# kadmin.local Authenticating as principal hdfs/admin@CLOUDERA.COM with password. Kadmin.local: listprincs hdfs* hdfs/ip-172-31-18-97.ap-southeast-1.compute.internal@CLOUDERA.COM hdfs/ip-172-31-19-209.ap-southeast-1.compute.internal@CLOUDERA.COM hdfs/ip-172-31-28-67.ap-southeast-1.compute.internal@CLOUDERA.COM hdfs/ip-172-31-30-214.ap-southeast-1.compute.internal@CLOUDERA.COM

Root@ip-172-31-18-97 577-hdfs-NAMENODE# kinit-kt hdfs.keytab hdfs/ip-172-31-18-97.ap-southeast-1.compute.internal@CLOUDERA.COM root@ip-172-31-18-97 577-hdfs-NAMENODE# klist Ticket cache: FILE:/tmp/krb5cc\ _ 0 Default principal: hdfs/ip-172-31-18-97.ap-southeast-1.compute.internal@CLOUDERA.COM Valid starting Expires Service Principal07/26/2017 11:23:08 07/27/2017 11:23:08 krbtgt/CLOUDERA.COM@CLOUDERA.COM renew until 07/31/2017 11:23:08

Note: initialization can be done using the hdfs identity listed by listprincs

4. Copy the hdfs.keytab file to the Windows machine and initialize it with the CMD command

5. Access the HDFS service again through the FireFox browser, and visit normally

6. Whether it is normal to access Yarn resource management service

Cannot be accessed properly when viewing job details because the hostname information was not added to the network.negotiate-auth.trusted-uris configuration of FireFox

Add the hostname to the configuration

The second visit was successful

5 merger and use of keytab

1. Put the keytab files that need to be merged in the same directory

two。 Use the ktutil command to merge keytab files as follows

Root@ip-172-31-18-97 keytab# ktutil ktutil: rkt hbase.keytab ktutil: rkt hive.keytab ktutil: rkt hdfs.keytab ktutil: rkt yarn.keytab ktutil: wkt test-new.keytab ktutil:

Generate test-new.keytab files in the current directory

Note: since the generated keytab file permissions are 600 by default, non-root users need to use sudo to execute ktutil commands or increase keytab permissions to more than 644.

3. Copy the generated file to the Windows machine and initialize it with kinit

Initialize with hdfs credentials

The service access of HDFS is successful

Initialize with hive credentials

Access to HDFS service succeeded

Initialize with hbase credentials

Access to HDFS service succeeded

Initialize with yarn credentials

Access to HDFS service succeeded

Through the above tests, the HDFS service can be accessed normally regardless of any identity using hdfs/yarn/hbase/hive, so it can be concluded that Kerberos only does identity authentication and does not authorize the page.

6. common problem

1. Unable to install Kerberos with you must install a windows service error

It may be due to the problem of Window version. The installation package selected in this document does not support Windows Server2008 (the kernel version of windows is 6.0, which is the same as windows vista), so it is recommended that client machines choose Windows7 or above.

two。 Set the Kerberos Credential Cache File path

Configure the environment variables for Windows

KRB5_CONFIG: Path for the kerberos inifile. KRB5CCNAME: Path for the kerberoscredential cache file.

After configuring the environment variables, you need to restart the machine for the environment variables to take effect. Note that the default "KRB5CCNAME" directory is in C:\ temp.

3. Misconfiguration of krb5.ini under windows caused MIT Kerberos not to start

If you directly rename the krb5.conf file to the ini file and replace the krb5.ini file, the file format problem will cause MIT Kerberos to fail to start normally. The configuration information is as follows:

# Configuration snippets may be placed in this directory as well [libdefaults] default_realm = CLOUDERA.COM dns_lookup_realm = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true rdns = EXAMPLE.COM # default_ccache_name = KEYRING:persistent:% {uid} [realms] # EXAMPLE.COM = {# kdc = kerberos.example.com# admin_server = kerberos.example.com#} CLOUDERA.COM = {kdc = ip-172-31-18-97.ap-southeast -1.compute.internal admin_server = ip-172-31-18-97.ap-southeast-1.compute.internal} [domain_realm] # .example.com = EXAMPLE.COM# example.com = EXAMPLE.COM. IP-31-18-97.ap-southeast-1.compute.internal = CLOUDERA.COM ip-172-31-18-97.ap-southeast-1.compute.internal = CLOUDERA.COM

4. After the credential is created, and after the keytab file is created with the credential, the kinit credential reports an incorrect password

You need to add the parameter "- norandkey" when generating the keytab file, otherwise it will prompt for a password error when initializing directly with kinit test@CLOUDERA.com.

5. Some of the functions are normal and some are abnormal when accessing the Hadoop service through the browser

You need to confirm whether the accessed host address is added to the network.negotiate-auth.trusted-uris. This parameter mainly configures the hostname that needs to access the service, and is divided by "," if you need to access multiple host.

6. Instead of merging the keytab of all Hadoop-related services to access different pages, you can successfully initialize a user such as HDFS.

7 reference documentation

Http://doc.mapr.com/display/MapR/Configuring+Kerberos+Authentication+for+Windows

Https://www.cloudera.com/documentation/enterprise/latest/topics/cdh\_sg\_browser\_access\_kerberos\_protected\_url.html

Https://community.hortonworks.com/articles/28537/user-authentication-from-windows-workstation-to-hd.html

Http://web.mit.edu/kerberos/dist/index.html#kfw-4.0

Drunken whips are famous horses, and teenagers are so pompous! Lingnan Huan Xisha, under the vomiting liquor store! The best friend refuses to let go, the flower of data play!

Warm Tip: to see the high-definition no-code picture, please open it with your mobile phone and click the picture to enlarge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.