Shulou(Shulou.com)06/01 Report--

Editor to share with you how to install and configure CDH5.X kerberos authentication, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

1. Background

Before the Hadoop1.0.0 or CDH3 version, there was no security certification for hadoop. All nodes in the default cluster are reliable and trustworthy. The user does not need to authenticate when interacting with HDFS or Mmax R. This leads to malicious users pretending to be real users or servers invading the hadoop cluster, maliciously submitting jobs, modifying JobTracker status, tampering with data on HDFS, pretending to be NameNode or TaskTracker to accept tasks, and so on. Although HDFS increases the permissions of files and directories after version 0.16, there is no guarantee of strong authentication, and these permissions can only protect against accidental data loss. Malicious users can easily tamper with permissions by pretending to be other users, so that the permission settings are virtually non-existent. There is no security guarantee for the Hadoop cluster.

After the Hadoop1.0.0 or CDH3 version, the Kerberos authentication mechanism is added. So that the nodes in the cluster are what they claim to be and are trusted. Kerberos can put the authenticated key on a reliable node first in cluster deployment. When the cluster is running, the nodes in the cluster are authenticated using the key. Only authenticated nodes can be used properly. The nodes trying to impersonate cannot communicate with the nodes within the cluster because they do not have the key information in advance. It prevents the malicious use or tampering of the Hadoop cluster and ensures the reliability and security of the Hadoop cluster.

two。 Installation process (requires the following 8 steps)

Configuring a Kerberos 5 Server

When setting up Kerberos, install the KDC first. If it is necessary to set up slave servers, install the master first.

To configure the first Kerberos KDC, follow these steps:

1. Ensure normal time synchronization and DNS configuration of all clients and servers prior to Kerberos. Pay special attention to time synchronization between the Kerberos server and the customer. If the time difference between the server and the client is greater than 5 minutes (this is configurable Kerberos 5), the Kerberos client cannot authenticate to the server. This time synchronization is necessary to prevent attackers from impersonating legitimate users with an old Kerberos ticket.

It is recommended to establish a network time protocol (NTP) compatible client / server network even if Kerberos is not used. Red Hat Enterprise Linux includes national TB control planning programmes. Refers to / usr/share/doc/ntp-- / index. Details on how to set up a network time protocol server for html, which is the version number package installed on your system, and http: / / www.ntp.org for more information about national TB control plans.

two。 Install the krb5-libs,krb5-server and krb5-workstation packages to run KDC on a dedicated machine. This machine needs to be secure-if possible, it should not run any service other than KDC.

3. Editing the / etc/krb5.conf and / var/kerberos/krb5kdc/kdc.conf configuration files reflects the domain name and domain-to-realm mapping. An example of instance construction in which a simple domain can be replaced. COM and example.com use the correct domain names and be sure to keep uppercase and lowercase names in the correct format by changing the name from the kerberos.example.com KDC Kerberos server. By convention, all realm names are uppercase and DNS hostnames and domain names are lowercase. For complete details of these configuration file formats, please refer to their respective man pages.

4. Create the database using the kdb5_util utility from a shell prompt:

/ usr/kerberos/sbin/kdb5_util create-s

The create command creates the key to the database storage Kerberos domain. -s forces the creation of a master server key store for hidden files. If there is no key to reading hidden files, the Kerberos server (krb5kdc) prompts the user for the master server password (the key that can be used for regeneration) to start each time.

5. Edit the / var/kerberos/krb5kdc/kadm5.acl file, which is used by kadmind to determine which of the most important managers has access to the Kerberos database. Most organizations can rely on one line:

* / admin@EXAMPLE.COM *

Most users are represented in the database by a principal (NULL or empty, instance, such as joe@EXAMPLE.COM). In this configuration, the user and the second major management instance (for example, Joe / admin@EXAMPLE.COM) can use the Kerberos database in the full power domain.

After the kadmind server has started, any user can access its service on any client or server running on the kadmin domain. However, only users are listed in kadm5. The acl file can modify the database in any way except to change its own password.

Attention please

The kadmin utility handles authentication with the kadmind server through the network, communication, and the use of Kerberos. Therefore, first of all, the principal must already exist before connecting to the server through the network to manage it. Create the first and kadmin principal. The local command, which is specifically designed to KDC on the same host and not to use Kerberos authentication.

The following kadmin types. The local command creates the first primary KDC terminal:

/ usr/kerberos/sbin/kadmin.local-Q "addprinc username/admin"

6.Start Kerberos using the following commands:

/ sbin/service krb5kdc start

/ sbin/service kadmin start

/ sbin/service krb524 start

7. Use the addprinc command to add principals for users. In the command line interface of kadmin.kadmin and kaadmin.local KDC. As a result, many commands, such as addprinc--, are available with the kadmin program after launch. Refers to the kadmin man page for more information.

8. Confirm that KDC issues lottery tickets. First, run kinit to get a ticket and store it in a credential cache file. Next, use the list to see the credentials in the cache, and use kdestroy to break the cache and the credentials it contains.

Attention please

By default, kinit attempts to use the same system login authentication username (not the Kerberos server). If the user name does not correspond to an error message for kinit problems that are primarily in the Kerberos database. If this happens, kinit provides the correct name for the command line (kinit) that is primarily used as an argument.

1. Install Kerberos.

No. 3.2. Installing Kerberos

Kerberos packages may be installed by default, but make sure that the appropriate packages are installed for the Kerberos server or client being configured.

To install packages for a Kerberos server:

# yum install krb5-server krb5-libs krb5-auth-dialog

To install packages for a Kerberos client:

# yum install krb5-workstation krb5-libs krb5-auth-dialog

If the Red Hat Enterprise Linux system will use Kerberos as part of single sign-on with smart cards, then also install the required PKI/OpenSSL package:

# yum install krb5-pkinit-openssl

/ / download jce to replace ocal_policy.jar and US_export_policy.jar files on each host by extracting the contents of the downloaded package into the $JAVA_HOME/jre/lib/security/ directory.

After downloading http://www.oracle.com/technetwork/java/javase/downloads/jce-6-download-429243.html / /, copy the following .jar to the java_home/jre/lib/security/ file directory:

[root@master ~] # ll

Total dosage 16

-rw-rw-r-- 1 root root 3035 December 21 2013 local_policy.jar

-rw-r--r-- 1 root root 7323 December 21 2013 README.txt

-rw-rw-r-- 1 root root 3023 December 21 2013 US_export_policy.jar

[root@master ~] #

[root@master ~] # cp / root/UnlimitedJCEPolicyJDK8/*.jar / usr/java/default/jre/lib/security/

[root@slave1 ~] # yum install krb5-server krb5-libs krb5-auth-dialog / / install server because there are many services on the master server, now use one of the slave as a kerberos server.

/ / after the server is installed, install the following client on the client.

[root@slave2 ~] # yum install krb5-workstation krb5-libs krb5-auth-dialog

[root@master ~] # yum install krb5-workstation krb5-libs krb5-auth-dialog

/ / modify / etc/krb5.conf

Vim / etc/krb5.conf

[root@master ~] # cat / etc/krb5.conf

[logging]

Default = FILE:/var/log/krb5libs.log

Kdc = FILE:/var/log/krb5kdc.log

Admin_server = FILE:/var/log/kadmind.log

[libdefaults]

Default_realm = master

Dns_lookup_kdc = false

Dns_lookup_realm = false

Clockskew = 120

Ticket_lifetime = 86400

Renew_lifetime = 604800

Forwardable = true

Renewable = true

# default_tgs_enctypes = rc4-hmac

# default_tkt_enctypes = rc4-hmac

# permitted_enctypes = rc4-hmac

# udp_preference_limit = 1

[realms]

Master = {

Kdc = 192.168.8.94purl 88

Admin_server = 192.168.8.94Viru 749

}

[domain_realm]

[kdc]

Profile=/var/kerberos/krb5kdc/kdc.conf

After the modification, copy it to other machines.

/ / modify / var/kerberos/krb5kdc/kdc.conf

Vim / var/kerberos/krb5kdc/kdc.conf

[root@master ~] # cat / var/kerberos/krb5kdc/kdc.conf

[kdcdefaults]

Kdc_ports = 88

Kdc_tcp_ports = 88

[realms]

Master = {

Master_key_type = aes256-cts

Max_life = 25h

Max_renewable_life = 4w

Acl_file = / var/kerberos/krb5kdc/kadm5.acl

Dict_file = / usr/share/dict/words

Admin_keytab = / var/kerberos/krb5kdc/kadm5.keytab

Supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal

Max_life = 24 hours

Max_renewable_life = 10d

# default_principal_flags = + renewable, + forwardable

}

/ / modify / etc/

[root@master ~] # vim / var/kerberos/krb5kdc/kadm5.acl

* / admin@master *

After the above three files are configured, you only need to copy krb5.conf to other machines in the cluster.

[root@master ~] # scp / etc/krb5.conf slave1:/etc/

Krb5.conf 100% 422 0.4KB/s 00:00

[root@master ~] # scp / etc/krb5.conf slave2:/etc/

Krb5.conf 100% 422 0.4KB/s 00:00

[root@master ~] #

/ / start the kdc server

/ sbin/service krb5kdc start

/ sbin/service kadmin start

/ / the password for creating a kerberos database is: 123456

[root@master] # / usr/sbin/kdb5_util create-r master-s

[root@master ~] # kadmin.local / / create an administrator account for remote management

[root@master ~] # kadmin.local

Authenticating as principal test/admin@master with password.

Kadmin.local: listprincs / / list all user listprincs

K/M@master

Hdfs/slave1@master

Kadmin/admin@master

Kadmin/changepw@master

Kadmin/master@master

Krbtgt/master@master

Test@master

Kadmin.local: addprinc hadoop/admin@master / / create test user hadoop

WARNING: no policy specified for wwn@master; defaulting to no policy

Enter password for principal "hadoop@master":

Re-enter password for principal "hadoop@master":

Principal "hadoop@master" created.

Kadmin.local: listprincs / / View, the last wwn@master user was created successfully.

K/M@master

Hdfs/slave1@master

Kadmin/admin@master

Kadmin/changepw@master

Kadmin/master@master

Krbtgt/master@master

Test@master

Hadoop/admin@master

[root@master ~] # kinit hadoop / / Test: use the previously created wwn user, enter the password and do not report an error.

Password for wwn@master:

[root@master ~] # klist-e / / View the cached ticket subscription: you can see that you have successfully logged in as hadoop@master.

Ticket cache: FILE:/tmp/krb5cc_0

Default principal: hadoop@master

Valid starting Expires Service principal

12-26-14 08:24:48 12-27-14 08:24:48 krbtgt/master@master

Renew until 08:24:52 on 12-26-14, Etype (skey, tkt): arcfour-hmac, arcfour-hmac

[root@master krb5kdc] # kinit hadoop

Kinit: Client not found in Kerberos database while getting initial credentials

[root@master krb5kdc] # kinit hadoop/admin@master

Password for hadoop/admin@master:

[root@master krb5kdc] # klist-e

Ticket cache: FILE:/tmp/krb5cc_0

Default principal: hadoop/admin@master

Valid starting Expires Service principal

01/05/15 11:16:56 01/06/15 11:16:56 krbtgt/master@master

Renew until 01ax 05amp 15 11:17:00, Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96

4. Configure KDC

1. Create a bill account

[root@master ~] # kadmin.local

Addprinc-randkey hdfs/master@master

Addprinc-randkey hdfs/slave1@master

Addprinc-randkey hdfs/slave2@master

Addprinc-randkey mapred/master@master

Addprinc-randkey mapred/slave1@master

Addprinc-randkey mapred/slave2@master

Addprinc-randkey host/master@master

Addprinc-randkey host/slave1@master

Addprinc-randkey host/slave2@master

two。 Create a keytab file, which is a key that includes bill account number + bill account password encryption

[root@master ~] # kadmin.local

Xst-norandkey-k hdfs.keytab hdfs/master host/master

Xst-norandkey-k hdfs.keytab hdfs/slave1 host/slave1

Xst-norandkey-k hdfs.keytab hdfs/slave2 host/slave2

Xst-norandkey-k mapred.keytab mapred/master host/master

Xst-norandkey-k mapred.keytab mapred/slave1 host/slave1

Xst-norandkey-k mapred.keytab mapred/slave2 host/slave2

3. Verification

[root@master] # kinit-k-t / root/hdfs.keytab hdfs/master@master

As long as the report is correct.

4. Copy hdfs.keytab and mapred.keytab to / usr/lib/hadoop-0.20/conf/ of the cluster

[root@master ~] # cp hdfs.keytab / etc/hadoop/conf/

[root@master ~] # cp mapred.keytab / etc/hadoop/conf/

[root@master conf] # chown hdfs:hadoop hdfs.keytab

[root@master conf] # chown mapred:hadoop mapred.keytab

[root@master conf] # chmod 400. / * .keytab

5. Configure hadoop (manual configuration is complicated. You need to modify the following file. )

Stop the hadoop cluster

Vim / etc/hadoop/conf/core-site.xml / / modify simple to: kerberos, and change the false of item 2 to true

Hadoop.security.authentication

Kerberos

Hadoop.security.authorization

True

[root@master conf] # vim / etc/hadoop/conf/hdfs-site.xml

After the final configuration is completed, the users to view the generated are:

[root@master ~] # kadmin.local

Authenticating as principal root/admin@master with password.

Kadmin.local: listprincs

HTTP/master@master

HTTP/slave1@master

HTTP/slave2@master

K/M@master

Hadoop/admin@master

Hbase/master@master

Hbase/slave1@master

Hbase/slave2@master

Hdfs/master@master

Hdfs/slave1@master

Hdfs/slave2@master

Hive/master@master

Host/master@master

Host/slave1@master

Host/slave2@master

Httpfs/master@master

Hue/master@master

Impala/master@master

Impala/slave1@master

Impala/slave2@master

Kadmin/admin@master

Kadmin/changepw@master

Kadmin/master@master

Krbtgt/master@master

Mapred/master@master

Mapred/slave1@master

Mapred/slave2@master

Oozie/master@master

Solr/master@master

Solr/slave1@master

Solr/slave2@master

Spark/master@master

Test@master

Yarn/master@master

Yarn/slave1@master

Yarn/slave2@master

Zookeeper/master@master

Zookeeper/slave1@master

Zookeeper/slave2@master

The CDH configuration steps are relatively simple: the following 8 steps are required: screenshots are as follows:

The above is all the contents of the article "how to install and configure kerberos Certification for CDH5.X". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.