Shulou(Shulou.com)06/01 Report--

This article introduces how to enable Auto-TLS in the existing CDP-DC cluster, the content is very detailed, interested friends can refer to, I hope it can be helpful to you.

The purpose of the documentation is to focus on how to enable TLS on an existing CDP-DC cluster. Content Overview 1) TLS Overview 2) Level-0: basic TLS/SSL configuration (on CM server hosts) 3) Level-1: enable encryption test environment for cluster 1) CM and Cloudera Runtime version 7.1.12) Redhat7.73) administrator account with Cloudera Manager 4) Cluster does not enable Kerberos and LDAP authentication

TLS Overview background

After deploying Kerberos, CM will indicate that there is a security risk, which requires at least one level of TLS encryption. Describes Transport layer Security (TLS) that provides encryption and authentication in communication between ClouderaManager servers and agents. Encryption prevents traffic listening, and authentication helps prevent malicious servers or agents from causing problems in the cluster. Cloudera Manager supports three levels of TLS security, which must be configured step by step.

Level 1 (good)-this level only configures encrypted communication between the browser and the ClouderaManager and between the agent and the ClouderaManager server. See configure TLS encryption for Cloudera Manager only, and then configure TLS encryption for Cloudera Manager agents at level 1: for instructions. Level 1 encryption prevents snooping of communications between agents and Cloudera Manager.

Level 2 (better)-this level includes encrypted communication between the agent and the server, as well as strong authentication of the Cloudera Manager server certificate by the agent. See level 2: TLS authentication for the ClouderaManager server configured by the agent. Level 2 provides an additional level of security for the agent by verifying the trust of the certificate provided by the Cloudera Manager server.

Level 3 (Best)-encrypted communication between the agent and the server. Level 3 TLS includes encrypted communication between the agent and the server, the Cloudera Manager server certificate is strongly verified by the agent, and the agent is verified to the Cloudera Manager server using a self-signed or CA-signed certificate. See level 3: configure TLS authentication for the agent to the Cloudera Manager server. Level 3 addresses untrusted network scenarios where you need to prevent clustered servers from being spoofed by untrusted agents running on the host. Cloudera recommends that you configure level 3 TLS encryption for untrusted network environments before enabling Kerberos authentication. This provides secure communication between the keytab between the Cloudera Manager server and the authenticated agent in the cluster.

Cloudera strongly recommends that you set up a fully functional CDH cluster and Cloudera Manager before you start configuring Cloudera Manager servers and agents to use TLS. Cloudera Manager will continue to receive HTTP requests on port 7180 (the default), but once TLS is enabled, it will immediately redirect the client to port 7183 for HTTPS connections. Once level 3 TLS is configured, if you want to add a new host running the agent, you must manually deploy the ClouderaManager agent and daemon package for your platform, issue a new certificate for the host, configure / etc / cloudera-scm-agent/config.ini to use SSL / TLS, and then bring the host online. Instead, you can disable TLS adding hosts, configure new hosts for TLS, and then re-enable the appropriate configuration. Either method is effective, according to your needs. For all hosts running the agent, Cloudera recommends that you first use Java to create a KeyStore, and then use openSSL to export keys and certificates for use by agents or hue.

Tool Overview Java Keytool and OpenSSL are key management tools that allow you to create the security artifacts needed for TLS / SSL. In addition to the following two short summaries, see "how to encode TLS / SSL certificates and key conversion files (DER, JKS, PEM)" for more information. Java KeytoolOracle Java keytool is a utility shipped with Oracle JDK to create and manage encryption keys and certificates. During the process of configuring TLS / SSL for a Cloudera Manager cluster, you will create private key pairs, KeyStore, certificate signing requests, and use this software tool to create truststores for cluster-specific use, as described in the steps in this guide. Java Keytool for Cloudera Manager TLS / SSL configuration requires that for any step in using Java Keytool, make sure that you use a tool such as Oracle Java keytool instead of OpenJDK. Use the Oracle JDK:/usr/java/jdk1.8.0_232-cloudera/bin/jre/lib/security provided by Cloudera in this default path on the JDK or Cloudera Manager server host downloaded from Oracle. All steps use the same version of Java keytool. If more than one JDK is installed on the host, set the PATH variable so that Oracle JDK is called first, as shown in this example: export JAVA_HOME=/usr/java/jdk1.8.0_232-clouderaexport PATH=$JAVA_HOME/bin:$PATH uses the same password in any command that calls the-keypass and-storepass options. Cloudera Manager requires that the key and its KeyStore use the same password. OpenSSLOpenSSL is an open source encryption and TLS / SSL toolkit that has been widely used since its inception in 1999. Like Java Keytool, OpenSSL allows you to create private keys, certificate requests, and KeyStore, and provides options for verifying certificates. During RPC client and server communication, the Cloudera Manager Agent host acts as the client of the Cloudera Manager Server host. Proxy hosts, Hue, Impala, and other Python-based services require keys and certificates (PKCS#8) in PEM format, which is why the following steps include using this tool to transform some JKS artifacts. For more information, see how to encode a TLS / SSL Certificate and key conversion File (DER,JKS,PEM).

How to use self-signed certificates for TLS self-signed certificates does not apply to production deployments. The self-signed certificate is created and stored in the specified KeyStore during key generation and should be replaced with a signed certificate. Using self-signed certificates requires the generation and distribution of certificates and the establishment of explicit trust for certificates. However, using a self-signed certificate makes it easy to obtain a certificate for TLS / SSL configuration and may be suitable for non-production or test settings. "for more information, see configuring encryption for manually." Replace paths, file names, aliases, and other examples for your system in the following command. 1) create a certificate directory: mkdir-p / opt/cloudera/security/x509/ / opt/cloudera/security/jks/ grants Cloudera Manager access to the directory, sets the correct permissions, and then changes to the directory: sudo chown-R cloudera-scm:cloudera-scm / opt/cloudera/security/jkssudo umask 0700cd / opt/cloudera/security/jks2) generates a key pair and self-signed certificate, and stores everything in the KeyStore using the same password as the KeyStore and storepass As shown below. Use the FQDN of the current host for CN to avoid throwing a java.io.IOException: HTTPS hostname wrong exception. Replace OU,O,L,ST and C values with entries appropriate to your environment: keytool-genkeypair-alias cmhost-keyalg RSA-keysize 2048-dname "cn=cm01.example.com, ou=Department,o=Company, l=City, st=State, c=US"-keypass password-keystore example.jks-storepass password

[root@ip-10-0-0168 jks] # keytool-genkeypair-alias cmhost-keyalg RSA-keysize 2048-dname "cn=ap-southeast-1.compute.internal, ou=ipausers, > o=Cloudera, l=Shanghai, st=Shanghai, c=CN"-keypass cloudera-keystore example.jks-storepass cloudera

Warning:The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool-importkeystore-srckeystore example.jks-destkeystore example.jks-deststoretype pkcs12". [root @ ip-10-0-0168 jks] # 3) copy the default Java truststore (cacerts) to the standby system truststore (jssecacerts): export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdksource / etc/profilesudo cp $JAVA_HOME/jre/lib/security/cacerts $JAVA_HOME/jre/lib/ Security/jssecacerts4) exports the certificate from the KeyStore (example.jks). Keytool-export-alias cmhost-keystore example.jks-rfc-file selfsigned.cer

[root@ip-10-0-0168 jks] # keytool-export-alias cmhost-keystore example.jks-rfc-file selfsigned.cerEnter keystore password: Certificate stored in file

Warning:The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool-importkeystore-srckeystore example.jks-destkeystore example.jks-deststoretype pkcs12". [root @ ip-10-0-0-168 jks] # 5) copy the self-signed certificate (selfsigned.cer) to the / opt/ cloudera / security/x509/ directory. Cp selfsigned.cer / opt/cloudera/security/x509/cmhost.pem6) imports the public key into the alternate system truststore (jssecacerts) so that any process running with Java on this computer will trust the key. The default password for the Java truststore is changeit. Do not use the password you created for the KeyStore in step 2. $keytool-import-alias cmhost-file / opt/cloudera/security/jks/selfsigned.cer-keystore $JAVA_HOME/jre/lib/security/jssecacerts-storepass changeit

[root@ip-10-0-0168 jks] # keytool-import-alias cmhost-file / opt/cloudera/security/jks/selfsigned.cerEnter keystore password: Re-enter new password:Owner: CN=ap-southeast-1.compute.internal, OU=ipausers, O=Cloudera, L=Shanghai, ST=Shanghai, C=CNIssuer: CN=ap-southeast-1.compute.internal, OU=ipausers, O=Cloudera, L=Shanghai, ST=Shanghai C=CNSerial number: 33f6581eValid from: Sun Jul 19 06:07:24 UTC 2020 until: Sat Oct 17 06:07:24 UTC 2020Certificate fingerprints: MD5: 41:C5:94:8B:32:D2:95:67:1D:A2:12:75:6E:05:22:E0 SHA1: 8B:BE:F7:7F:75:A0:9B:55:0E:A7:6C:6E:2D:CD:32:CB:79:41:9C:EF SHA256: 54:ED:FE:C2 : FA:89:27:DC:3B:06:27:5C:EA:FB:93:2A:8B:A4:6B:27:4A:6E:13:DF:36:DB:76:E9:DE:33:10:55Signature algorithm name: SHA256withRSASubject Public Key Algorithm: 2048-bit RSA keyVersion: 3

Extensions:

# 1: ObjectId: 2.5.29.14 Criticality=falseSubjectKeyIdentifier [KeyIdentifier [0000: 6F C9 8E D3 60 A8 EA 33 BB 44 01 C8 34 5C 14 B1 o.`3.D.. 40010: E9 CF 6D 1C.. m.]

Trust this certificate? [no]: yesCertificate was added to Keystore [root @ ip-10-0,0,168 jks] # it is important to repeat this process on each host in the cluster. 7) rename the KeyStore: mv / opt/cloudera/security/jks/example.jks / opt/cloudera/security/jks/cmhost-keystore.jks8) you can also delete the certificate because it was copied to the appropriate path in step 5. Rm / opt/cloudera/security/jks/selfsigned.cer9) self-signed certificate is set up.

CM configuration updates go to the CM Management main page-> manage-> Security Click Enable Auto-TLS, enter the Auto-TLS configuration page and fill in the configuration information. Click next: follow the prompts, log in to the Cloudera-Manager server server, and then restart the Cloudera Manager Server service. When systemctl restart cloudera-scm-server restarts CM Server's service and continues to log in using the original login port, the browser will automatically jump to port 7183. Click Advanced Click to continue to … Link, go to the main page of CM, you can see from the main page, there are many out-of-date configurations. Click the restart button for expired configuration to restart the service (both the cluster and CMS services need to be restarted) if you have updated TLS Certificates information, click Rotate TLS Certificates through Auto-TLS to simply start the TLS of the entire cluster, reducing the complexity of manual configuration. On the existing CDP-DC cluster how to enable Auto-TLS to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.