Shulou(Shulou.com)06/03 Report--

Add kerberos authentication and ACL permission control to a running kafka cluster while ensuring that all previous producer\ consumer services are uninterrupted

Solution: make the kafka cluster listen on two ports, one is an unauthenticated connection and the other is an authenticated connection of kerberos

At this point, there was a problem configuring ACL:

If I connect to the kafka cluster with kerberos authentication, my user name is the primary part of principal. For example, principal is kafka/master@HZ.DATA.COM, then my user name is kafka. At this time, all I have to do is to configure the appropriate permissions for the user kafka.

But when I connect to the kafka cluster in an unauthenticated way, I get the following error:

[WARN] 16:06:55,440, [Class] NetworkClient, Error while fetching metadata with correlation id 1: {test=UNKNOWN_TOPIC_OR_PARTITION} org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [test]

Or the error in console-producer is as follows:

WARN Error while fetching metadata with correlation id 2: {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-08-22 15 org.apache.kafka.clients.NetworkClient] WARN Error while fetching metadata with correlation id 4: {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-08-22 15 WARN Error while fetching metadata with correlation id] WARN Error while fetching metadata with correlation id 6: {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-08-22 15 test=UNKNOWN_TOPIC_OR_PARTITION 17 test=UNKNOWN_TOPIC_OR_PARTITION] WARN Error while fetching metadata with correlation id 7: {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient) [2017-08-22 15 15 Switzerland 17 test=UNKNOWN_TOPIC_OR_PARTITION 27993] WARN Error while fetching metadata with correlation id 8: {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)

Reason: once ACL is configured for a resource (such as topic) for a user of kerberos, the corresponding data cannot be accessed because there is no permission for non-authenticated access.

Solution: set the log of the kafka cluster to debug level and restart it. You can see that when you access the kafka cluster in an unauthenticated way, the user is ANONYMOUS.

1. Configure the appropriate permissions for ANONYMOUS.

2. After all, it is not convenient for each topic to set permissions for ANONYMOUS. You can set ANONYMOUS to super.users.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.