Shulou(Shulou.com)06/01 Report--

This article mainly explains "What are Kafka's interview questions?" Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let Xiaobian take you to learn "what are Kafka's interview questions"!

1. The difference between RangeAssign and RoundRobinAssign, feels no different

Answer: Difference:

Distribute according to scope:

Sort all consumers lexicographically and partition values in each topic.

Divide the partitions of the theme into piles, one for each consumer.

theme-based;

In the case of a single theme, the partition of the theme is assigned to each consumer of the consumption group:

Guarantees that the difference in the number of partitions allocated to consumers is at most 1

If the consumer group subscribes to two topics, the partitions for each topic are assigned by region:

will result in a difference in the number of partitions allocated to different consumers in the consumer group greater than 1.

Polling mode lists all subscribed topic partitions, lists all consumer threads

Sort by topic, sort by consumer.

The maximum number of partitions allocated by different consumers is one, and the maximum difference in the number of partitions allocated by consumers is 1.

When a consumer group subscribes to multiple partitions:

2. When producing messages, the retry times are up, but it still fails. Will the data in the buffer be cleared? Is it up to the client to catch the exception and save the message?

A: Yes, it needs to be handled by the client, but it is not necessary to catch exceptions. It can be judged directly.

The following is the Sender thread to send messages, processing messages retry the main logic of the source code display:

Sender thread:

Responsible for getting messages from message accumulators and sending:

Implementation of the run method:

At the end of the method shown above, send the message:

Implementation of sendProducerData:

Finally, call the sendProduceRequests method to send the message:

The implementation of the sendProduceRequest method in the figure above: (Note that this is singular, not plural, meaning that the method sends a single batch of messages.)

Finally, create the request object and send the request:

callback handleProduceResponse method implementation:

CompleteBatch method implementation:

If there is an error when the message batch is sent, it is judged whether it can be retried. If it can be retried, the message batch is re-listed and waiting for retransmission:

The canRetrieval implementation in the above image:

Determine whether the current message batch can be retried. If it can be retried, retry is required:

To reorder message batches:

Re-queue batches of messages that need to be retried for retry:

In the completeBatch method, if the retry count is exhausted, execute the following code:

)

Implementation of failBatch:

Methods invoked in the above figure:

Finally, in the method shown in the figure above, the failed message batch is released from the accumulator.

The question is: if we run out of retries for the messages we sent, how do I know which message failed?

If I want to put the failed message somewhere else, such as for manual compensation, how do I get the failed message?

3. dome1 does not start, you need to configure advertised.listeners to start What is the reason Name Description Type Default Importance advertised.listeners If the listener address for the client is different from the listener configuration value, you need to use this attribute to register the listener on zookeeper. In an IaaS environment, this value is typically different from the address bound by the broker. If not set, use the listeners value. Unlike listeners, this address is not allowed to be specified to 0.0.0.0. stringnullhighlisteners Comma-separated list of listener addresses and listener names. If the listener is not using a security protocol, you also need to set the value of listener.security.protocol.map. Setting hostname to 0.0.0.0 binds to all network interfaces. If hostname is not set, bind to default network interface. Example: PLAINTEXT://myhost:9092,SSL://:9091CLIENT://0.0.0: 9092, RESOLUTION://localhost:9093stringnullhighinter.broker.listener.namebroker The listener name used for communication. If this property is not set, the value set by security.inter.broker.protocol is used. Do not use it with the security.inter.broker.protocol attribute. stringnullmediumsecurity.inter.broker.protocolbroker The security protocol used for communication between stringnullmediumsecurity. inter.broker.protocolbroker. Options include PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. Do not configure it with the inter.broker.listener.name property. stringPLAINTEXTmediumlistener.security.protocol.map Mapping between listener names and security protocols. When multiple ports or IP addresses use the same security protocol, it must be configured. For example, internal and external communications can be isolated using this configuration, even if SSL is used in both. Specifically, the user can define listeners with listener names INTERNAL and EXTERNAL, and the configuration of this attribute is: INTERNAL:SSL,EXTERNAL:SSL. Keys and values are separated by colons, and key-value pairs are separated by commas. Each listener name can appear only once in the configuration. The configuration of different security protocols (SSL and SASL) can be configured by adding a normalized prefix (listener name lowercase) to each listener. For example: To configure a different keystore for the INTERNAL listener, the name is: listener.name.internal.ssl.keystore.location If no listener name is configured, the configuration uses a generic configuration, such as ssl.keystore.location. stringPLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSLlow

Configure the virtual machine with two IP addresses:

kafka server.properties configuration:

Default configuration:

Publish services on all network interfaces using PLAINTEXT://:9092.

Zookeeper's information:

By default, both IP addresses 106 and 116 can be used:

public class MyProducer { public static void main(String[] args) throws ExecutionException, InterruptedException { Map configs = new HashMap();// configs.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.100.106:9092"); configs.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.100.116:9092"); configs.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); configs.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class); KafkaProducer producer = new KafkaProducer(configs); RecordMetadata metadata = producer.send( new ProducerRecord("tp_demo_01", "hello lagou 106-1") ).get(); System.out.println(metadata.topic() + "\t" + metadata.partition() + "\t" + metadata.offset()); producer.close(); }}

If the server has two IP addresses, one is an intranet and the other is an extranet, how to configure it so that the communication between brokers goes through the intranet IP address, while the extranet address is used by the extranet client?

First you need to configure two listeners:

listeners=PLAINTEXT://192.168.100.106:9092,PLAINTEXT://192.168.100.116:9093advertised.listeners=PLAINTEXT://192.168.100.116:9093

Error: Because PLAINTEXT in listeners is the name of the listener.

If two listeners cannot have the same name, improve the configuration:

listeners=PLAINTEXT://192.168.100.106:9092,MYCLIENT://192.168.100.116:9093advertised.listeners=MYCLIENT://192.168.100.116:9093

Error: Because no security protocol is configured for the listener with the MYCLIENT name.

listeners=PLAINTEXT://192.168.100.106:9092,MYCLIENT://192.168.100.116:9093advertised.listeners=MYCLIENT://192.168.100.116:9093listener.security.protocol.map=MYCLIENT:PLAINTEXT,PLAINTEXT:PLAINTEXTinter.broker.listener.name=MYCLIENT

zk Results:

Use the code to access:

192.168.100.116:9093 accessible, 192.168.100.106:9092 inaccessible.

Then 192.168.100.116:9093 is exposed to the client for use.

Ultimate configuration:

listeners=PLAINTEXT://192.168.100.106:9092,MYCLIENT://192.168.100.116:9093advertised.listeners=PLAINTEXT://192.168.100.106:9092,MYCLIENT://192.168.100.116:9093listener.security.protocol.map=MYCLIENT:PLAINTEXT,PLAINTEXT:PLAINTEXTinter.broker.listener.name=PLAINTEXT

A server has two IP addresses, one is an intranet address and the other is an extranet address.

Advertised.listeners Specifies the listeners to publish to zookeeper for client use.

When kafka starts using listeners, listen to the specified IP address and port using the specified protocol.

Also specify the name of the listener used for communication between brokers, which uses an intranet IP address.

The other of the advertised.listeners is for clients to use.

At this point, I believe everyone has a deeper understanding of "what Kafka's interview questions are." You may wish to actually operate them! Here is the website, more related content can enter the relevant channels for inquiry, pay attention to us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.