Shulou(Shulou.com)06/03 Report--

Pulsar official documentation Concepts and Architecture-key elements in Messaging Concepts

1 message composition

Make up the data carried by Value / data payload messages. All pulsar messages carry the original bytes, but the message data also needs to follow the data. ShcemaKey messages can be tagged by Key. This can work for things such as topic compression optional key / value mapping of Properties user-defined attributes the name of the producer of the message generated by Producer name (the producer is automatically given a default name, but you can also explicitly apply your own name) Sequence ID each message belongs to an ordered sequence on its subject. The sequence ID of the message is its sequential Publish time message release timestamp (applied automatically by the producer) Event time an optional timestamp that the application can attach to the message, indicating the time when an event occurred, such as when the message was processed. If it is not explicitly set, the event time of the message is 0.

2 producer sending mode

The pattern states that the synchronous sending producer will wait for confirmation from the agent after sending each message. If no acknowledgement is received, the producer will assume that the send operation failed and the asynchronous sending producer will put the message in the blocking queue and return immediately. The client library then sends the message to the agent in the background. If the queue is full (the maximum size is configurable, the producer may be blocked or fail immediately when calling API, depending on the parameters passed to the producer

2.1 message compression support

LZ4

ZLIB

ZSTD

SNAPPY

2.2 support for batch processing

If batch processing is enabled, the producer accumulates and sends a batch of messages in a single request. The batch size is defined by the maximum number of messages and the maximum release delay.

3 consumer reception mode

The pattern states that synchronous reception will be blocked until a message is available, asynchronous reception will immediately return a futurevalue, such as CompletableFuture in Java, which is completed when a new message is available

3.1 Monitoring

The client library provides the listener implementation for the user. For example, the Java client provides a MessageListener interface. In this interface, the received method is called whenever a new message is received.

3.2 confirm

When the consumer successfully uses the message, the consumer sends an acknowledgement request to the agent so that the agent discards the message. Otherwise, it stores messages.

Messages can be confirmed one by one, or they can be cumulative. For cumulative confirmation, the consumer only needs to confirm the last message it received. All messages in the stream up to (including) the messages provided will not be re-delivered to the consumer

(cumulative confirmation cannot be used with shared subscription mode, because shared mode involves multiple users who have access to the same subscription)

In shared subscription mode, messages can be confirmed separately.

3.3 negative confirmation

When the consumer fails to use the message once and wants to use the message again, the consumer can send a negative confirmation to the agent, and the agent will resubmit the message.

Messages can be denied or cumulatively acknowledged one by one, depending on the consumption subscription model.

In the exclusive subscription model and the failover subscription model, consumers only passively acknowledge the last message they receive.

In the "shared" and "secret sharing" subscription modes, you can confirm the message negatively, respectively

3.4 confirm timeout

When the message is not used successfully and you want to trigger the agent to automatically redeliver the message, you can use the automatic redelivery mechanism of unacknowledged messages. The client will track the unacknowledged message throughout the acknowledgement timeout range and automatically send the agent a request to redeliver the unacknowledged message when the specified acknowledgement timeout occurs.

Be careful

Use negative confirmation before the confirmation timeout. Negative acknowledgement controls the redelivery of a single message in a more precise way and avoids invalid redelivery when the message processing time exceeds the acknowledgement timeout.

4 Dead letter theme

When some messages cannot be used by consumers, new messages will be generated. In this mechanism, unusable messages are stored in a separate topic, called a dead letter topic. You can decide what to do with messages in a dead letter topic

The following example shows how to enable a dead letter theme in a Java client using the default dead letter theme

Consumer consumer = pulsarClient.newConsumer (Schema.BYTES)

.topic (topic)

.subscriptionName ("my-subscription")

.subscriptionType (SubscriptionType.Shared)

.roomLetterPolicy (DeadLetterPolicy.builder ()

.maxRedeliverCount (maxRedeliveryCount)

.build ()

.subscribe ()

The default dead letter theme uses the following format:

-- DLQ

If you want to specify the name of a dead letter topic, use the following Java client example:

Consumer consumer = pulsarClient.newConsumer (Schema.BYTES)

.topic (topic)

.subscriptionName ("my-subscription")

.subscriptionType (SubscriptionType.Shared)

.roomLetterPolicy (DeadLetterPolicy.builder ()

.maxRedeliverCount (maxRedeliveryCount)

.roomLetterTopic ("your-topic-name")

.build ()

.subscribe ()

The subject of the dead letter depends on the redelivery of the message. The message will be redelivered due to a confirmation timeout or a negative confirmation. If you want to use negative acknowledgements for messages, be sure to do negative acknowledgments before the acknowledgement times out.

Be careful

Currently, dead letter topics are only enabled in shared subscription mode

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.