Shulou(Shulou.com)06/01 Report--

What are the seven reasons for choosing Pulsar instead of Kafka? in view of this question, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible way.

Guide: for developers of cloud-native distributed applications, to focus more on application and micro-service development rather than wasting time dealing with complex messaging infrastructures, they need a solution to help manage these infrastructures.

The first step in building a messaging infrastructure is to choose the appropriate message middleware technology. There are many options, from a variety of open source frameworks (such as RabbitMQ, ActiveMQ, NATS) to some commercial products (such as IBM MQ or RedHat AMQ), in addition to Kafka. In the end, however, instead of using Kafka, we chose Pulsar.

Why did you finally choose Pulsar? Here are 7 reasons to choose Pulsar over Kafka.

1. Combination of streaming and queuing

Pulsar is like an all-in-one product that not only handles high-speed real-time scenarios like Kafka, but also supports standard message queuing patterns, such as multi-consumers, failed backup subscriptions, and message fanouts. Pulsar automatically tracks the read location of the client and stores this information in a high-performance distributed ledger (BookKeeper).

Unlike Kafka, Pulsar has the functions of traditional message queues, such as RabbitMQ, so you only need to run a Pulsar system to handle both real-time streams and message queues.

two。 Support for partitions, but not required

If you have ever used Kafka, you must know what partitioning is all about. All topics in Kafka are partitioned, which increases throughput. By partitioning into different broker, the processing speed of a single topic can be greatly improved. But what if some topics don't require too high processing rates? In such cases, wouldn't it be better to ignore partitioning and avoid the API and administrative work that comes with it?

Pulsar can do it. If you need only one theme, you can use one theme without using partitions. If you need to maintain the processing rate of multiple consumer instances and do not need to use partitions, Pulsar's shared subscriptions can do so.

If partitioning is really needed to further improve performance, Pulsar can also support the use of partitions.

3. The log is good, but ledger is better.

The Kafka development team foresaw the importance of logs for a real-time data exchange system. The log is written to the system by append, and the writing speed is very fast. The data in the log is serial and can be quickly read in the order in which it is written. Serial reads and writes are faster than random reads and writes. For any system that provides data assurance, persistent storage interaction is a bottleneck, and log abstraction maximizes the efficiency of this aspect.

Logs are good, but when the amount of data is too large, it will also bring us some trouble, and keeping all logs on a single server has become a challenge. What should I do after the log fills up the server storage? How to expand the capacity? Or what should I do when the server that keeps the log goes down and needs to recreate a new server from the copy? Copying logs from one server to another takes a long time, especially if you want to maintain the system's real-time data at the same time.

Pulsar segments the logs, thus avoiding copying large chunks of logs. With BookKeeper, Pulsar divides the logs into segments across multiple different servers. In other words, the log will not be saved on a single server, and any server will not become the bottleneck of the whole system. This makes fault handling and capacity expansion easier, requiring only the addition of new servers without rebalancing.

4. Stateless

For cloud native application developers, the favorite thing is statelessness. Stateless components can be started quickly, can be replaced, and can be expanded seamlessly. Wouldn't it be better if messaging middleware were also stateless?

Kafka is not stateless, each broker contains all the logs of the partition, and if a broker goes down, not all broker can take its place. If the workload is too high, you cannot add a new broker to share at will, but you must synchronize the status with the broker that holds a copy of its partition.

In Pulsar architecture, broker is stateless. But completely stateless systems cannot persist messages, so Pulsar does not rely on broker for message persistence. In Pulsar architecture, the distribution and preservation of data are independent of each other. Broker receives data from producers and sends it to consumers, but the data is stored in BookKeeper.

Pulsar's broker is stateless, so if the workload is high, you can add a new broker directly and quickly take over the workload.

5. Simple cross-domain replication

Cross-domain replication is Pulsar's specialty. Pulsar took this feature into account at the beginning of its design, and it is easy to configure. Whether it is a global distributed application or a disaster recovery solution, it can be done through Pulsar.

6. Steady performance

Benchmark testing (http://openmessaging.cloud/docs/benchmarks/pulsar/) shows that Pulsar can provide higher throughput while maintaining low latency.

7. Completely open source

Pulsar provides many features similar to Kafka, such as cross-domain replication, streaming message processing (Pulsar Functions), connectors (Pulsar IO), SQL-based topic queries (Pulsar SQL), schema registry, and some features that Kafka does not have, such as tiered storage and multi-tenancy. Even better, these features are open source.

The answers to the seven questions about choosing Pulsar instead of Kafka are shared here. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.