Mastering Apache Pulsar

Chapter 5. Consumers

So far, we’ve talked about consuming messages in Pulsar, but we haven’t discussed the specifics of how messages can be consumed in Pulsar. It’s worth taking a moment to specify what a consumer is in Pulsar.

A Pulsar consumer is any process that subscribes to a Pulsar topic. Consumers are typically native consumers (Java, Python, or Go processes that subscribe to topics), Pulsar Functions, Pulsar IO, or subscribers via HTTP proxy. How message brokers manage consumers has a material impact on the types of applications built on top of the messaging system. As such, Pulsar’s philosophy about consuming is to be as flexible as the system can warrant. Pulsar achieves this flexibility through a novel subscription model, acknowledgment schemes, configurable consumption modes, and a tunable way to manage message age. These design decisions play a crucial role in enabling the rich applications that run on Pulsar today.

From the perspective of a messaging system, you may be asking why we aren’t starting with producers before consumers. After all, we do need to get data into the system before being able to consume it. There are some good reasons to introduce consumers first, but the main reason is because there are fewer concepts to cover. Producers introduce concepts such as partitioned topics, and we’ll save that until we have a better idea of some of the more rudimentary client configurations you’ll find for a consumer. In this chapter you can expect a good deal more ...