Mastering Kafka Streams and ksqlDB

Chapter 3. Stateless Processing

The simplest form of stream processing requires no memory of previously seen events. Each event is consumed, processed,¹ and subsequently forgotten. This paradigm is called stateless processing, and Kafka Streams includes a rich set of operators for working with data in a stateless way.

In this chapter, we will explore the stateless operators that are included in Kafka Streams, and in doing so, we’ll see how some of the most common stream processing tasks can be tackled with ease. The topics we will explore include:

• Filtering records

• Adding and removing fields

• Rekeying records

• Branching streams

• Merging streams

• Transforming records into one or more outputs

• Enriching records, one at a time

We’ll take a tutorial-based approach for introducing these concepts. Specifically, we’ll be streaming data about cryptocurrencies from Twitter and applying some stateless operators to convert the raw data into something more meaningful: investment signals. By the end of this chapter, you will understand how to use stateless operators in Kafka Streams to enrich and transform raw data, which will prepare you for the more advanced concepts that we will explore in later chapters.

Before we jump into the tutorial, let’s get a better frame of reference for what stateless processing is by comparing it to the other form of stream processing: stateful processing.

Stateless Versus Stateful Processing

One of the most important things you should consider when ...