Mastering Kafka Streams and ksqlDB

Chapter 5. Windows and Time

Time is such an important concept that we measure our lives by the passing of it. Each year, half a dozen people stand around me, singing happy birthday, and as the last flat note dissipates from the air, a cake is offered at the feet of this mysterious force. I like to think the cake is for me, but it’s for time.

Not only is time so intricately woven into the physical world, but it also permeates our event streams. In order to unlock the full power of Kafka Streams, we must understand the relationship between events and time. This chapter explores this relationship in detail, and will give us hands-on experience with something called windows. Windows allow us to group events into explicit time buckets, and can be used for creating more advanced joins and aggregations (which we first explored in the previous chapter).

By the end of this chapter, you will understand the following:

• The differences between event time, ingestion time, and processing time

• How to build a custom timestamp extractor for associating events with a particular timestamp and time semantic

• How time controls the flow of data through Kafka Streams

• What types of windows are supported in Kafka Streams

• How to perform windowed joins

• How to perform windowed aggregations

• What strategies are available for dealing with late and out-of-order events

• How to use the suppress operator to process the final results of windows

• How to query windowed key-value stores

As with previous ...