Introduction to Apache Flink

Chapter 5. Stateful Computation

Streaming computation can be either stateless or stateful. A stateless program looks at each individual event and creates some output based on that last event. For example, a streaming program might receive temperature readings from a sensor and raise an alert if the temperature goes beyond 90 degrees. A stateful program creates output based on multiple events taken together. Examples of stateful programs include:

• All types of windows that we discussed in Chapter 4. For example, getting the average temperature reported by a sensor over the last hour is a stateful computation.

• All kinds of state machines used for complex event processing (CEP). For example, creating an alert after receiving 2 temperature readings that differ by more than 20 degrees within 1 minute is a stateful computation.

• All kinds of joins between streams as well as joins between streams, and static or slowly changing tables.

Figure 5-1 exemplifies the main difference between stateless and stateful stream processing. A stateless program (a transformation of black records to white records in the figure) receives each record separately (black input) and produces each output record based on the last input record alone (white records). A stateful program maintains state that is updated based on every input and produces output (gray records) based on the last input and the current value of the state.

Figure 5-1. Stateless and stateful processing are compared here. Input records ...