The Kappa Architecture is simpler than the Lambda pattern as it comprises the Speed and Serving Layers only. All the computations occur as stream processing and there are no batch re-computations done on the full Dataset. Recomputations are only done to support changes and new requirements.
Typically, the incoming real-time data stream is processed in memory and is persisted in a database or HDFS to support queries, as illustrated in the following figure:
The Kappa Architecture can be realized by using Apache Spark combined with a queuing solution, such as Apache Kafka. If the data retention times are bound ...
