Stream processing

The stream processing component itself consists of three main sub-components, which are:

  • The Broker: that collects and holds the events or data streams from the data collection agents
  • The Processing Engine: that actually transforms, correlates, aggregates the data, and performs other necessary operations
  • The Distributed Cache: that actually serves as a mechanism for maintaining common datasets across all distributed components of the Processing Engine

The same aspects of the stream processing component are zoomed out and depicted in the diagram that follows:

There are a few key attributes that should be catered for by the ...

Get Practical Real-time Data Processing and Analytics now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.