Apache Kafka Series - Kafka Streams for Data Processing

Apache Kafka Series - Kafka Streams for Data Processing

by Stéphane Maarek
Released May 2018
Publisher(s): Packt Publishing
ISBN: 9781789343496

Watch it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Video description

The new volume in the Apache Kafka Series! Learn the Kafka Streams data-processing library, for Apache Kafka. Join hundreds of knowledge savvy students in learning one of the most promising data-processing libraries on Apache Kafka. This course is based on Java 8, and will include one example in Scala. Kafka Streams is Java-based and therefore is not suited for any other programming language. This course is the first and only available Kafka Streams course on the web. Get it now to become a Kafka expert!

What You Will Learn

  • Write four Kafka Streams application in Java 8.
  • Configure Kafka Streams to use exactly once semantics.
  • Scale Kafka Streams applications.
  • Program with the high-level DSL of Kafka Streams.
  • Build and package your application.
  • Write tests for your Kafka Streams Topology and so much more!

Audience

Developers and devops who would like to learn how to write, package, deploy and run a Kafka Streams applications. Architects who would like to understand how Kafka Streams works and its position in the Kafka-centered data pipeline and enterprise architecture. People who fully understand the basics of Kafka. Java 8 and Scala developers only

About The Author

Stéphane Maarek: Stéphane Maarek is a solutions architect, consultant, and software developer who has a particular interest in all things related to big data and analytics. He is also a bestseller instructor on Udemy for his courses on Apache Kafka, Apache NiFi, and AWS Lambda. He loves Apache Kafka and regularly contributes to the Apache Kafka project.

Stéphane has also written a guest blog post that was featured on the Confluent website, the company behind Apache Kafka. He is also an AWS Certified Solutions Architect and has many years of experience with technologies such as Apache Kafka, Apache NiFi, Apache Spark, Hadoop, PostgreSQL, Tableau, Spotfire, Docker, Ansible, and more.

Table of contents

  1. Chapter 1 : Kafka Streams - First Look
    1. What is Kafka Streams?
    2. Course Objective / Prerequisites / Target Students
    3. About your Instructor: Stephane Maarek
    4. Running your first Kafka Streams Application: WordCount
    5. Kafka Streams vs other stream processing libraries (Spark Streaming, NiFi, Flink
    6. Code Download for this Class
  2. Chapter 2 : End to End Kafka Streams Application - Word Count
    1. Section Objective
    2. Kafka Streams Core Concepts
    3. Environment and IDE Setup: Java 8, Maven, IntelliJ IDEA
    4. Starter Project Setup
    5. Kafka Streams Application Properties
    6. Java 8 Lambda Functions - quick overview
    7. Word Count Application Topology
    8. Printing the Kafka Streams Topology
    9. Kafka Streams Graceful Shutdown
    10. Running Application from IntelliJ IDEA
    11. Debugging Application from IntelliJ IDEA
    12. Internal Topics for our Kafka Streams Application
    13. Packaging the application as Fat Jar Running the Fat Jar
    14. Scaling our Application
    15. Section Wrap-Up
  3. Chapter 3 : KStreams and KTables Simple Operations (Stateless)
    1. Section Objectives
    2. KStream KTables
    3. Stateless vs Stateful Operations
    4. MapValues / Map
    5. Filter / FilterNot
    6. FlatMapValues / FlatMap
    7. Branch
    8. SelectKey
    9. Reading from Kafka
    10. Writing to Kafka
    11. Streams Marked for Re-Partition
    12. Refresher on Log Compaction
    13. KStream and KTables Duality
    14. Transforming a KTable to a KStream
    15. Transforming a KStream to a KTable
    16. Section Summary
  4. Chapter 4 : Practice Exercise – FavouriteColour
    1. FavouriteColour - Practice Exercise Description Guidance
    2. Stuck? Here are some Hints!
    3. Java Solution
    4. Running the application
    5. Scala Solution
  5. Chapter 5 : KStreams and KTables Advanced Operations (Stateful)
    1. Section Objective
    2. KTable groupBy
    3. KGroupedStream / KGroupedTable Count
    4. KGroupedStream / KGroupedTable Aggregate
    5. KGroupedStream / KGroupedTable Reduce
    6. KStream peek
    7. KStream Transform / TransformValues
    8. What if I want to write to an external System?
    9. Summary Diagram
  6. Chapter 6 : Exactly Once Semantics – Theory
    1. What's Exactly Once?
    2. Exactly Once in Kafka 0.11
    3. What's the problem with at least once anyway?
    4. How to do exactly once in Kafka Streams
  7. Chapter 7 : Exactly Once Semantics - Practice Exercise – BankBalance
    1. BankBalance - Exercise Overview
    2. Kafka Producer Guidance
    3. Kafka Producer Solution
    4. Kafka Streams Guidance Hints
    5. Kafka Streams Solution
    6. Running the BankBalance Application
    7. Section Summary
  8. Chapter 8 : Joins - KStream to GlobalKTable example
    1. What are joins in Kafka Streams?
    2. Join Constraints and GlobalKTables
    3. The different types of joins: Inner Join, Left Join, Outer Join
    4. Creating a join with UserEnrich Kafka Streams App
    5. Running the Kafka Streams Join application
  9. Chapter 9 : Testing your Kafka Streams Application
    1. Testing in Kafka Streams
    2. Setup your Kafka Streams project
    3. Hands-On: Test your WordCount application
  10. Chapter 10 : Next Steps
    1. Congratulations and next steps

Product information

  • Title: Apache Kafka Series - Kafka Streams for Data Processing
  • Author(s): Stéphane Maarek
  • Release date: May 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781789343496