Book description
Simplify real-time data processing by leveraging the power of Apache Kafka 1.0
About This Book
- Use Kafka 1.0 features such as Confluent platforms and Kafka streams to build efficient streaming data applications to handle and process your data
- Integrate Kafka with other Big Data tools such as Apache Hadoop, Apache Spark, and more
- Hands-on recipes to help you design, operate, maintain, and secure your Apache Kafka cluster with ease
Who This Book Is For
This book is for developers and Kafka administrators who are looking for quick, practical solutions to problems encountered while operating, managing or monitoring Apache Kafka. If you are a developer, some knowledge of Scala or Java will help, while for administrators, some working knowledge of Kafka will be useful.
What You Will Learn
- Install and configure Apache Kafka 1.0 to get optimal performance
- Create and configure Kafka Producers and Consumers
- Operate your Kafka clusters efficiently by implementing the mirroring technique
- Work with the new Confluent platform and Kafka streams, and achieve high availability with Kafka
- Monitor Kafka using tools such as Graphite and Ganglia
- Integrate Kafka with third-party tools such as Elasticsearch, Logstash, Apache Hadoop, Apache Spark, and more
In Detail
Apache Kafka provides a unified, high-throughput, low-latency platform to handle real-time data feeds. This book will show you how to use Kafka efficiently, and contains practical solutions to the common problems that developers and administrators usually face while working with it.
This practical guide contains easy-to-follow recipes to help you set up, configure, and use Apache Kafka in the best possible manner. You will use Apache Kafka Consumers and Producers to build effective real-time streaming applications. The book covers the recently released Kafka version 1.0, the Confluent Platform and Kafka Streams. The programming aspect covered in the book will teach you how to perform important tasks such as message validation, enrichment and composition.Recipes focusing on optimizing the performance of your Kafka cluster, and integrate Kafka with a variety of third-party tools such as Apache Hadoop, Apache Spark, and Elasticsearch will help ease your day to day collaboration with Kafka greatly. Finally, we cover tasks related to monitoring and securing your Apache Kafka cluster using tools such as Ganglia and Graphite.
If you're looking to become the go-to person in your organization when it comes to working with Apache Kafka, this book is the only resource you need to have.
Style and approach
Following a cookbook recipe-based approach, we'll teach you how to solve everyday difficulties and struggles you encounter using Kafka through hands-on examples.
Table of contents
- Preface
-
Configuring Kafka
- Introduction
- Installing Kafka
- Running Kafka
- Configuring Kafka brokers
- Configuring Kafka topics
- Creating a message console producer
- Creating a message console consumer
- Configuring the broker settings
- Configuring threads and performance
- Configuring the log settings
- Configuring the replica settings
- Configuring the ZooKeeper settings
- Configuring other miscellaneous parameters
- Kafka Clusters
- Message Validation
- Message Enrichment
- The Confluent Platform
- Kafka Streams
- Managing Kafka
- Operating Kafka
- Monitoring and Security
-
Third-Party Tool Integration
- Introduction
- Moving data between Kafka nodes with Flume
- Writing to an HDFS cluster with Gobblin
- Moving data from Kafka to Elastic with Logstash
- Connecting Spark streams and Kafka
- Ingesting data from Kafka to Storm
- Pushing data from Kafka to Elastic
- Inserting data from Kafka to SolrCloud
- Building a Kafka producer with Akka
- Building a Kafka consumer with Akka
- Storing data in Cassandra
- Running Kafka on Mesos
- Reading Kafka with Apache Beam
- Writing to Kafka from Apache Beam
Product information
- Title: Apache Kafka 1.0 Cookbook
- Author(s):
- Release date: December 2017
- Publisher(s): Packt Publishing
- ISBN: 9781787286849
You might also like
book
Apache Kafka Quick Start Guide
Process large volumes of data in real-time while building high performance and robust data stream processing …
video
Introduction to Apache Kafka
Currently one of the hottest projects across the Hadoop ecosystem, Apache Kafka is a distributed, real-time …
video
Apache Kafka A-Z with Hands-On Learning
Do you want to learn Apache Kafka to store and process multiple, nonstop streams of information …
video
Apache Kafka Series - Kafka Streams for Data Processing
The new volume in the Apache Kafka Series! Learn the Kafka Streams data-processing library, for Apache …