Название: Kafka Connect: Build Data Pipelines by Integrating Existing Systems (Sixth Early Release)
Автор: Mickael Maison, Kate Stanley
Издательство: O’Reilly Media, Inc.
Год: 2022-10-06
Страниц: 260
Язык: английский
Формат: epub (true), mobi
Размер: 10.08 MB

Used by more than 80% of Fortune 100 companies, Apache Kafka has become the de facto event streaming platform. Kafka Connect is a key component of Kafka that lets you flow data between your existing systems and Kafka to process data in real time.

With this practical guide, authors Mickael Maison and Kate Stanley show data engineers, site reliability engineers, and application developers how to build data pipelines between Kafka clusters and a variety of data sources and sinks. Connect allows you to quickly adopt Kafka by tapping into existing data and enabling many advanced use cases. No matter where you are in your event streaming journey, Kafka Connect is the ideal tool for building a modern data pipeline.

Connect is one of the components of the Apache Kafka project. While you don’t need to be a Kafka expert to use Connect, it’s useful to have a basic understanding of the main concepts in order to build reliable data pipelines. In the Chapter 1 we will give a quick overview of Kafka and you will learn the basics in order to fully understand the rest of this book. We will explain what Kafka is, its use cases and briefly introduce some of its inner workings. Finally we will discuss the different Kafka clients, including Kafka Streams, and show you how to run them against a local Kafka cluster.

Learn Connect's capabilities, main concepts, and terminology
Design data and event streaming pipelines that use Connect
Configure and operate Connect environments at scale
Deploy secured and highly available Connect clusters
Build sink and source connectors and single message transforms and converters