Data Streaming with Event Hubs
for Kafka
Dinusha Kumarasiri

About Me
 Microsoft MVP for Microsoft Azure
 Microsoft Certified Trainer (MCT)
 Senior Consultant at Dialog Information Technology
 Cloud Enthusiast
 Love to share what I learn

Agenda
 Event Driven Architecture
 Event Streaming
 Event Streaming with Apache Kafka
 Event Streaming with Azure Event Hubs
 Event Hubs for Kafka
 Demo

Event Driven Architecture
 Traditional API Driven
Problems
 Dependency among APIs
 Difficult to Scale each component
 Lack of Resiliency
Motivation
 Easy to build for simple
operations

Event Driven Architecture
 Event Driven
Motivation
 Components are Loosely Coupled
 Components communicate through Events
 Can Scale individually
 Improved Resiliency

Event Streaming
 Continuous Flow of information
 Events are meaningful when it composes an event Stream
 Event processing can combine multiple event streams
Producer 1
Producer 2
Producer 3
Event Stream
Platform
Consumer 1
Consumer 2
Consumer 3
Fraud Detection
Predictive
Maintenance
Real-Time
Monitoring
Supply Chain
Optimization
Customer
Experience

Event Streaming with Apache Kafka
 Open Source
 Can build Kafka cluster with multiple nodes (Brokers)
 Hold a stream of events in Topics, which can be built with multiple Partitions
 Fully managed Kafka cluster can be built with Apache Kafka for Confluent Cloud

Event Streaming with Azure Event Hub
 Cloud Native offering for Event Streaming in Microsoft Azure
 Designed for High-volume event Ingestion and Processing
 Scalable and Reliable platform
 Data automatically partitioned and distributed across multiple instances

Azure Event Hubs for Apache Kafka
 Fully managed Kafka-Compatible service provided by Microsoft Azure
 Allows Kafka APIs and Tools to produce & consume data with Event Hubs
 Enables the migration of Kafka applications to Azure

Demo https://github.com/Azure/azure-event-hubs-for-kafka

Event Streaming – Big Picture
Event Producers Event Ingestion Stream Processing Insights

dinushaonline.blogspot.com @kumarasiri048 dinushak Dinusha Kumarasiri