Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

Kafka at the Edge: an IoT scenario with OpenShift Streams for Apache Kafka | DevNation Tech Talk

Kafka at the Edge: an IoT scenario with OpenShift Streams for Apache Kafka | DevNation Tech Talk

Descargar para leer sin conexión

Apache Kafka is taking the world by storm and is rapidly becoming the de-facto event bus for event-driven and streaming applications that respond to events and data in real time. OpenShift Streams for Apache Kafka is Red Hat's fully hosted and managed Apache Kafka service targeting development teams that want to incorporate streaming data and scalable messaging in their applications, without the burden of setting up and maintaining a Kafka cluster infrastructure.

In this session you will discover how Apache Kafka can be used in an IoT scenario to ingest data from devices and make them available in real-time to other applications.

More specifically you will learn how to:

Simulate devices that send MQTT messages to a MQTT broker
Use Apache Camel and Camel-K to bridge MQTT with Apache Kafka
Use Kafka Streams in a Quarkus application to process the device messages
Query the state of the devices using GraphQ

Apache Kafka is taking the world by storm and is rapidly becoming the de-facto event bus for event-driven and streaming applications that respond to events and data in real time. OpenShift Streams for Apache Kafka is Red Hat's fully hosted and managed Apache Kafka service targeting development teams that want to incorporate streaming data and scalable messaging in their applications, without the burden of setting up and maintaining a Kafka cluster infrastructure.

In this session you will discover how Apache Kafka can be used in an IoT scenario to ingest data from devices and make them available in real-time to other applications.

More specifically you will learn how to:

Simulate devices that send MQTT messages to a MQTT broker
Use Apache Camel and Camel-K to bridge MQTT with Apache Kafka
Use Kafka Streams in a Quarkus application to process the device messages
Query the state of the devices using GraphQ

Más Contenido Relacionado

Presentaciones para usted

Más de Red Hat Developers

Audiolibros relacionados

Gratis con una prueba de 30 días de Scribd

Ver todo

Kafka at the Edge: an IoT scenario with OpenShift Streams for Apache Kafka | DevNation Tech Talk

  1. 1. Kafka at the Edge: An IoT scenario with OpenShift Streams for Apache Kafka Bernard Tison Red Hat 1
  2. 2. Agenda 2 Agenda ▸ What is Apache Kafka, and how are we using it? ▸ IoT use case ▸ Demo ▸ Questions
  3. 3. 3 Apache Kafka is a distributed system designed for streams. It is built to be an high-availability, horizontally-scalable, fault-tolerant, commit log, and allows distributed data streams and stream processing applications. Known use cases are: ▸ High-performance data pipelines ▸ Streaming analytics ▸ Data integration ▸ Event bus in Event Driven Architecture What is Kafka? What is Apache Kafka? Open-source distributed event streaming platform Learn more on the Apache Kafka community page: https://kafka.apache.org/
  4. 4. Source: https://kafka.apache.org/ Brokers, topics, partitions, producers, and consumers! What is Kafka? 4 What is Apache Kafka? Kafka Cluster Broker A Broker B Broker C Consumer Group Topic: Clicks Topic: Purchases P1 P2 P3 P1 P2 P3 Producers
  5. 5. Source: https://kafka.apache.org/intro What is Kafka? 5 APIs What is Apache Kafka? ▸ The Admin API to manage and inspect topics, brokers, and other Kafka objects. ▸ The Producer API to publish (write) a stream of events to one or more Kafka topics. ▸ The Consumer API to subscribe to (read) one or more topics and to process the stream of events produced to them. ▸ The Kafka Streams API to implement stream processing applications and microservices. ▸ The Kafka Connect API to build and run reusable data import/export connectors that consume (read) or produce (write) streams of events from and to external systems and applications so they can integrate with Kafka.
  6. 6. High-throughput, high-availability, and decoupled data pipeline. What is Kafka? 6 Hub and Spoke Architecture 10s of thousands of events Producers/Data Sources Consumers/Data Sinks
  7. 7. OpenShift Streams for Apache Kafka 7 https://red.ht/TryKafka Development Preview trial available for free at:
  8. 8. Red Hat OpenShift Streams for Apache Kafka Complete solution for stream-based applications OpenShift Streams for Apache Kafka 8 Metrics & monitoring Configuration mgmt Hosted & managed (99.95% SLA) Kafka cluster Broker(s) Topic(s) Streamlined developer experience: a curated solution with a developer-first, consistent experience Delivered as a service, managed by Red Hat SRE - 24x7 global support and a 99.95% service-level agreement (SLA) Real-time, streaming data broker - Dedicated Apache Kafka cluster deliver as a service in the cloud and location of choice ▸ Access to Kafka brokers, topics, and partitions ▸ Configuration management ▸ Metrics and monitoring ▸ UI / CLI / API / service bindings ▸ Integrated identity & access management STREAMLINED DEVELOPER EXPERIENCE UI API CLI Service Binding https://red.ht/TryKafka
  9. 9. 9 ▸ Based on the Strimzi community project that provides an optimized version of Apache Kafka on Kubernetes ▸ Access to customer dedicated Kafka instances. Each instance provides Kafka core capabilities ▸ Kafka instances are configured based on opinionated recommendations made by Red Hat experts OpenShift Streams for Apache Kafka A dedicated, fully managed, Apache Kafka cluster An opinionated experience for your Kafka service Apache Kafka implementation optimized for OpenShift https://red.ht/TryKafka
  10. 10. 10 There is More ▸ AMQ Broker: MQTT broker ▸ Apache Camel: bridge MQTT to Kafka ▸ Debezium: captures row-level changes to database tables and passes corresponding change events to Kafka ▸ GraphQL: query language and server-side runtime for APIs that prioritizes giving clients exactly the data they request and no more.
  11. 11. 11
  12. 12. Demo 12 Source: https://crimson-ceremony.net
  13. 13. 13 Product Overview Try it out! https://red.ht/TryKafka
  14. 14. 14 Resources ▸ GitHub repo: https://github.com/rh-bu-cs-rhosak-iot (WIP) ▸ Try Kafka: https://red.ht/TryKafka ▸ Streams for Apache Kafka YouTube playlist: https://www.youtube.com/playlist?list=PLf3vm0UK6HKqZ3Vi7h1Ynfbi0TpdXUr25 ▸ Streams for Apache Kafka getting started blog post: https://developers.redhat.com/articles/2021/07/07/getting-started-red-hat-openshift-str eams-apache-kafka
  15. 15. linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat 15 Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you

×