Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Edge2AI delivered by Cloudera Edge Management(CEM) 

193 visualizaciones

Publicado el

Data collection, processing and intelligence at the edge.

Publicado en: Datos y análisis
  • Sé el primero en comentar

Edge2AI delivered by Cloudera Edge Management(CEM) 

  1. 1. © Cloudera, Inc. All rights reserved. EDGE2AI DELIVERED BY CLOUDERA EDGE MANAGEMENT (CEM) George Vetticaden VP of Product Management, Data In Motion (DIM) BU
  2. 2. © Cloudera, Inc. All rights reserved. 2 STREAMING ANALYTICS REFERENCE ARCHITECTURE IOT Ingest Gateway Powered by Kafka Kafka Topic gateway- freightliner- sensors- Kafka Topic gateway- mack- sensors- Kafka Topic gateway- tata- sensors- Kafka Cluster Geo 2 Data Syndicate Services Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- geo Kafka Topic syndicate freightliner- geo Kafka Cluster Geo 1 Data Syndicate Services Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- geo Kafka Topic syndicate freightliner- geo Data Flow Apps Powered by NiFi Data Collection at the Edge Freightliner Fleet Truck Mack Fleet Truck Tata Fleet Truck C++ Agent C++ Agent C++ Agent Streaming Analytics Apps / Data Lakes Analytics App Micro Services DS / AI Workbench CDSW Data Lake CDH / HDP Analytics App
  3. 3. © Cloudera, Inc. All rights reserved. 3 MANAGING EDGE DATA COLLECTION IS A HARD PROBLEM Kafka Topic gateway- freightliner- sensors- Kafka Topic gateway- mack- sensors- Kafka Topic gateway- tata- sensors- Kafka Cluster Geo 2 Data Syndicate Services Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- geo Kafka Topic syndicate freightliner- geo Kafka Cluster Geo 1 Data Syndicate Services Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- geo Kafka Topic syndicate freightliner- geo Subscribing Streaming Analytics Apps Subcribing Streaming Analytics Apps Analytics App 4 Analytics App 1 Analytics App 2 Micro Services Analytics App 3 Data Flow Apps Powered by NiFi Freightliner Fleet Truck Mack Fleet Truck Tata Fleet Truck C++ Agent C++ Agent C++ Agent Data Collection at the Edge IOT Ingest Gateway Powered by Kafka Challenge #1: Developing Edge Flows How do I develop flows meant to run on the edge using NiFi-like flow based programming with no code?
  4. 4. © Cloudera, Inc. All rights reserved. 4 MANAGING EDGE DATA COLLECTION IS A HARD PROBLEM Kafka Topic gateway- freightliner- sensors- Kafka Topic gateway- mack- sensors- Kafka Topic gateway- tata- sensors- Kafka Cluster Geo 2 Data Syndicate Services Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- geo Kafka Topic syndicate freightliner- geo Kafka Cluster Geo 1 Data Syndicate Services Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- geo Kafka Topic syndicate freightliner- geo Subscribing Streaming Analytics Apps Subcribing Streaming Analytics Apps Analytics App 4 Analytics App 1 Analytics App 2 Micro Services Analytics App 3 Data Flow Apps Powered by NiFi Freightliner Fleet Truck Mack Fleet Truck Tata Fleet Truck C++ Agent C++ Agent C++ Agent Data Collection at the Edge IOT Ingest Gateway Powered by Kafka Challenge #2: Deploying Flows/ Over the Air Updates (OTA) How do I deploy data collection flows to the edge? How do I do updates to only the Freightliner class of trucks?
  5. 5. © Cloudera, Inc. All rights reserved. 5 MANAGING EDGE DATA COLLECTION IS A HARD PROBLEM Kafka Topic gateway- freightliner- sensors- Kafka Topic gateway- mack- sensors- Kafka Topic gateway- tata- sensors- Kafka Cluster Geo 2 Data Syndicate Services Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- geo Kafka Topic syndicate freightliner- geo Kafka Cluster Geo 1 Data Syndicate Services Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- speed Kafka Topic syndicate freightliner- geo Kafka Topic syndicate freightliner- geo Subscribing Streaming Analytics Apps Subcribing Streaming Analytics Apps Analytics App 4 Analytics App 1 Analytics App 2 Micro Services Analytics App 3 Data Flow Apps Powered by NiFi Freightliner Fleet Truck Mack Fleet Truck Tata Fleet Truck C++ Agent C++ Agent C++ Agent Data Collection at the Edge IOT Ingest Gateway Powered by Kafka Challenge #3: Monitoring Agents and Flows How do I manage hundreds of thousands of edge agents and the flows that they are running?
  6. 6. © Cloudera, Inc. All rights reserved. 6 INTRODUCING CLOUDERA EDGE MANAGEMENT (CEM) What is CEM?  New product offering is an edge management solution consisting of edge agents + edge management hub.  What does CEM do? Manage, Control and Monitor edge agents to collect data from edge devices and push intelligence back to the edge.  How does CEM do it? Develop, Deploy, Run & Monitor edge flow apps on thousands of edge devices. N ew
  7. 7. © Cloudera, Inc. All rights reserved. 7 INTRODUCING CLOUDERA EDGE MANAGEMENT (CEM) Key Benefits  Agility - Build edge data collections with the same agility as they build flow apps with NiFi.  Key Enabler for IOT Initiatives - Build successful IoT initiatives by collecting, curating and analyzing data from thousands of edge devices.  Operational Confidence - Gain complete operational confidence with your edge implementation by gaining visibility into all of your deployed agents. N ew
  8. 8. © Cloudera, Inc. All rights reserved. 8 The EDGE: Truck Fleet COMMON EDGE DATAFLOW REQUIREMENTS Use Case Setup #1: Company X has 1000s of vehicles in its fleet with a sensors streaming various telemetric data #2: The Fleet consists of 4 different truck manufacturers each with its own slightly different data and flow requirements.
  9. 9. © Cloudera, Inc. All rights reserved. 9 COMMON EDGE DATAFLOW REQUIREMENTS Edge Flow Requirements 1. Edge collection agents need to be lightweight, resource efficient (size, memory, processing) and be able to run in diverse environments like a gateway on truck. 3. Each truck manufacturer has different sensors and data ingest and processing requirements 2. Each agent should be able perform local collection, filter/routing, & transformation functions, and send data to the Cloud for further more complex processing/analytics 4. Development and deployment flows to 1000s of agents grouped my manufacturer must be made easy. 5. The edge data processing needs to support intermittent network connectivity. The EDGE: Truck Fleet
  10. 10. © Cloudera, Inc. All rights reserved. 10 The EDGE: Truck Fleet Agent Class: Mack Agent Class: Freightliner Agent Class: Benz Agent Class: Tata IMPLEMENTING REQUIREMENTS WITH CEM Steps 1. Install MiNifi C++ agent on each truck 2. On each agent configure a class that represent the Truck manufacturer type. This allows users to to push out different flows to the edge based on the truck type. 4. Develop the edge flow using NiFi like flow based development paradigm 5. Publish/Update the Flow. This will push out the flow to every agent that is configured with the flow’s class. 6. Monitor the agents and the flows running on each truck 3. In Edge Flow Manager (EFM), select an class of Agents you are going to develop an edge flow N ew
  11. 11. © Cloudera, Inc. All rights reserved. 11 OTA UPDATE TO ALL THE FREIGHTLINER TRUCKS MADE EASY BY CEM Update the Edge Flow with a new/improved predictive model Capture/Ingest New Speed Sensor. Filter events whose speed > 50
  12. 12. © Cloudera, Inc. All rights reserved. DEMO CEM DEVELOP, DEPLOY & MANAGE AN EDGE “SMART” FLOW FOR THE FREIGHTLINER CLASS OF MINIFI AGENTS
  13. 13. © Cloudera, Inc. All rights reserved. THANK YOU

×