Blockchain and Apache NiFi
•
5 recomendaciones•1,280 vistas
The document discusses Timothy Spann, a Senior Solutions Engineer at Cloudera who has been running Big Data meetups in Princeton since 2015. It provides links to his profiles on various websites and details some of the topics he has spoken about at conferences, including Apache NiFi, Deep Learning, and Streaming. The rest of the document focuses on use cases for data integration and movement using Apache NiFi, as well as concepts like blockchain, distributed data stores, and accessing blockchain and Ethereum data.
Denunciar
Compartir
Denunciar
Compartir
Descargar para leer sin conexión
Recomendados
Hyperledger Architecture Vol 2 > Smart Contracts
Overview of Hyperledger Architecture & Introduction to Smart Contracts
Hyperledger Architecture Working Group
Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose
An introduction to Apache NiFi presented at the Apache NiFi Meetup as part of the 2016 Hadoop Summit held in San Jose.
AppDynamics VS New Relic – The Complete Guide
This document compares the application performance monitoring (APM) tools AppDynamics and New Relic. It discusses their supported languages and environments, key features for backend, frontend, and mobile monitoring, how each tool helps users solve errors, pricing differences, and concludes that AppDynamics is better for on-premise use while New Relic caters more to startups and smaller businesses.
Application Archaeology: Accelerating App Modernization at DICK’S Sporting Goods
SpringOne Platform 2019
Session Title: Application Archaeology: Accelerating App Modernization at DICK’S Sporting Goods
Speakers: Sean Graham, Application Transformation Practice Lead, DICK'S Sporting Goods and Brandon Blincoe, Solutions Architect, Application Transformation, Pivotal
Youtube: https://youtu.be/J66S0qP7DO4
Monitoring Kubernetes with Prometheus
Presented at GDG Devfest Ukraine 2018.
Prometheus has become the defacto monitoring system for cloud native applications, with systems like Kubernetes and Etcd natively exposing Prometheus metrics. In this talk Tom will explore all the moving part for a working Prometheus-on-Kubernetes monitoring system, including kube-state-metrics, node-exporter, cAdvisor and Grafana. You will learn about the various methods for getting to a working setup: the manual approach, using CoreOS’s Prometheus Operator, or using Prometheus Ksonnet Mixin. Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.
Apache NiFi Crash Course Intro
The document provides an introduction and overview of Apache NiFi and its architecture. It discusses how NiFi can be used to effectively manage and move data between different producers and consumers. It also summarizes key NiFi features like guaranteed delivery, data buffering, prioritization, and data provenance. Finally, it briefly outlines the NiFi architecture and components as well as opportunities for the future of the MiniFi project.
Kafka error handling patterns and best practices | Hemant Desale and Aruna Ka...
Transaction Banking from Goldman Sachs is a high volume, latency sensitive digital banking platform offering. We have chosen an event driven architecture to build highly decoupled and independent microservices in a cloud native manner and are designed to meet the objectives of Security, Availability Latency and Scalability. Kafka was a natural choice – to decouple producers and consumers and to scale easily for high volume processing. However, there are certain aspects that require careful consideration – handling errors and partial failures, managing downtime of consumers, secure communication between brokers and producers / consumers. In this session, we will present the patterns and best practices that helped us build robust event driven applications. We will also present our solution approach that has been reused across multiple application domains. We hope that by sharing our experience, we can establish a reference implementation that application developers can benefit from.
Introducing Change Data Capture with Debezium
This document discusses change data capture (CDC) and how it can be used to stream change events from databases. It introduces Debezium, an open source CDC platform that captures change events from transaction logs. Debezium supports capturing changes from multiple databases and transmitting them as a stream of events. The summary discusses how CDC can be used for data replication between databases, auditing, and in microservices architectures. It also covers deployment of CDC on Kubernetes using OpenShift.
Recomendados
Hyperledger Architecture Vol 2 > Smart Contracts
Overview of Hyperledger Architecture & Introduction to Smart Contracts
Hyperledger Architecture Working Group
Dataflow with Apache NiFi - Apache NiFi Meetup - 2016 Hadoop Summit - San Jose
An introduction to Apache NiFi presented at the Apache NiFi Meetup as part of the 2016 Hadoop Summit held in San Jose.
AppDynamics VS New Relic – The Complete Guide
This document compares the application performance monitoring (APM) tools AppDynamics and New Relic. It discusses their supported languages and environments, key features for backend, frontend, and mobile monitoring, how each tool helps users solve errors, pricing differences, and concludes that AppDynamics is better for on-premise use while New Relic caters more to startups and smaller businesses.
Application Archaeology: Accelerating App Modernization at DICK’S Sporting Goods
SpringOne Platform 2019
Session Title: Application Archaeology: Accelerating App Modernization at DICK’S Sporting Goods
Speakers: Sean Graham, Application Transformation Practice Lead, DICK'S Sporting Goods and Brandon Blincoe, Solutions Architect, Application Transformation, Pivotal
Youtube: https://youtu.be/J66S0qP7DO4
Monitoring Kubernetes with Prometheus
Presented at GDG Devfest Ukraine 2018.
Prometheus has become the defacto monitoring system for cloud native applications, with systems like Kubernetes and Etcd natively exposing Prometheus metrics. In this talk Tom will explore all the moving part for a working Prometheus-on-Kubernetes monitoring system, including kube-state-metrics, node-exporter, cAdvisor and Grafana. You will learn about the various methods for getting to a working setup: the manual approach, using CoreOS’s Prometheus Operator, or using Prometheus Ksonnet Mixin. Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.
Apache NiFi Crash Course Intro
The document provides an introduction and overview of Apache NiFi and its architecture. It discusses how NiFi can be used to effectively manage and move data between different producers and consumers. It also summarizes key NiFi features like guaranteed delivery, data buffering, prioritization, and data provenance. Finally, it briefly outlines the NiFi architecture and components as well as opportunities for the future of the MiniFi project.
Kafka error handling patterns and best practices | Hemant Desale and Aruna Ka...
Transaction Banking from Goldman Sachs is a high volume, latency sensitive digital banking platform offering. We have chosen an event driven architecture to build highly decoupled and independent microservices in a cloud native manner and are designed to meet the objectives of Security, Availability Latency and Scalability. Kafka was a natural choice – to decouple producers and consumers and to scale easily for high volume processing. However, there are certain aspects that require careful consideration – handling errors and partial failures, managing downtime of consumers, secure communication between brokers and producers / consumers. In this session, we will present the patterns and best practices that helped us build robust event driven applications. We will also present our solution approach that has been reused across multiple application domains. We hope that by sharing our experience, we can establish a reference implementation that application developers can benefit from.
Introducing Change Data Capture with Debezium
This document discusses change data capture (CDC) and how it can be used to stream change events from databases. It introduces Debezium, an open source CDC platform that captures change events from transaction logs. Debezium supports capturing changes from multiple databases and transmitting them as a stream of events. The summary discusses how CDC can be used for data replication between databases, auditing, and in microservices architectures. It also covers deployment of CDC on Kubernetes using OpenShift.
GDG Cloud Southlake #8 Steve Cravens: Infrastructure as-Code (IaC) in 2022: ...
Infrastructure as Code (IaC) is a concept that has been around for a while now and much research has been done to not only prove out the value but also how to enhance IaC implementations. We have a full guest list including Steve Cravens, who can speak to the school of hard knocks of why IaC is important. Stenio Ferreira, who prior to Google worked at Hashicorp and has vast experience on how to successfully implement IaC with Terraform. Lastly, Josh Addington, who is an Sr. Solutions Engineer at Hashicorp and will be speaking to the Day 2 operations as well as other offerings that can enhance IaC implementations.
Here is the high level overview:
• IaC overview
• Terraform Tactical
• IaC day 2 and Governance
NiFi Developer Guide
This document summarizes a presentation on developing with Apache NiFi. It discusses NiFi's REST API for programmatic access, the NiFi developer guide for building custom processors, and tips for contributing to the NiFi project through the GitHub pull request process. Key aspects of the NiFi architecture like its repositories and FlowFile lifecycle are also overviewed.
IT and OT Convergence
OpsRamp and Mystic River are joining forces to bring you this interactive webinar. Can we create gateways between IT and OT? IT/OT convergence has been defined as the integration of information technology (IT) systems used for data-centric computing with operational technology (OT) systems used to monitor events, processes and devices and make adjustments in enterprise and industrial operations. But what this will look like and what is the new role of IT operations management? What is changing in IT and OT to align these worlds? This Tech Talk features our partner Mystic River Consulting, a firm with proven, repeatable service delivery methodologies and a proprietary platform that drives transformational IT project results, at a game changing speed. We’ll discuss the convergence of IT and OT and dive into a demonstration to show what’s possible with OpsRamp.
Watch the recording: https://www.brighttalk.com/webcast/17791/416457
Learn more at https://www.opsramp.com
Also, follow us on social media channels to learn about product highlights, news, announcements, events, conferences and more:
Twitter - https://www.twitter.com/OpsRamp
LinkedIn - https://www.linkedin.com/company/opsramp
Facebook - https://www.facebook.com/OpsRampHQ/
Intro to Delta Lake
Delta Lake brings reliability, performance, and security to data lakes. It provides ACID transactions, schema enforcement, and unified handling of batch and streaming data to make data lakes more reliable. Delta Lake also features lightning fast query performance through its optimized Delta Engine. It enables security and compliance at scale through access controls and versioning of data. Delta Lake further offers an open approach and avoids vendor lock-in by using open formats like Parquet that can integrate with various ecosystems.
Data and AI summit: data pipelines observability with open lineage
Presentation of Data lineage an Observability with OpenLineage at the "Data and AI summit" (formerly Spark summit). With a focus on the Apache Spark integration for OpenLineage
Creating Connector to Bridge the Worlds of Kafka and gRPC at Wework (Anoop Di...
What do you do when you've two different technologies on the upstream and the downstream that are both rapidly being adopted industrywide? How do you bridge them scalably and robustly? At Wework, the upstream data was being brokered by Kafka and the downstream consumers were highly scalable gRPC services. While Kafka was capable of efficiently channeling incoming events in near real-time from a variety of sensors that were used in select Wework spaces, the downstream gRPC services that were user-facing were exceptionally good at serving requests in a concurrent and robust manner. This was a formidable combination, if only there was a way to effectively bridge these two in an optimized way. Luckily, sink Connectors came to the rescue. However, there weren't any for gRPC sinks! So we wrote one.
In this talk, we will briefly focus on the advantages of using Connectors, creating new Connectors, and specifically spend time on gRPC sink Connector and its impact on Wework's data pipeline.
Stability Patterns for Microservices
Communication between Microservices is inherently unreliable. These integration points may produce cascading failures, slow responses, service outages. We will walk through stability patterns like timeouts, circuit breaker, bulkheads and discuss how they improve stability of Microservices.
Prometheus Multi Tenancy
Natan Yellin discusses options for gathering Prometheus metrics from multiple Kubernetes tenants in a multi-tenant environment. There are three main approaches: solving it outside of Prometheus using other tools; using multiple Prometheus instances with a centralized Prometheus; or using a single Prometheus instance with built-in multi-tenancy. The most mature option currently is to use multiple Prometheus instances with a central Prometheus for long-term storage and unified queries. Tools like Thanos, Cortex, and Mimir provide ways to implement this approach.
Apache Tez - A unifying Framework for Hadoop Data Processing
This document provides an overview of Apache Tez, a framework for building data processing applications on Hadoop YARN. It describes how Tez allows applications to define complex data flows as directed acyclic graphs (DAGs) and handles distributed execution, fault tolerance, and resource management. Tez has improved the performance of Apache Hive and Pig by an order of magnitude by enabling more flexible DAG definitions and runtime optimizations. It also supports integration with other data processing engines like Spark, Storm and interactive SQL queries. The document outlines how Tez works and provides guidance on how developers can contribute to the open source project.
FIWARE Global Summit - Idra: A Solution for Open Data Interoperability
Idra is an open source platform that provides a single access point for open datasets from different public organizations and data portals, which use various formats and platforms. It addresses the fragmentation of open data technologies that limits data access and reuse. Idra imports only metadata from the original sources and federates it through its portal and APIs. This allows users to search across datasets from different portals in a standardized way, and developers to build applications accessing unified open data.
Building Event-Driven (Micro) Services with Apache Kafka
The document discusses building event-driven microservices using Apache Kafka. It describes how microservices can interact asynchronously through events published to an event hub like Apache Kafka. This allows for loose coupling between services and the ability to integrate new services by consuming past events from the event log. The document also discusses how Apache Kafka can be used for change data capture from legacy systems, streaming data integration, and providing a unified platform for real-time event processing and historical analytics.
API Gateway How-To: The Many Ways to Apply the Gateway Pattern
The document discusses different patterns for using an API gateway, including:
1) API management for existing APIs to handle cross-cutting concerns in one place
2) Ingress for APIs and services to manage traffic entering a cluster
3) Application modernization to help split a monolith into microservices gradually
4) Simplified aggregated API to simplify access to multiple complex internal services
5) Using a service mesh for internal east-west traffic between services while using a gateway for north-south external traffic
The document provides examples and use cases for each pattern and concludes that the right approach depends on specific needs, and that gateways and service meshes can overlap on common concerns like security and observability.
Hadoop and Manufacturing
Manufacturers have an abundance of data, whether from connected sensors, plant systems, manufacturing systems, claims systems and external data from industry and government. Manufacturers face increased challenges from continually improving product quality, reducing warranty and recall costs to efficiently leveraging their supply chain. For example, giving the manufacturer a complete view of the product and customer information integrating manufacturing and plant floor data, with as built product configurations with sensor data from customer use to efficiently analyze warranty claim information to reduce detection to correction time, detect fraud and even become proactive around issues requires a capable enterprise data hub that integrates large volumes of both structured and unstructured information. Learn how an enterprise data hub built on Hadoop provides the tools to support analysis at every level in the manufacturing organization.
DW Migration Webinar-March 2022.pptx
The document discusses migrating a data warehouse to the Databricks Lakehouse Platform. It outlines why legacy data warehouses are struggling, how the Databricks Platform addresses these issues, and key considerations for modern analytics and data warehousing. The document then provides an overview of the migration methodology, approach, strategies, and key takeaways for moving to a lakehouse on Databricks.
Ethereum in Enterprise.pptx
Ethereum and other blockchains are finding their way into the enterprise world. We look into common use cases, blockchains, and standard approaches to deploy and access enterprise blockchains
Supercharge Splunk with Cloudera
Cloudera can help optimize Splunk deployments by providing more cost-effective scalability, increased data flexibility, and enhanced analytics capabilities. Cloudera can ingest data from Splunk indexes and apply enrichment using open-source machine learning before storing the data in its data hub. This provides a single platform for advanced analytics like SQL and Python/R scripts across both historical and new data. Initial use cases include offloading event data from Splunk to reduce costs and loading additional context sources to gain better insights.
Cloud Monitoring tool Grafana
Installation of Grafana on linux ; connectivity with Prometheus database , installation of Prometheus ; Installation of node_exporter ,Tomcat-exporter ; installation and configuration of alert manager .. Detailed step by step installation and working
Introduction to Apache NiFi dws19 DWS - DC 2019
A quick introduction to Apache NiFi and it's ecosystem. Also a hands on demo on using processors, examining provenance, ingesting REST Feeds, XML, Cameras, Files, Running TensorFlow, Running Apache MXNet, integrating with Spark and Kafka. Storing to HDFS, HBase, Phoenix, Hive and S3.
Open core summit: Observability for data pipelines with OpenLineage
This document discusses Open Lineage and the Marquez project for collecting metadata and data lineage information from data pipelines. It describes how Open Lineage defines a standard model and protocol for instrumentation to collect metadata on jobs, datasets, and runs in a consistent way. This metadata can then provide context on the data source, schema, owners, usage, and changes. The document outlines how Marquez implements the Open Lineage standard by defining entities, relationships, and facets to store this metadata and enable use cases like data governance, discovery, and debugging. It also positions Marquez as a centralized but modular framework to integrate various data platforms and extensions like Datakin's lineage analysis tools.
Kafka internals
This presentation contains our understanding of Kafka internals, its performance. We also present Nest Logic Active Data Profiling system.
Edge to AI: Analytics from Edge to Cloud with Efficient Movement of Machine ...
Apache NiFi, Apache MXNet, TensorFlow, Apache Spark, Python, Cloudera CDF, Cloudera CDH, Cloudera Data Science Workbench, CDSW, MiNiFi, Cloudera EFM, Apache Zeppelin, YOLO, Computer Vision, REST API, JSON, Integration, Java, Big Data, Phoenix, Hive, HDFS, Parquet, AVRO. Cloudera. Hortonworks. John Kuchmek. Tim Spann.
Data Science in Enterprise
This document discusses enterprise data science and machine learning. It begins by noting that data is now more plentiful and machine learning opportunities are everywhere. However, challenges remain around scaling data science work, making models production-ready, and meeting different team needs. The document then introduces Cloudera's Data Science Workbench for addressing these challenges. It claims the Workbench provides a secure, self-service environment allowing data scientists direct access to enterprise data and tools while meeting IT requirements. Examples are given of how it supports the full data science pipeline from exploration to production. In demos, it highlights features like connecting to Hadoop clusters securely and enabling collaboration. Overall, the document pitches Cloudera's Workbench as a solution
Más contenido relacionado
La actualidad más candente
GDG Cloud Southlake #8 Steve Cravens: Infrastructure as-Code (IaC) in 2022: ...
Infrastructure as Code (IaC) is a concept that has been around for a while now and much research has been done to not only prove out the value but also how to enhance IaC implementations. We have a full guest list including Steve Cravens, who can speak to the school of hard knocks of why IaC is important. Stenio Ferreira, who prior to Google worked at Hashicorp and has vast experience on how to successfully implement IaC with Terraform. Lastly, Josh Addington, who is an Sr. Solutions Engineer at Hashicorp and will be speaking to the Day 2 operations as well as other offerings that can enhance IaC implementations.
Here is the high level overview:
• IaC overview
• Terraform Tactical
• IaC day 2 and Governance
NiFi Developer Guide
This document summarizes a presentation on developing with Apache NiFi. It discusses NiFi's REST API for programmatic access, the NiFi developer guide for building custom processors, and tips for contributing to the NiFi project through the GitHub pull request process. Key aspects of the NiFi architecture like its repositories and FlowFile lifecycle are also overviewed.
IT and OT Convergence
OpsRamp and Mystic River are joining forces to bring you this interactive webinar. Can we create gateways between IT and OT? IT/OT convergence has been defined as the integration of information technology (IT) systems used for data-centric computing with operational technology (OT) systems used to monitor events, processes and devices and make adjustments in enterprise and industrial operations. But what this will look like and what is the new role of IT operations management? What is changing in IT and OT to align these worlds? This Tech Talk features our partner Mystic River Consulting, a firm with proven, repeatable service delivery methodologies and a proprietary platform that drives transformational IT project results, at a game changing speed. We’ll discuss the convergence of IT and OT and dive into a demonstration to show what’s possible with OpsRamp.
Watch the recording: https://www.brighttalk.com/webcast/17791/416457
Learn more at https://www.opsramp.com
Also, follow us on social media channels to learn about product highlights, news, announcements, events, conferences and more:
Twitter - https://www.twitter.com/OpsRamp
LinkedIn - https://www.linkedin.com/company/opsramp
Facebook - https://www.facebook.com/OpsRampHQ/
Intro to Delta Lake
Delta Lake brings reliability, performance, and security to data lakes. It provides ACID transactions, schema enforcement, and unified handling of batch and streaming data to make data lakes more reliable. Delta Lake also features lightning fast query performance through its optimized Delta Engine. It enables security and compliance at scale through access controls and versioning of data. Delta Lake further offers an open approach and avoids vendor lock-in by using open formats like Parquet that can integrate with various ecosystems.
Data and AI summit: data pipelines observability with open lineage
Presentation of Data lineage an Observability with OpenLineage at the "Data and AI summit" (formerly Spark summit). With a focus on the Apache Spark integration for OpenLineage
Creating Connector to Bridge the Worlds of Kafka and gRPC at Wework (Anoop Di...
What do you do when you've two different technologies on the upstream and the downstream that are both rapidly being adopted industrywide? How do you bridge them scalably and robustly? At Wework, the upstream data was being brokered by Kafka and the downstream consumers were highly scalable gRPC services. While Kafka was capable of efficiently channeling incoming events in near real-time from a variety of sensors that were used in select Wework spaces, the downstream gRPC services that were user-facing were exceptionally good at serving requests in a concurrent and robust manner. This was a formidable combination, if only there was a way to effectively bridge these two in an optimized way. Luckily, sink Connectors came to the rescue. However, there weren't any for gRPC sinks! So we wrote one.
In this talk, we will briefly focus on the advantages of using Connectors, creating new Connectors, and specifically spend time on gRPC sink Connector and its impact on Wework's data pipeline.
Stability Patterns for Microservices
Communication between Microservices is inherently unreliable. These integration points may produce cascading failures, slow responses, service outages. We will walk through stability patterns like timeouts, circuit breaker, bulkheads and discuss how they improve stability of Microservices.
Prometheus Multi Tenancy
Natan Yellin discusses options for gathering Prometheus metrics from multiple Kubernetes tenants in a multi-tenant environment. There are three main approaches: solving it outside of Prometheus using other tools; using multiple Prometheus instances with a centralized Prometheus; or using a single Prometheus instance with built-in multi-tenancy. The most mature option currently is to use multiple Prometheus instances with a central Prometheus for long-term storage and unified queries. Tools like Thanos, Cortex, and Mimir provide ways to implement this approach.
Apache Tez - A unifying Framework for Hadoop Data Processing
This document provides an overview of Apache Tez, a framework for building data processing applications on Hadoop YARN. It describes how Tez allows applications to define complex data flows as directed acyclic graphs (DAGs) and handles distributed execution, fault tolerance, and resource management. Tez has improved the performance of Apache Hive and Pig by an order of magnitude by enabling more flexible DAG definitions and runtime optimizations. It also supports integration with other data processing engines like Spark, Storm and interactive SQL queries. The document outlines how Tez works and provides guidance on how developers can contribute to the open source project.
FIWARE Global Summit - Idra: A Solution for Open Data Interoperability
Idra is an open source platform that provides a single access point for open datasets from different public organizations and data portals, which use various formats and platforms. It addresses the fragmentation of open data technologies that limits data access and reuse. Idra imports only metadata from the original sources and federates it through its portal and APIs. This allows users to search across datasets from different portals in a standardized way, and developers to build applications accessing unified open data.
Building Event-Driven (Micro) Services with Apache Kafka
The document discusses building event-driven microservices using Apache Kafka. It describes how microservices can interact asynchronously through events published to an event hub like Apache Kafka. This allows for loose coupling between services and the ability to integrate new services by consuming past events from the event log. The document also discusses how Apache Kafka can be used for change data capture from legacy systems, streaming data integration, and providing a unified platform for real-time event processing and historical analytics.
API Gateway How-To: The Many Ways to Apply the Gateway Pattern
The document discusses different patterns for using an API gateway, including:
1) API management for existing APIs to handle cross-cutting concerns in one place
2) Ingress for APIs and services to manage traffic entering a cluster
3) Application modernization to help split a monolith into microservices gradually
4) Simplified aggregated API to simplify access to multiple complex internal services
5) Using a service mesh for internal east-west traffic between services while using a gateway for north-south external traffic
The document provides examples and use cases for each pattern and concludes that the right approach depends on specific needs, and that gateways and service meshes can overlap on common concerns like security and observability.
Hadoop and Manufacturing
Manufacturers have an abundance of data, whether from connected sensors, plant systems, manufacturing systems, claims systems and external data from industry and government. Manufacturers face increased challenges from continually improving product quality, reducing warranty and recall costs to efficiently leveraging their supply chain. For example, giving the manufacturer a complete view of the product and customer information integrating manufacturing and plant floor data, with as built product configurations with sensor data from customer use to efficiently analyze warranty claim information to reduce detection to correction time, detect fraud and even become proactive around issues requires a capable enterprise data hub that integrates large volumes of both structured and unstructured information. Learn how an enterprise data hub built on Hadoop provides the tools to support analysis at every level in the manufacturing organization.
DW Migration Webinar-March 2022.pptx
The document discusses migrating a data warehouse to the Databricks Lakehouse Platform. It outlines why legacy data warehouses are struggling, how the Databricks Platform addresses these issues, and key considerations for modern analytics and data warehousing. The document then provides an overview of the migration methodology, approach, strategies, and key takeaways for moving to a lakehouse on Databricks.
Ethereum in Enterprise.pptx
Ethereum and other blockchains are finding their way into the enterprise world. We look into common use cases, blockchains, and standard approaches to deploy and access enterprise blockchains
Supercharge Splunk with Cloudera
Cloudera can help optimize Splunk deployments by providing more cost-effective scalability, increased data flexibility, and enhanced analytics capabilities. Cloudera can ingest data from Splunk indexes and apply enrichment using open-source machine learning before storing the data in its data hub. This provides a single platform for advanced analytics like SQL and Python/R scripts across both historical and new data. Initial use cases include offloading event data from Splunk to reduce costs and loading additional context sources to gain better insights.
Cloud Monitoring tool Grafana
Installation of Grafana on linux ; connectivity with Prometheus database , installation of Prometheus ; Installation of node_exporter ,Tomcat-exporter ; installation and configuration of alert manager .. Detailed step by step installation and working
Introduction to Apache NiFi dws19 DWS - DC 2019
A quick introduction to Apache NiFi and it's ecosystem. Also a hands on demo on using processors, examining provenance, ingesting REST Feeds, XML, Cameras, Files, Running TensorFlow, Running Apache MXNet, integrating with Spark and Kafka. Storing to HDFS, HBase, Phoenix, Hive and S3.
Open core summit: Observability for data pipelines with OpenLineage
This document discusses Open Lineage and the Marquez project for collecting metadata and data lineage information from data pipelines. It describes how Open Lineage defines a standard model and protocol for instrumentation to collect metadata on jobs, datasets, and runs in a consistent way. This metadata can then provide context on the data source, schema, owners, usage, and changes. The document outlines how Marquez implements the Open Lineage standard by defining entities, relationships, and facets to store this metadata and enable use cases like data governance, discovery, and debugging. It also positions Marquez as a centralized but modular framework to integrate various data platforms and extensions like Datakin's lineage analysis tools.
Kafka internals
This presentation contains our understanding of Kafka internals, its performance. We also present Nest Logic Active Data Profiling system.
La actualidad más candente (20)
GDG Cloud Southlake #8 Steve Cravens: Infrastructure as-Code (IaC) in 2022: ...
GDG Cloud Southlake #8 Steve Cravens: Infrastructure as-Code (IaC) in 2022: ...
Data and AI summit: data pipelines observability with open lineage
Data and AI summit: data pipelines observability with open lineage
Creating Connector to Bridge the Worlds of Kafka and gRPC at Wework (Anoop Di...
Creating Connector to Bridge the Worlds of Kafka and gRPC at Wework (Anoop Di...
Apache Tez - A unifying Framework for Hadoop Data Processing
Apache Tez - A unifying Framework for Hadoop Data Processing
FIWARE Global Summit - Idra: A Solution for Open Data Interoperability
FIWARE Global Summit - Idra: A Solution for Open Data Interoperability
Building Event-Driven (Micro) Services with Apache Kafka
Building Event-Driven (Micro) Services with Apache Kafka
API Gateway How-To: The Many Ways to Apply the Gateway Pattern
API Gateway How-To: The Many Ways to Apply the Gateway Pattern
Open core summit: Observability for data pipelines with OpenLineage
Open core summit: Observability for data pipelines with OpenLineage
Similar a Blockchain and Apache NiFi
Edge to AI: Analytics from Edge to Cloud with Efficient Movement of Machine ...
Apache NiFi, Apache MXNet, TensorFlow, Apache Spark, Python, Cloudera CDF, Cloudera CDH, Cloudera Data Science Workbench, CDSW, MiNiFi, Cloudera EFM, Apache Zeppelin, YOLO, Computer Vision, REST API, JSON, Integration, Java, Big Data, Phoenix, Hive, HDFS, Parquet, AVRO. Cloudera. Hortonworks. John Kuchmek. Tim Spann.
Data Science in Enterprise
This document discusses enterprise data science and machine learning. It begins by noting that data is now more plentiful and machine learning opportunities are everywhere. However, challenges remain around scaling data science work, making models production-ready, and meeting different team needs. The document then introduces Cloudera's Data Science Workbench for addressing these challenges. It claims the Workbench provides a secure, self-service environment allowing data scientists direct access to enterprise data and tools while meeting IT requirements. Examples are given of how it supports the full data science pipeline from exploration to production. In demos, it highlights features like connecting to Hadoop clusters securely and enabling collaboration. Overall, the document pitches Cloudera's Workbench as a solution
Part 1: Introducing the Cloudera Data Science Workbench
3 Things to Learn About:
*The emergence of open source tools for data science
*Common gaps in the ecosystem
*Introduction of a new tool from Cloudera
Preparing for the Cybersecurity Renaissance
We are in the midst of a fundamental shift in the way in which organizations protect themselves from the modern adversary.
Traditional rules based cybersecurity applications of the past are not able to protect organizations in the new mobile, social, and hyper-connected world they now operate within. However, the convergence of big data technology, analytic advancements, and a variety of other factors have sparked a cybersecurity renaissance that will forever change the way in which organizations protect themselves.
Join Rocky DeStefano, Cloudera's Cybersecurity subject matter expert, as he explores how modern organizations are protecting themselves from more frequent, sophisticated attacks.
During this webinar you will learn about:
The current challenges cybersecurity professionals are facing today
How big data technologies are extending the capabilities of cybersecurity applications
Cloudera customers that are future proofing their cybersecurity posture with Cloudera’s next generation data and analytics management system
Effective IoT System on Openstack
This document discusses building an effective IoT system on OpenStack. It describes key IoT use cases and requirements, such as high data volume, velocity, and variety. The proposed architecture uses OpenStack services like Nova, Neutron, Swift, and Ceilometer to provide scalable infrastructure, networking, storage, and monitoring for IoT workloads. The document outlines how OpenStack can support broker integration, device management, flexible data stores, external connectivity, and data federation to realize a full-featured IoT platform. Future work involves proof-of-concept testing of the integrated architecture.
Meetup Streaming Data Pipeline Development
Meetup Streaming Data Pipeline Development
28 June 2023 6pm EST
Milwaukee meetup
https://www.meetup.com/futureofdata-princeton/events/292976004/
Details
This will be a hybrid event with a Zoom. The in-person event will be in Milwaukee.
In this interactive session, Tim will lead participants through how to best build streaming data pipelines. He will cover how to build applications from some common use cases and highlight tips, tricks, best practices and patterns.
He will show how to build the easy way and then dive deep into the underlying open source technologies including Apache NiFi, Apache Flink, Apache Kafka and Apache Iceberg.
If you wish to follow along, please download open source projects beforehand. You can also download this helpful streaming platform: https://docs.cloudera.com/csp-ce/latest/installation/topics/csp-ce-installing-ce.html
All source code and slides will be shared for those interested in building their own FLaNK Apps. https://www.flankstack.dev/
https://www.thecapitalgrille.com/locations/wi/milwaukee/milwaukee/8027
The Capital Grille 310 W Wisconsin Ave, Milwaukee, WI 53203
limited seating, preference will be given to NLIT attendees
A peak at the menu (Not Pizza)
RISOTTO FRITTERS WITH FRESH MOZZARELLA AND PROSCIUTTO
SLICED SIRLOIN WITH ROQUEFORT AND BALSAMIC ONIONS
MINIATURE LOBSTER AND CRAB CAKES
WILD MUSHROOM AND HERBED CHEESE
You can join the meeting virtually here (no meat or cheese virtually):
Future of Data Milwaukee Meetup Streaming Data Pipeline Development 28 June 2023
Future of Data Milwaukee Meetup Streaming Data Pipeline Development 28 June 2023
Future of Data: New Jersey - Princeton, Edison, Holmdel
This will be a hybrid event with a Zoom. The in-person event will be in Milwaukee.
In this interactive session, Tim will lead participants through how to best build streaming data pipelines. He will cover how to build applications from some common use cases and highlight tips, tricks, best practices and patterns.
He will show how to build the easy way and then dive deep into the underlying open source technologies including Apache NiFi, Apache Flink, Apache Kafka and Apache Iceberg.
If you wish to follow along, please download open source projects beforehand. You can also download this helpful streaming platform: https://docs.cloudera.com/csp-ce/latest/installation/topics/csp-ce-installing-ce.html
All source code and slides will be shared for those interested in building their own FLaNK Apps. https://www.flankstack.dev/
https://www.thecapitalgrille.com/locations/wi/milwaukee/milwaukee/8027
The Capital Grille 310 W Wisconsin Ave, Milwaukee, WI 53203
limited seating, preference will be given to NLIT attendees
A peak at the menu (Not Pizza)
RISOTTO FRITTERS WITH FRESH MOZZARELLA AND PROSCIUTTO
SLICED SIRLOIN WITH ROQUEFORT AND BALSAMIC ONIONS
MINIATURE LOBSTER AND CRAB CAKES
WILD MUSHROOM AND HERBED CHEESE
You can join the meeting virtually here (no meat or cheese virtually):
The Edge to AI Deep Dive Barcelona Meetup March 2019
The Edge to AI Deep Dive Barcelona Meetup March 2019
A deep dive demo of using MiNiFi, NiFi, CDSW for real-time AI at the edge, in a local cluster, in the cloud and in a Data Science platform at scale with real-time streaming and data storage.
Apache NiFi, MiNiFi, NiFi Registry, Cloudera Data Science Workbench (CDSW), Python, Pyspark, Spark SQL, Apache Calcite, Apache Parquet, Apache MXNet, GluonCV.
Where in the world is your Corporate data?
Your employees – and your company data – are on the go every day. As a result, your employees are relying on the use of 3rd party online services without IT approval – that is Shadow IT in your own organization. That’s some risky business. Where in the world is your Corporate Data?
With TeraGo Cloud Drive we are giving you back control of your most valuable asset, your data.
In this webinar you will learn about:
How Shadow IT is picking up velocity due to the accessibility and ease of cloud applications
Consequences of weak corporate security mechanisms
How to give your IT department control of your data and its’ security
Study on Secure Cryptographic Techniques in Cloud
Cloud Computing is turning into an increasing number of popular day by day. If the safety parameters are taken care properly many organizations and authorities corporations will flow into cloud technology.one usage of cloud computing is statistics storage. Cloud affords considerable capability of garage for cloud users. It is more reliable and flexible to users to shop and retrieve their facts at whenever and everywhere. It is an increasingly more growing technology. Mariam Fatima | M Ganeshan | Saif Ulla Shariff "Study on Secure Cryptographic Techniques in Cloud" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-5 | Issue-4 , June 2021, URL: https://www.ijtsrd.compapers/ijtsrd42363.pdf Paper URL: https://www.ijtsrd.comcomputer-science/computer-network/42363/study-on-secure-cryptographic-techniques-in-cloud/mariam-fatima
FullDay Faeder on Friday
Inaugural Edition of Weekly Symantec Cyber Security topics and events. This weeks is primarily focused on Cloud Security and 3 Organizations transforming the world as we know it
FullDay on Fridays Feb. 3, 2017
Inaugural Edition of FullDay Faeder on Fridays Weekly Symantec Cyber Security topics and events. This weeks is primarily focused on Cloud Security and 3 Organizations transforming the world as we know it
Cloudera streaming with flink oct 29, 2020 meetup london
Cloudera streaming with flink oct 29, 2020 meetup london
Future of Data: London presents: Cloudera Streaming with Flink
Enterprise machine learning on k8s lessons learned and the road ahead
This document discusses lessons learned from deploying machine learning workloads on Kubernetes. It describes how enterprise machine learning requires considering the full data and machine learning lifecycle. It also discusses how Cloudera Machine Learning focuses on supporting common machine learning workflows rather than specific frameworks. The document outlines Cloudera's approach to running Spark on Kubernetes and its roadmap, which includes unified data engineering and machine learning workflows, dependency management, elastic computing, and AI-driven optimization of workloads and infrastructure using AIOPS.
Sqrrl February Webinar: Breaking Down Data Silos
In this talk, Adam Fuchs, the CTO of Sqrrl and co-founder of the Accumulo project discusses some of the lessons learned for properly architecting, applying, and managing cell-level security labels in customer environments.
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
https://www.youtube.com/watch?v=Yeua8NlzQ3Y
https://www.conf42.com/Large_Language_Models_LLMs_2024_Tim_Spann_generative_ai_streaming
Adding Generative AI to Real-Time Streaming Pipelines
Abstract
Let’s build streaming pipelines that convert streaming events into prompts, call LLMs, and process the results.
Summary
Tim Spann: My talk is adding generative AI to real time streaming pipelines. I'm going to discuss a couple of different open source technologies. We'll touch on Kafka, Nifi, Flink, Python, Iceberg. All the slides, all the code and GitHub are out there.
Llm, if you didn't know, is rapidly evolving. There's a lot of different ways to interact with models. That enrichment, transformation, processing really needs tools. The amount of models and projects and software that are available is massive.
Nifi supports hundreds of different inputs and can convert them on the fly. Great way to distribute your data quickly to whoever needs it without duplication, without tight coupling. Fun to find new things to integrate into.
So what we can do is, well, I want to get a meetup chat going. I have a processor here that just listens for events as they come from slack. And then I'm going to clean it up, add a couple fields and push that out to slack. Every model is a little bit of different tweaking.
Nifi acts as a whole website. And as you see here, it can be get, post, put, whatever you want. We send that response back to flink and it shows up here. Thank you for attending this talk. I'm going to be speaking at some other events very shortly.
Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hi, Tim Spann here. My talk is adding generative AI to real time streaming pipelines, and we're here for the large language model conference at Comp 42, which is always a nice one, great place to be. I'm going to discuss a couple of different open source technologies that work together to enable you to build real time pipelines using large language models. So we'll touch on Kafka, Nifi, Flink, Python, Iceberg, and I'll show you a little bit of each one in the demos. I've been working with data machine learning, streaming IoT, some other things for a number of years, and you could contact me at any of these places, whether Twitter or whatever it's called, some different blogs, or in person at my meetups and at different conferences around the world. I do a weekly newsletter, cover streaming ML, a lot of LLM, open source, Python, Java, all kinds of fun stuff, as I mentioned, do a bunch of different meetups. They are not just in the east coast of the US, they are available virtually live, and I also put them on YouTube, and if you need them somewhere else, let me know. We publish all the slides, all the code and GitHub. Everything you need is out there. Let's get into the talk. Llm, if you didn't know, is rapidly evolving. While you're typing down the things that you use, it
Encryption in the Public Cloud: 16 Bits of Advice for Security Techniques
Dave Asprey, VP of Cloud Security at Trend Micro, presented on encryption techniques for securing data and applications in public clouds. He outlined 16 pieces of advice, including encrypting network traffic, files systems, and data in shared storage. Only allow decryption keys to enter the cloud during decryption. Minimize the number of services per virtual machine instance and only open necessary ports. Use host-based intrusion detection and system hardening tools.
Meetup: Streaming Data Pipeline Development
The document discusses streaming data pipelines and includes information about:
- The FLaNK stack which is comprised of Apache NiFi, Apache Kafka, Apache Flink, and Java.
- SQL Stream Builder which allows developers, analysts, and data scientists to write streaming applications using standard SQL without needing to write Java or Scala code.
- Apache Kafka as a distributed, partitioned, and replicated publish-subscribe messaging system.
- Apache Flink which is a framework for distributed stream and batch data processing.
Cloud-Native Machine Learning: Emerging Trends and the Road Ahead
Big data platforms are being asked to support an ever increasing range of workloads and compute environments, including large-scale machine learning and public and private clouds. In this talk, we will discuss some emerging capabilities around cloud-native machine learning and data engineering, including running machine learning and Spark workloads directly on Kubernetes, and share our vision of the road ahead for ML and AI in the cloud.
Fighting cyber fraud with hadoop v2
This document discusses using Hadoop to fight cyber fraud by analyzing big data. It explains that big data technologies provide powerful tools for services but also enable malicious cyber attacks by sophisticated attackers. Hadoop allows analyzing large datasets to detect fraud and security threats through techniques like machine learning, anomaly detection, and predicting real-time and historical patterns. The document advocates asking bigger questions to innovate solutions and gain operational and business advantages from big data analytics.
Similar a Blockchain and Apache NiFi (20)
Edge to AI: Analytics from Edge to Cloud with Efficient Movement of Machine ...
Edge to AI: Analytics from Edge to Cloud with Efficient Movement of Machine ...
Part 1: Introducing the Cloudera Data Science Workbench
Part 1: Introducing the Cloudera Data Science Workbench
Future of Data Milwaukee Meetup Streaming Data Pipeline Development 28 June 2023
Future of Data Milwaukee Meetup Streaming Data Pipeline Development 28 June 2023
The Edge to AI Deep Dive Barcelona Meetup March 2019
The Edge to AI Deep Dive Barcelona Meetup March 2019
Cloudera streaming with flink oct 29, 2020 meetup london
Cloudera streaming with flink oct 29, 2020 meetup london
Enterprise machine learning on k8s lessons learned and the road ahead
Enterprise machine learning on k8s lessons learned and the road ahead
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Encryption in the Public Cloud: 16 Bits of Advice for Security Techniques
Encryption in the Public Cloud: 16 Bits of Advice for Security Techniques
Cloud-Native Machine Learning: Emerging Trends and the Road Ahead
Cloud-Native Machine Learning: Emerging Trends and the Road Ahead
Más de Timothy Spann
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
by
Timothy Spann
Principal Developer Advocate
https://budapestdata.hu/2024/en/
https://budapestml.hu/2024/en/
tim.spann@zilliz.com
https://www.linkedin.com/in/timothyspann/
https://x.com/paasdev
https://github.com/tspannhw
https://www.youtube.com/@flank-stack
milvus
vector database
gen ai
generative ai
deep learning
machine learning
apache nifi
apache pulsar
apache kafka
apache flink
DSSML24_tspann_CodelessGenerativeAIPipelines
Codeless Generative AI Pipelines
(GenAI with Milvus)
https://ml.dssconf.pl/user.html#!/lecture/DSSML24-041a/rate
Discover the potential of real-time streaming in the context of GenAI as we delve into the intricacies of Apache NiFi and its capabilities. Learn how this tool can significantly simplify the data engineering workflow for GenAI applications, allowing you to focus on the creative aspects rather than the technical complexities. I will guide you through practical examples and use cases, showing the impact of automation on prompt building. From data ingestion to transformation and delivery, witness how Apache NiFi streamlines the entire pipeline, ensuring a smooth and hassle-free experience.
Timothy Spann
https://www.youtube.com/@FLaNK-Stack
https://medium.com/@tspann
https://www.datainmotion.dev/
milvus, unstructured data, vector database, zilliz, cloud, vectors, python, deep learning, generative ai, genai, nifi, kafka, flink, streaming, iot, edge
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Discussion on Vector Databases, Unstructured Data and AI
https://www.meetup.com/unstructured-data-meetup-new-york/
This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs.This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
Building Real-Time Pipelines With FLaNK
Timothy Spann, Principal Developer Advocate, Streaming - Cloudera Future of Data meetup, startup grind, AI Camp
The combination of Apache Flink, Apache NiFi, and Apache Kafka for building real-time data processing pipelines is extremely powerful, as demonstrated by this case study using the FLaNK-MTA project. The project leverages these technologies to process and analyze real-time data from the New York City Metropolitan Transportation Authority (MTA). FLaNK-MTA demonstrates how to efficiently collect, transform, and analyze high-volume data streams, enabling timely insights and decision-making.
Apache NiFi
Apache Kafka
Apache Flink
Apache Iceberg
LLM
Generative AI
Slack
Postgresql
Generative AI on Enterprise Cloud with NiFi and Milvus
Gen AI on Enterprise Cloud
Apache NiFi
Milvus
Apache Kafka
Apache Flink
Cloudera Machine Learning
Cloudera DataFlow
https://medium.com/@tspann/building-a-milvus-connector-for-nifi-34372cb3c7fa
https://www.meetup.com/futureofdata-princeton/events/300737266/
https://lu.ma/q7pcfyjn?source=post_page-----34372cb3c7fa--------------------------------&tk=TTyakY
If you're interested in working with Generative AI on the cloud, this virtual workshop is for you.
Tim Spann from Cloudera and Yujian Tang from Zilliz will cover how you can implement your own GenAI workflows on the cloud at enterprise scale.
9:00 - 9:05: Intro
9:05 - 9:15: What is Milvus
9:15 - 9:25: Cloudera Development Platform
9:25 - 10:00: Demo
Location
https://www.youtube.com/watch?v=IfWIzKsoHnA
https://github.com/tspannhw/SpeakerProfile
https://www.linkedin.com/in/yujiantang/
April 2024 - NLIT Cloudera Real-Time LLM Streaming 2024
https://www.fbcinc.com/e/nlit/agenda.aspx
Cloudera booth
data in motion
tim spann
seattle
April 2024
Real-Time AI Streaming - AI Max Princeton
AI Max Conference Princeton
https://www.startupgrind.com/events/details/startup-grind-princeton-presents-startup-grind-hosts-ai-max-summit/
2024 XTREMEJ_ Building Real-time Pipelines with FLaNK_ A Case Study with Tra...
2024 XTREMEJ_ Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
https://xtremej.dev/2023/schedule/
Building Real-time Pipelines with FLaNK: A Case Study with Transit Data
Overview of the problem, the application (code walkthru and running), overview of FLaNK, introduction to NiFi, introduction to Kafka, and introduction to Flink.
28March2024-Codeless-Generative-AI-Pipelines
28March2024-Codeless-Generative-AI-Pipelines
https://www.meetup.com/futureofdata-princeton/events/299440871/
https://www.meetup.com/real-time-analytics-meetup-ny/events/299290822/
******Note*****
The event is seat-limited, therefore please complete your registration here. Only people completing the form will be able to attend.
-----------------------
We're excited to invite you to join us in-person, for a Real-Time Analytics exploration!
Join us for an evening of insights, networking as we delve into the OSS technologies shaping the field!
Agenda:
05:30-06:00: Pizza and friends
06:00- 06:40: Codeless GenAI Pipelines with Flink, Kafka, NiFi
06:40- 07:20 Real-Time Analytics in the Corporate World: How Apache Pinot® Powers Industry Leaders
07:20-07:30 QNA
Codeless GenAI Pipelines with Flink, Kafka, NiFi | Tim Spann, Cloudera
Explore the power of real-time streaming with GenAI using Apache NiFi. Learn how NiFi simplifies data engineering workflows, allowing you to focus on creativity over technical complexities. I'll guide you through practical examples, showcasing NiFi's automation impact from ingestion to delivery. Whether you're a seasoned data engineer or new to GenAI, this talk offers valuable insights into optimizing workflows. Join us to unlock the potential of real-time streaming and witness how NiFi makes data engineering a breeze for GenAI applications!
Real-Time Analytics in the Corporate World: How Apache Pinot® Powers Industry Leaders | Viktor Gamov, StarTree
Explore how industry leaders like LinkedIn, Uber Eats, and Stripe are mastering real-time data with Viktor as your guide. Discover how Apache Pinot transforms data into actionable insights instantly. Viktor will showcase Pinot's features, including the Star-Tree Index, and explain why it's a game-changer in data strategy. This session is for everyone, from data geeks to business gurus, eager to uncover the future of tech. Join us and be wowed by the power of real-time analytics with Apache Pinot!
-------
Tim Spann is a Principal Developer Advocate in Data In Motion for Cloudera.
He works with Apache NiFi, Apache Kafka, Apache Pulsar, Apache Flink, Flink SQL, Apache Pinot, Trino, Apache Iceberg, DeltaLake, Apache Spark, Big Data, IoT, Cloud, AI/DL, machine learning, and deep learning. Tim has over ten years of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming. Previously, he was a Developer Advocate at StreamNative, Principal DataFlow Field Engineer at Cloudera, a Senior Solutions Engineer at Hortonworks, a Senior Solutions Architect at AirisData, a Senior Field Engineer at Pivotal and a Team Leader at HPE. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton & NYC on Big Data, Cloud, IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as ApacheCon, DeveloperWeek, Pulsar Summit and many more.
TCFPro24 Building Real-Time Generative AI Pipelines
https://princetonacm.acm.org/tcfpro/
18th Annual IEEE IT Professional Conference (ITPC)
Armstrong Hall at The College of New Jersey
Friday, March 15th, 2024 | 10:00 AM to 5:00 PM
IT Professional Conference at Trenton Computer Festival
IEEE Information Technology Professional Conference on Friday, March 15th, 2024
TCFPro24 Building Real-Time Generative AI Pipelines
Building Real-Time Generative AI Pipelines
In this talk, Tim will delve into the exciting realm of building real-time generative AI pipelines with streaming capabilities. The discussion will revolve around the integration of cutting-edge technologies to create dynamic and responsive systems that harness the power of generative algorithms.
From leveraging streaming data sources to implementing advanced machine learning models, the presentation will explore the key components necessary for constructing a robust real-time generative AI pipeline. Practical insights, use cases, and best practices will be shared, offering a comprehensive guide for developers and data scientists aspiring to design and implement dynamic AI systems in a streaming environment.
Tim will show a live demo showing we can use Apache NiFi to provide a live chat between a person in Slack and several LLM models all orchestrated with Apache NiFi, Apache Kafka and Python. We will use RAG against Chroma and Pinecone vector data stores, Hugging Face and WatsonX.AI LLM, and add additional context with NiFi lookups of stocks, weather and other data streams in real-time.
Timothy Spann
Tim Spann is a Principal Developer Advocate in Data In Motion for Cloudera. He works with Apache NiFi, Apache Pulsar, Apache Kafka, Apache Flink, Flink SQL, Apache Pinot, Trino, Apache Iceberg, DeltaLake, Apache Spark, Big Data, IoT, Cloud, AI/DL, machine learning, and deep learning. Tim has over ten years of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming.
Previously, he was a Developer Advocate at StreamNative, Principal DataFlow Field Engineer at Cloudera, a Senior Solutions Engineer at Hortonworks, a Senior Solutions Architect at AirisData, a Senior Field Engineer at Pivotal and a Team Leader at HPE. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton & NYC on Big Data, Cloud, IoT, deep learning, streaming, NiFi, the blockchain, and Spark.
Tim is a frequent speaker at conferences such as ApacheCon, DeveloperWeek, Pulsar Summit and many more. He holds a BS and MS in computer science.
2024 Build Generative AI for Non-Profits
2024_BuildGenerativeAIforNon-Profits
https://www.meetup.com/futureofdata-princeton/events/299515503/
https://www.meetup.com/applied-generative-artificial-intelligence-applications/events/299486639/
LLM
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipelines
https://www.meetup.com/futureofdata-newyork/events/298660453/
Unlocking Financial Data with Real-Time Pipelines
(Flink Analytics on Stocks with SQL )
By Timothy Spann
Financial institutions thrive on accurate and timely data to drive critical decision-making processes, risk assessments, and regulatory compliance. However, managing and processing vast amounts of financial data in real-time can be a daunting task. To overcome this challenge, modern data engineering solutions have emerged, combining powerful technologies like Apache Flink, Apache NiFi, Apache Kafka, and Iceberg to create efficient and reliable real-time data pipelines. In this talk, we will explore how this technology stack can unlock the full potential of financial data, enabling organizations to make data-driven decisions swiftly and with confidence.
Introduction: Financial institutions operate in a fast-paced environment where real-time access to accurate and reliable data is crucial. Traditional batch processing falls short when it comes to handling rapidly changing financial markets and responding to customer demands promptly. In this talk, we will delve into the power of real-time data pipelines, utilizing the strengths of Apache Flink, Apache NiFi, Apache Kafka, and Iceberg, to unlock the potential of financial data. I will be utilizing NiFi 2.0 with Python and Vector Databases.
Timothy Spann
Principal Developer Advocate, Cloudera
Tim Spann is a Principal Developer Advocate in Data In Motion for Cloudera. He works with Apache NiFi, Apache Kafka, Apache Pulsar, Apache Flink, Flink SQL, Apache Pinot, Trino, Apache Iceberg, DeltaLake, Apache Spark, Big Data, IoT, Cloud, AI/DL, machine learning, and deep learning. Tim has over ten years of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming. Previously, he was a Developer Advocate at StreamNative, Principal DataFlow Field Engineer at Cloudera, a Senior Solutions Engineer at Hortonworks, a Senior Solutions Architect at AirisData, a Senior Field Engineer at Pivotal and a Team Leader at HPE. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton & NYC on Big Data, Cloud, IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as ApacheCon, DeveloperWeek, Pulsar Summit and many more. He holds a BS and MS in computer science.
https://twitter.com/PaaSDev
https://www.linkedin.com/in/timothyspann/
https://medium.com/@tspann
https://github.com/tspannhw/FLiPStackWeekly/
Conf42-Python-Building Apache NiFi 2.0 Python Processors
Conf42-Python-Building Apache NiFi 2.0 Python Processors
https://www.conf42.com/Python_2024_Tim_Spann_apache_nifi_2_processors
Building Apache NiFi 2.0 Python Processors
Abstract
Let’s enhance real-time streaming pipelines with smart Python code. Adding code for vector databases and LLM.
Summary
Tim Spann: I'm going to be talking today, be building Apache 9520 Python processors. One of the main purposes of supporting Python in the streaming tool Apache Nifi is to interface with new machine learning and AI and Gen AI. He says Python is a real game changer for Cloudera.
You're just going to add some metadata around it. It's a great way to pass a file along without changing it too substantially. We really need you to have Python 310 and again JDK 21 on your machine. You got to be smart about how you use these models.
There are a ton of python processors available. You can use them in multiple ways. We're still in the early world of Python processors, so now's the time to start putting yours out there. Love to see a lot of people write their own.
When we are parsing documents here, again, this is the Python one I'm picking PDF. Lots of different things you could do. If you're interested on writing your own python code for Apache Nifi, definitely reach out and thank.
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg with Stock Data and LLM
Abstract
In this talk, we’ll discuss how to use Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg to process and analyze stock data. We demonstrated the ingestion, processing, and analysis of stock data. Additionally, we illustrated how to use an LLM to generate predictions from the analyzed data.
Karin Wolok
Developer Relations, Dev Marketing, and Community Programming @ Project Elevate
Karin Wolok's LinkedIn account Karin Wolok's twitter account
Tim Spann
Principal Developer Advocate @ Cloudera
Tim Spann's LinkedIn account Tim Spann's twitter account
https://www.conf42.com/Python_2024_Karin_Wolok_Tim_Spann_nifi__kafka_risingwave_iceberg_llm
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
https://www.aicamp.ai/event/eventdetails/W2024022214
apache nifi
llm
generative ai
gen ai
ml
dl
machine learning
apache kafka
apache flink
postgresql
python
AI Meetup (NYC): GenAI, LLMs, ML and Data
Feb 22, 05:30 PM EST
Welcome to the monthly in-person AI meetup in New York City, in collaboration with Microsoft. Join us for deep dive tech talks on AI, GenAI, LLMs and machine learning, food/drink, networking with speakers and fellow developers
Agenda:
* 5:30pm~6:00pm: Checkin, Food/drink and networking
* 6:00pm~6:10pm: Welcome/community update
* 6:10pm~8:30pm: Tech talks
* 8:30pm: Q&A, Open discussion
Tech Talk: Searching and Reasoning Over Multimedia Data with Vector Databases and LMMs
Speaker: Zain Hasan (Weaviate LinkedIn)
Abstract: In this talk, Zain Hasan will discuss how we can use open-source multimodal embedding models in conjunction with large generative multimodal models that can that can see, hear, read, and feel data(!), to perform cross-modal search(searching audio with images, videos with text etc.) and multimodal retrieval augmented generation (MM-RAG) at the billion-object scale with the help of open source vector databases. I will also demonstrate, with live code demos, how being able to perform this cross-modal retrieval in real-time can enables users to use LLMs that can reason over their enterprise multimodal data. This talk will revolve around how we can scale the usage of multimodal embedding and generative models in production.
Tech Talk: Codeless Generative AI Pipelines
Speaker: Timothy Spann (Cloudera LinkedIn)
Abstract: Join us for an insightful talk on leveraging the power of real-time streaming tools, specifically Apache NiFi, to revolutionize GenAI data engineering. In this session, we’ll explore how the integration of Apache NiFi can automate the entire process of prompt building, making it a seamless and efficient task.
Speakers/Topics:
Stay tuned as we are updating speakers and schedules. If you have a keen interest in speaking to our community, we invite you to submit topics for consideration: Submit Topics
Sponsors:
We are actively seeking sponsors to support our community. Whether it is by offering venue spaces, providing food/drink, or cash sponsorship. Sponsors will have the chance to speak at the meetups, receive prominent recognition, and gain exposure to our extensive membership base of 20,000+ local or 300K+ developers worldwide.
Venue:
Microsoft NYC - Times Square, 11 Times Square, New York, NY 10036
Room Name: Central Park West 6501
Community on Slack/Discord
- Event chat: chat and connect with speakers and attendees
- Sharing blogs, events, job openings, projects collaborations
Join Slack (search and join the #newyork channel) | Join Discord
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
20-Feb-2024
In this talk, I will walk through how someone can set up and run continuous SQL queries against Kafka topics utilizing Apache Flink. We will walk through creating Kafka topics, schemas, and publishing data.
We will then cover consuming Kafka data, joining Kafka topics, and inserting new events into Kafka topics as they arrive. This basic overview will show hands-on techniques, tips, and examples of how to do this.
Tim Spann
Tim Spann is the Principal Developer Advocate for Data in Motion @ Cloudera where he works with Apache Kafka, Apache Flink, Apache NiFi, Apache Iceberg, TensorFlow, Apache Spark, big data, the IoT, machine learning, and deep learning. Tim has over a decade of experience with the IoT, big data, distributed computing, streaming technologies, and Java programming. Previously, he was a Developer Advocate at StreamNative, Principal Field Engineer at Cloudera, a Senior Solutions Architect at AirisData and a senior field engineer at Pivotal. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton on big data, the IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as IoT Fusion, Strata, ApacheCon, Data Works Summit Berlin, DataWorks Summit Sydney, and Oracle Code NYC. He holds a BS and MS in computer science.
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK_ A Case Study with Transit Data
OSACon 2023_ Unlocking Financial Data with Real-Time Pipelines
OSACon 2023_ Unlocking Financial Data with Real-Time Pipelines
Unlocking Financial Data with Real-Time Pipelines
Financial institutions thrive on accurate and timely data to drive critical decision-making processes, risk assessments, and regulatory compliance. However, managing and processing vast amounts of financial data in real-time can be a daunting task. To overcome this challenge, modern data engineering solutions have emerged, combining powerful technologies like Apache Flink, Apache NiFi, Apache Kafka, and Iceberg to create efficient and reliable real-time data pipelines. In this talk, we will explore how this technology stack can unlock the full potential of financial data, enabling organizations to make data-driven decisions swiftly and with confidence.
Introduction: Financial institutions operate in a fast-paced environment where real-time access to accurate and reliable data is crucial. Traditional batch processing falls short when it comes to handling rapidly changing financial markets and responding to customer demands promptly. In this talk, we will delve into the power of real-time data pipelines, utilizing the strengths of Apache Flink, Apache NiFi, Apache Kafka, and Iceberg, to unlock the potential of financial data.
Key Points to be Covered:
Introduction to Real-Time Data Pipelines: a. The limitations of traditional batch processing in the financial domain. b. Understanding the need for real-time data processing.
Apache Flink: Powering Real-Time Stream Processing: a. Overview of Apache Flink and its role in real-time stream processing. b. Use cases for Apache Flink in the financial industry. c. How Flink enables fast, scalable, and fault-tolerant processing of streaming financial data.
Apache Kafka: Building Resilient Event Streaming Platforms: a. Introduction to Apache Kafka and its role as a distributed streaming platform. b. Kafka's capabilities in handling high-throughput, fault-tolerant, and real-time data streaming. c. Integration of Kafka with financial data sources and consumers.
Apache NiFi: Data Ingestion and Flow Management: a. Overview of Apache NiFi and its role in data ingestion and flow management. b. Data integration and transformation capabilities of NiFi for financial data. c. Utilizing NiFi to collect and process financial data from diverse sources.
Iceberg: Efficient Data Lake Management: a. Understanding Iceberg and its role in managing large-scale data lakes. b. Iceberg's schema evolution and table-level metadata capabilities. c. How Iceberg simplifies data lake management in financial institutions.
Real-World Use Cases: a. Real-time fraud detection using Flink, Kafka, and NiFi. b. Portfolio risk analysis with Iceberg and Flink. c. Streamlined regulatory reporting leveraging all four technologies.
Best Practices and Considerations: a. Architectural considerations when building real-time financial data pipelines. b. Ensuring data integrity, security, and compliance in real-time pipelines. c. Scalability an
Building Real-Time Travel Alerts
Building Real-time Travel Alerts
In this session, we will walk through how to build a complete streaming application to send alerts based on travel advisories from public data. We will also join in other data sources of relevance and push out alerts.
We will show you how to build this streaming application with Apache NiFi, Apache Kafka, and Apache Flink and show you when/why/how, and what to build to maximize performance, productivity, and ease of development.
Let's get streaming.
Apache Flink
Apache Kafka
Apache NiFi
FLaNK Stack
Tim Spann
Big Data Conference Europe 2023
Más de Timothy Spann (20)
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
Generative AI on Enterprise Cloud with NiFi and Milvus
Generative AI on Enterprise Cloud with NiFi and Milvus
April 2024 - NLIT Cloudera Real-Time LLM Streaming 2024
April 2024 - NLIT Cloudera Real-Time LLM Streaming 2024
2024 XTREMEJ_ Building Real-time Pipelines with FLaNK_ A Case Study with Tra...
2024 XTREMEJ_ Building Real-time Pipelines with FLaNK_ A Case Study with Tra...
TCFPro24 Building Real-Time Generative AI Pipelines
TCFPro24 Building Real-Time Generative AI Pipelines
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
Conf42-Python-Building Apache NiFi 2.0 Python Processors
Conf42-Python-Building Apache NiFi 2.0 Python Processors
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...
Conf42Python -Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg...
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
2024 Feb AI Meetup NYC GenAI_LLMs_ML_Data Codeless Generative AI Pipelines
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...
NY Open Source Data Meetup Feb 8 2024 Building Real-time Pipelines with FLaNK...
OSACon 2023_ Unlocking Financial Data with Real-Time Pipelines
OSACon 2023_ Unlocking Financial Data with Real-Time Pipelines
Último
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
Regarding mobile operating systems, two major players dominate our thoughts: Android and iPhone. With Android leading the market, software development companies are focused on delivering apps compatible with this OS. Ensuring an app's functionality across various Android devices, OS versions, and hardware specifications is critical, making Android app testing essential.
Requirement Traceability in Xen Functional Safety
Using OpenFastTrace to trace requirements in Xen functional safety
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
Presented at NLJUG's J-Spring 2024.
SMS API Integration in Saudi Arabia| Best SMS API Service
Discover the benefits and implementation of SMS API integration in the UAE and Middle East. This comprehensive guide covers the importance of SMS messaging APIs, the advantages of bulk SMS APIs, and real-world case studies. Learn how CEQUENS, a leader in communication solutions, can help your business enhance customer engagement and streamline operations with innovative CPaaS, reliable SMS APIs, and omnichannel solutions, including WhatsApp Business. Perfect for businesses seeking to optimize their communication strategies in the digital age.
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版一模一样【微信：741003700 】【美国纽约州立大学奥尔巴尼分校毕业证学位证书】【微信：741003700 】学位证，留信认证（真实可查，永久存档）offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原海外各大学 Bachelor Diploma degree, Master Degree Diploma
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Salesforce Healthcare CRM, implemented by VALiNTRY360, revolutionizes patient management by enhancing patient engagement, streamlining administrative processes, and improving care coordination. Its advanced analytics, robust security, and seamless integration with telehealth services ensure that healthcare providers can deliver personalized, efficient, and secure patient care. By automating routine tasks and providing actionable insights, Salesforce Healthcare CRM enables healthcare providers to focus on delivering high-quality care, leading to better patient outcomes and higher satisfaction. VALiNTRY360's expertise ensures a tailored solution that meets the unique needs of any healthcare practice, from small clinics to large hospital systems.
For more info visit us https://valintry360.com/solutions/health-life-sciences
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
Learn about the latest innovations in and around OpenUI5/SAPUI5: UI5 Tooling, UI5 linter, UI5 Web Components, Web Components Integration, UI5 2.x, UI5 GenAI.
Recording:
https://www.youtube.com/live/MSdGLG2zLy8?si=INxBHTqkwHhxV5Ta&t=0
Unveiling the Advantages of Agile Software Development.pdf
Learn about Agile Software Development's advantages. Simplify your workflow to spur quicker innovation. Jump right in! We have also discussed the advantages.
Transform Your Communication with Cloud-Based IVR Solutions
Discover the power of Cloud-Based IVR Solutions to streamline communication processes. Embrace scalability and cost-efficiency while enhancing customer experiences with features like automated call routing and voice recognition. Accessible from anywhere, these solutions integrate seamlessly with existing systems, providing real-time analytics for continuous improvement. Revolutionize your communication strategy today with Cloud-Based IVR Solutions. Learn more at: https://thesmspoint.com/channel/cloud-telephony
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech is a premier mobile app development company in Noida, providing cutting-edge solutions for businesses.
Visit Us For : https://www.dronainfotech.com/mobile-application-development/
316895207-SAP-Oil-and-Gas-Downstream-Training.pptx
316895207-SAP-Oil-and-Gas-Downstream-Training.pptx
Hand Rolled Applicative User Validation Code Kata
Could you use a simple piece of Scala validation code (granted, a very simplistic one too!) that you can rewrite, now and again, to refresh your basic understanding of Applicative operators <*>, <*, *>?
The goal is not to write perfect code showcasing validation, but rather, to provide a small, rough-and ready exercise to reinforce your muscle-memory.
Despite its grandiose-sounding title, this deck consists of just three slides showing the Scala 3 code to be rewritten whenever the details of the operators begin to fade away.
The code is my rough and ready translation of a Haskell user-validation program found in a book called Finding Success (and Failure) in Haskell - Fall in love with applicative functors.
Modelling Up - DDDEurope 2024 - Amsterdam
What to do when you have a perfect model for your software but you are constrained by an imperfect business model?
This talk explores the challenges of bringing modelling rigour to the business and strategy levels, and talking to your non-technical counterparts in the process.
Using Query Store in Azure PostgreSQL to Understand Query Performance
Microsoft has added an excellent new extension in PostgreSQL on their Azure Platform. This session, presented at Posette 2024, covers what Query Store is and the types of information you can get out of it.
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
The UI5 tooling is the development and build tooling of UI5. It is built in a modular and extensible way so that it can be easily extended by your needs. This session will showcase various tooling extensions which can boost your development experience by far so that you can really work offline, transpile your code in your project to use even newer versions of EcmaScript (than 2022 which is supported right now by the UI5 tooling), consume any npm package of your choice in your project, using different kind of proxies, and even stitching UI5 projects during development together to mimic your target environment.
WWDC 2024 Keynote Review: For CocoaCoders Austin
Overview of WWDC 2024 Keynote Address.
Covers: Apple Intelligence, iOS18, macOS Sequoia, iPadOS, watchOS, visionOS, and Apple TV+.
Understandable dialogue on Apple TV+
On-device app controlling AI.
Access to ChatGPT with a guest appearance by Chief Data Thief Sam Altman!
App Locking! iPhone Mirroring! And a Calculator!!
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Presentation from Ghazal Aakel and Eric Jochum at the GSD Community Stage Meetup on June 6th, 2024
Artificia Intellicence and XPath Extension Functions
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
GreenCode-A-VSCode-Plugin--Dario-Jurisic
Presentation about a VSCode plugin from Dario Jurisic at the GSD Community Stage meetup
Último (20)
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdf
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
316895207-SAP-Oil-and-Gas-Downstream-Training.pptx
316895207-SAP-Oil-and-Gas-Downstream-Training.pptx
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Blockchain and Apache NiFi
- 1. © Cloudera, Inc. All rights reserved. Timothy Spann Senior Solutions Engineer tspann@cloudera.com @PaasDev
- 2. © Cloudera, Inc. All rights reserved. Introduction Tim Spann has been running meetups in Princeton on Big Data technologies since 2015. Tim has spoken at several international conferences on Apache NiFi, Deep Learning and Streaming. https://community.hortonworks.com/users/9304/tspann.html https://dzone.com/users/297029/bunkertor.html https://www.meetup.com/futureofdata-princeton/ https://twitter.com/PaaSDev https://dzone.com/articles/integrating-keras-tensorflow-yolov3-into-apache-ni https://conferences.oreilly.com/strata/strata-ny/public/schedule/speaker/185963
- 3. © Cloudera, Inc. All rights reserved. https://www.cloudera.com/about/events.html
- 4. © Cloudera, Inc. All rights reserved.
- 5. © Cloudera, Inc. All rights reserved.
- 6. © Cloudera, Inc. All rights reserved. 6© Cloudera, Inc. All rights reserved. DATAFLOW USE CASES Data Movement Optimize resource utilization by moving data between data centers or between on-premises infrastructure and cloud infrastructure Optimize Log Collection & Analysis Optimize log analytics solutions such as Splunk by using DataFlow as a single platform to collect and deliver multiple data sources and using HDP for lower cost storage options Gain key insights with Streaming Analytics Accelerate big data ROI by analyzing streaming data for patterns, comparing with ML models and delivering actionable intelligence Single view / 360° view of customer Ingest, transform and combine customer data from multiple sources into a single data view / lake Stream Processing Combine multiple streams of data in real-time, enrich the data and route it to different end points based on rules Capture IoT Data Ingest sensor data from IoT devices and stream it for further processing and comprehensive analysis
- 8. © Cloudera, Inc. All rights reserved. 8© Cloudera, Inc. All rights reserved. MODERN DATA ARCHITECTURE DATA CENTER Machine Learning/ Artificial Intelligence Telemetry – Connected Devices Time Series Databases Stream Analytics Deep Historical Analysis Exception Monitoring Legacy/ Operational Data Sensors, Control Systems Cyber Security Edge Analytics Social Mobile IoT IoT CLOUD Geo Location
- 9. © Cloudera, Inc. All rights reserved.
- 10. © Cloudera, Inc. All rights reserved. 10
- 11. © Cloudera, Inc. All rights reserved. Blockchain https://hortonworks.com/article/the-advantages-of-blockchain-technology/ Blockchain ensures data objectivity—a single source of truth. Blockchain also represents a security layer that ensures that data is encrypted in such a way that only the people you want to can read your data. It makes it next to impossible for people to corrupt or manipulate the data—or even gain wrongful access to it—because the system raises an instant red flag when a problem occurs, and it uses a new, advanced encryption method to secure the data. https://hortonworks.com/blog/blockchain-driven-data-marketplaces-reference-architecture/ https://hortonworks.com/blog/blockchain-driven-data-marketplaces-reference-architecture/ https://en.wikipedia.org/wiki/Blockchain http://vision.cloudera.com/data-driven-innovation-in-healthcare-where-weve-been-and-where-were-headed/
- 12. © Cloudera, Inc. All rights reserved. “Ethereum is a decentralized platform that runs smart contracts: applications that run exactly as programmed without any possibility of downtime, censorship, fraud or third party interference.” https://community.hortonworks.com/articles/191255/ethereum-accessing-feeds-from-etherscan-on-volume.html https://github.com/ethereum/wiki/wiki/White-Paper https://community.hortonworks.com/content/kbentry/191146/accessing-feeds-from-etherdelta-on-trades-funds-bu. html
- 13. © Cloudera, Inc. All rights reserved. Blockchain Is Everywhere Distributed DataStore https://bitdb.network/ https://ipfs.io/A peer-to-peer hypermedia protocol https://github.com/ipfs/java-ipfs-api
- 14. © Cloudera, Inc. All rights reserved. • https://community.hortonworks.com/articles/1995 70/ingest-btccom-and-blockchaincom-data-via-apa che-ni.html Blockchain Data Access
- 15. © Cloudera, Inc. All rights reserved. • https://community.hortonworks.com/articles/199566/ingesting-infura-rest-apis-to-access-the-ethereum.html Ethereum Data Access
- 16. © Cloudera, Inc. All rights reserved. • https://community.hortonworks.com/articles/191255/ethereum-accessing-feeds-from-etherscan-on-volume.html Ethereum Data Access https://community.hortonworks.com/articles/191146/accessing-feeds-from-etherdelta-on-trades-funds-bu.html