Building Modern Streaming Analytics with Confluent on AWS

confluent
confluentconfluent
© 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Nae Ittaboon (he/him)
Solutions Architect
Amazon Web Services (AWS)
Building Modern Streaming Analytics
with Confluent on AWS
#DataInMotionTour
© 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Agenda
2
❑ Develop a modern data strategy
❑ Act on events in real time with data streaming
❑ Build seamless streaming with AWS and Confluent
❑ Innovate together to power customer success
❑ Key takeaways
© 2023, Amazon Web Services, Inc. or its affiliates.
Challenges of data analytics at scale
3
Data volume and velocity
Multiple analytics needs
Variety of sources and data types
Difficult to manage systems
Complex to scale
Inflexible tools
Security, compliance
Slow performance Increasing and unpredictable cost
© 2023, Amazon Web Services, Inc. or its affiliates.
Catalog
Governance
Data
sources
People,
apps, and
devices
Analytics
Machine
learning
Databases
Data
lakes
Modern data strategy
© 2023, Amazon Web Services, Inc. or its affiliates.
Amazon
OpenSearch
Service
Amazon
Aurora
Amazon
EMR
Amazon
SageMaker
Amazon
DynamoDB
Amazon
Redshift
Amazon S3
Modern data architecture on AWS
Key Pillars
Data at any scale
Seamless data access and movement
Purpose-built for best price performance
Built-in ML to solve business challenges
Unified governance
© 2023, Amazon Web Services, Inc. or its affiliates.
The benefits of data lakes
Data lake
Catalog
Store all your data in open formats
Decouple storage from compute
Cost-effectively scale storage to exabytes
Process data in place
Choice of analytical and ML engines
© 2023, Amazon Web Services, Inc. or its affiliates.
Sharing across data movement with Data Mesh
Data producers Data mesh Data consumers
Unique modern data architecture
Suited to business function
Teams that want to share data
Unique modern data architecture
Suited to business function
Team that runs the marketplace Teams that want to use data
© 2023, Amazon Web Services, Inc. or its affiliates.
Source: Perishable insights, Mike Gualtieri, Forrester
Real time Seconds Minutes Hours Days Months
Value
of
data
to
decision-making
Preventive/predictive
Actionable Reactive Historical
Time-critical decisions Traditional “batch” business intelligence
Information half-life
in decision-making
Why real-time data streaming analytics ?
Data loses value quickly over time
© 2023, Amazon Web Services, Inc. or its affiliates.
Common real-time analytics use cases
Anomaly and fraud detection
Empowering IoT analytics
Nourishing marketing campaigns
Real-time personalization
Tailoring customer experience in real time
Supporting healthcare and emergency services
© 2023, Amazon Web Services, Inc. or its affiliates.
Devices and/or
applications
that produce
real-time
data at high
velocity
Data from tens of
thousands of data
sources can be written to
a single stream
Data are stored in the
order they were
received for a set
duration
of time and can be
replayed indefinitely
during that time
Records are read in
the order they are
produced, enabling real-
time analytics or
streaming ETL
Database (NoSQL
most common)
Data lake
Data warehouse
Event driven
Applications
`
Analytics
dashboard
Real-time streaming analytics pipeline
Ingest, Process & Analyze High Volumes of High-Velocity Data from Various Sources in Real Time
© 2023, Amazon Web Services, Inc. or its affiliates.
Mobile device
Metering
Click streams
IoT sensors
AWS SDKs
Kinesis Agent/KPL
`
Apache Kafka
Amazon Kinesis Data
Streams
Amazon Kinesis Data
Firehose
Apache Kafka
Amazon Kinesis Data
Streams
Amazon Kinesis
Data Analytics
AWS Glue Streaming
Amazon EMR
Amazon DynamoDB
Amazon EMR
Amazon S3
Amazon Redshift Amazon OpenSearch
Amazon QuickSight
Apache Kafka
Streaming analytics on AWS
© 2023, Amazon Web Services, Inc. or its affiliates.
Rich front-end
customer
experiences
Real-time
Event Streams and Analysis
A Sale A shipment
A Trade
A Customer
Experience
Real-time backend
operations
Event streaming with Kafka to set Data in Motion:
Continuously processing evolving streams of data in real time with Confluent
© 2023, Amazon Web Services, Inc. or its affiliates.
Out-of-box integration with
popular services
Certified and validated by
AWS
AWS Native Services
Top-5 Global ISV for S3 Data Volume
3rd-Party ISV Services
• Amazon RDS
Ready
• AWS Lambda
Ready
• Amazon Redshift
Ready
• AWS PrivateLink
Ready
• AWS Outposts
Ready
Validated Service Designations
Confluent integrations with AWS
© 2023, Amazon Web Services, Inc. or its affiliates.
S
Confluent Cloud on AWS – reference architecture
S3
© 2023, Amazon Web Services, Inc. or its affiliates.
Confluent + AWS: Accelerate Customer Business Outcomes
Topline Impacting New
Experiences
● Event-driven & real-time
● Unify data across org. w/ Kafka
data fabric (Schema Reg,..)
● AWS Analytics, Redshift, ML
connectors
Mitigate Risk
● Higher Service Quality &
Resilience with 99.99% SLA
● Deep Kafka expertise & innovation
● Elastic billing/pricing
Developer Agility
● Focus on innovation (not data
infrastructure)
● Leverage full Kafka OSS
ecosystem + AWS services
Faster Time to Market
● ~50-75% faster time to market*
● Streamline hybrid cloud
migration with no complex lift-n-
shift
● Maintain business continuity
Lower Kafka TCO
● ~25-50% lower TCO *
● GBps-scale & fast deployments
for global expansion
● Deploy Kafka at scale in 1 week
Maximize ROI
● ~200% ROI per Forrester study
● Save 10s of $Ms with legacy
offload to AWS with Confluent
Replicator
* For customers that don’t already have Kafka based system in-market
* TCO assessment to be analyzed for specific customer scenarios
© 2023, Amazon Web Services, Inc. or its affiliates.
Accelerate modernization from on-prem to AWS
Redshift Sink
Lambda Sink
AWS Direct
Connect
Replicator
Legacy Applications
Mainframe
Legacy Data Systems
JDBC / CDC
connectors
Connect
Leverage 120+ Confluent pre-built connectors
Modernize
Value added apps, increase agility, reduce TCO
On-prem AWS Cloud
Bridge
Hybrid cloud streaming
Amazon Athena
AWS Glue
Amazon
SageMaker
AWS Lake
Formation
Amazon
DynamoDB
Amazon
Amazon
Aurora
S3 Sink
Data Streams
Apps
ksqlDB
© 2023, Amazon Web Services, Inc. or its affiliates.
Increase developer agility & speed of innovation
Serverless integration
Connect for effortless integrations with Lambda & data stores
AWS serverless platform
Free up backend operation & Infras. management
Apps
Microservices
ksqlDB
Schema
Registry
COMPUTE
AWS
Lambda
Data stores
REST Proxy
& Clients
Source
Connectors
Lambda
Sink
DATA STORES
Amazon
DynamoDB
Amazon
Aurora
STORAGE
Amazon
S3
S3 Sink
ANALYTICS
Amazon
Athena
Amazon
Redshift
© 2023, Amazon Web Services, Inc. or its affiliates.
Amazon Redshift Warehousing with Confluent Cloud
Serverless with AWS and Confluent Cloud
Real-time Sentiment Analysis with Confluent
Amazon ElastiCache and Confluent Cloud
confluent.awsworkshop.io
Try it out yourself
© 2023, Amazon Web Services, Inc. or its affiliates.
Learn more
Working with streaming data on AWS
https://aws.amazon.com/streaming-data/
Modern Data Architecture on AWS
https://go.aws/3OJDhFk
Build Modern Data Streaming Analytics
Architectures on AWS
https://go.aws/3bt0HAm
Derive Insights from Modern Data
https://go.aws/3xVU3dn
© 2023, Amazon Web Services, Inc. or its affiliates.
Free trial available through the
AWS Marketplace
$400 in free credits to spend
during your first 30 days
© 2023, Amazon Web Services, Inc. or its affiliates.
© 2023, Amazon Web Services, Inc. or its affiliates.
Thank you!
Nae Ittaboon
ittaboon@amazon.co.th
1 de 21

Recomendados

Disaster Recovery Strategies - AWS Siklab 2022.pptx por
Disaster Recovery Strategies - AWS Siklab 2022.pptxDisaster Recovery Strategies - AWS Siklab 2022.pptx
Disaster Recovery Strategies - AWS Siklab 2022.pptxJohn Louis Garcia
133 vistas18 diapositivas
Getting started on your AWS migration journey por
Getting started on your AWS migration journeyGetting started on your AWS migration journey
Getting started on your AWS migration journeyAmazon Web Services
451 vistas55 diapositivas
Webinar Data Mesh - Part 3 por
Webinar Data Mesh - Part 3Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3Jeffrey T. Pollock
1K vistas18 diapositivas
Cloudamize Platform Training for Azure.pptx por
Cloudamize Platform Training for Azure.pptxCloudamize Platform Training for Azure.pptx
Cloudamize Platform Training for Azure.pptxSasikumarPalanivel3
165 vistas106 diapositivas
AWS Cloud Migration Insights Forum por
AWS Cloud Migration Insights ForumAWS Cloud Migration Insights Forum
AWS Cloud Migration Insights ForumAmazon Web Services
5.2K vistas56 diapositivas
Data Catalog for Better Data Discovery and Governance por
Data Catalog for Better Data Discovery and GovernanceData Catalog for Better Data Discovery and Governance
Data Catalog for Better Data Discovery and GovernanceDenodo
1.9K vistas19 diapositivas

Más contenido relacionado

La actualidad más candente

Building a modern data warehouse por
Building a modern data warehouseBuilding a modern data warehouse
Building a modern data warehouseJames Serra
15.6K vistas57 diapositivas
Databricks Platform.pptx por
Databricks Platform.pptxDatabricks Platform.pptx
Databricks Platform.pptxAlex Ivy
3.2K vistas46 diapositivas
Collibra - Forrester Presentation : Data Governance 2.0 por
Collibra - Forrester Presentation : Data Governance 2.0Collibra - Forrester Presentation : Data Governance 2.0
Collibra - Forrester Presentation : Data Governance 2.0Guillaume LE GALIARD
1.4K vistas19 diapositivas
Get Savvy with Snowflake por
Get Savvy with SnowflakeGet Savvy with Snowflake
Get Savvy with SnowflakeMatillion
583 vistas19 diapositivas
Getting Started with AWS Database Migration Service por
Getting Started with AWS Database Migration ServiceGetting Started with AWS Database Migration Service
Getting Started with AWS Database Migration ServiceAmazon Web Services
6.1K vistas27 diapositivas
SAP Solutions on AWS Overview por
SAP Solutions on AWS Overview SAP Solutions on AWS Overview
SAP Solutions on AWS Overview Amazon Web Services
2.8K vistas19 diapositivas

La actualidad más candente(20)

Building a modern data warehouse por James Serra
Building a modern data warehouseBuilding a modern data warehouse
Building a modern data warehouse
James Serra15.6K vistas
Databricks Platform.pptx por Alex Ivy
Databricks Platform.pptxDatabricks Platform.pptx
Databricks Platform.pptx
Alex Ivy3.2K vistas
Collibra - Forrester Presentation : Data Governance 2.0 por Guillaume LE GALIARD
Collibra - Forrester Presentation : Data Governance 2.0Collibra - Forrester Presentation : Data Governance 2.0
Collibra - Forrester Presentation : Data Governance 2.0
Guillaume LE GALIARD1.4K vistas
Get Savvy with Snowflake por Matillion
Get Savvy with SnowflakeGet Savvy with Snowflake
Get Savvy with Snowflake
Matillion583 vistas
Getting Started with AWS Database Migration Service por Amazon Web Services
Getting Started with AWS Database Migration ServiceGetting Started with AWS Database Migration Service
Getting Started with AWS Database Migration Service
Amazon Web Services6.1K vistas
What is Talend | Talend Tutorial for Beginners | Talend Online Training | Edu... por Edureka!
What is Talend | Talend Tutorial for Beginners | Talend Online Training | Edu...What is Talend | Talend Tutorial for Beginners | Talend Online Training | Edu...
What is Talend | Talend Tutorial for Beginners | Talend Online Training | Edu...
Edureka!381 vistas
Enterprise Architecture vs. Data Architecture por DATAVERSITY
Enterprise Architecture vs. Data ArchitectureEnterprise Architecture vs. Data Architecture
Enterprise Architecture vs. Data Architecture
DATAVERSITY789 vistas
Monitor All Your Things: Amazon CloudWatch in Action with BBC (DEV302) - AWS ... por Amazon Web Services
Monitor All Your Things: Amazon CloudWatch in Action with BBC (DEV302) - AWS ...Monitor All Your Things: Amazon CloudWatch in Action with BBC (DEV302) - AWS ...
Monitor All Your Things: Amazon CloudWatch in Action with BBC (DEV302) - AWS ...
Amazon Web Services4.3K vistas
Cloud Operating Model Design por Joseph Schwartz
Cloud Operating Model DesignCloud Operating Model Design
Cloud Operating Model Design
Joseph Schwartz10.1K vistas
Five Things to Consider About Data Mesh and Data Governance por DATAVERSITY
Five Things to Consider About Data Mesh and Data GovernanceFive Things to Consider About Data Mesh and Data Governance
Five Things to Consider About Data Mesh and Data Governance
DATAVERSITY1.7K vistas
네이버 클라우드 플랫폼의 서비스 전략(공공, Cloud Connect) por KINX
네이버 클라우드 플랫폼의 서비스 전략(공공, Cloud Connect)네이버 클라우드 플랫폼의 서비스 전략(공공, Cloud Connect)
네이버 클라우드 플랫폼의 서비스 전략(공공, Cloud Connect)
KINX1.4K vistas
Azure data platform overview por James Serra
Azure data platform overviewAzure data platform overview
Azure data platform overview
James Serra19.4K vistas
Building the Data Lake with Azure Data Factory and Data Lake Analytics por Khalid Salama
Building the Data Lake with Azure Data Factory and Data Lake AnalyticsBuilding the Data Lake with Azure Data Factory and Data Lake Analytics
Building the Data Lake with Azure Data Factory and Data Lake Analytics
Khalid Salama13K vistas
Understanding DataOps and Its Impact on Application Quality por DevOps.com
Understanding DataOps and Its Impact on Application QualityUnderstanding DataOps and Its Impact on Application Quality
Understanding DataOps and Its Impact on Application Quality
DevOps.com533 vistas

Similar a Building Modern Streaming Analytics with Confluent on AWS

Single View of Data por
Single View of DataSingle View of Data
Single View of Dataconfluent
71 vistas20 diapositivas
Build real-time streaming data pipelines to AWS with Confluent por
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluentconfluent
66 vistas53 diapositivas
Confluent_AWS_ImmersionDay_Q42023.pdf por
Confluent_AWS_ImmersionDay_Q42023.pdfConfluent_AWS_ImmersionDay_Q42023.pdf
Confluent_AWS_ImmersionDay_Q42023.pdfAhmed791434
5 vistas53 diapositivas
Confluent:AWS - GameDay.pptx por
 Confluent:AWS - GameDay.pptx Confluent:AWS - GameDay.pptx
Confluent:AWS - GameDay.pptxAhmed791434
15 vistas67 diapositivas
The Future of Mainframe Is in the Cloud por
The Future of Mainframe Is in the CloudThe Future of Mainframe Is in the Cloud
The Future of Mainframe Is in the CloudPrecisely
35 vistas21 diapositivas
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture por
Serverless Kafka on AWS as Part of a Cloud-native Data Lake ArchitectureServerless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Serverless Kafka on AWS as Part of a Cloud-native Data Lake ArchitectureKai Wähner
4.7K vistas30 diapositivas

Similar a Building Modern Streaming Analytics with Confluent on AWS(20)

Single View of Data por confluent
Single View of DataSingle View of Data
Single View of Data
confluent71 vistas
Build real-time streaming data pipelines to AWS with Confluent por confluent
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluent
confluent66 vistas
Confluent_AWS_ImmersionDay_Q42023.pdf por Ahmed791434
Confluent_AWS_ImmersionDay_Q42023.pdfConfluent_AWS_ImmersionDay_Q42023.pdf
Confluent_AWS_ImmersionDay_Q42023.pdf
Ahmed7914345 vistas
Confluent:AWS - GameDay.pptx por Ahmed791434
 Confluent:AWS - GameDay.pptx Confluent:AWS - GameDay.pptx
Confluent:AWS - GameDay.pptx
Ahmed79143415 vistas
The Future of Mainframe Is in the Cloud por Precisely
The Future of Mainframe Is in the CloudThe Future of Mainframe Is in the Cloud
The Future of Mainframe Is in the Cloud
Precisely35 vistas
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture por Kai Wähner
Serverless Kafka on AWS as Part of a Cloud-native Data Lake ArchitectureServerless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Kai Wähner4.7K vistas
Building real-time serverless data applications with Confluent and AWS - Lond... por Ahmed791434
Building real-time serverless data applications with Confluent and AWS - Lond...Building real-time serverless data applications with Confluent and AWS - Lond...
Building real-time serverless data applications with Confluent and AWS - Lond...
Ahmed7914345 vistas
Building real-time serverless data applications with Confluent and AWS.pptx por Ahmed791434
Building real-time serverless data applications with Confluent and AWS.pptxBuilding real-time serverless data applications with Confluent and AWS.pptx
Building real-time serverless data applications with Confluent and AWS.pptx
Ahmed7914345 vistas
Data Con LA 2022 - Modern Data Strategy por Data Con LA
Data Con LA 2022 - Modern Data StrategyData Con LA 2022 - Modern Data Strategy
Data Con LA 2022 - Modern Data Strategy
Data Con LA19 vistas
AWS reInvent 2022 reCap AI/ML and Data por Chris Fregly
AWS reInvent 2022 reCap AI/ML and DataAWS reInvent 2022 reCap AI/ML and Data
AWS reInvent 2022 reCap AI/ML and Data
Chris Fregly335 vistas
Module 3 - QuickSight Overview por Lam Le
Module 3 - QuickSight OverviewModule 3 - QuickSight Overview
Module 3 - QuickSight Overview
Lam Le273 vistas
Improve Time to Market with Real-Time Analytics on Time-Series Data por Vin Dahake
Improve Time to Market with Real-Time Analytics on Time-Series DataImprove Time to Market with Real-Time Analytics on Time-Series Data
Improve Time to Market with Real-Time Analytics on Time-Series Data
Vin Dahake32 vistas
AWSome Day Singapore Keynote 2015 por Hwee Bee Tan
AWSome Day Singapore Keynote 2015AWSome Day Singapore Keynote 2015
AWSome Day Singapore Keynote 2015
Hwee Bee Tan559 vistas
How TrueCar Gains Actionable Insights with Splunk Cloud PPT por Amazon Web Services
How TrueCar Gains Actionable Insights with Splunk Cloud PPTHow TrueCar Gains Actionable Insights with Splunk Cloud PPT
How TrueCar Gains Actionable Insights with Splunk Cloud PPT
Amazon Web Services607 vistas
Enabling Your Organization’s Amazon Redshift Adoption – Going from Zero to He... por Amazon Web Services
Enabling Your Organization’s Amazon Redshift Adoption – Going from Zero to He...Enabling Your Organization’s Amazon Redshift Adoption – Going from Zero to He...
Enabling Your Organization’s Amazon Redshift Adoption – Going from Zero to He...
Amazon Web Services463 vistas
AWS webinar what is cloud computing 13 09 11 por Amazon Web Services
AWS webinar what is cloud computing 13 09 11AWS webinar what is cloud computing 13 09 11
AWS webinar what is cloud computing 13 09 11
Amazon Web Services1.6K vistas
AWS Webcast - Informatica - Big Data Solutions Showcase por Amazon Web Services
AWS Webcast - Informatica - Big Data Solutions ShowcaseAWS Webcast - Informatica - Big Data Solutions Showcase
AWS Webcast - Informatica - Big Data Solutions Showcase
Amazon Web Services3.3K vistas
ISV Best Practices - AWS Partner Summit Mumbai 2018.pdf por Amazon Web Services
ISV Best Practices - AWS Partner Summit Mumbai 2018.pdfISV Best Practices - AWS Partner Summit Mumbai 2018.pdf
ISV Best Practices - AWS Partner Summit Mumbai 2018.pdf
Amazon Web Services788 vistas

Más de confluent

Citi TechTalk Session 2: Kafka Deep Dive por
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Diveconfluent
17 vistas60 diapositivas
Q&A with Confluent Professional Services: Confluent Service Mesh por
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Meshconfluent
67 vistas69 diapositivas
Citi Tech Talk: Event Driven Kafka Microservices por
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservicesconfluent
23 vistas29 diapositivas
Confluent & GSI Webinars series - Session 3 por
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3confluent
15 vistas59 diapositivas
Citi Tech Talk: Messaging Modernization por
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernizationconfluent
17 vistas39 diapositivas
Citi Tech Talk: Data Governance for streaming and real time data por
Citi Tech Talk: Data Governance for streaming and real time dataCiti Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time dataconfluent
21 vistas24 diapositivas

Más de confluent(20)

Citi TechTalk Session 2: Kafka Deep Dive por confluent
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Dive
confluent17 vistas
Q&A with Confluent Professional Services: Confluent Service Mesh por confluent
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Mesh
confluent67 vistas
Citi Tech Talk: Event Driven Kafka Microservices por confluent
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservices
confluent23 vistas
Confluent & GSI Webinars series - Session 3 por confluent
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3
confluent15 vistas
Citi Tech Talk: Messaging Modernization por confluent
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernization
confluent17 vistas
Citi Tech Talk: Data Governance for streaming and real time data por confluent
Citi Tech Talk: Data Governance for streaming and real time dataCiti Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time data
confluent21 vistas
Confluent & GSI Webinars series: Session 2 por confluent
Confluent & GSI Webinars series: Session 2Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2
confluent15 vistas
Data In Motion Paris 2023 por confluent
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023
confluent224 vistas
The Future of Application Development - API Days - Melbourne 2023 por confluent
The Future of Application Development - API Days - Melbourne 2023The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
confluent68 vistas
The Playful Bond Between REST And Data Streams por confluent
The Playful Bond Between REST And Data StreamsThe Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data Streams
confluent49 vistas
The Journey to Data Mesh with Confluent por confluent
The Journey to Data Mesh with ConfluentThe Journey to Data Mesh with Confluent
The Journey to Data Mesh with Confluent
confluent69 vistas
Citi Tech Talk: Monitoring and Performance por confluent
Citi Tech Talk: Monitoring and PerformanceCiti Tech Talk: Monitoring and Performance
Citi Tech Talk: Monitoring and Performance
confluent39 vistas
Citi Tech Talk Disaster Recovery Solutions Deep Dive por confluent
Citi Tech Talk  Disaster Recovery Solutions Deep DiveCiti Tech Talk  Disaster Recovery Solutions Deep Dive
Citi Tech Talk Disaster Recovery Solutions Deep Dive
confluent66 vistas
Citi Tech Talk: Hybrid Cloud por confluent
Citi Tech Talk: Hybrid CloudCiti Tech Talk: Hybrid Cloud
Citi Tech Talk: Hybrid Cloud
confluent43 vistas
Confluent Partner Tech Talk with QLIK por confluent
Confluent Partner Tech Talk with QLIKConfluent Partner Tech Talk with QLIK
Confluent Partner Tech Talk with QLIK
confluent90 vistas
Real-time Streaming for Government and the Public Sector por confluent
Real-time Streaming for Government and the Public SectorReal-time Streaming for Government and the Public Sector
Real-time Streaming for Government and the Public Sector
confluent41 vistas
Confluent Partner Tech Talk with SVA por confluent
Confluent Partner Tech Talk with SVAConfluent Partner Tech Talk with SVA
Confluent Partner Tech Talk with SVA
confluent95 vistas
How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re... por confluent
How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re...How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re...
How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re...
confluent28 vistas
Leveraging streaming data in real-time to build a Single View of Customer (SVOC) por confluent
Leveraging streaming data in real-time to build a Single View of Customer (SVOC)Leveraging streaming data in real-time to build a Single View of Customer (SVOC)
Leveraging streaming data in real-time to build a Single View of Customer (SVOC)
confluent21 vistas
Real-time Network Streaming Innovation & Insights por confluent
Real-time Network Streaming Innovation & InsightsReal-time Network Streaming Innovation & Insights
Real-time Network Streaming Innovation & Insights
confluent18 vistas

Último

From chaos to control: Managing migrations and Microsoft 365 with ShareGate! por
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!sammart93
9 vistas39 diapositivas
The Research Portal of Catalonia: Growing more (information) & more (services) por
The Research Portal of Catalonia: Growing more (information) & more (services)The Research Portal of Catalonia: Growing more (information) & more (services)
The Research Portal of Catalonia: Growing more (information) & more (services)CSUC - Consorci de Serveis Universitaris de Catalunya
73 vistas25 diapositivas
6g - REPORT.pdf por
6g - REPORT.pdf6g - REPORT.pdf
6g - REPORT.pdfLiveplex
9 vistas23 diapositivas
Microsoft Power Platform.pptx por
Microsoft Power Platform.pptxMicrosoft Power Platform.pptx
Microsoft Power Platform.pptxUni Systems S.M.S.A.
47 vistas38 diapositivas
SAP Automation Using Bar Code and FIORI.pdf por
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdfVirendra Rai, PMP
19 vistas38 diapositivas
DALI Basics Course 2023 por
DALI Basics Course  2023DALI Basics Course  2023
DALI Basics Course 2023Ivory Egg
14 vistas12 diapositivas

Último(20)

From chaos to control: Managing migrations and Microsoft 365 with ShareGate! por sammart93
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
sammart939 vistas
6g - REPORT.pdf por Liveplex
6g - REPORT.pdf6g - REPORT.pdf
6g - REPORT.pdf
Liveplex9 vistas
SAP Automation Using Bar Code and FIORI.pdf por Virendra Rai, PMP
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdf
Virendra Rai, PMP19 vistas
DALI Basics Course 2023 por Ivory Egg
DALI Basics Course  2023DALI Basics Course  2023
DALI Basics Course 2023
Ivory Egg14 vistas
handbook for web 3 adoption.pdf por Liveplex
handbook for web 3 adoption.pdfhandbook for web 3 adoption.pdf
handbook for web 3 adoption.pdf
Liveplex19 vistas
STPI OctaNE CoE Brochure.pdf por madhurjyapb
STPI OctaNE CoE Brochure.pdfSTPI OctaNE CoE Brochure.pdf
STPI OctaNE CoE Brochure.pdf
madhurjyapb12 vistas
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas... por Bernd Ruecker
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
Bernd Ruecker26 vistas
Unit 1_Lecture 2_Physical Design of IoT.pdf por StephenTec
Unit 1_Lecture 2_Physical Design of IoT.pdfUnit 1_Lecture 2_Physical Design of IoT.pdf
Unit 1_Lecture 2_Physical Design of IoT.pdf
StephenTec11 vistas
Special_edition_innovator_2023.pdf por WillDavies22
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdf
WillDavies2216 vistas
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive por Network Automation Forum
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Report 2030 Digital Decade por Massimo Talia
Report 2030 Digital DecadeReport 2030 Digital Decade
Report 2030 Digital Decade
Massimo Talia14 vistas
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院 por IttrainingIttraining
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院

Building Modern Streaming Analytics with Confluent on AWS

  • 1. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. Nae Ittaboon (he/him) Solutions Architect Amazon Web Services (AWS) Building Modern Streaming Analytics with Confluent on AWS #DataInMotionTour
  • 2. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. Agenda 2 ❑ Develop a modern data strategy ❑ Act on events in real time with data streaming ❑ Build seamless streaming with AWS and Confluent ❑ Innovate together to power customer success ❑ Key takeaways
  • 3. © 2023, Amazon Web Services, Inc. or its affiliates. Challenges of data analytics at scale 3 Data volume and velocity Multiple analytics needs Variety of sources and data types Difficult to manage systems Complex to scale Inflexible tools Security, compliance Slow performance Increasing and unpredictable cost
  • 4. © 2023, Amazon Web Services, Inc. or its affiliates. Catalog Governance Data sources People, apps, and devices Analytics Machine learning Databases Data lakes Modern data strategy
  • 5. © 2023, Amazon Web Services, Inc. or its affiliates. Amazon OpenSearch Service Amazon Aurora Amazon EMR Amazon SageMaker Amazon DynamoDB Amazon Redshift Amazon S3 Modern data architecture on AWS Key Pillars Data at any scale Seamless data access and movement Purpose-built for best price performance Built-in ML to solve business challenges Unified governance
  • 6. © 2023, Amazon Web Services, Inc. or its affiliates. The benefits of data lakes Data lake Catalog Store all your data in open formats Decouple storage from compute Cost-effectively scale storage to exabytes Process data in place Choice of analytical and ML engines
  • 7. © 2023, Amazon Web Services, Inc. or its affiliates. Sharing across data movement with Data Mesh Data producers Data mesh Data consumers Unique modern data architecture Suited to business function Teams that want to share data Unique modern data architecture Suited to business function Team that runs the marketplace Teams that want to use data
  • 8. © 2023, Amazon Web Services, Inc. or its affiliates. Source: Perishable insights, Mike Gualtieri, Forrester Real time Seconds Minutes Hours Days Months Value of data to decision-making Preventive/predictive Actionable Reactive Historical Time-critical decisions Traditional “batch” business intelligence Information half-life in decision-making Why real-time data streaming analytics ? Data loses value quickly over time
  • 9. © 2023, Amazon Web Services, Inc. or its affiliates. Common real-time analytics use cases Anomaly and fraud detection Empowering IoT analytics Nourishing marketing campaigns Real-time personalization Tailoring customer experience in real time Supporting healthcare and emergency services
  • 10. © 2023, Amazon Web Services, Inc. or its affiliates. Devices and/or applications that produce real-time data at high velocity Data from tens of thousands of data sources can be written to a single stream Data are stored in the order they were received for a set duration of time and can be replayed indefinitely during that time Records are read in the order they are produced, enabling real- time analytics or streaming ETL Database (NoSQL most common) Data lake Data warehouse Event driven Applications ` Analytics dashboard Real-time streaming analytics pipeline Ingest, Process & Analyze High Volumes of High-Velocity Data from Various Sources in Real Time
  • 11. © 2023, Amazon Web Services, Inc. or its affiliates. Mobile device Metering Click streams IoT sensors AWS SDKs Kinesis Agent/KPL ` Apache Kafka Amazon Kinesis Data Streams Amazon Kinesis Data Firehose Apache Kafka Amazon Kinesis Data Streams Amazon Kinesis Data Analytics AWS Glue Streaming Amazon EMR Amazon DynamoDB Amazon EMR Amazon S3 Amazon Redshift Amazon OpenSearch Amazon QuickSight Apache Kafka Streaming analytics on AWS
  • 12. © 2023, Amazon Web Services, Inc. or its affiliates. Rich front-end customer experiences Real-time Event Streams and Analysis A Sale A shipment A Trade A Customer Experience Real-time backend operations Event streaming with Kafka to set Data in Motion: Continuously processing evolving streams of data in real time with Confluent
  • 13. © 2023, Amazon Web Services, Inc. or its affiliates. Out-of-box integration with popular services Certified and validated by AWS AWS Native Services Top-5 Global ISV for S3 Data Volume 3rd-Party ISV Services • Amazon RDS Ready • AWS Lambda Ready • Amazon Redshift Ready • AWS PrivateLink Ready • AWS Outposts Ready Validated Service Designations Confluent integrations with AWS
  • 14. © 2023, Amazon Web Services, Inc. or its affiliates. S Confluent Cloud on AWS – reference architecture S3
  • 15. © 2023, Amazon Web Services, Inc. or its affiliates. Confluent + AWS: Accelerate Customer Business Outcomes Topline Impacting New Experiences ● Event-driven & real-time ● Unify data across org. w/ Kafka data fabric (Schema Reg,..) ● AWS Analytics, Redshift, ML connectors Mitigate Risk ● Higher Service Quality & Resilience with 99.99% SLA ● Deep Kafka expertise & innovation ● Elastic billing/pricing Developer Agility ● Focus on innovation (not data infrastructure) ● Leverage full Kafka OSS ecosystem + AWS services Faster Time to Market ● ~50-75% faster time to market* ● Streamline hybrid cloud migration with no complex lift-n- shift ● Maintain business continuity Lower Kafka TCO ● ~25-50% lower TCO * ● GBps-scale & fast deployments for global expansion ● Deploy Kafka at scale in 1 week Maximize ROI ● ~200% ROI per Forrester study ● Save 10s of $Ms with legacy offload to AWS with Confluent Replicator * For customers that don’t already have Kafka based system in-market * TCO assessment to be analyzed for specific customer scenarios
  • 16. © 2023, Amazon Web Services, Inc. or its affiliates. Accelerate modernization from on-prem to AWS Redshift Sink Lambda Sink AWS Direct Connect Replicator Legacy Applications Mainframe Legacy Data Systems JDBC / CDC connectors Connect Leverage 120+ Confluent pre-built connectors Modernize Value added apps, increase agility, reduce TCO On-prem AWS Cloud Bridge Hybrid cloud streaming Amazon Athena AWS Glue Amazon SageMaker AWS Lake Formation Amazon DynamoDB Amazon Amazon Aurora S3 Sink Data Streams Apps ksqlDB
  • 17. © 2023, Amazon Web Services, Inc. or its affiliates. Increase developer agility & speed of innovation Serverless integration Connect for effortless integrations with Lambda & data stores AWS serverless platform Free up backend operation & Infras. management Apps Microservices ksqlDB Schema Registry COMPUTE AWS Lambda Data stores REST Proxy & Clients Source Connectors Lambda Sink DATA STORES Amazon DynamoDB Amazon Aurora STORAGE Amazon S3 S3 Sink ANALYTICS Amazon Athena Amazon Redshift
  • 18. © 2023, Amazon Web Services, Inc. or its affiliates. Amazon Redshift Warehousing with Confluent Cloud Serverless with AWS and Confluent Cloud Real-time Sentiment Analysis with Confluent Amazon ElastiCache and Confluent Cloud confluent.awsworkshop.io Try it out yourself
  • 19. © 2023, Amazon Web Services, Inc. or its affiliates. Learn more Working with streaming data on AWS https://aws.amazon.com/streaming-data/ Modern Data Architecture on AWS https://go.aws/3OJDhFk Build Modern Data Streaming Analytics Architectures on AWS https://go.aws/3bt0HAm Derive Insights from Modern Data https://go.aws/3xVU3dn
  • 20. © 2023, Amazon Web Services, Inc. or its affiliates. Free trial available through the AWS Marketplace $400 in free credits to spend during your first 30 days
  • 21. © 2023, Amazon Web Services, Inc. or its affiliates. © 2023, Amazon Web Services, Inc. or its affiliates. Thank you! Nae Ittaboon ittaboon@amazon.co.th