Data in Motion Tour ANZ Sydney 2023 Keynote.pdf

confluent
confluentconfluent
Data in Motion Tour ANZ Sydney 2023 Keynote.pdf
Welcome Address
Jade Bowen
Senior Manager, Customer Success, ANZ
Confluent
Kai Waehner
Field CTO, Confluent
Jun Rao
Co-founder,
Confluent
John Heaton
Chief Technology Officer
Alex Bank
Gayle Amara
Principal Architect Living
Network, Optus
Matt Moore
Sr Manager Technology
Partnerships
AWS
James Gollan
nterprise Solutions
Engineer ANZ, Confluent
Saul Caganoff
Principal Cloud
Engineering, Deloitte
Diego Bayona
Principal Software
Engineer Service NSW
Stuart Ward
Financial Services Industry
Lead APAC, Confluent
Eric Tschetter
Field CTO, Imply
Engin Cukuroglu
Manager- Solutions
Engineering, Confluent
Warren Vella
Solution Engineer, Confluent
REAL-TIME DATA = REAL-WORLD BUSINESS VALUE
● 74% of organisations saw 2-5x ROI
● 89% said data streaming is top priority
for IT investments
● 76% found team and project silos a
challenge
3 Key Findings for APAC
Data in Motion Tour ANZ Sydney 2023 Keynote.pdf
Get your passport card stamped at each sponsor table for a chance to win a
$300 Gift Card*
!
Passport Card
*Must be present to win. Raffle will occur during afternoon reception
*Must be present to win. Raffle will occur during afternoon reception
Reinventing Kafka in the Data Streaming Era
Jun Rao
Co-founder, Confluent
Loyalty Rewards
Curbside Pickup
Trending Now
Popular on Netflix
Top Picks for Joshua
Created by the founders of
Confluent while at LinkedIn
Apache Kafka has ushered in the
data streaming era…
>70%
of the Fortune 500
>100,000+
Organizations
>41,000
Kafka Meetup Attendees
>200
Global Meetup Groups
>750
Kafka Improvement Proposals (KIPs)
>12,000
Jiras for Apache Kafka
>32,000
Stack Overflow Questions
Real-time Trades
Ride ETA
Personalized Recommendations
Data in Motion Tour ANZ Sydney 2023 Keynote.pdf
Data in Motion Tour ANZ Sydney 2023 Keynote.pdf
Data in Motion Tour ANZ Sydney 2023 Keynote.pdf
The need for a cloud-native, data streaming platform
Connecting all your apps, systems and data into a central nervous system
Self-managing Kafka
comes with cost and
complexity
● Cluster sizing
● Cluster provisioning
● Architecture planning
● Upgrades / patches
● Source / sink connectors
● Monitoring / reporting
● Security / reliability
management
● Mirroring
● Expansion planning
● Load rebalancing
● Utilization optimization
● Cluster migrations
● Performance enhancements
● Zookeeper management
● Multi-tenancy
Each new project adds complexity…
Client 1
…
Client n
Networking spend for
increased throughput
LB
Storage
Disks
Broker 1 Broker 2 Broker n
Excess, underutilized
storage/compute
Disk n
…
…
Example of a typical Kafka deployment in the cloud
Resources to manage
infrastructure failures
PUTTING KAFKA IN THE CLOUD…
ISN’T JUST PUTTING KAFKA IN THE CLOUD.
Confluent Cloud
COMPLETE EVERYWHERE
CLOUD-NATIVE
Kora: the Cloud-Native Engine for Apache Kafka
Serverless
● Elastic scaling up & down from
0 to GBps
● Auto capacity mgmt, load
balancing, and upgrades
Infinite Storage
● Store data cost- effectively at
any scale without growing
compute
Resilience
● Multi-AZ and multi-region
replication
● Durability self-validation
High Availability
● 99.99% SLA
● Multi-region / AZ availability across
cloud providers
● Patches deployed in Confluent
Cloud before Apache Kafka
Network Flexibility
● Public, VPC, and Private Link
● Seamlessly link across clouds
and on-prem with Cluster
Linking
How Kafka Stores Data
Application Application Application Application Application
Cost
Elasticity
Broker Broker Broker
Infinite Storage with Confluent Cloud
Broker Broker
Application Application Application Application
Cost
efficiency
Improved
Elasticity
Broker
local
hotset
remote
local
hotset
remote
local
hotset
remote
Object Storage
Broker
local
hotset
remote
Benefits from Cloud Native
30XELASTICITY
50
40
30
20
10
0
OSS Kafka Confluent Cloud
Hours required to scale 3 brokers to 4, replication factor of 3,
30-day retention, 100 MBps throughput, 10GBps network
100X RESILIENCY -X STORAGE
0
22.5 45 67.5 90
Confluent
Cloud
OSS Kafka
Other Kafka
Service
99.99%2
99.9%
99%1
10X 100X
Minimum downtime commitment by
Kafka service based on SLA
Infinite Storage
AWS GA
Infinite Storage
GCP GA
Infinite Storage
Azure GA
Time
Hours required to scale 3 brokers to 4, replication factor of 3,
30-day retention, 100 MBps throughput, 10GBps network
AVG Storage
per Cluster
AWS
GCP
AZURE
Performance benefits too!
Large performance gains are realized too,
with an open to path to even more.
CCloud (Next-gen
Replication Engine)
CCloud (Present)
Apache Kafka
No more spending on excess, wasted resources - storage
and compute costs scale up and down with demand
Solving all aspects of the Kafka infra costs with networking
optimizations that solve 90% of the Kafka TCO problem
Common
Breakdown
Of Kafka Infra Costs
(1GBps, 3x fanout)
Networking: 90.8% Compute: 3.0% Storage: 6.2%
Confluent has the lowest TCO for Apache Kafka
Our fully managed SaaS offering solves all aspects of the TCO challenge for Apache Kafka
COMPLETE EVERYWHERE
CLOUD-NATIVE
Confluent Cloud
70+ FULLY MANAGED CONNECTORS
Amazon S3
Custom Connectors
ANNOUNCING
You need to trust and understand the data moving
through your data streaming platform
Schemas Data Lineage
Metadata Integrity & Validation
Email Address →
→
→
Does it contain @ and domain?
Is the right digits given the country?
Is it the right length and format given the country?
Data Quality Rules in Action
Online Retail Example
Postal Code
Phone Number
val fraudulentPayments: KStream[String, Payment] = builder
.stream[String, Payment](“payments-kafka-topic”)
.filter((_ ,payment) => payment.fraudProbability > 0.8)
fraudulentPayments.to(“fraudulent-payments-topic”)
CREATE STREAM fraudulent_payments AS
SELECT * FROM payments
WHERE fraudProbability > 0.8;
STANDALONE APPS FRAMEWORKS
SPRING 2023 FALL 2023 WINTER 2023 2024
We plan to GA our SQL Flink service
in Winter 2023
SQL Early Access SQL Public Preview SQL General Availability Java + Python
Provision a new cluster
*Must be present to win. Raffle will occur during afternoon reception
ANNOUNCING STREAM SHARING
Easily share real-time data
without delays in a few
clicks from Confluent to any
Kafka client
Safely share and protect your data
with robust authenticated sharing,
access management, and layered
encryption controls
Trust the quality and compatibility
of shared data by enforcing
consistent schemas across users,
teams, and organizations
Confluent Cloud
COMPLETE EVERYWHERE
CLOUD-NATIVE
Any Cloud. Any Region.
Everywhere You Want To Be
Why Confluent is the world’s most trusted
data streaming platform
Focus & Expertise
Only company focused on data in motion:
● Founded in 2014 by the Original Creators of
Apache Kafka
● Over 80% of Kafka commits are by
Confluent employees
● Advised on thousands of real-world Kafka
deployments across a wide range of
patterns & industries
Building and supporting a world class product:
● >9 million engineering hours spent
building building our product
● We internally manage >15,000 clusters (and
counting) in Confluent Cloud
● Over 1 million cumulative hours of Kafka
expertise within Confluent support &
services
Execution at Scale
Start your free trial of
Confluent Cloud & get $400 in
credits
Get started with Confluent Cloud is easy!
confluent.io/confluent-cloud/tryfree/
Request a personalized TCO
assessment for your current
Kafka needs or future Data
Streaming Platform
Thank You
1 de 36

Recomendados

Reinventing Kafka in the Data Streaming Era - Jun Rao por
Reinventing Kafka in the Data Streaming Era - Jun RaoReinventing Kafka in the Data Streaming Era - Jun Rao
Reinventing Kafka in the Data Streaming Era - Jun Raoconfluent
143 vistas30 diapositivas
Gartner IT Symposium 2013: Delivering IT-as-a-Service with Cloud Brokering an... por
Gartner IT Symposium 2013: Delivering IT-as-a-Service with Cloud Brokering an...Gartner IT Symposium 2013: Delivering IT-as-a-Service with Cloud Brokering an...
Gartner IT Symposium 2013: Delivering IT-as-a-Service with Cloud Brokering an...Gravitant, Inc.
2.2K vistas9 diapositivas
Confluent:AWS - GameDay.pptx por
 Confluent:AWS - GameDay.pptx Confluent:AWS - GameDay.pptx
Confluent:AWS - GameDay.pptxAhmed791434
17 vistas67 diapositivas
Data In Motion Paris 2023 por
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023confluent
233 vistas278 diapositivas
Skysphere leverage the cloud por
Skysphere leverage the cloudSkysphere leverage the cloud
Skysphere leverage the cloudCornerStone
205 vistas23 diapositivas
Build real-time streaming data pipelines to AWS with Confluent por
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluentconfluent
73 vistas53 diapositivas

Más contenido relacionado

Similar a Data in Motion Tour ANZ Sydney 2023 Keynote.pdf

Streaming Time Series Data With Kenny Gorman and Elena Cuevas | Current 2022 por
Streaming Time Series Data With Kenny Gorman and Elena Cuevas | Current 2022Streaming Time Series Data With Kenny Gorman and Elena Cuevas | Current 2022
Streaming Time Series Data With Kenny Gorman and Elena Cuevas | Current 2022HostedbyConfluent
341 vistas35 diapositivas
Bridge Your Kafka Streams to Azure Webinar por
Bridge Your Kafka Streams to Azure WebinarBridge Your Kafka Streams to Azure Webinar
Bridge Your Kafka Streams to Azure Webinarconfluent
408 vistas41 diapositivas
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture por
Serverless Kafka on AWS as Part of a Cloud-native Data Lake ArchitectureServerless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Serverless Kafka on AWS as Part of a Cloud-native Data Lake ArchitectureKai Wähner
4.7K vistas30 diapositivas
A New Approach to Continuous Monitoring in the Cloud por
A New Approach to Continuous Monitoring in the CloudA New Approach to Continuous Monitoring in the Cloud
A New Approach to Continuous Monitoring in the CloudNETSCOUT
1.3K vistas24 diapositivas
AWS Keynote 1 - AWS Enterprise Summit - Hong Kong por
AWS Keynote 1 - AWS Enterprise Summit - Hong KongAWS Keynote 1 - AWS Enterprise Summit - Hong Kong
AWS Keynote 1 - AWS Enterprise Summit - Hong KongAmazon Web Services
574 vistas34 diapositivas
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself por
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it YourselfWhy Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it YourselfDATAVERSITY
522 vistas32 diapositivas

Similar a Data in Motion Tour ANZ Sydney 2023 Keynote.pdf(20)

Streaming Time Series Data With Kenny Gorman and Elena Cuevas | Current 2022 por HostedbyConfluent
Streaming Time Series Data With Kenny Gorman and Elena Cuevas | Current 2022Streaming Time Series Data With Kenny Gorman and Elena Cuevas | Current 2022
Streaming Time Series Data With Kenny Gorman and Elena Cuevas | Current 2022
HostedbyConfluent341 vistas
Bridge Your Kafka Streams to Azure Webinar por confluent
Bridge Your Kafka Streams to Azure WebinarBridge Your Kafka Streams to Azure Webinar
Bridge Your Kafka Streams to Azure Webinar
confluent408 vistas
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture por Kai Wähner
Serverless Kafka on AWS as Part of a Cloud-native Data Lake ArchitectureServerless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Serverless Kafka on AWS as Part of a Cloud-native Data Lake Architecture
Kai Wähner4.7K vistas
A New Approach to Continuous Monitoring in the Cloud por NETSCOUT
A New Approach to Continuous Monitoring in the CloudA New Approach to Continuous Monitoring in the Cloud
A New Approach to Continuous Monitoring in the Cloud
NETSCOUT1.3K vistas
AWS Keynote 1 - AWS Enterprise Summit - Hong Kong por Amazon Web Services
AWS Keynote 1 - AWS Enterprise Summit - Hong KongAWS Keynote 1 - AWS Enterprise Summit - Hong Kong
AWS Keynote 1 - AWS Enterprise Summit - Hong Kong
Amazon Web Services574 vistas
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself por DATAVERSITY
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it YourselfWhy Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself
Why Cloud-Native Kafka Matters: 4 Reasons to Stop Managing it Yourself
DATAVERSITY522 vistas
Unlock value with Confluent and AWS.pptx por Ahmed791434
Unlock value with Confluent and AWS.pptxUnlock value with Confluent and AWS.pptx
Unlock value with Confluent and AWS.pptx
Ahmed79143413 vistas
Building real-time serverless data applications with Confluent and AWS.pptx por Ahmed791434
Building real-time serverless data applications with Confluent and AWS.pptxBuilding real-time serverless data applications with Confluent and AWS.pptx
Building real-time serverless data applications with Confluent and AWS.pptx
Ahmed7914345 vistas
How Discovery Migrated 80% of Their IT to AWS with Cloudreach por Amazon Web Services
How Discovery Migrated 80% of Their IT to AWS with CloudreachHow Discovery Migrated 80% of Their IT to AWS with Cloudreach
How Discovery Migrated 80% of Their IT to AWS with Cloudreach
Amazon Web Services1.5K vistas
Building real-time serverless data applications with Confluent and AWS - Lond... por Ahmed791434
Building real-time serverless data applications with Confluent and AWS - Lond...Building real-time serverless data applications with Confluent and AWS - Lond...
Building real-time serverless data applications with Confluent and AWS - Lond...
Ahmed7914345 vistas
Leverage the cloud 2012 li por CornerStone
Leverage the cloud 2012 liLeverage the cloud 2012 li
Leverage the cloud 2012 li
CornerStone286 vistas
Flexiant and 451 Research Discuss the Service Provider Path from VPS to IaaS por Flexiant
Flexiant and 451 Research Discuss the Service Provider Path from VPS to IaaSFlexiant and 451 Research Discuss the Service Provider Path from VPS to IaaS
Flexiant and 451 Research Discuss the Service Provider Path from VPS to IaaS
Flexiant801 vistas
The SnapLogic Integration Cloud for ServiceNow por SnapLogic
The SnapLogic Integration Cloud for ServiceNowThe SnapLogic Integration Cloud for ServiceNow
The SnapLogic Integration Cloud for ServiceNow
SnapLogic1.7K vistas
Building Modern Applications on AWS.pptx por Nelson Kimathi
Building Modern Applications on AWS.pptxBuilding Modern Applications on AWS.pptx
Building Modern Applications on AWS.pptx
Nelson Kimathi28 vistas
Confluent & GSI Webinars series - Session 3 por confluent
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3
confluent17 vistas
Webinar: How Microsoft is changing the game with Windows Azure por Common Sense
Webinar: How Microsoft is changing the game with Windows AzureWebinar: How Microsoft is changing the game with Windows Azure
Webinar: How Microsoft is changing the game with Windows Azure
Common Sense1.2K vistas
Keepler | Full-Stack Serverless Applications on GCP por Keepler Data Tech
Keepler | Full-Stack Serverless Applications on GCPKeepler | Full-Stack Serverless Applications on GCP
Keepler | Full-Stack Serverless Applications on GCP
Keepler Data Tech278 vistas
Microsoft Windows Azure - Platfrom Appfabric Service Bus And Access Control P... por Microsoft Private Cloud
Microsoft Windows Azure - Platfrom Appfabric Service Bus And Access Control P...Microsoft Windows Azure - Platfrom Appfabric Service Bus And Access Control P...
Microsoft Windows Azure - Platfrom Appfabric Service Bus And Access Control P...
Three Key Steps for Moving Your Branches to the Cloud por Zscaler
Three Key Steps for Moving Your Branches to the CloudThree Key Steps for Moving Your Branches to the Cloud
Three Key Steps for Moving Your Branches to the Cloud
Zscaler648 vistas

Más de confluent

Citi TechTalk Session 2: Kafka Deep Dive por
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Diveconfluent
19 vistas60 diapositivas
Q&A with Confluent Professional Services: Confluent Service Mesh por
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Meshconfluent
67 vistas69 diapositivas
Citi Tech Talk: Event Driven Kafka Microservices por
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservicesconfluent
25 vistas29 diapositivas
Citi Tech Talk: Messaging Modernization por
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernizationconfluent
17 vistas39 diapositivas
Citi Tech Talk: Data Governance for streaming and real time data por
Citi Tech Talk: Data Governance for streaming and real time dataCiti Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time dataconfluent
21 vistas24 diapositivas
Confluent & GSI Webinars series: Session 2 por
Confluent & GSI Webinars series: Session 2Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2confluent
16 vistas31 diapositivas

Más de confluent(20)

Citi TechTalk Session 2: Kafka Deep Dive por confluent
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Dive
confluent19 vistas
Q&A with Confluent Professional Services: Confluent Service Mesh por confluent
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Mesh
confluent67 vistas
Citi Tech Talk: Event Driven Kafka Microservices por confluent
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservices
confluent25 vistas
Citi Tech Talk: Messaging Modernization por confluent
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernization
confluent17 vistas
Citi Tech Talk: Data Governance for streaming and real time data por confluent
Citi Tech Talk: Data Governance for streaming and real time dataCiti Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time data
confluent21 vistas
Confluent & GSI Webinars series: Session 2 por confluent
Confluent & GSI Webinars series: Session 2Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2
confluent16 vistas
The Future of Application Development - API Days - Melbourne 2023 por confluent
The Future of Application Development - API Days - Melbourne 2023The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
confluent68 vistas
The Playful Bond Between REST And Data Streams por confluent
The Playful Bond Between REST And Data StreamsThe Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data Streams
confluent49 vistas
The Journey to Data Mesh with Confluent por confluent
The Journey to Data Mesh with ConfluentThe Journey to Data Mesh with Confluent
The Journey to Data Mesh with Confluent
confluent73 vistas
Citi Tech Talk: Monitoring and Performance por confluent
Citi Tech Talk: Monitoring and PerformanceCiti Tech Talk: Monitoring and Performance
Citi Tech Talk: Monitoring and Performance
confluent47 vistas
Citi Tech Talk Disaster Recovery Solutions Deep Dive por confluent
Citi Tech Talk  Disaster Recovery Solutions Deep DiveCiti Tech Talk  Disaster Recovery Solutions Deep Dive
Citi Tech Talk Disaster Recovery Solutions Deep Dive
confluent68 vistas
Citi Tech Talk: Hybrid Cloud por confluent
Citi Tech Talk: Hybrid CloudCiti Tech Talk: Hybrid Cloud
Citi Tech Talk: Hybrid Cloud
confluent43 vistas
Confluent Partner Tech Talk with QLIK por confluent
Confluent Partner Tech Talk with QLIKConfluent Partner Tech Talk with QLIK
Confluent Partner Tech Talk with QLIK
confluent90 vistas
Real-time Streaming for Government and the Public Sector por confluent
Real-time Streaming for Government and the Public SectorReal-time Streaming for Government and the Public Sector
Real-time Streaming for Government and the Public Sector
confluent41 vistas
Confluent Partner Tech Talk with SVA por confluent
Confluent Partner Tech Talk with SVAConfluent Partner Tech Talk with SVA
Confluent Partner Tech Talk with SVA
confluent95 vistas
How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re... por confluent
How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re...How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re...
How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re...
confluent28 vistas
Single View of Data por confluent
Single View of DataSingle View of Data
Single View of Data
confluent71 vistas
Leveraging streaming data in real-time to build a Single View of Customer (SVOC) por confluent
Leveraging streaming data in real-time to build a Single View of Customer (SVOC)Leveraging streaming data in real-time to build a Single View of Customer (SVOC)
Leveraging streaming data in real-time to build a Single View of Customer (SVOC)
confluent21 vistas
Real-time Network Streaming Innovation & Insights por confluent
Real-time Network Streaming Innovation & InsightsReal-time Network Streaming Innovation & Insights
Real-time Network Streaming Innovation & Insights
confluent18 vistas
Smart Digital Receipts OnePass por confluent
Smart Digital Receipts OnePassSmart Digital Receipts OnePass
Smart Digital Receipts OnePass
confluent50 vistas

Último

Data Integrity for Banking and Financial Services por
Data Integrity for Banking and Financial ServicesData Integrity for Banking and Financial Services
Data Integrity for Banking and Financial ServicesPrecisely
78 vistas26 diapositivas
Igniting Next Level Productivity with AI-Infused Data Integration Workflows por
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
385 vistas86 diapositivas
Keynote Talk: Open Source is Not Dead - Charles Schulz - Vates por
Keynote Talk: Open Source is Not Dead - Charles Schulz - VatesKeynote Talk: Open Source is Not Dead - Charles Schulz - Vates
Keynote Talk: Open Source is Not Dead - Charles Schulz - VatesShapeBlue
210 vistas15 diapositivas
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O... por
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...ShapeBlue
88 vistas13 diapositivas
Confidence in CloudStack - Aron Wagner, Nathan Gleason - Americ por
Confidence in CloudStack - Aron Wagner, Nathan Gleason - AmericConfidence in CloudStack - Aron Wagner, Nathan Gleason - Americ
Confidence in CloudStack - Aron Wagner, Nathan Gleason - AmericShapeBlue
88 vistas9 diapositivas
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue por
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlueShapeBlue
103 vistas23 diapositivas

Último(20)

Data Integrity for Banking and Financial Services por Precisely
Data Integrity for Banking and Financial ServicesData Integrity for Banking and Financial Services
Data Integrity for Banking and Financial Services
Precisely78 vistas
Igniting Next Level Productivity with AI-Infused Data Integration Workflows por Safe Software
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software385 vistas
Keynote Talk: Open Source is Not Dead - Charles Schulz - Vates por ShapeBlue
Keynote Talk: Open Source is Not Dead - Charles Schulz - VatesKeynote Talk: Open Source is Not Dead - Charles Schulz - Vates
Keynote Talk: Open Source is Not Dead - Charles Schulz - Vates
ShapeBlue210 vistas
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O... por ShapeBlue
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...
Declarative Kubernetes Cluster Deployment with Cloudstack and Cluster API - O...
ShapeBlue88 vistas
Confidence in CloudStack - Aron Wagner, Nathan Gleason - Americ por ShapeBlue
Confidence in CloudStack - Aron Wagner, Nathan Gleason - AmericConfidence in CloudStack - Aron Wagner, Nathan Gleason - Americ
Confidence in CloudStack - Aron Wagner, Nathan Gleason - Americ
ShapeBlue88 vistas
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue por ShapeBlue
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue
ShapeBlue103 vistas
Updates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBIT por ShapeBlue
Updates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBITUpdates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBIT
Updates on the LINSTOR Driver for CloudStack - Rene Peinthor - LINBIT
ShapeBlue166 vistas
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda... por ShapeBlue
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...
ShapeBlue120 vistas
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas... por Bernd Ruecker
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
Bernd Ruecker50 vistas
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R... por ShapeBlue
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
ShapeBlue132 vistas
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive por Network Automation Forum
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ... por ShapeBlue
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...
ShapeBlue144 vistas
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue por ShapeBlue
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlueElevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue
Elevating Privacy and Security in CloudStack - Boris Stoyanov - ShapeBlue
ShapeBlue179 vistas
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit... por ShapeBlue
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
ShapeBlue117 vistas
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue por ShapeBlue
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlueVNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue
ShapeBlue163 vistas
"Surviving highload with Node.js", Andrii Shumada por Fwdays
"Surviving highload with Node.js", Andrii Shumada "Surviving highload with Node.js", Andrii Shumada
"Surviving highload with Node.js", Andrii Shumada
Fwdays53 vistas
The Power of Heat Decarbonisation Plans in the Built Environment por IES VE
The Power of Heat Decarbonisation Plans in the Built EnvironmentThe Power of Heat Decarbonisation Plans in the Built Environment
The Power of Heat Decarbonisation Plans in the Built Environment
IES VE69 vistas
CloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlue por ShapeBlue
CloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlueCloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlue
CloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlue
ShapeBlue94 vistas

Data in Motion Tour ANZ Sydney 2023 Keynote.pdf

  • 2. Welcome Address Jade Bowen Senior Manager, Customer Success, ANZ Confluent
  • 3. Kai Waehner Field CTO, Confluent Jun Rao Co-founder, Confluent John Heaton Chief Technology Officer Alex Bank Gayle Amara Principal Architect Living Network, Optus Matt Moore Sr Manager Technology Partnerships AWS James Gollan nterprise Solutions Engineer ANZ, Confluent Saul Caganoff Principal Cloud Engineering, Deloitte Diego Bayona Principal Software Engineer Service NSW Stuart Ward Financial Services Industry Lead APAC, Confluent Eric Tschetter Field CTO, Imply Engin Cukuroglu Manager- Solutions Engineering, Confluent Warren Vella Solution Engineer, Confluent
  • 4. REAL-TIME DATA = REAL-WORLD BUSINESS VALUE ● 74% of organisations saw 2-5x ROI ● 89% said data streaming is top priority for IT investments ● 76% found team and project silos a challenge 3 Key Findings for APAC
  • 6. Get your passport card stamped at each sponsor table for a chance to win a $300 Gift Card* ! Passport Card *Must be present to win. Raffle will occur during afternoon reception
  • 7. *Must be present to win. Raffle will occur during afternoon reception
  • 8. Reinventing Kafka in the Data Streaming Era Jun Rao Co-founder, Confluent
  • 9. Loyalty Rewards Curbside Pickup Trending Now Popular on Netflix Top Picks for Joshua Created by the founders of Confluent while at LinkedIn Apache Kafka has ushered in the data streaming era… >70% of the Fortune 500 >100,000+ Organizations >41,000 Kafka Meetup Attendees >200 Global Meetup Groups >750 Kafka Improvement Proposals (KIPs) >12,000 Jiras for Apache Kafka >32,000 Stack Overflow Questions Real-time Trades Ride ETA Personalized Recommendations
  • 13. The need for a cloud-native, data streaming platform Connecting all your apps, systems and data into a central nervous system
  • 14. Self-managing Kafka comes with cost and complexity ● Cluster sizing ● Cluster provisioning ● Architecture planning ● Upgrades / patches ● Source / sink connectors ● Monitoring / reporting ● Security / reliability management ● Mirroring ● Expansion planning ● Load rebalancing ● Utilization optimization ● Cluster migrations ● Performance enhancements ● Zookeeper management ● Multi-tenancy Each new project adds complexity… Client 1 … Client n Networking spend for increased throughput LB Storage Disks Broker 1 Broker 2 Broker n Excess, underutilized storage/compute Disk n … … Example of a typical Kafka deployment in the cloud Resources to manage infrastructure failures
  • 15. PUTTING KAFKA IN THE CLOUD… ISN’T JUST PUTTING KAFKA IN THE CLOUD.
  • 17. Kora: the Cloud-Native Engine for Apache Kafka Serverless ● Elastic scaling up & down from 0 to GBps ● Auto capacity mgmt, load balancing, and upgrades Infinite Storage ● Store data cost- effectively at any scale without growing compute Resilience ● Multi-AZ and multi-region replication ● Durability self-validation High Availability ● 99.99% SLA ● Multi-region / AZ availability across cloud providers ● Patches deployed in Confluent Cloud before Apache Kafka Network Flexibility ● Public, VPC, and Private Link ● Seamlessly link across clouds and on-prem with Cluster Linking
  • 18. How Kafka Stores Data Application Application Application Application Application Cost Elasticity Broker Broker Broker
  • 19. Infinite Storage with Confluent Cloud Broker Broker Application Application Application Application Cost efficiency Improved Elasticity Broker local hotset remote local hotset remote local hotset remote Object Storage Broker local hotset remote
  • 20. Benefits from Cloud Native 30XELASTICITY 50 40 30 20 10 0 OSS Kafka Confluent Cloud Hours required to scale 3 brokers to 4, replication factor of 3, 30-day retention, 100 MBps throughput, 10GBps network 100X RESILIENCY -X STORAGE 0 22.5 45 67.5 90 Confluent Cloud OSS Kafka Other Kafka Service 99.99%2 99.9% 99%1 10X 100X Minimum downtime commitment by Kafka service based on SLA Infinite Storage AWS GA Infinite Storage GCP GA Infinite Storage Azure GA Time Hours required to scale 3 brokers to 4, replication factor of 3, 30-day retention, 100 MBps throughput, 10GBps network AVG Storage per Cluster AWS GCP AZURE
  • 21. Performance benefits too! Large performance gains are realized too, with an open to path to even more. CCloud (Next-gen Replication Engine) CCloud (Present) Apache Kafka
  • 22. No more spending on excess, wasted resources - storage and compute costs scale up and down with demand Solving all aspects of the Kafka infra costs with networking optimizations that solve 90% of the Kafka TCO problem Common Breakdown Of Kafka Infra Costs (1GBps, 3x fanout) Networking: 90.8% Compute: 3.0% Storage: 6.2% Confluent has the lowest TCO for Apache Kafka Our fully managed SaaS offering solves all aspects of the TCO challenge for Apache Kafka
  • 24. 70+ FULLY MANAGED CONNECTORS Amazon S3
  • 26. You need to trust and understand the data moving through your data streaming platform Schemas Data Lineage Metadata Integrity & Validation
  • 27. Email Address → → → Does it contain @ and domain? Is the right digits given the country? Is it the right length and format given the country? Data Quality Rules in Action Online Retail Example Postal Code Phone Number
  • 28. val fraudulentPayments: KStream[String, Payment] = builder .stream[String, Payment](“payments-kafka-topic”) .filter((_ ,payment) => payment.fraudProbability > 0.8) fraudulentPayments.to(“fraudulent-payments-topic”) CREATE STREAM fraudulent_payments AS SELECT * FROM payments WHERE fraudProbability > 0.8; STANDALONE APPS FRAMEWORKS
  • 29. SPRING 2023 FALL 2023 WINTER 2023 2024 We plan to GA our SQL Flink service in Winter 2023 SQL Early Access SQL Public Preview SQL General Availability Java + Python
  • 30. Provision a new cluster *Must be present to win. Raffle will occur during afternoon reception
  • 31. ANNOUNCING STREAM SHARING Easily share real-time data without delays in a few clicks from Confluent to any Kafka client Safely share and protect your data with robust authenticated sharing, access management, and layered encryption controls Trust the quality and compatibility of shared data by enforcing consistent schemas across users, teams, and organizations
  • 33. Any Cloud. Any Region. Everywhere You Want To Be
  • 34. Why Confluent is the world’s most trusted data streaming platform Focus & Expertise Only company focused on data in motion: ● Founded in 2014 by the Original Creators of Apache Kafka ● Over 80% of Kafka commits are by Confluent employees ● Advised on thousands of real-world Kafka deployments across a wide range of patterns & industries Building and supporting a world class product: ● >9 million engineering hours spent building building our product ● We internally manage >15,000 clusters (and counting) in Confluent Cloud ● Over 1 million cumulative hours of Kafka expertise within Confluent support & services Execution at Scale
  • 35. Start your free trial of Confluent Cloud & get $400 in credits Get started with Confluent Cloud is easy! confluent.io/confluent-cloud/tryfree/ Request a personalized TCO assessment for your current Kafka needs or future Data Streaming Platform