Real-time Streaming for Government and the Public Sector

confluent
confluentconfluent
Data Streaming in Government
Will LaForest
Field CTO
will@confluent.io
Data is the Competitive Advantage
Trending Now
Popular on Netflix
Top Picks for Joshua
Curbside Pickup
Loyalty Rewards
Personalized Recommendations
Real-time Trades
Ride ETA
Product of Evidence Act
• Passed January 2019
• Evidence Based Government
All agencies expected to Adhere
Mission Statement
Fully leverage the value of federal data
for mission, service, and the public
good by guiding the Federal
Government in practicing ethical
governance, conscious design, and a
learning culture.
Data as a Strategic Asset
Data is the Competitive Advantage
Real-time Streaming for Government and the Public Sector
“Insights that can provide incredible value, but the
value expires and evaporate once the moment is
gone” (Forrester)
The value of data is
locked in silos of data estates
Real-time Streaming for Government and the Public Sector
Why Didn’t It Work Before?
Messaging ESB ETL
The Rise of Data Streaming
2010
created at LinkedIn by
Confluent founders
2014
2020
80%
Fortune 100
Companies
The Trinity of Data Streaming
01
Publish & Subscribe
to Streams of Events
02
Brokered & Stored
your Event Streams
03
Process & Analyze
your Events Streams
Real-time and Historic
13
1 2 3 4 5 6 8
7 10
9 11 12 New Data
Slow
Consumer
Reprocessing
Historical
Time
Real-TIme
Consumer
Real-TIme
Consumer
What Are Events
Transportation
TPMS sensor in Carol’s car detected low tire-pressure at 5:11am.
Kafka
Banking
Alice sent $250 to Bob on Friday at 7:34pm.
Kafka
Retail
Sabine’s order of a Fujifilm camera was shipped at 9:10am.
Kafka
Streams of Events Tell a Story
15
{
datetime: 1970-01-01T00:00,
airport: QXYZ,
runways: [
{id: 32,
length:3500,
status: active}
]
fuel: [Grade80],
ILS: None
}
{
datetime: 2020-05-30T12:32,
airport: QXYZ,
runways: [
{id: 32,
length: 7500,
surface: grooved,
lights: HI}
]
fuel: [Grade80, 100LL, A],
ILS: None
}
{
datetime: 2018-01-01T12:32,
airport: QXYZ,
runways: [
{id: 32,
length:3500,
status: active}
]
fuel: [Grade80],
ILS:
{id: ils_1,
type: Super1,
status: active}
}
1 2 3
Examples
Real-time Streaming for Government and the Public Sector
Data Sharing For USCIS
18
19
JADC2
“DOD uses
ride-sharing
service Uber as
an analogy to
describe its
desired end
state for JADC2”
21
DoD Global Cyber Data Fabric
Thank you
Will LaForest
Field CTO
will@confluent.io
1 de 22

Recomendados

Dull, Difficult, and Essential: Managing Public Records por
Dull,  Difficult,  and Essential: Managing Public RecordsDull,  Difficult,  and Essential: Managing Public Records
Dull, Difficult, and Essential: Managing Public RecordsPaul W. Taylor
696 vistas39 diapositivas
InsideView Clean Data por
InsideView Clean DataInsideView Clean Data
InsideView Clean DataInsideView
209 vistas2 diapositivas
Better Online Strategy Presentation Melb 22 Oct por
Better Online Strategy Presentation Melb 22 OctBetter Online Strategy Presentation Melb 22 Oct
Better Online Strategy Presentation Melb 22 OctCSIRO National AI Centre
515 vistas43 diapositivas
Move It Don't Lose It: Is Your Big Data Collecting Dust? por
Move It Don't Lose It: Is Your Big Data Collecting Dust?Move It Don't Lose It: Is Your Big Data Collecting Dust?
Move It Don't Lose It: Is Your Big Data Collecting Dust?Jennifer Walker
324 vistas21 diapositivas
Data Refinery Is Fuelling Next Generation Big Data Analytics por
Data Refinery Is Fuelling Next Generation Big Data AnalyticsData Refinery Is Fuelling Next Generation Big Data Analytics
Data Refinery Is Fuelling Next Generation Big Data AnalyticsJean-Michel Franco
1.4K vistas11 diapositivas
The future of bi isn't a bi tool por
The future of bi isn't a bi toolThe future of bi isn't a bi tool
The future of bi isn't a bi toolDATAVERSITY
894 vistas59 diapositivas

Más contenido relacionado

Similar a Real-time Streaming for Government and the Public Sector

Joe C por
Joe CJoe C
Joe CHilary Ip
342 vistas18 diapositivas
Big Data por
Big DataBig Data
Big DataReddhi Basu
758 vistas24 diapositivas
What is Big Data? - Business Plans por
What is Big Data? - Business PlansWhat is Big Data? - Business Plans
What is Big Data? - Business PlansOur Business Ladder
124 vistas5 diapositivas
Day 2 aziz apj aziz_big_datakeynote_press por
Day 2 aziz apj aziz_big_datakeynote_pressDay 2 aziz apj aziz_big_datakeynote_press
Day 2 aziz apj aziz_big_datakeynote_pressIntelAPAC
1.6K vistas22 diapositivas
Hawaii Pacific GIS Conference 2012: Plenary Session Keynote - Next Generation... por
Hawaii Pacific GIS Conference 2012: Plenary Session Keynote - Next Generation...Hawaii Pacific GIS Conference 2012: Plenary Session Keynote - Next Generation...
Hawaii Pacific GIS Conference 2012: Plenary Session Keynote - Next Generation...Hawaii Geographic Information Coordinating Council
924 vistas40 diapositivas
Noise to Signal - The Biggest Problem in Data por
Noise to Signal - The Biggest Problem in DataNoise to Signal - The Biggest Problem in Data
Noise to Signal - The Biggest Problem in DataDATAVERSITY
671 vistas28 diapositivas

Similar a Real-time Streaming for Government and the Public Sector(20)

Day 2 aziz apj aziz_big_datakeynote_press por IntelAPAC
Day 2 aziz apj aziz_big_datakeynote_pressDay 2 aziz apj aziz_big_datakeynote_press
Day 2 aziz apj aziz_big_datakeynote_press
IntelAPAC1.6K vistas
Noise to Signal - The Biggest Problem in Data por DATAVERSITY
Noise to Signal - The Biggest Problem in DataNoise to Signal - The Biggest Problem in Data
Noise to Signal - The Biggest Problem in Data
DATAVERSITY671 vistas
Data Intelligence: How the Amalgamation of Data, Science, and Technology is C... por Caserta
Data Intelligence: How the Amalgamation of Data, Science, and Technology is C...Data Intelligence: How the Amalgamation of Data, Science, and Technology is C...
Data Intelligence: How the Amalgamation of Data, Science, and Technology is C...
Caserta 1.1K vistas
From IoT to IoTA por Striim
From IoT to IoTAFrom IoT to IoTA
From IoT to IoTA
Striim326 vistas
The Sherpa Approach: Meeting the Demands of the Digital Age por Sherpa Software
The Sherpa Approach:  Meeting the Demands of the Digital AgeThe Sherpa Approach:  Meeting the Demands of the Digital Age
The Sherpa Approach: Meeting the Demands of the Digital Age
Sherpa Software848 vistas
Usama Fayyad talk in South Africa: From BigData to Data Science por Usama Fayyad
Usama Fayyad talk in South Africa:  From BigData to Data ScienceUsama Fayyad talk in South Africa:  From BigData to Data Science
Usama Fayyad talk in South Africa: From BigData to Data Science
Usama Fayyad3.2K vistas
Open government international garry lloyd por Garry Lloyd
Open government international   garry lloydOpen government international   garry lloyd
Open government international garry lloyd
Garry Lloyd357 vistas
Business intelligence with web data gabc may por Semetis
Business intelligence with web data   gabc mayBusiness intelligence with web data   gabc may
Business intelligence with web data gabc may
Semetis467 vistas
BUS 5114 DISCUSSION ASSIGNMENT 01 - 09.docx por SheuBasharu1
BUS 5114 DISCUSSION ASSIGNMENT 01 - 09.docxBUS 5114 DISCUSSION ASSIGNMENT 01 - 09.docx
BUS 5114 DISCUSSION ASSIGNMENT 01 - 09.docx
SheuBasharu16 vistas
Experian Discussion Paper_Managing data as a strategic asset in Local Go... por Martin Soley
Experian Discussion Paper_Managing data as a strategic asset in Local Go...Experian Discussion Paper_Managing data as a strategic asset in Local Go...
Experian Discussion Paper_Managing data as a strategic asset in Local Go...
Martin Soley125 vistas
The Business of Big Data (IA Ventures) por Ben Siscovick
The Business of Big Data (IA Ventures)The Business of Big Data (IA Ventures)
The Business of Big Data (IA Ventures)
Ben Siscovick2.4K vistas
Orbyfy Overview - Solutions_vF_x.pdf por Orbyfy
Orbyfy Overview - Solutions_vF_x.pdfOrbyfy Overview - Solutions_vF_x.pdf
Orbyfy Overview - Solutions_vF_x.pdf
Orbyfy71 vistas

Más de confluent

Citi TechTalk Session 2: Kafka Deep Dive por
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Diveconfluent
17 vistas60 diapositivas
Build real-time streaming data pipelines to AWS with Confluent por
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluentconfluent
69 vistas53 diapositivas
Q&A with Confluent Professional Services: Confluent Service Mesh por
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Meshconfluent
67 vistas69 diapositivas
Citi Tech Talk: Event Driven Kafka Microservices por
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservicesconfluent
23 vistas29 diapositivas
Confluent & GSI Webinars series - Session 3 por
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3confluent
15 vistas59 diapositivas
Citi Tech Talk: Messaging Modernization por
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernizationconfluent
17 vistas39 diapositivas

Más de confluent(20)

Citi TechTalk Session 2: Kafka Deep Dive por confluent
Citi TechTalk Session 2: Kafka Deep DiveCiti TechTalk Session 2: Kafka Deep Dive
Citi TechTalk Session 2: Kafka Deep Dive
confluent17 vistas
Build real-time streaming data pipelines to AWS with Confluent por confluent
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluent
confluent69 vistas
Q&A with Confluent Professional Services: Confluent Service Mesh por confluent
Q&A with Confluent Professional Services: Confluent Service MeshQ&A with Confluent Professional Services: Confluent Service Mesh
Q&A with Confluent Professional Services: Confluent Service Mesh
confluent67 vistas
Citi Tech Talk: Event Driven Kafka Microservices por confluent
Citi Tech Talk: Event Driven Kafka MicroservicesCiti Tech Talk: Event Driven Kafka Microservices
Citi Tech Talk: Event Driven Kafka Microservices
confluent23 vistas
Confluent & GSI Webinars series - Session 3 por confluent
Confluent & GSI Webinars series - Session 3Confluent & GSI Webinars series - Session 3
Confluent & GSI Webinars series - Session 3
confluent15 vistas
Citi Tech Talk: Messaging Modernization por confluent
Citi Tech Talk: Messaging ModernizationCiti Tech Talk: Messaging Modernization
Citi Tech Talk: Messaging Modernization
confluent17 vistas
Citi Tech Talk: Data Governance for streaming and real time data por confluent
Citi Tech Talk: Data Governance for streaming and real time dataCiti Tech Talk: Data Governance for streaming and real time data
Citi Tech Talk: Data Governance for streaming and real time data
confluent21 vistas
Confluent & GSI Webinars series: Session 2 por confluent
Confluent & GSI Webinars series: Session 2Confluent & GSI Webinars series: Session 2
Confluent & GSI Webinars series: Session 2
confluent16 vistas
Data In Motion Paris 2023 por confluent
Data In Motion Paris 2023Data In Motion Paris 2023
Data In Motion Paris 2023
confluent224 vistas
The Future of Application Development - API Days - Melbourne 2023 por confluent
The Future of Application Development - API Days - Melbourne 2023The Future of Application Development - API Days - Melbourne 2023
The Future of Application Development - API Days - Melbourne 2023
confluent68 vistas
The Playful Bond Between REST And Data Streams por confluent
The Playful Bond Between REST And Data StreamsThe Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data Streams
confluent49 vistas
The Journey to Data Mesh with Confluent por confluent
The Journey to Data Mesh with ConfluentThe Journey to Data Mesh with Confluent
The Journey to Data Mesh with Confluent
confluent70 vistas
Citi Tech Talk: Monitoring and Performance por confluent
Citi Tech Talk: Monitoring and PerformanceCiti Tech Talk: Monitoring and Performance
Citi Tech Talk: Monitoring and Performance
confluent40 vistas
Citi Tech Talk Disaster Recovery Solutions Deep Dive por confluent
Citi Tech Talk  Disaster Recovery Solutions Deep DiveCiti Tech Talk  Disaster Recovery Solutions Deep Dive
Citi Tech Talk Disaster Recovery Solutions Deep Dive
confluent66 vistas
Citi Tech Talk: Hybrid Cloud por confluent
Citi Tech Talk: Hybrid CloudCiti Tech Talk: Hybrid Cloud
Citi Tech Talk: Hybrid Cloud
confluent43 vistas
Confluent Partner Tech Talk with QLIK por confluent
Confluent Partner Tech Talk with QLIKConfluent Partner Tech Talk with QLIK
Confluent Partner Tech Talk with QLIK
confluent90 vistas
Confluent Partner Tech Talk with SVA por confluent
Confluent Partner Tech Talk with SVAConfluent Partner Tech Talk with SVA
Confluent Partner Tech Talk with SVA
confluent95 vistas
How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re... por confluent
How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re...How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re...
How to Build Real-Time Analytics Applications like Netflix, Confluent, and Re...
confluent28 vistas
Single View of Data por confluent
Single View of DataSingle View of Data
Single View of Data
confluent71 vistas
Leveraging streaming data in real-time to build a Single View of Customer (SVOC) por confluent
Leveraging streaming data in real-time to build a Single View of Customer (SVOC)Leveraging streaming data in real-time to build a Single View of Customer (SVOC)
Leveraging streaming data in real-time to build a Single View of Customer (SVOC)
confluent21 vistas

Último

MariaDB stored procedures and why they should be improved por
MariaDB stored procedures and why they should be improvedMariaDB stored procedures and why they should be improved
MariaDB stored procedures and why they should be improvedFederico Razzoli
8 vistas32 diapositivas
SAP FOR CONTRACT MANUFACTURING.pdf por
SAP FOR CONTRACT MANUFACTURING.pdfSAP FOR CONTRACT MANUFACTURING.pdf
SAP FOR CONTRACT MANUFACTURING.pdfVirendra Rai, PMP
11 vistas2 diapositivas
Dapr Unleashed: Accelerating Microservice Development por
Dapr Unleashed: Accelerating Microservice DevelopmentDapr Unleashed: Accelerating Microservice Development
Dapr Unleashed: Accelerating Microservice DevelopmentMiroslav Janeski
10 vistas29 diapositivas
Headless JS UG Presentation.pptx por
Headless JS UG Presentation.pptxHeadless JS UG Presentation.pptx
Headless JS UG Presentation.pptxJack Spektor
7 vistas24 diapositivas
HarshithAkkapelli_Presentation.pdf por
HarshithAkkapelli_Presentation.pdfHarshithAkkapelli_Presentation.pdf
HarshithAkkapelli_Presentation.pdfharshithakkapelli
11 vistas16 diapositivas
Software testing company in India.pptx por
Software testing company in India.pptxSoftware testing company in India.pptx
Software testing company in India.pptxSakshiPatel82
7 vistas9 diapositivas

Último(20)

MariaDB stored procedures and why they should be improved por Federico Razzoli
MariaDB stored procedures and why they should be improvedMariaDB stored procedures and why they should be improved
MariaDB stored procedures and why they should be improved
Federico Razzoli8 vistas
Dapr Unleashed: Accelerating Microservice Development por Miroslav Janeski
Dapr Unleashed: Accelerating Microservice DevelopmentDapr Unleashed: Accelerating Microservice Development
Dapr Unleashed: Accelerating Microservice Development
Miroslav Janeski10 vistas
Headless JS UG Presentation.pptx por Jack Spektor
Headless JS UG Presentation.pptxHeadless JS UG Presentation.pptx
Headless JS UG Presentation.pptx
Jack Spektor7 vistas
Software testing company in India.pptx por SakshiPatel82
Software testing company in India.pptxSoftware testing company in India.pptx
Software testing company in India.pptx
SakshiPatel827 vistas
DSD-INT 2023 Salt intrusion Modelling of the Lauwersmeer, towards a measureme... por Deltares
DSD-INT 2023 Salt intrusion Modelling of the Lauwersmeer, towards a measureme...DSD-INT 2023 Salt intrusion Modelling of the Lauwersmeer, towards a measureme...
DSD-INT 2023 Salt intrusion Modelling of the Lauwersmeer, towards a measureme...
Deltares5 vistas
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx por animuscrm
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx
animuscrm14 vistas
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J... por Deltares
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...
DSD-INT 2023 3D hydrodynamic modelling of microplastic transport in lakes - J...
Deltares9 vistas
Software evolution understanding: Automatic extraction of software identifier... por Ra'Fat Al-Msie'deen
Software evolution understanding: Automatic extraction of software identifier...Software evolution understanding: Automatic extraction of software identifier...
Software evolution understanding: Automatic extraction of software identifier...
Team Transformation Tactics for Holistic Testing and Quality (Japan Symposium... por Lisi Hocke
Team Transformation Tactics for Holistic Testing and Quality (Japan Symposium...Team Transformation Tactics for Holistic Testing and Quality (Japan Symposium...
Team Transformation Tactics for Holistic Testing and Quality (Japan Symposium...
Lisi Hocke28 vistas
DSD-INT 2023 Wave-Current Interaction at Montrose Tidal Inlet System and Its ... por Deltares
DSD-INT 2023 Wave-Current Interaction at Montrose Tidal Inlet System and Its ...DSD-INT 2023 Wave-Current Interaction at Montrose Tidal Inlet System and Its ...
DSD-INT 2023 Wave-Current Interaction at Montrose Tidal Inlet System and Its ...
Deltares10 vistas
DSD-INT 2023 Machine learning in hydraulic engineering - Exploring unseen fut... por Deltares
DSD-INT 2023 Machine learning in hydraulic engineering - Exploring unseen fut...DSD-INT 2023 Machine learning in hydraulic engineering - Exploring unseen fut...
DSD-INT 2023 Machine learning in hydraulic engineering - Exploring unseen fut...
Deltares7 vistas
360 graden fabriek por info33492
360 graden fabriek360 graden fabriek
360 graden fabriek
info3349237 vistas
DSD-INT 2023 Simulation of Coastal Hydrodynamics and Water Quality in Hong Ko... por Deltares
DSD-INT 2023 Simulation of Coastal Hydrodynamics and Water Quality in Hong Ko...DSD-INT 2023 Simulation of Coastal Hydrodynamics and Water Quality in Hong Ko...
DSD-INT 2023 Simulation of Coastal Hydrodynamics and Water Quality in Hong Ko...
Deltares14 vistas
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with... por sparkfabrik
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
sparkfabrik5 vistas
Tridens DevOps por Tridens
Tridens DevOpsTridens DevOps
Tridens DevOps
Tridens9 vistas
DSD-INT 2023 Leveraging the results of a 3D hydrodynamic model to improve the... por Deltares
DSD-INT 2023 Leveraging the results of a 3D hydrodynamic model to improve the...DSD-INT 2023 Leveraging the results of a 3D hydrodynamic model to improve the...
DSD-INT 2023 Leveraging the results of a 3D hydrodynamic model to improve the...
Deltares6 vistas
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h... por Deltares
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...
Deltares5 vistas

Real-time Streaming for Government and the Public Sector

  • 1. Data Streaming in Government Will LaForest Field CTO will@confluent.io
  • 2. Data is the Competitive Advantage Trending Now Popular on Netflix Top Picks for Joshua Curbside Pickup Loyalty Rewards Personalized Recommendations Real-time Trades Ride ETA
  • 3. Product of Evidence Act • Passed January 2019 • Evidence Based Government All agencies expected to Adhere Mission Statement Fully leverage the value of federal data for mission, service, and the public good by guiding the Federal Government in practicing ethical governance, conscious design, and a learning culture.
  • 4. Data as a Strategic Asset
  • 5. Data is the Competitive Advantage
  • 7. “Insights that can provide incredible value, but the value expires and evaporate once the moment is gone” (Forrester)
  • 8. The value of data is locked in silos of data estates
  • 10. Why Didn’t It Work Before? Messaging ESB ETL
  • 11. The Rise of Data Streaming 2010 created at LinkedIn by Confluent founders 2014 2020 80% Fortune 100 Companies
  • 12. The Trinity of Data Streaming 01 Publish & Subscribe to Streams of Events 02 Brokered & Stored your Event Streams 03 Process & Analyze your Events Streams
  • 13. Real-time and Historic 13 1 2 3 4 5 6 8 7 10 9 11 12 New Data Slow Consumer Reprocessing Historical Time Real-TIme Consumer Real-TIme Consumer
  • 14. What Are Events Transportation TPMS sensor in Carol’s car detected low tire-pressure at 5:11am. Kafka Banking Alice sent $250 to Bob on Friday at 7:34pm. Kafka Retail Sabine’s order of a Fujifilm camera was shipped at 9:10am. Kafka
  • 15. Streams of Events Tell a Story 15 { datetime: 1970-01-01T00:00, airport: QXYZ, runways: [ {id: 32, length:3500, status: active} ] fuel: [Grade80], ILS: None } { datetime: 2020-05-30T12:32, airport: QXYZ, runways: [ {id: 32, length: 7500, surface: grooved, lights: HI} ] fuel: [Grade80, 100LL, A], ILS: None } { datetime: 2018-01-01T12:32, airport: QXYZ, runways: [ {id: 32, length:3500, status: active} ] fuel: [Grade80], ILS: {id: ils_1, type: Super1, status: active} } 1 2 3
  • 18. Data Sharing For USCIS 18
  • 19. 19
  • 20. JADC2 “DOD uses ride-sharing service Uber as an analogy to describe its desired end state for JADC2”
  • 21. 21 DoD Global Cyber Data Fabric
  • 22. Thank you Will LaForest Field CTO will@confluent.io