SlideShare a Scribd company logo
1 of 67
Download to read offline
THE MORE DATA YOU COLLECT 
THE MORE VALUE YOU CAN 
DERIVE FROM IT
THE COST OF DATA 
GENERATION IS FALLING
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
Lower cost, 
higher throughput 
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
Lower cost, 
higher throughput 
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE 
Highly 
constrained
+ ELASTIC AND HIGHLY SCALABLE 
+ NO UPFRONT CAPITAL EXPENSE 
+ ONLY PAY FOR WHAT YOU USE 
+ AVAILABLE ON-DEMAND 
! 
= REMOVE CONSTRAINTS
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
AWS Import / Export 
AWS Direct Connect 
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
Inbound data transfer is free 
Multipart upload to S3 
Physical media 
AWS Direct Connect
Amazon S3, 
Amazon Glacier, 
Amazon DynamoDB, 
Amazon RDS, 
Amazon Redshift, 
AWS Storage Gateway, 
Data on Amazon EC2 
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
AMAZON S3 
SIMPLE STORAGE SERVICE
CASE STUDY: 
SPOTIFY ADDS 20,000 TRACKS/DAY TO ITS CATALOGUE
AMAZON 
DYNAMODB" 
HIGH-PERFORMANCE, FULLY MANAGED 
NoSQL DATABASE SERVICE
DURABLE & 
AVAILABLE 
CONSISTENT, DISK-ONLY 
WRITES (SSD)
LOW LATENCY 
AVERAGE READS < 5MS, 
WRITES < 10MS
! 
! 
! 
NO ADMINISTRATION
CASE STUDY: 
SHAZAM SUPPORTED 500,000 WRITES/SEC 
DURING SUPER BOWL
AMAZON 
REDSHIFT" 
FULLY MANAGED, PETA-BYTE SCALE 
DATAWAREHOUSE ON AWS
30 MINUTES 
DOWN TO 
12 SECONDS
AMAZON REDSHIFT LETS YOU 
START SMALL AND GROW BIG 
Extra Large Node (HS1.XL) 
! 
Single Node (2 TB) 
! 
Cluster 2-32 Nodes (4 TB – 64 TB) 
Eight Extra Large Node (HS1.8XL) 
Cluster 2-100 Nodes (32 TB – 1.6 PB) 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 
XL 
XL XL XL XL XL XL XL XL 
XL XL XL XL XL XL XL XL 
XL XL XL XL XL XL XL XL 
XL XL XL XL XL XL XL XL
JDBC/ODBC 
! 
!
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE 
Amazon EC2 
Amazon Elastic 
MapReduce
AMAZON EC2 
ELASTIC COMPUTE CLOUD
3 HOURS 
FOR $4828.85/hr
Instead of 
$20+ MILLIONS 
in infrastructure
GPU INSTANCES" 
! 
G2" 
CG1 
1x NVIDIA Kepler GK104 
8 vCPU (Intel Xeon E5-2670) 
$0.65/h 
2x NVIDIA Fermi M2050 
16 vCPU (Intel Xeon X5570) 
$2.10/h
ON A SINGLE INSTANCE 
COMPUTE TIME: 4h 
COST: 4h x $2.1 = $8.4
ON MULTIPLE INSTANCES 
COMPUTE TIME: 1h 
COST: 1h x 4 x $2.1 = $8.4
AMAZON ELASTIC 
MAPREDUCE 
HADOOP AS A SERVICE
CASE STUDY: 
"WITH AMAZON EMR WE CAN ANALYZE 100% OF THE DATA, 
NOT JUST A SAMPLE" 
- Sanjeevan Bala, Head of Data Planning & Analytics, Channel 4
Amazon S3, 
Amazon DynamoDB, 
Amazon RDS, 
Amazon Redshift, 
Data on Amazon EC2 
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
PUBLIC DATA SETS 
http://aws.amazon.com/publicdatasets
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE 
BATCH 
PROCESSING
STREAM 
PROCESSING 
GENERATE ➔ ➔ SHARE
AMAZON KINESIS 
REAL-TIME DATA STREAM PROCESSING
Hourly server logs: how your 
systems went wrong an hour ago 
Weekly / Monthly Bill: What you 
spent this past billing cycle 
Daily customer report from your 
website: tells you what deal or ad 
to try next time 
Daily fraud reports: tells you if there 
was fraud yesterday 
Daily business reports: tells me 
how customers used AWS services 
yesterday 
Real-time metrics: what just went 
wrong now 
Real-time spending alerts/caps: 
guaranteeing you can’t 
overspend 
Real-time analysis: what to offer 
the current customer now 
Real-time detection: blocks 
fraudulent use now 
Fast ETL into Amazon Redshift: 
how are customers using 
services now
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
Amazon S3, 
Amazon DynamoDB, 
Amazon RDS, 
Amazon Redshift, 
Data on Amazon EC2 
Amazon S3, 
Amazon Glacier, 
Amazon DynamoDB, 
Amazon RDS, 
Amazon Redshift, 
AWS Storage Gateway, 
Data on Amazon EC2 
GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE 
Amazon EC2 
Amazon Elastic 
MapReduce 
AWS Import / Export 
AWS Direct Connect
STREAM 
PROCESSING 
GENERATE ➔ ➔ SHARE
STREAM 
PROCESSING 
Amazon S3, 
Amazon DynamoDB, 
Amazon RDS, 
Amazon Redshift, 
Data on Amazon EC2 
GENERATE ➔ ➔ SHARE 
Amazon Kinesis 
Stream Processing 
on Amazon EC2
FROM DATA TO 
ACTIONABLE 
INFORMATION
Workshop part2 – Big Data

More Related Content

Viewers also liked

(BDT311) MegaRun: Behind the 156,000 Core HPC Run on AWS and Experience of On...
(BDT311) MegaRun: Behind the 156,000 Core HPC Run on AWS and Experience of On...(BDT311) MegaRun: Behind the 156,000 Core HPC Run on AWS and Experience of On...
(BDT311) MegaRun: Behind the 156,000 Core HPC Run on AWS and Experience of On...
Amazon Web Services
 

Viewers also liked (20)

AWS Paris Summit 2014 - Closing Keynote Werner Vogels - Beyond the fridge
AWS Paris Summit 2014 - Closing Keynote Werner Vogels - Beyond the fridgeAWS Paris Summit 2014 - Closing Keynote Werner Vogels - Beyond the fridge
AWS Paris Summit 2014 - Closing Keynote Werner Vogels - Beyond the fridge
 
AWS Webcast - Webinar Series for State and Local Government #2: Discover the ...
AWS Webcast - Webinar Series for State and Local Government #2: Discover the ...AWS Webcast - Webinar Series for State and Local Government #2: Discover the ...
AWS Webcast - Webinar Series for State and Local Government #2: Discover the ...
 
Updating Security Operations for the Cloud - AWS Symposium 2014 - Washington ...
Updating Security Operations for the Cloud - AWS Symposium 2014 - Washington ...Updating Security Operations for the Cloud - AWS Symposium 2014 - Washington ...
Updating Security Operations for the Cloud - AWS Symposium 2014 - Washington ...
 
AWS Government, Education, and Nonprofits Symposium London, United Kingdom L...
 AWS Government, Education, and Nonprofits Symposium London, United Kingdom L... AWS Government, Education, and Nonprofits Symposium London, United Kingdom L...
AWS Government, Education, and Nonprofits Symposium London, United Kingdom L...
 
(BIZ306) Migrating Trimble Sketchup 3D Warehouse to AWS | AWS re:Invent 2014
(BIZ306) Migrating Trimble Sketchup 3D Warehouse to AWS | AWS re:Invent 2014(BIZ306) Migrating Trimble Sketchup 3D Warehouse to AWS | AWS re:Invent 2014
(BIZ306) Migrating Trimble Sketchup 3D Warehouse to AWS | AWS re:Invent 2014
 
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
 
AWS Public Sector Symposium | Effective Security Response in the Cloud - Sess...
AWS Public Sector Symposium | Effective Security Response in the Cloud - Sess...AWS Public Sector Symposium | Effective Security Response in the Cloud - Sess...
AWS Public Sector Symposium | Effective Security Response in the Cloud - Sess...
 
Fostering a Culture of Innovation with Cloud
Fostering a Culture of Innovation with CloudFostering a Culture of Innovation with Cloud
Fostering a Culture of Innovation with Cloud
 
AWS Service Drill Downs
AWS Service Drill DownsAWS Service Drill Downs
AWS Service Drill Downs
 
(SPOT207) State of the Union: AWS Database Services | AWS re:Invent 2014
(SPOT207) State of the Union: AWS Database Services | AWS re:Invent 2014(SPOT207) State of the Union: AWS Database Services | AWS re:Invent 2014
(SPOT207) State of the Union: AWS Database Services | AWS re:Invent 2014
 
(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014
(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014
(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014
 
Journey Through the Cloud - Digital Media
Journey Through the Cloud - Digital MediaJourney Through the Cloud - Digital Media
Journey Through the Cloud - Digital Media
 
Deployment with Elastic Beanstalk at Edinburgh Startup Event
Deployment with Elastic Beanstalk at Edinburgh Startup EventDeployment with Elastic Beanstalk at Edinburgh Startup Event
Deployment with Elastic Beanstalk at Edinburgh Startup Event
 
(AFF201) What the Top 50 Games Do with In-App Purchasing That the Rest of Us ...
(AFF201) What the Top 50 Games Do with In-App Purchasing That the Rest of Us ...(AFF201) What the Top 50 Games Do with In-App Purchasing That the Rest of Us ...
(AFF201) What the Top 50 Games Do with In-App Purchasing That the Rest of Us ...
 
(PFC307) Auto Scaling: A Machine Learning Approach | AWS re:Invent 2014
(PFC307) Auto Scaling: A Machine Learning Approach | AWS re:Invent 2014(PFC307) Auto Scaling: A Machine Learning Approach | AWS re:Invent 2014
(PFC307) Auto Scaling: A Machine Learning Approach | AWS re:Invent 2014
 
T2 – Continuous integration on aws
T2 – Continuous integration on awsT2 – Continuous integration on aws
T2 – Continuous integration on aws
 
(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014
(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014
(WEB305) Migrating Your Website to AWS | AWS re:Invent 2014
 
AWS Webcast - Janet Community Portal to Integrate AWS Cloud Services
AWS Webcast - Janet Community Portal to Integrate AWS Cloud ServicesAWS Webcast - Janet Community Portal to Integrate AWS Cloud Services
AWS Webcast - Janet Community Portal to Integrate AWS Cloud Services
 
What is Cloud Computing with AWS?
What is Cloud Computing with AWS?What is Cloud Computing with AWS?
What is Cloud Computing with AWS?
 
(BDT311) MegaRun: Behind the 156,000 Core HPC Run on AWS and Experience of On...
(BDT311) MegaRun: Behind the 156,000 Core HPC Run on AWS and Experience of On...(BDT311) MegaRun: Behind the 156,000 Core HPC Run on AWS and Experience of On...
(BDT311) MegaRun: Behind the 156,000 Core HPC Run on AWS and Experience of On...
 

Similar to Workshop part2 – Big Data

AWS Summit Tel Aviv - Startup Track - Data Analytics & Big Data
AWS Summit Tel Aviv - Startup Track - Data Analytics & Big DataAWS Summit Tel Aviv - Startup Track - Data Analytics & Big Data
AWS Summit Tel Aviv - Startup Track - Data Analytics & Big Data
Amazon Web Services
 

Similar to Workshop part2 – Big Data (20)

2014 Import.io Data Summit - Including Hadoop/Impala Getting Started Demo
2014 Import.io Data Summit - Including Hadoop/Impala Getting Started Demo2014 Import.io Data Summit - Including Hadoop/Impala Getting Started Demo
2014 Import.io Data Summit - Including Hadoop/Impala Getting Started Demo
 
Cloud World Forum: Large Scale Data Analysis on AWS
Cloud World Forum: Large Scale Data Analysis on AWSCloud World Forum: Large Scale Data Analysis on AWS
Cloud World Forum: Large Scale Data Analysis on AWS
 
Large Scale Data Analysis with AWS
Large Scale Data Analysis with AWSLarge Scale Data Analysis with AWS
Large Scale Data Analysis with AWS
 
Journey Through the AWS Cloud - Big Data Analysis
Journey Through the AWS Cloud - Big Data AnalysisJourney Through the AWS Cloud - Big Data Analysis
Journey Through the AWS Cloud - Big Data Analysis
 
Build Data Lakes and Analytics on AWS
Build Data Lakes and Analytics on AWS Build Data Lakes and Analytics on AWS
Build Data Lakes and Analytics on AWS
 
AWS Summit Tel Aviv - Startup Track - Data Analytics & Big Data
AWS Summit Tel Aviv - Startup Track - Data Analytics & Big DataAWS Summit Tel Aviv - Startup Track - Data Analytics & Big Data
AWS Summit Tel Aviv - Startup Track - Data Analytics & Big Data
 
Data Analytics on AWS
Data Analytics on AWSData Analytics on AWS
Data Analytics on AWS
 
From delivering plugins to delivering "as a Service" - Atlassian connect 2017
From delivering plugins to delivering "as a Service" - Atlassian connect 2017From delivering plugins to delivering "as a Service" - Atlassian connect 2017
From delivering plugins to delivering "as a Service" - Atlassian connect 2017
 
AWS re:Invent Recap from AWS User Group UK meetup #8
AWS re:Invent Recap from AWS User Group UK meetup #8AWS re:Invent Recap from AWS User Group UK meetup #8
AWS re:Invent Recap from AWS User Group UK meetup #8
 
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...
 
Build Data Lakes and Analytics on AWS: Patterns & Best Practices - BDA305 - A...
Build Data Lakes and Analytics on AWS: Patterns & Best Practices - BDA305 - A...Build Data Lakes and Analytics on AWS: Patterns & Best Practices - BDA305 - A...
Build Data Lakes and Analytics on AWS: Patterns & Best Practices - BDA305 - A...
 
AWS Enterprise Summit Netherlands - Big Data Architectural Patterns & Best Pr...
AWS Enterprise Summit Netherlands - Big Data Architectural Patterns & Best Pr...AWS Enterprise Summit Netherlands - Big Data Architectural Patterns & Best Pr...
AWS Enterprise Summit Netherlands - Big Data Architectural Patterns & Best Pr...
 
Big Data Architectural Patterns
Big Data Architectural PatternsBig Data Architectural Patterns
Big Data Architectural Patterns
 
(SOV202) Choosing Among AWS Managed Database Services | AWS re:Invent 2014
(SOV202) Choosing Among AWS Managed Database Services | AWS re:Invent 2014(SOV202) Choosing Among AWS Managed Database Services | AWS re:Invent 2014
(SOV202) Choosing Among AWS Managed Database Services | AWS re:Invent 2014
 
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...
 
Data Warehouses & Data Lakes: Data Analytics Week SF
Data Warehouses & Data Lakes: Data Analytics Week SFData Warehouses & Data Lakes: Data Analytics Week SF
Data Warehouses & Data Lakes: Data Analytics Week SF
 
Amazon RedShift - Ianni Vamvadelis
Amazon RedShift - Ianni VamvadelisAmazon RedShift - Ianni Vamvadelis
Amazon RedShift - Ianni Vamvadelis
 
Immersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analíticoImmersion Day - Como simplificar o acesso ao seu ambiente analítico
Immersion Day - Como simplificar o acesso ao seu ambiente analítico
 
Leveraging Amazon Redshift for Your Data Warehouse
Leveraging Amazon Redshift for Your Data WarehouseLeveraging Amazon Redshift for Your Data Warehouse
Leveraging Amazon Redshift for Your Data Warehouse
 
Run Your CI/CD Pipeline at Scale for a Fraction of the Cost - AWS Online Tech...
Run Your CI/CD Pipeline at Scale for a Fraction of the Cost - AWS Online Tech...Run Your CI/CD Pipeline at Scale for a Fraction of the Cost - AWS Online Tech...
Run Your CI/CD Pipeline at Scale for a Fraction of the Cost - AWS Online Tech...
 

More from Amazon Web Services

Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
Amazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Recently uploaded

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Recently uploaded (20)

Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 

Workshop part2 – Big Data

  • 1.
  • 2. THE MORE DATA YOU COLLECT THE MORE VALUE YOU CAN DERIVE FROM IT
  • 3.
  • 4.
  • 5. THE COST OF DATA GENERATION IS FALLING
  • 6.
  • 7.
  • 8. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
  • 9. Lower cost, higher throughput GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
  • 10. Lower cost, higher throughput GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE Highly constrained
  • 11. + ELASTIC AND HIGHLY SCALABLE + NO UPFRONT CAPITAL EXPENSE + ONLY PAY FOR WHAT YOU USE + AVAILABLE ON-DEMAND ! = REMOVE CONSTRAINTS
  • 12. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
  • 13. AWS Import / Export AWS Direct Connect GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
  • 14. Inbound data transfer is free Multipart upload to S3 Physical media AWS Direct Connect
  • 15. Amazon S3, Amazon Glacier, Amazon DynamoDB, Amazon RDS, Amazon Redshift, AWS Storage Gateway, Data on Amazon EC2 GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
  • 16. AMAZON S3 SIMPLE STORAGE SERVICE
  • 17. CASE STUDY: SPOTIFY ADDS 20,000 TRACKS/DAY TO ITS CATALOGUE
  • 18. AMAZON DYNAMODB" HIGH-PERFORMANCE, FULLY MANAGED NoSQL DATABASE SERVICE
  • 19. DURABLE & AVAILABLE CONSISTENT, DISK-ONLY WRITES (SSD)
  • 20. LOW LATENCY AVERAGE READS < 5MS, WRITES < 10MS
  • 21. ! ! ! NO ADMINISTRATION
  • 22. CASE STUDY: SHAZAM SUPPORTED 500,000 WRITES/SEC DURING SUPER BOWL
  • 23. AMAZON REDSHIFT" FULLY MANAGED, PETA-BYTE SCALE DATAWAREHOUSE ON AWS
  • 24.
  • 25.
  • 26. 30 MINUTES DOWN TO 12 SECONDS
  • 27.
  • 28. AMAZON REDSHIFT LETS YOU START SMALL AND GROW BIG Extra Large Node (HS1.XL) ! Single Node (2 TB) ! Cluster 2-32 Nodes (4 TB – 64 TB) Eight Extra Large Node (HS1.8XL) Cluster 2-100 Nodes (32 TB – 1.6 PB) 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL 8XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL XL
  • 30.
  • 31.
  • 32.
  • 33.
  • 34. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE Amazon EC2 Amazon Elastic MapReduce
  • 35. AMAZON EC2 ELASTIC COMPUTE CLOUD
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
  • 41. 3 HOURS FOR $4828.85/hr
  • 42. Instead of $20+ MILLIONS in infrastructure
  • 43. GPU INSTANCES" ! G2" CG1 1x NVIDIA Kepler GK104 8 vCPU (Intel Xeon E5-2670) $0.65/h 2x NVIDIA Fermi M2050 16 vCPU (Intel Xeon X5570) $2.10/h
  • 44. ON A SINGLE INSTANCE COMPUTE TIME: 4h COST: 4h x $2.1 = $8.4
  • 45. ON MULTIPLE INSTANCES COMPUTE TIME: 1h COST: 1h x 4 x $2.1 = $8.4
  • 46.
  • 47.
  • 48.
  • 49.
  • 50. AMAZON ELASTIC MAPREDUCE HADOOP AS A SERVICE
  • 51. CASE STUDY: "WITH AMAZON EMR WE CAN ANALYZE 100% OF THE DATA, NOT JUST A SAMPLE" - Sanjeevan Bala, Head of Data Planning & Analytics, Channel 4
  • 52. Amazon S3, Amazon DynamoDB, Amazon RDS, Amazon Redshift, Data on Amazon EC2 GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
  • 53. PUBLIC DATA SETS http://aws.amazon.com/publicdatasets
  • 54.
  • 55.
  • 56. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
  • 57. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE BATCH PROCESSING
  • 58. STREAM PROCESSING GENERATE ➔ ➔ SHARE
  • 59. AMAZON KINESIS REAL-TIME DATA STREAM PROCESSING
  • 60. Hourly server logs: how your systems went wrong an hour ago Weekly / Monthly Bill: What you spent this past billing cycle Daily customer report from your website: tells you what deal or ad to try next time Daily fraud reports: tells you if there was fraud yesterday Daily business reports: tells me how customers used AWS services yesterday Real-time metrics: what just went wrong now Real-time spending alerts/caps: guaranteeing you can’t overspend Real-time analysis: what to offer the current customer now Real-time detection: blocks fraudulent use now Fast ETL into Amazon Redshift: how are customers using services now
  • 61.
  • 62. GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE
  • 63. Amazon S3, Amazon DynamoDB, Amazon RDS, Amazon Redshift, Data on Amazon EC2 Amazon S3, Amazon Glacier, Amazon DynamoDB, Amazon RDS, Amazon Redshift, AWS Storage Gateway, Data on Amazon EC2 GENERATE ➔ STORE ➔ ANALYZE ➔ SHARE Amazon EC2 Amazon Elastic MapReduce AWS Import / Export AWS Direct Connect
  • 64. STREAM PROCESSING GENERATE ➔ ➔ SHARE
  • 65. STREAM PROCESSING Amazon S3, Amazon DynamoDB, Amazon RDS, Amazon Redshift, Data on Amazon EC2 GENERATE ➔ ➔ SHARE Amazon Kinesis Stream Processing on Amazon EC2
  • 66. FROM DATA TO ACTIONABLE INFORMATION