SlideShare una empresa de Scribd logo
1 de 69
Descargar para leer sin conexión
Karan Bhatia, PhD
Introducing Elastic MapReduce
Big Data Solutions Practice
Vários Tutoriais , treinamentos e mentoria em
português
Inscreva-se agora !!
http://awshub.com.br
Introducing Elastic MapReduce
4 bytes x 1,000,000 households x 1 measurement/month x 10 years
480 MBytes
4 bytes x 1,000,000 households x 1 measurement/min x 10 years
220 TBytes
Big Data as Business Transformation
Generated data
Available for analysis
Data volume
Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011
IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares
AWS Elastic MapReduce
Map reduce
HDFS
Thousands of customers, 2 million+ clusters in 2012
EMR Sample Use Cases
Apontador e MapLink
e AWS
Apoio:
• O que conheço do usuário?
{"BaseLogId":"RmlpbjZkWVhCM0NxckNjYjF3eFU0dGNTYnhJPQ","TrackUserId":"a18e0672-ad07-4f28-
b447-fc0cba90ee17","SiteId":"apto-
dv01","SessionId":"1369827720327:f52c5b","ExternalId":"1933510381","Hostname":"integra01.aponta
dor.lan","Path":"/local/sp/sao_paulo/bares_e_casas_noturnas/QYN7825H/","Referer":null,"PageTitle":"L
ocais, Eventos, Endereços, Mapas -
Apontador.com","IpAddress":"200.150.177.249","AgentInfo":"Mozilla/5.0 (Macintosh; Intel Mac OS X
10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116Safari/537.36","Position":"{
"lat": -23.5934691, "lon": -46.6882606, "acc":
36}","SearchInfo":null,"RawRequestInfo":”RawRequest”: ","CreateAt":"2013-06-24T14:39:46.7082358Z"}
•O que mais?
Ações, cliques, buscas
COMO trazer o melhor para o usuário?
• O que recebemos para determinar o transito?
<Route><Category>1</Category><DateTime>0001-01-01T00:00:00</DateTime><Destination
xmlns:a="http://schemas.datacontract.org/2004/07/SwissKnife.Spatial"><a:Lat>-
8.150483</a:Lat><a:Lng>-35.420284</a:Lng></Destination><Origin
xmlns:a="http://schemas.datacontract.org/2004/07/SwissKnife.Spatial"><a:Lat>-
8.149973</a:Lat><a:Lng>-35.41825</a:Lng></Origin>
COMO descobrir o trânsito?
Teorema de Bayes:
O MODELO estatístico
• Hive (~ 40 instancias spot m3.large)
90% - Utilidades diárias
• Streaming
10% - Solr, MapReduces mais complexos (MCMC, FastFourier, e.g.)
• Estrutura usada
Hive ( ~ 40 instancias spot m3.large), Elastic MapReduce
S3 (aproximadamente 7 Tb de dados estruturados em diversos buckets)
RDS (dados de organização dos dados do S3)
O QUE usamos?
• A Chaordic é a empresa líder
em personalização para e-
commerce no Brasil, tendo
como clientes 9 dos 15 maiores
players do país.
• Os produtos desenvolvidos pela
Chaordic se integram aos
maiores sites de e-commerce
brasileiros e precisam de uma
infra-estrutura confiável,
rápida, escalável e de baixo
custo.
“ Com a AWS conseguimos construir um único sistema para
atender a demanda dos maiores sites de e-commerce do Brasil a
um custo relativamente baixo”.
“Construir um data
center próprio para
atender nossa
demanda seria
economicamente
inviável”
- João Bosco, CTO
O Desafio
• Atender dezenas de milhões de
usuários únicos por mês;
• Processamento de Big Data;
• Responder em menos de 100ms;
• Escalar bem em momentos de pico
de acesso;
• Tudo isto a um custo acessível.
Sobre o Papel da AWS e
Benefícios alcançados
• 4 bilhões de requisições por
mês;
• +300 mil requisições por
minuto;
• +200 milhões de
recomendações todos os dias;
• Spot instances: -20% custo aws.
Map Reduce
Introducing Elastic MapReduce
Map Shuffle Reduce
AWS Elastic MapReduce
Managed Hadoop analytics
Input data
S3, DynamoDB, Redshift
Elastic
MapReduce
Code
Input data
S3, DynamoDB, Redshift
Elastic
MapReduce
Code Name
node
Input data
S3, DynamoDB, Redshift
Elastic
MapReduce
Code Name
node
Input data
Elastic
cluster
S3, DynamoDB, Redshift
S3/HDFS
Elastic
MapReduce
Code Name
node
Input data
S3/HDFS
Queries
+ BI
Via JDBC, Pig, Hive
S3, DynamoDB, Redshift
Elastic
cluster
Elastic
MapReduce
Code Name
node
Output
Input data
Queries
+ BI
Via JDBC, Pig, Hive
S3, DynamoDB, Redshift
Elastic
cluster
S3/HDFS
Output
Input data
S3, DynamoDB, Redshift
Introducing Elastic MapReduce
Introducing Elastic MapReduce
Introducing Elastic MapReduce
Introducing Elastic MapReduce
1
2
4
8
16
32
64
128
256
1 2 4 8 16 32 64 128
Memory(GB)
EC2 Compute Units
Instance Types
Standard 2nd Gen Standard Micro High-Memory High-CPU Cluster Compute Cluster GPU High I/O High-Storage Cluster High-Mem
hi1.4xlarge
60.5 GB of memory
35 EC2 Compute
Units
2x1024 GB SSD
instance storage
64-bit platform
cc1.4xlarge
23 GB of memory
33.5 EC2 Compute
Units
1690 GB of instance
storage
64-bit platform
c1.xlarge
7 GB of memory
20 EC2 Compute
Units
1690 GB of
instance storage
64-bit platform
m1.small
1.7 GB memory
1 EC2 Compute Unit
160 GB instance
storage
32-bit or 64-bit
m1.medium
3.75 GB memory
2 EC2 Compute Unit
410 GB instance
storage
32-bit or 64-bit
platform
m1.large
EBS Optimizable
7.5 GB memory
4 EC2 Compute Units
850 GB instance
storage
64-bit platform
m1.xlarge
EBS Optimizable
15 GB memory
8 EC2 Compute Units
1,690 GB instance
storage
64-bit platform
m2.xlarge
17.1 GB of memory
6.5 EC2 Compute
Units
420 GB of instance
storage
64-bit platform
m2.2xlarge
34.2 GB of memory
13 EC2 Compute
Units
850 GB of instance
storage
64-bit platform
m2.4xlarge
EBS Optimizable
68.4 GB of memory
26 EC2 Compute
Units
1690 GB of instance
storage
64-bit platform
t1.micro
613 MB memory
Up to 2 EC2 Compute
Units
EBS storage only
32-bit or 64-bit platform
c1.medium
1.7 GB of memory
5 EC2 Compute Units
350 GB of instance
storage
32-bit or 64-bit
platform
cg1.4xlarge
22 GB of memory
33.5 EC2 Compute
Units
2 x NVIDIA Tesla
“Fermi”  M2050 GPUs
1690 GB of instance
storage
64-bit platform
cc2.8xlarge
60.5 GB of memory
88 EC2 Compute
Units
3370 GB of instance
storage
64-bit platformm3.xlarge
15 GB of memory
13 EC2 Compute
Units
m3.2xlarge
EBS Optimizable
30 GB of
memory
26 EC2 Compute
Units
hs1.8xlarge
117 GB of memory
35 EC2 Compute
Units
24x2 TB instance
storage
64-bit platform
cr1.8xlarge
244 GB of memory
88 EC2 Compute Units
2x120 GB SSD
instance storage
64-bit platform
Introducing Elastic MapReduce
Introducing Elastic MapReduce
Introducing Elastic MapReduce
Introducing Elastic MapReduce
Introducing Elastic MapReduce
Introducing Elastic MapReduce
1. Elastic clusters
10 hours
5 hours
Peak capacity
2. Rapid, tuned provisioning
Tedious.
Remove undifferentiated
heavy lifting.
3. Hadoop all the way down
Robust ecosystem.
Databases, machine learning, segmentation,
clustering, analytics, metadata stores,
exchange formats, and so on...
4. Agility for experimentation
Instance choice.
Stay flexible on instance type & number.
5. Cost optimizations
Built for Spot.
Name-your-price supercomputing.
1. Elastic clusters
2. Rapid, tuned provisioning
3. Hadoop all the way down
4. Agility for experimentation.
5. Cost optimizations
Data, data, everywhere...
Data is stored in silos.
S3
DynamoDB EMR
HBase on EMR RDS
Redshift
On-premises
S3
DynamoDB EMR
HBase on EMR RDS
Redshift
On-premises
S3
DynamoDB EMR
HBase on EMR RDS
Redshift
On premises
S3
DynamoDB EMR
HBase on EMR RDS
Redshift
On premises
S3
DynamoDB EMR
HBase on EMR RDS
Redshift
On premises
AWS Data Pipeline
Announced in November, available now.
Orchestration for data-intensive workloads.
AWS Data Pipeline
Data-intensive orchestration and automation
Reliable and scheduled
Easy to use, drag and drop
Execution and retry logic
Map data dependencies
Create and manage temporary compute
resources
Anatomy of a pipeline
Additional checks and notifications
Arbitrarily complex pipelines
aws.amazon.com/datapipeline
aws.amazon.com/big-data
Thanks
karanb@amazon.com

Más contenido relacionado

La actualidad más candente

2017 AWS DB Day | Amazon Database Migration Service (DMS) 소개 및 실습
2017 AWS DB Day | Amazon Database Migration Service (DMS) 소개 및 실습2017 AWS DB Day | Amazon Database Migration Service (DMS) 소개 및 실습
2017 AWS DB Day | Amazon Database Migration Service (DMS) 소개 및 실습Amazon Web Services Korea
 
Getting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDBGetting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDBAmazon Web Services
 
Path to the future #4 - Ingestão, processamento e análise de dados em tempo real
Path to the future #4 - Ingestão, processamento e análise de dados em tempo realPath to the future #4 - Ingestão, processamento e análise de dados em tempo real
Path to the future #4 - Ingestão, processamento e análise de dados em tempo realAmazon Web Services LATAM
 
Melhores práticas de data warehouse no Amazon Redshift
Melhores práticas de data warehouse no Amazon RedshiftMelhores práticas de data warehouse no Amazon Redshift
Melhores práticas de data warehouse no Amazon RedshiftAmazon Web Services LATAM
 
Strategic Uses for Cost Efficient Long-Term Cloud Storage
Strategic Uses for Cost Efficient Long-Term Cloud StorageStrategic Uses for Cost Efficient Long-Term Cloud Storage
Strategic Uses for Cost Efficient Long-Term Cloud StorageAmazon Web Services
 
AWS를 활용한 첫 빅데이터 프로젝트 시작하기(김일호)- AWS 웨비나 시리즈 2015
AWS를 활용한 첫 빅데이터 프로젝트 시작하기(김일호)- AWS 웨비나 시리즈 2015AWS를 활용한 첫 빅데이터 프로젝트 시작하기(김일호)- AWS 웨비나 시리즈 2015
AWS를 활용한 첫 빅데이터 프로젝트 시작하기(김일호)- AWS 웨비나 시리즈 2015Amazon Web Services Korea
 
Em tempo real: Ingestão, processamento e analise de dados
Em tempo real: Ingestão, processamento e analise de dadosEm tempo real: Ingestão, processamento e analise de dados
Em tempo real: Ingestão, processamento e analise de dadosAmazon Web Services LATAM
 
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍Amazon Web Services Korea
 
ENT313 Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum E...
ENT313 Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum E...ENT313 Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum E...
ENT313 Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum E...Amazon Web Services
 
클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)
클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)
클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)Amazon Web Services Korea
 
Understand AWS Pricing
Understand AWS PricingUnderstand AWS Pricing
Understand AWS PricingLynn Langit
 
getting started with amazon aurora
getting started with amazon auroragetting started with amazon aurora
getting started with amazon auroraAmazon Web Services
 
Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015Amazon Web Services
 
Advanced Autoscaling for Kubernetes & Amazon ECS
Advanced Autoscaling for Kubernetes & Amazon ECS Advanced Autoscaling for Kubernetes & Amazon ECS
Advanced Autoscaling for Kubernetes & Amazon ECS Amazon Web Services
 
2017 AWS DB Day | Amazon Redshift 자세히 살펴보기
2017 AWS DB Day | Amazon Redshift 자세히 살펴보기2017 AWS DB Day | Amazon Redshift 자세히 살펴보기
2017 AWS DB Day | Amazon Redshift 자세히 살펴보기Amazon Web Services Korea
 
2017 AWS DB Day | Amazon Redshift 소개 및 실습
2017 AWS DB Day | Amazon Redshift  소개 및 실습2017 AWS DB Day | Amazon Redshift  소개 및 실습
2017 AWS DB Day | Amazon Redshift 소개 및 실습Amazon Web Services Korea
 
Getting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDBGetting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDBAmazon Web Services
 
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of ThingsDay 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of ThingsAmazon Web Services
 

La actualidad más candente (20)

2017 AWS DB Day | Amazon Database Migration Service (DMS) 소개 및 실습
2017 AWS DB Day | Amazon Database Migration Service (DMS) 소개 및 실습2017 AWS DB Day | Amazon Database Migration Service (DMS) 소개 및 실습
2017 AWS DB Day | Amazon Database Migration Service (DMS) 소개 및 실습
 
Getting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDBGetting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDB
 
Path to the future #4 - Ingestão, processamento e análise de dados em tempo real
Path to the future #4 - Ingestão, processamento e análise de dados em tempo realPath to the future #4 - Ingestão, processamento e análise de dados em tempo real
Path to the future #4 - Ingestão, processamento e análise de dados em tempo real
 
Melhores práticas de data warehouse no Amazon Redshift
Melhores práticas de data warehouse no Amazon RedshiftMelhores práticas de data warehouse no Amazon Redshift
Melhores práticas de data warehouse no Amazon Redshift
 
Strategic Uses for Cost Efficient Long-Term Cloud Storage
Strategic Uses for Cost Efficient Long-Term Cloud StorageStrategic Uses for Cost Efficient Long-Term Cloud Storage
Strategic Uses for Cost Efficient Long-Term Cloud Storage
 
AWS를 활용한 첫 빅데이터 프로젝트 시작하기(김일호)- AWS 웨비나 시리즈 2015
AWS를 활용한 첫 빅데이터 프로젝트 시작하기(김일호)- AWS 웨비나 시리즈 2015AWS를 활용한 첫 빅데이터 프로젝트 시작하기(김일호)- AWS 웨비나 시리즈 2015
AWS를 활용한 첫 빅데이터 프로젝트 시작하기(김일호)- AWS 웨비나 시리즈 2015
 
Em tempo real: Ingestão, processamento e analise de dados
Em tempo real: Ingestão, processamento e analise de dadosEm tempo real: Ingestão, processamento e analise de dados
Em tempo real: Ingestão, processamento e analise de dados
 
Introduction on Amazon EC2
 Introduction on Amazon EC2 Introduction on Amazon EC2
Introduction on Amazon EC2
 
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
AWS 마이그레이션 서비스 - 김일호 :: 2015 리인벤트 리캡 게이밍
 
ENT313 Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum E...
ENT313 Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum E...ENT313 Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum E...
ENT313 Deploying a Disaster Recovery Site on AWS: Minimal Cost with Maximum E...
 
클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)
클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)
클라우드 기반 데이터 분석 및 인공 지능을 위한 비지니스 혁신 - 윤석찬 (AWS 테크에반젤리스트)
 
Understand AWS Pricing
Understand AWS PricingUnderstand AWS Pricing
Understand AWS Pricing
 
getting started with amazon aurora
getting started with amazon auroragetting started with amazon aurora
getting started with amazon aurora
 
Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015Getting Started with Big Data and HPC in the Cloud - August 2015
Getting Started with Big Data and HPC in the Cloud - August 2015
 
Amazon Aurora (Debanjan Saha) - AWS DB Day
Amazon Aurora (Debanjan Saha) - AWS DB DayAmazon Aurora (Debanjan Saha) - AWS DB Day
Amazon Aurora (Debanjan Saha) - AWS DB Day
 
Advanced Autoscaling for Kubernetes & Amazon ECS
Advanced Autoscaling for Kubernetes & Amazon ECS Advanced Autoscaling for Kubernetes & Amazon ECS
Advanced Autoscaling for Kubernetes & Amazon ECS
 
2017 AWS DB Day | Amazon Redshift 자세히 살펴보기
2017 AWS DB Day | Amazon Redshift 자세히 살펴보기2017 AWS DB Day | Amazon Redshift 자세히 살펴보기
2017 AWS DB Day | Amazon Redshift 자세히 살펴보기
 
2017 AWS DB Day | Amazon Redshift 소개 및 실습
2017 AWS DB Day | Amazon Redshift  소개 및 실습2017 AWS DB Day | Amazon Redshift  소개 및 실습
2017 AWS DB Day | Amazon Redshift 소개 및 실습
 
Getting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDBGetting Started with Amazon DynamoDB
Getting Started with Amazon DynamoDB
 
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of ThingsDay 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
Day 4 - Big Data on AWS - RedShift, EMR & the Internet of Things
 

Destacado

Construindo sua Infraestrutura de Videos e Conteudos na Nuvem
Construindo sua Infraestrutura de Videos e Conteudos na NuvemConstruindo sua Infraestrutura de Videos e Conteudos na Nuvem
Construindo sua Infraestrutura de Videos e Conteudos na NuvemAmazon Web Services LATAM
 
Otimizando e reduzindo custos na Nuvem da AWS
Otimizando e reduzindo custos na Nuvem da AWSOtimizando e reduzindo custos na Nuvem da AWS
Otimizando e reduzindo custos na Nuvem da AWSAmazon Web Services LATAM
 
Conhecendo as Opcoes de Bancos de Dados na Nuvem da AWS
Conhecendo as Opcoes de Bancos de Dados na Nuvem da AWSConhecendo as Opcoes de Bancos de Dados na Nuvem da AWS
Conhecendo as Opcoes de Bancos de Dados na Nuvem da AWSAmazon Web Services LATAM
 
Conhecendo as opcoes de Storage na Nuvem da AWS
Conhecendo as opcoes de Storage na Nuvem da AWSConhecendo as opcoes de Storage na Nuvem da AWS
Conhecendo as opcoes de Storage na Nuvem da AWSAmazon Web Services LATAM
 
(BDT303) Construct Your ETL Pipeline with AWS Data Pipeline, Amazon EMR, and ...
(BDT303) Construct Your ETL Pipeline with AWS Data Pipeline, Amazon EMR, and ...(BDT303) Construct Your ETL Pipeline with AWS Data Pipeline, Amazon EMR, and ...
(BDT303) Construct Your ETL Pipeline with AWS Data Pipeline, Amazon EMR, and ...Amazon Web Services
 
AWS Webcast - Getting Started with AWS OpsWorks
AWS Webcast - Getting Started with AWS OpsWorksAWS Webcast - Getting Started with AWS OpsWorks
AWS Webcast - Getting Started with AWS OpsWorksAmazon Web Services
 
Configuration Management with AWS OpsWorks - November 2016 Webinar Series
Configuration Management with AWS OpsWorks - November 2016 Webinar SeriesConfiguration Management with AWS OpsWorks - November 2016 Webinar Series
Configuration Management with AWS OpsWorks - November 2016 Webinar SeriesAmazon Web Services
 

Destacado (20)

Introducing AWS OpsWorks
Introducing AWS OpsWorksIntroducing AWS OpsWorks
Introducing AWS OpsWorks
 
Whole Site Delivery with Amazon CloudFront
Whole Site Delivery with Amazon CloudFrontWhole Site Delivery with Amazon CloudFront
Whole Site Delivery with Amazon CloudFront
 
Construindo sua Infraestrutura de Videos e Conteudos na Nuvem
Construindo sua Infraestrutura de Videos e Conteudos na NuvemConstruindo sua Infraestrutura de Videos e Conteudos na Nuvem
Construindo sua Infraestrutura de Videos e Conteudos na Nuvem
 
Primeira Semana na Nuvem da AWS
Primeira Semana na Nuvem da AWSPrimeira Semana na Nuvem da AWS
Primeira Semana na Nuvem da AWS
 
Otimizando e reduzindo custos na Nuvem da AWS
Otimizando e reduzindo custos na Nuvem da AWSOtimizando e reduzindo custos na Nuvem da AWS
Otimizando e reduzindo custos na Nuvem da AWS
 
Conhecendo o AWS Elastic Beanstalk
Conhecendo o AWS Elastic BeanstalkConhecendo o AWS Elastic Beanstalk
Conhecendo o AWS Elastic Beanstalk
 
AWS_Data_Pipeline
AWS_Data_PipelineAWS_Data_Pipeline
AWS_Data_Pipeline
 
Arquiteturas de Alta Disponibilidade na AWS
Arquiteturas de Alta Disponibilidade na AWSArquiteturas de Alta Disponibilidade na AWS
Arquiteturas de Alta Disponibilidade na AWS
 
Continuous Deployment e DevOps na Nuvem
Continuous Deployment e DevOps na NuvemContinuous Deployment e DevOps na Nuvem
Continuous Deployment e DevOps na Nuvem
 
Backup e Disaster Recovery na Nuvem
Backup e Disaster Recovery na NuvemBackup e Disaster Recovery na Nuvem
Backup e Disaster Recovery na Nuvem
 
Conhecendo as Opcoes de Bancos de Dados na Nuvem da AWS
Conhecendo as Opcoes de Bancos de Dados na Nuvem da AWSConhecendo as Opcoes de Bancos de Dados na Nuvem da AWS
Conhecendo as Opcoes de Bancos de Dados na Nuvem da AWS
 
Conhecendo o DynamoDB
Conhecendo o DynamoDBConhecendo o DynamoDB
Conhecendo o DynamoDB
 
Conhecendo as opcoes de Storage na Nuvem da AWS
Conhecendo as opcoes de Storage na Nuvem da AWSConhecendo as opcoes de Storage na Nuvem da AWS
Conhecendo as opcoes de Storage na Nuvem da AWS
 
AmazonRedshift
AmazonRedshiftAmazonRedshift
AmazonRedshift
 
AWS Cloud Security
AWS Cloud SecurityAWS Cloud Security
AWS Cloud Security
 
Introducao ao Amazon Redshift
Introducao ao Amazon RedshiftIntroducao ao Amazon Redshift
Introducao ao Amazon Redshift
 
Big Data na Nuvem
Big Data na NuvemBig Data na Nuvem
Big Data na Nuvem
 
(BDT303) Construct Your ETL Pipeline with AWS Data Pipeline, Amazon EMR, and ...
(BDT303) Construct Your ETL Pipeline with AWS Data Pipeline, Amazon EMR, and ...(BDT303) Construct Your ETL Pipeline with AWS Data Pipeline, Amazon EMR, and ...
(BDT303) Construct Your ETL Pipeline with AWS Data Pipeline, Amazon EMR, and ...
 
AWS Webcast - Getting Started with AWS OpsWorks
AWS Webcast - Getting Started with AWS OpsWorksAWS Webcast - Getting Started with AWS OpsWorks
AWS Webcast - Getting Started with AWS OpsWorks
 
Configuration Management with AWS OpsWorks - November 2016 Webinar Series
Configuration Management with AWS OpsWorks - November 2016 Webinar SeriesConfiguration Management with AWS OpsWorks - November 2016 Webinar Series
Configuration Management with AWS OpsWorks - November 2016 Webinar Series
 

Similar a Introducing Elastic MapReduce

Launching Your First Big Data Project on AWS
Launching Your First Big Data Project on AWSLaunching Your First Big Data Project on AWS
Launching Your First Big Data Project on AWSAmazon Web Services
 
Big Data and High Performance Computing Solutions in the AWS Cloud
Big Data and High Performance Computing Solutions in the AWS CloudBig Data and High Performance Computing Solutions in the AWS Cloud
Big Data and High Performance Computing Solutions in the AWS CloudAmazon Web Services
 
Getting Started with Amazon EC2 and Compute Services
Getting Started with Amazon EC2 and Compute ServicesGetting Started with Amazon EC2 and Compute Services
Getting Started with Amazon EC2 and Compute ServicesAmazon Web Services
 
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech TalksDeep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech TalksAmazon Web Services
 
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...Amazon Web Services
 
AWS Presentation at JasperWorld APAC
AWS Presentation at JasperWorld APACAWS Presentation at JasperWorld APAC
AWS Presentation at JasperWorld APACAmazon Web Services
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Building a Big Data & Analytics Platform using AWS
Building a Big Data & Analytics Platform using AWS Building a Big Data & Analytics Platform using AWS
Building a Big Data & Analytics Platform using AWS Amazon Web Services
 
cloud conference 2013 - Infrastructure as a Service in Amazon Web Services
cloud conference 2013 - Infrastructure as a Service in Amazon Web Servicescloud conference 2013 - Infrastructure as a Service in Amazon Web Services
cloud conference 2013 - Infrastructure as a Service in Amazon Web ServicesVMEngine
 
MongoDB Evenings Houston: Implementing EDW Using MongoDB by Purvesh Patel, Ch...
MongoDB Evenings Houston: Implementing EDW Using MongoDB by Purvesh Patel, Ch...MongoDB Evenings Houston: Implementing EDW Using MongoDB by Purvesh Patel, Ch...
MongoDB Evenings Houston: Implementing EDW Using MongoDB by Purvesh Patel, Ch...MongoDB
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)
AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)
AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)Amazon Web Services
 
Data & Analytics - Session 2 - Introducing Amazon Redshift
Data & Analytics - Session 2 - Introducing Amazon RedshiftData & Analytics - Session 2 - Introducing Amazon Redshift
Data & Analytics - Session 2 - Introducing Amazon RedshiftAmazon Web Services
 
Realtime Analytics on AWS
Realtime Analytics on AWSRealtime Analytics on AWS
Realtime Analytics on AWSSungmin Kim
 
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...Amazon Web Services
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Time Series Analytics Azure ADX
Time Series Analytics Azure ADXTime Series Analytics Azure ADX
Time Series Analytics Azure ADXRiccardo Zamana
 
Getting Started with AWS Compute Services
Getting Started with AWS Compute ServicesGetting Started with AWS Compute Services
Getting Started with AWS Compute ServicesAmazon Web Services
 

Similar a Introducing Elastic MapReduce (20)

Launching Your First Big Data Project on AWS
Launching Your First Big Data Project on AWSLaunching Your First Big Data Project on AWS
Launching Your First Big Data Project on AWS
 
Big Data and High Performance Computing Solutions in the AWS Cloud
Big Data and High Performance Computing Solutions in the AWS CloudBig Data and High Performance Computing Solutions in the AWS Cloud
Big Data and High Performance Computing Solutions in the AWS Cloud
 
Getting Started with Amazon EC2 and Compute Services
Getting Started with Amazon EC2 and Compute ServicesGetting Started with Amazon EC2 and Compute Services
Getting Started with Amazon EC2 and Compute Services
 
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech TalksDeep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
 
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
 
AWS Presentation at JasperWorld APAC
AWS Presentation at JasperWorld APACAWS Presentation at JasperWorld APAC
AWS Presentation at JasperWorld APAC
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Building a Big Data & Analytics Platform using AWS
Building a Big Data & Analytics Platform using AWS Building a Big Data & Analytics Platform using AWS
Building a Big Data & Analytics Platform using AWS
 
cloud conference 2013 - Infrastructure as a Service in Amazon Web Services
cloud conference 2013 - Infrastructure as a Service in Amazon Web Servicescloud conference 2013 - Infrastructure as a Service in Amazon Web Services
cloud conference 2013 - Infrastructure as a Service in Amazon Web Services
 
MongoDB Evenings Houston: Implementing EDW Using MongoDB by Purvesh Patel, Ch...
MongoDB Evenings Houston: Implementing EDW Using MongoDB by Purvesh Patel, Ch...MongoDB Evenings Houston: Implementing EDW Using MongoDB by Purvesh Patel, Ch...
MongoDB Evenings Houston: Implementing EDW Using MongoDB by Purvesh Patel, Ch...
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)
AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)
AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)
 
Data & Analytics - Session 2 - Introducing Amazon Redshift
Data & Analytics - Session 2 - Introducing Amazon RedshiftData & Analytics - Session 2 - Introducing Amazon Redshift
Data & Analytics - Session 2 - Introducing Amazon Redshift
 
Realtime Analytics on AWS
Realtime Analytics on AWSRealtime Analytics on AWS
Realtime Analytics on AWS
 
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...
AWS Summit 2013 | India - Petabyte Scale Data Warehousing at Low Cost, Abhish...
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Time Series Analytics Azure ADX
Time Series Analytics Azure ADXTime Series Analytics Azure ADX
Time Series Analytics Azure ADX
 
Introduction on Amazon EC2
Introduction on Amazon EC2Introduction on Amazon EC2
Introduction on Amazon EC2
 
Getting Started with AWS Compute Services
Getting Started with AWS Compute ServicesGetting Started with AWS Compute Services
Getting Started with AWS Compute Services
 

Más de Amazon Web Services LATAM

AWS para terceiro setor - Sessão 1 - Introdução à nuvem
AWS para terceiro setor - Sessão 1 - Introdução à nuvemAWS para terceiro setor - Sessão 1 - Introdução à nuvem
AWS para terceiro setor - Sessão 1 - Introdução à nuvemAmazon Web Services LATAM
 
AWS para terceiro setor - Sessão 2 - Armazenamento e Backup
AWS para terceiro setor - Sessão 2 - Armazenamento e BackupAWS para terceiro setor - Sessão 2 - Armazenamento e Backup
AWS para terceiro setor - Sessão 2 - Armazenamento e BackupAmazon Web Services LATAM
 
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.Amazon Web Services LATAM
 
AWS para terceiro setor - Sessão 1 - Introdução à nuvem
AWS para terceiro setor - Sessão 1 - Introdução à nuvemAWS para terceiro setor - Sessão 1 - Introdução à nuvem
AWS para terceiro setor - Sessão 1 - Introdução à nuvemAmazon Web Services LATAM
 
AWS para terceiro setor - Sessão 2 - Armazenamento e Backup
AWS para terceiro setor - Sessão 2 - Armazenamento e BackupAWS para terceiro setor - Sessão 2 - Armazenamento e Backup
AWS para terceiro setor - Sessão 2 - Armazenamento e BackupAmazon Web Services LATAM
 
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.Amazon Web Services LATAM
 
Automatice el proceso de entrega con CI/CD en AWS
Automatice el proceso de entrega con CI/CD en AWSAutomatice el proceso de entrega con CI/CD en AWS
Automatice el proceso de entrega con CI/CD en AWSAmazon Web Services LATAM
 
Automatize seu processo de entrega de software com CI/CD na AWS
Automatize seu processo de entrega de software com CI/CD na AWSAutomatize seu processo de entrega de software com CI/CD na AWS
Automatize seu processo de entrega de software com CI/CD na AWSAmazon Web Services LATAM
 
Ransomware: como recuperar os seus dados na nuvem AWS
Ransomware: como recuperar os seus dados na nuvem AWSRansomware: como recuperar os seus dados na nuvem AWS
Ransomware: como recuperar os seus dados na nuvem AWSAmazon Web Services LATAM
 
Ransomware: cómo recuperar sus datos en la nube de AWS
Ransomware: cómo recuperar sus datos en la nube de AWSRansomware: cómo recuperar sus datos en la nube de AWS
Ransomware: cómo recuperar sus datos en la nube de AWSAmazon Web Services LATAM
 
Aprenda a migrar y transferir datos al usar la nube de AWS
Aprenda a migrar y transferir datos al usar la nube de AWSAprenda a migrar y transferir datos al usar la nube de AWS
Aprenda a migrar y transferir datos al usar la nube de AWSAmazon Web Services LATAM
 
Aprenda como migrar e transferir dados ao utilizar a nuvem da AWS
Aprenda como migrar e transferir dados ao utilizar a nuvem da AWSAprenda como migrar e transferir dados ao utilizar a nuvem da AWS
Aprenda como migrar e transferir dados ao utilizar a nuvem da AWSAmazon Web Services LATAM
 
Cómo mover a un almacenamiento de archivos administrados
Cómo mover a un almacenamiento de archivos administradosCómo mover a un almacenamiento de archivos administrados
Cómo mover a un almacenamiento de archivos administradosAmazon Web Services LATAM
 
Os benefícios de migrar seus workloads de Big Data para a AWS
Os benefícios de migrar seus workloads de Big Data para a AWSOs benefícios de migrar seus workloads de Big Data para a AWS
Os benefícios de migrar seus workloads de Big Data para a AWSAmazon Web Services LATAM
 

Más de Amazon Web Services LATAM (20)

AWS para terceiro setor - Sessão 1 - Introdução à nuvem
AWS para terceiro setor - Sessão 1 - Introdução à nuvemAWS para terceiro setor - Sessão 1 - Introdução à nuvem
AWS para terceiro setor - Sessão 1 - Introdução à nuvem
 
AWS para terceiro setor - Sessão 2 - Armazenamento e Backup
AWS para terceiro setor - Sessão 2 - Armazenamento e BackupAWS para terceiro setor - Sessão 2 - Armazenamento e Backup
AWS para terceiro setor - Sessão 2 - Armazenamento e Backup
 
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
 
AWS para terceiro setor - Sessão 1 - Introdução à nuvem
AWS para terceiro setor - Sessão 1 - Introdução à nuvemAWS para terceiro setor - Sessão 1 - Introdução à nuvem
AWS para terceiro setor - Sessão 1 - Introdução à nuvem
 
AWS para terceiro setor - Sessão 2 - Armazenamento e Backup
AWS para terceiro setor - Sessão 2 - Armazenamento e BackupAWS para terceiro setor - Sessão 2 - Armazenamento e Backup
AWS para terceiro setor - Sessão 2 - Armazenamento e Backup
 
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
AWS para terceiro setor - Sessão 3 - Protegendo seus dados.
 
Automatice el proceso de entrega con CI/CD en AWS
Automatice el proceso de entrega con CI/CD en AWSAutomatice el proceso de entrega con CI/CD en AWS
Automatice el proceso de entrega con CI/CD en AWS
 
Automatize seu processo de entrega de software com CI/CD na AWS
Automatize seu processo de entrega de software com CI/CD na AWSAutomatize seu processo de entrega de software com CI/CD na AWS
Automatize seu processo de entrega de software com CI/CD na AWS
 
Cómo empezar con Amazon EKS
Cómo empezar con Amazon EKSCómo empezar con Amazon EKS
Cómo empezar con Amazon EKS
 
Como começar com Amazon EKS
Como começar com Amazon EKSComo começar com Amazon EKS
Como começar com Amazon EKS
 
Ransomware: como recuperar os seus dados na nuvem AWS
Ransomware: como recuperar os seus dados na nuvem AWSRansomware: como recuperar os seus dados na nuvem AWS
Ransomware: como recuperar os seus dados na nuvem AWS
 
Ransomware: cómo recuperar sus datos en la nube de AWS
Ransomware: cómo recuperar sus datos en la nube de AWSRansomware: cómo recuperar sus datos en la nube de AWS
Ransomware: cómo recuperar sus datos en la nube de AWS
 
Ransomware: Estratégias de Mitigação
Ransomware: Estratégias de MitigaçãoRansomware: Estratégias de Mitigação
Ransomware: Estratégias de Mitigação
 
Ransomware: Estratégias de Mitigación
Ransomware: Estratégias de MitigaciónRansomware: Estratégias de Mitigación
Ransomware: Estratégias de Mitigación
 
Aprenda a migrar y transferir datos al usar la nube de AWS
Aprenda a migrar y transferir datos al usar la nube de AWSAprenda a migrar y transferir datos al usar la nube de AWS
Aprenda a migrar y transferir datos al usar la nube de AWS
 
Aprenda como migrar e transferir dados ao utilizar a nuvem da AWS
Aprenda como migrar e transferir dados ao utilizar a nuvem da AWSAprenda como migrar e transferir dados ao utilizar a nuvem da AWS
Aprenda como migrar e transferir dados ao utilizar a nuvem da AWS
 
Cómo mover a un almacenamiento de archivos administrados
Cómo mover a un almacenamiento de archivos administradosCómo mover a un almacenamiento de archivos administrados
Cómo mover a un almacenamiento de archivos administrados
 
Simplifique su BI con AWS
Simplifique su BI con AWSSimplifique su BI con AWS
Simplifique su BI con AWS
 
Simplifique o seu BI com a AWS
Simplifique o seu BI com a AWSSimplifique o seu BI com a AWS
Simplifique o seu BI com a AWS
 
Os benefícios de migrar seus workloads de Big Data para a AWS
Os benefícios de migrar seus workloads de Big Data para a AWSOs benefícios de migrar seus workloads de Big Data para a AWS
Os benefícios de migrar seus workloads de Big Data para a AWS
 

Último

Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 

Último (20)

Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 

Introducing Elastic MapReduce

  • 1. Karan Bhatia, PhD Introducing Elastic MapReduce Big Data Solutions Practice
  • 2. Vários Tutoriais , treinamentos e mentoria em português Inscreva-se agora !! http://awshub.com.br
  • 4. 4 bytes x 1,000,000 households x 1 measurement/month x 10 years 480 MBytes
  • 5. 4 bytes x 1,000,000 households x 1 measurement/min x 10 years 220 TBytes
  • 6. Big Data as Business Transformation
  • 7. Generated data Available for analysis Data volume Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011 IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares
  • 9. Thousands of customers, 2 million+ clusters in 2012
  • 10. EMR Sample Use Cases
  • 11. Apontador e MapLink e AWS Apoio:
  • 12. • O que conheço do usuário? {"BaseLogId":"RmlpbjZkWVhCM0NxckNjYjF3eFU0dGNTYnhJPQ","TrackUserId":"a18e0672-ad07-4f28- b447-fc0cba90ee17","SiteId":"apto- dv01","SessionId":"1369827720327:f52c5b","ExternalId":"1933510381","Hostname":"integra01.aponta dor.lan","Path":"/local/sp/sao_paulo/bares_e_casas_noturnas/QYN7825H/","Referer":null,"PageTitle":"L ocais, Eventos, Endereços, Mapas - Apontador.com","IpAddress":"200.150.177.249","AgentInfo":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116Safari/537.36","Position":"{ "lat": -23.5934691, "lon": -46.6882606, "acc": 36}","SearchInfo":null,"RawRequestInfo":”RawRequest”: ","CreateAt":"2013-06-24T14:39:46.7082358Z"} •O que mais? Ações, cliques, buscas COMO trazer o melhor para o usuário?
  • 13. • O que recebemos para determinar o transito? <Route><Category>1</Category><DateTime>0001-01-01T00:00:00</DateTime><Destination xmlns:a="http://schemas.datacontract.org/2004/07/SwissKnife.Spatial"><a:Lat>- 8.150483</a:Lat><a:Lng>-35.420284</a:Lng></Destination><Origin xmlns:a="http://schemas.datacontract.org/2004/07/SwissKnife.Spatial"><a:Lat>- 8.149973</a:Lat><a:Lng>-35.41825</a:Lng></Origin> COMO descobrir o trânsito?
  • 14. Teorema de Bayes: O MODELO estatístico
  • 15. • Hive (~ 40 instancias spot m3.large) 90% - Utilidades diárias • Streaming 10% - Solr, MapReduces mais complexos (MCMC, FastFourier, e.g.) • Estrutura usada Hive ( ~ 40 instancias spot m3.large), Elastic MapReduce S3 (aproximadamente 7 Tb de dados estruturados em diversos buckets) RDS (dados de organização dos dados do S3) O QUE usamos?
  • 16. • A Chaordic é a empresa líder em personalização para e- commerce no Brasil, tendo como clientes 9 dos 15 maiores players do país. • Os produtos desenvolvidos pela Chaordic se integram aos maiores sites de e-commerce brasileiros e precisam de uma infra-estrutura confiável, rápida, escalável e de baixo custo. “ Com a AWS conseguimos construir um único sistema para atender a demanda dos maiores sites de e-commerce do Brasil a um custo relativamente baixo”. “Construir um data center próprio para atender nossa demanda seria economicamente inviável” - João Bosco, CTO
  • 17. O Desafio • Atender dezenas de milhões de usuários únicos por mês; • Processamento de Big Data; • Responder em menos de 100ms; • Escalar bem em momentos de pico de acesso; • Tudo isto a um custo acessível.
  • 18. Sobre o Papel da AWS e Benefícios alcançados • 4 bilhões de requisições por mês; • +300 mil requisições por minuto; • +200 milhões de recomendações todos os dias; • Spot instances: -20% custo aws.
  • 28. Elastic MapReduce Code Name node Input data S3/HDFS Queries + BI Via JDBC, Pig, Hive S3, DynamoDB, Redshift Elastic cluster
  • 29. Elastic MapReduce Code Name node Output Input data Queries + BI Via JDBC, Pig, Hive S3, DynamoDB, Redshift Elastic cluster S3/HDFS
  • 35. 1 2 4 8 16 32 64 128 256 1 2 4 8 16 32 64 128 Memory(GB) EC2 Compute Units Instance Types Standard 2nd Gen Standard Micro High-Memory High-CPU Cluster Compute Cluster GPU High I/O High-Storage Cluster High-Mem hi1.4xlarge 60.5 GB of memory 35 EC2 Compute Units 2x1024 GB SSD instance storage 64-bit platform cc1.4xlarge 23 GB of memory 33.5 EC2 Compute Units 1690 GB of instance storage 64-bit platform c1.xlarge 7 GB of memory 20 EC2 Compute Units 1690 GB of instance storage 64-bit platform m1.small 1.7 GB memory 1 EC2 Compute Unit 160 GB instance storage 32-bit or 64-bit m1.medium 3.75 GB memory 2 EC2 Compute Unit 410 GB instance storage 32-bit or 64-bit platform m1.large EBS Optimizable 7.5 GB memory 4 EC2 Compute Units 850 GB instance storage 64-bit platform m1.xlarge EBS Optimizable 15 GB memory 8 EC2 Compute Units 1,690 GB instance storage 64-bit platform m2.xlarge 17.1 GB of memory 6.5 EC2 Compute Units 420 GB of instance storage 64-bit platform m2.2xlarge 34.2 GB of memory 13 EC2 Compute Units 850 GB of instance storage 64-bit platform m2.4xlarge EBS Optimizable 68.4 GB of memory 26 EC2 Compute Units 1690 GB of instance storage 64-bit platform t1.micro 613 MB memory Up to 2 EC2 Compute Units EBS storage only 32-bit or 64-bit platform c1.medium 1.7 GB of memory 5 EC2 Compute Units 350 GB of instance storage 32-bit or 64-bit platform cg1.4xlarge 22 GB of memory 33.5 EC2 Compute Units 2 x NVIDIA Tesla “Fermi”  M2050 GPUs 1690 GB of instance storage 64-bit platform cc2.8xlarge 60.5 GB of memory 88 EC2 Compute Units 3370 GB of instance storage 64-bit platformm3.xlarge 15 GB of memory 13 EC2 Compute Units m3.2xlarge EBS Optimizable 30 GB of memory 26 EC2 Compute Units hs1.8xlarge 117 GB of memory 35 EC2 Compute Units 24x2 TB instance storage 64-bit platform cr1.8xlarge 244 GB of memory 88 EC2 Compute Units 2x120 GB SSD instance storage 64-bit platform
  • 46. 2. Rapid, tuned provisioning
  • 49. 3. Hadoop all the way down
  • 50. Robust ecosystem. Databases, machine learning, segmentation, clustering, analytics, metadata stores, exchange formats, and so on...
  • 51. 4. Agility for experimentation
  • 52. Instance choice. Stay flexible on instance type & number.
  • 55. 1. Elastic clusters 2. Rapid, tuned provisioning 3. Hadoop all the way down 4. Agility for experimentation. 5. Cost optimizations
  • 56. Data, data, everywhere... Data is stored in silos.
  • 57. S3 DynamoDB EMR HBase on EMR RDS Redshift On-premises
  • 58. S3 DynamoDB EMR HBase on EMR RDS Redshift On-premises
  • 59. S3 DynamoDB EMR HBase on EMR RDS Redshift On premises
  • 60. S3 DynamoDB EMR HBase on EMR RDS Redshift On premises
  • 61. S3 DynamoDB EMR HBase on EMR RDS Redshift On premises
  • 62. AWS Data Pipeline Announced in November, available now. Orchestration for data-intensive workloads.
  • 63. AWS Data Pipeline Data-intensive orchestration and automation Reliable and scheduled Easy to use, drag and drop Execution and retry logic Map data dependencies Create and manage temporary compute resources
  • 64. Anatomy of a pipeline
  • 65. Additional checks and notifications