Enviar búsqueda
Cargar
Data Catalog & ETL - Glue & Athena
•
4 recomendaciones
•
1,735 vistas
Amazon Web Services
Seguir
Data Catalog & ETL - Glue & Athena
Leer menos
Leer más
Denunciar
Compartir
Denunciar
Compartir
1 de 26
Descargar ahora
Descargar para leer sin conexión
Recomendados
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
Amazon Web Services
DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptx
Databricks
AWS Lake Formation Deep Dive
AWS Lake Formation Deep Dive
Cobus Bernard
Introduction to AWS Glue
Introduction to AWS Glue
Amazon Web Services
Making Data Timelier and More Reliable with Lakehouse Technology
Making Data Timelier and More Reliable with Lakehouse Technology
Matei Zaharia
Introduction to AWS Glue
Introduction to AWS Glue
Amazon Web Services
Building-a-Data-Lake-on-AWS
Building-a-Data-Lake-on-AWS
Amazon Web Services
Introduction to AWS Lake Formation.pptx
Introduction to AWS Lake Formation.pptx
SwathiPonugumati
Recomendados
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017
Amazon Web Services
DW Migration Webinar-March 2022.pptx
DW Migration Webinar-March 2022.pptx
Databricks
AWS Lake Formation Deep Dive
AWS Lake Formation Deep Dive
Cobus Bernard
Introduction to AWS Glue
Introduction to AWS Glue
Amazon Web Services
Making Data Timelier and More Reliable with Lakehouse Technology
Making Data Timelier and More Reliable with Lakehouse Technology
Matei Zaharia
Introduction to AWS Glue
Introduction to AWS Glue
Amazon Web Services
Building-a-Data-Lake-on-AWS
Building-a-Data-Lake-on-AWS
Amazon Web Services
Introduction to AWS Lake Formation.pptx
Introduction to AWS Lake Formation.pptx
SwathiPonugumati
Modernizing to a Cloud Data Architecture
Modernizing to a Cloud Data Architecture
Databricks
Snowflake Overview
Snowflake Overview
Snowflake Computing
Building Serverless ETL Pipelines with AWS Glue
Building Serverless ETL Pipelines with AWS Glue
Amazon Web Services
Building A Modern Data Analytics Architecture on AWS
Building A Modern Data Analytics Architecture on AWS
Amazon Web Services
Building a Data Lake on AWS
Building a Data Lake on AWS
Gary Stafford
Moving to Databricks & Delta
Moving to Databricks & Delta
Databricks
Databricks on AWS.pptx
Databricks on AWS.pptx
Wasm1953
Building Data Lakes for Analytics on AWS
Building Data Lakes for Analytics on AWS
Amazon Web Services
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Amazon Web Services
Introducing Databricks Delta
Introducing Databricks Delta
Databricks
AWS Glue - let's get stuck in!
AWS Glue - let's get stuck in!
Chris Taylor
(BDT208) A Technical Introduction to Amazon Elastic MapReduce
(BDT208) A Technical Introduction to Amazon Elastic MapReduce
Amazon Web Services
BDA311 Introduction to AWS Glue
BDA311 Introduction to AWS Glue
Amazon Web Services
Implementing a Data Lake
Implementing a Data Lake
Amazon Web Services
Introduction to Amazon Athena
Introduction to Amazon Athena
Amazon Web Services
Achieving Lakehouse Models with Spark 3.0
Achieving Lakehouse Models with Spark 3.0
Databricks
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
Databricks
Module 2 - Datalake
Module 2 - Datalake
Lam Le
Modern Data Platform on AWS
Modern Data Platform on AWS
Amazon Web Services
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
Databricks
在 AWS 上構建無服務器分析
在 AWS 上構建無服務器分析
Amazon Web Services
Cutting to the chase for Machine Learning Analytics Ecosystem & AWS Lake Form...
Cutting to the chase for Machine Learning Analytics Ecosystem & AWS Lake Form...
AWS Riyadh User Group
Más contenido relacionado
La actualidad más candente
Modernizing to a Cloud Data Architecture
Modernizing to a Cloud Data Architecture
Databricks
Snowflake Overview
Snowflake Overview
Snowflake Computing
Building Serverless ETL Pipelines with AWS Glue
Building Serverless ETL Pipelines with AWS Glue
Amazon Web Services
Building A Modern Data Analytics Architecture on AWS
Building A Modern Data Analytics Architecture on AWS
Amazon Web Services
Building a Data Lake on AWS
Building a Data Lake on AWS
Gary Stafford
Moving to Databricks & Delta
Moving to Databricks & Delta
Databricks
Databricks on AWS.pptx
Databricks on AWS.pptx
Wasm1953
Building Data Lakes for Analytics on AWS
Building Data Lakes for Analytics on AWS
Amazon Web Services
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Amazon Web Services
Introducing Databricks Delta
Introducing Databricks Delta
Databricks
AWS Glue - let's get stuck in!
AWS Glue - let's get stuck in!
Chris Taylor
(BDT208) A Technical Introduction to Amazon Elastic MapReduce
(BDT208) A Technical Introduction to Amazon Elastic MapReduce
Amazon Web Services
BDA311 Introduction to AWS Glue
BDA311 Introduction to AWS Glue
Amazon Web Services
Implementing a Data Lake
Implementing a Data Lake
Amazon Web Services
Introduction to Amazon Athena
Introduction to Amazon Athena
Amazon Web Services
Achieving Lakehouse Models with Spark 3.0
Achieving Lakehouse Models with Spark 3.0
Databricks
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
Databricks
Module 2 - Datalake
Module 2 - Datalake
Lam Le
Modern Data Platform on AWS
Modern Data Platform on AWS
Amazon Web Services
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
Databricks
La actualidad más candente
(20)
Modernizing to a Cloud Data Architecture
Modernizing to a Cloud Data Architecture
Snowflake Overview
Snowflake Overview
Building Serverless ETL Pipelines with AWS Glue
Building Serverless ETL Pipelines with AWS Glue
Building A Modern Data Analytics Architecture on AWS
Building A Modern Data Analytics Architecture on AWS
Building a Data Lake on AWS
Building a Data Lake on AWS
Moving to Databricks & Delta
Moving to Databricks & Delta
Databricks on AWS.pptx
Databricks on AWS.pptx
Building Data Lakes for Analytics on AWS
Building Data Lakes for Analytics on AWS
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Introducing Databricks Delta
Introducing Databricks Delta
AWS Glue - let's get stuck in!
AWS Glue - let's get stuck in!
(BDT208) A Technical Introduction to Amazon Elastic MapReduce
(BDT208) A Technical Introduction to Amazon Elastic MapReduce
BDA311 Introduction to AWS Glue
BDA311 Introduction to AWS Glue
Implementing a Data Lake
Implementing a Data Lake
Introduction to Amazon Athena
Introduction to Amazon Athena
Achieving Lakehouse Models with Spark 3.0
Achieving Lakehouse Models with Spark 3.0
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
Module 2 - Datalake
Module 2 - Datalake
Modern Data Platform on AWS
Modern Data Platform on AWS
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
Similar a Data Catalog & ETL - Glue & Athena
在 AWS 上構建無服務器分析
在 AWS 上構建無服務器分析
Amazon Web Services
Cutting to the chase for Machine Learning Analytics Ecosystem & AWS Lake Form...
Cutting to the chase for Machine Learning Analytics Ecosystem & AWS Lake Form...
AWS Riyadh User Group
Data_Analytics_and_AI_ML
Data_Analytics_and_AI_ML
Amazon Web Services
Building a Modern Data Platform in the Cloud
Building a Modern Data Platform in the Cloud
Amazon Web Services
Building-Serverless-Analytics-On-AWS
Building-Serverless-Analytics-On-AWS
Amazon Web Services
AWS Data Lake: data analysis @ scale
AWS Data Lake: data analysis @ scale
Amazon Web Services
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWS
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWS
Steven Hsieh
Automate Business Insights on AWS - Simple, Fast, and Secure Analytics Platforms
Automate Business Insights on AWS - Simple, Fast, and Secure Analytics Platforms
Amazon Web Services
Implementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdf
Amazon Web Services
Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28
Amazon Web Services
AWS Floor 28 - Building Data lake on AWS
AWS Floor 28 - Building Data lake on AWS
Adir Sharabi
Immersion Day - Como gerenciar seu catálogo de dados e processo de transform...
Immersion Day - Como gerenciar seu catálogo de dados e processo de transform...
Amazon Web Services LATAM
Wild Rydes with Big Data/Kinesis focus: AWS Serverless Workshop
Wild Rydes with Big Data/Kinesis focus: AWS Serverless Workshop
AWS Germany
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Amazon Web Services
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Amazon Web Services
Value of Data Beyond Analytics by Darin Briskman
Value of Data Beyond Analytics by Darin Briskman
Sameer Kenkare
Building Serverless Analytics Solutions with Amazon QuickSight (ANT391) - AWS...
Building Serverless Analytics Solutions with Amazon QuickSight (ANT391) - AWS...
Amazon Web Services
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Amazon Web Services
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Amazon Web Services
Data Warehouses and Data Lakes
Data Warehouses and Data Lakes
Amazon Web Services
Similar a Data Catalog & ETL - Glue & Athena
(20)
在 AWS 上構建無服務器分析
在 AWS 上構建無服務器分析
Cutting to the chase for Machine Learning Analytics Ecosystem & AWS Lake Form...
Cutting to the chase for Machine Learning Analytics Ecosystem & AWS Lake Form...
Data_Analytics_and_AI_ML
Data_Analytics_and_AI_ML
Building a Modern Data Platform in the Cloud
Building a Modern Data Platform in the Cloud
Building-Serverless-Analytics-On-AWS
Building-Serverless-Analytics-On-AWS
AWS Data Lake: data analysis @ scale
AWS Data Lake: data analysis @ scale
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWS
AWS 2019 Taipei Summit - Building Serverless Analytics Platform on AWS
Automate Business Insights on AWS - Simple, Fast, and Secure Analytics Platforms
Automate Business Insights on AWS - Simple, Fast, and Secure Analytics Platforms
Implementazione di una soluzione Data Lake.pdf
Implementazione di una soluzione Data Lake.pdf
Building Data Lake on AWS | AWS Floor28
Building Data Lake on AWS | AWS Floor28
AWS Floor 28 - Building Data lake on AWS
AWS Floor 28 - Building Data lake on AWS
Immersion Day - Como gerenciar seu catálogo de dados e processo de transform...
Immersion Day - Como gerenciar seu catálogo de dados e processo de transform...
Wild Rydes with Big Data/Kinesis focus: AWS Serverless Workshop
Wild Rydes with Big Data/Kinesis focus: AWS Serverless Workshop
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Value of Data Beyond Analytics by Darin Briskman
Value of Data Beyond Analytics by Darin Briskman
Building Serverless Analytics Solutions with Amazon QuickSight (ANT391) - AWS...
Building Serverless Analytics Solutions with Amazon QuickSight (ANT391) - AWS...
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Building Data Lakes and Analytics on AWS
Data Warehouses and Data Lakes
Data Warehouses and Data Lakes
Más de Amazon Web Services
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Amazon Web Services
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Amazon Web Services
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
Amazon Web Services
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
Amazon Web Services
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
Amazon Web Services
Open banking as a service
Open banking as a service
Amazon Web Services
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Amazon Web Services
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
Amazon Web Services
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Amazon Web Services
Computer Vision con AWS
Computer Vision con AWS
Amazon Web Services
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
Amazon Web Services
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Amazon Web Services
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
Amazon Web Services
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Amazon Web Services
Tools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
How to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
Building a web application without servers
Building a web application without servers
Amazon Web Services
Fundraising Essentials
Fundraising Essentials
Amazon Web Services
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
Amazon Web Services
Más de Amazon Web Services
(20)
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
Open banking as a service
Open banking as a service
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Computer Vision con AWS
Computer Vision con AWS
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Tools for building your MVP on AWS
Tools for building your MVP on AWS
How to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Building a web application without servers
Building a web application without servers
Fundraising Essentials
Fundraising Essentials
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
Data Catalog & ETL - Glue & Athena
1.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark HC Lo, Solutions Architect Data Catalog & ETL - Glue & Athena September 12, 2019
2.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark What is AWS Glue Data Catalog? Unified metadata repository across relational databases, Amazon RDS, Amazon Redshift, and Amazon S3…with support for more coming! • Get a single view into your data, no matter where it is stored • Automatically classify your data in one central list that is searchable • Track data evolution using schema versioning • Query your data using Amazon Athena or Amazon Redshift Spectrum • Hive metastore compatible; can be used as an external Hive Metastore for applications running on Amazon EMR
3.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark What is a Data Lake Architectural pattern enabling: • Ubiquitous storage at any scale • Consolidated data processing • Collaborate and analyze data in different ways leading to better, faster decision making
4.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Most comprehensive Broadest and deepest portfolio, purpose-built for builders Migration & Streaming Services Infrastructure Data Catalog & ETL Security & Management Data Warehousing Big Data Processing Interactive Query Operational Analytics Real time Analytics Serverless Data processing Data Movement Analytics Data Lake Infrastructure & Management Dashboards Predictive Analytics Visualization, Engagement, & Machine Learning Digital User Engagement
5.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Data Movement Analytics Most comprehensive Broadest and deepest portfolio, purpose-built for builders + 11 more Redshift EMR (Spark & Hadoop) Athena Elasticsearch Service Kinesis Data Analytics Glue (Spark & Python) S3/Glacier GlueLake Formation Visualization, Engagement, & Machine Learning QuickSight SageMaker Comprehend Lex Polly Rekognition Translate Transcribe Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams | Managed Streaming for Kafka Data Lake Infrastructure & Management Pinpoint
6.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Popular Customer Use Cases
7.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data Lake on AWS On premises data Web app data Amazon RDS Other databases Streaming data Your data AWS GLUE ETL Amazon QuickSight Amazon SageMaker
8.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Log Aggregation AWS Service Logs Web Application Logs Server Logs S3 Athena New File Trigger Update table partition Create partition on S3 Copy to new partition Query data S3 Lambda Glue Data Catalog
9.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Log Aggregation with ETL AWS Service Logs Web Application Logs Server Logs S3 Athena Glue Crawler Update table partition Create partition on S3 Query data S3 Glue ETL Glue Data Catalog
10.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Real-Time Data Collection S3 Athena Real-time events Store partitioned in S3 Trigger Job Update table partition Query data Kinesis Glue ETL Glue Data Catalog
11.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data Export S3 Athena Database Migration Exported tables in S3 Trigger Job Update table partition Query data Database Migration Service Glue ETL Glue Data Catalog
12.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SaaS Model S3 Athena Query data Hot data Warn & cold dataApplication request Glue Data Catalog
13.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data Science S3 Athena Application Data S3 Glue ETL Athena SageMaker EMR Enrichment Feature Store Glue Data Catalog
14.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. S3S3 AWS Glue ETL Athena Amazon Reviews Dataset Glue Data Catalog 1 Comprehend 2 3 Glue Crawler 4 QuickSight 5 Data Enrichment – Amazon Comprehend
15.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Connect Kinesis Data Streams Agent Events Kinesis Data Firehose S3 Athena AWS Glue Data Catalog Firehouse Output Schema Parquet 1 2 3 4 5 Redshift Spectrum Data Ingest in Parquet Format
16.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Analytics Reporting Athena Redshift Spectrum EMR API QuickSight Glue Data Catalog
17.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Amazon Athena is an interactive query service that makes it easy to analyze data directly on Amazon S3 using Standard SQL
18.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Why Amazon Athena ? • Decouple storage from compute • Serverless – No infrastructure or resources to manage • Pay only for data scanned • Schema on read – Same data, many views • Secure – IAM for authentication; Encryption at rest & in transit • Standard compliant and open storage file formats • Built on powerful community supported OSS solutions
19.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Familiar Technologies Under the Covers Used for SQL Queries In-memory distributed query engine ANSI-SQL compatible with extensions (Eg. SELECT * FROM tableName) Used for DDL functionality Complex data types Multitude of formats Supports data partitioning (Eg. CREATE TABLE, ALTER TABLE, MSCK REPAIR)
20.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Presto SQL • ANSI SQL compliant • Complex joins, nested queries & window functions • Complex data types (arrays, structs, maps) • Presto built-in functions • File Formats: CSV, JSON, RegEx, Parquet, Avro, ORC, CloudTrail • Compression: GZIP, Zlib, LZO, Snappy • Integrated with AWS Glue Data Catalog
21.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark A Better Model Old Methodology • Analyst asks for a report • Developer writes code • Code executes on shared cluster for several hours • Analyst reviews report • Analyst asks for more… With Amazon Athena • Analyst creates table • Analyst iterates • Generate final report Simple, Quick and No Infrastructure or Developer to Manage
22.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Simple Pricing • DDL operations – FREE • SQL operations – FREE • Query concurrency – FREE • Data scanned - $5 / TB • Standard S3 rates for storage, requests, and data transfer apply
23.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Security and Access Control • Encryption – SSE, SSE-KMS, CSE-KMS • Auto detect source bucket KMS key • Destination bucket may use separate key • Access Control • IAM • S3 ACL • S3 bucket policies • Coming… Athorization with Glue Data Catalog • Database level • Table level
24.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Cost Monitoring • Billing console provides spend per account • Athena APIs are logged in CloudTrail • Combine CloudTrail and Athena API for per IAM user cost • More cost controls to come…
25.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark LAB 2 - Guide http://bit.ly/2md1R9z
26.
© 2019, Amazon
Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2019, Amazon Web Services, Inc. or its Affiliates. 【AWS 亞馬遜雲端聚落】 意猶未盡 ? 立即加入LINE好友 >>掌握AWS最新消息 ! Thank you!
Descargar ahora