In this chalk talk, we take a deep dive on Amazon Redshift architecture and the latest performance enhancements that give you faster insights into your data. We also cover Amazon Redshift Spectrum, a feature of Amazon Redshift that enables you to analyze data across Amazon Redshift and your Amazon S3 data lake to deliver unique insights not possible by analyzing independent data silos.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Greg Khairallah (gregkh@amazon.com)
SRV337
Building a Modern Data Warehouse
Head of Business Development, Database, and Analytics, Amazon Web Services

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Driving value from data
Traditional analytics
Data lakes extend the traditional approach
Amazon Redshift & Amazon Redshift Spectrum
Recently released features
Agenda

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Organizations that successfully generate business
value from their data will outperform their peers. An
Aberdeen survey saw organizations who implemented
a data lake outperforming similar companies by 9% in
organic revenue growth.*
24%
15%
Leaders Followers
Organic revenue growth
*Aberdeen: Angling for Insight in Today’s Data Lake, Michael Lock, SVP Analytics and Business Intelligence
Most Important: Driving Value from Data

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Is Changing
Capture and store
new data at PB-EB scale
Do new type of analytics in
a cost effective way
• Machine learning
• Big data processing
• Real-time analytics
• Full-text search
New types of
analytics

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Traditionally, Analytics Used to Look Like This
OLTP ERP CRM LOB
Data warehouse
Business intelligence • Relational data
• TBs–PBs scale
• Schema defined prior to data load
• Operational reporting and ad hoc
• Large initial capex + $10K–$50K/TB/year

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Lakes Extend the Traditional Approach
Data warehouse
Business intelligence
OLTP ERP CRM LOB
• Relational and non-relational data
• TBs–EBs scale
• Diverse analytical engines
• Low-cost storage & analytics
Devices Web Sensors Social
Data lake
Big data processing,
real-time, machine learning

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Lakes from AWS
Analytics
• Unmatched durability, and availability at EB scale
• Best security, compliance, and audit capabilities
• Object-level controls for fine-grain access
• Fastest performance by retrieving subsets of data
• The most ways to bring data in
• 2x as many integrations with partners
• Analyze with broadest set of analytics & ML services
Machine
learning
Real-time dataOn-premises
Data lake
on AWS
movementdata movement

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS makes data management and analysis easy
Central Storage
Secure, cost-effective storage with
life cycle policies
Data Ingestion
Get your data into Amazon S3 quickly and securely
Amazon Kinesis Date Firehose, AWS Direct Connect, AWS
Snowball, AWS Database Migration Service
Catalog & Search
Access and search metadata
Processing & Analytics
Predictive and prescriptive analytics
AWS Glue, Amazon DynamoDB,
Amazon Elasticsearch Service
Amazon Athena, Amazon QuickSight,
Amazon EMR, Amazon Redshift
IAM, Amazon CloudWatch, AWS CloudTrail, AWS KMS
Protect & Secure
Use entitlements to ensure data is secure and users’
identities are verified
Amazon S3
Amazon Glacier,
Glacier Select, and
Vault Lock
Archive
Extremely low-cost, durable storage

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift
10x faster at 1/10th the cost
Fast
Delivers fast results for all types
of workloads
Cost-effective
No upfront costs, start small,
and pay as you go
Integrated Secure
Audit everything; encrypt data
end-to-end; extensive
certification and compliance
Integrated with S3 data lakes, AWS
services and third-party tools
$
Simple
Create and start using a data
warehouse in minutes
Scalable
Gigabytes to petabytes
to exabytes

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift Spectrum
Extend the data warehouse to your Amazon S3 data lake
Scale compute and storage separately
Join data across Amazon Redshift and Amazon S3
Exabyte-scale Redshift SQL queries against Amazon S3
Stable query performance and unlimited concurrency
Parquet, ORC, JSON, Grok, Avro, & CSV formats
Pay only for the amount of data scanned
S3 data lakeRedshift data
Redshift Spectrum
query engine

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Glue
Amazon
S3
Data
sources
Amazon
Redshift
Redshift
Spectrum
BI Tools
Amazon Redshift Spectrum is a game changer for us. Reports that took minutes to produce are now delivered in
seconds. We like the ability to scale compute on-demand to query Petabytes of data in Amazon S3 in various
open file formats.”
NUVIAD is a mobile marketing platform providing
professional marketers, agencies, and local businesses with
hyper-targeted analytics at petabyte scale
Data Lake Analytics with Amazon Redshift Spectrum
• Seamlessly analyzing open file formats directly
in Amazon S3 to provide fresh, up-to-the-
minute insights
• Unlimited analytics and query concurrency
with Amazon Redshift
• Unlimited data capacity with Amazon S3
• 80% performance gain using Parquet data
format
– Rafi Ton
CEO, NUVIAD
“

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Query
SELECT COUNT(*)
FROM S3.EXT_TABLE
GROUP BY…
Life of a Spectrum
Query
Amazon
Redshift
JDBC/ODBC
...
1 2 3 4 N
Amazon S3
Exabyte-scale object storage
Data Catalog
Apache Hive Metastore
1
Query is optimized and compiled at
the leader node. Determine what gets
run locally and what goes to Amazon
Redshift Spectrum.
2
Query plan is sent to
all compute nodes.
3
Compute nodes obtain partition info from
Data Catalog; dynamically prune partitions.
4
Each compute node issues multiple
requests to the Amazon Redshift
Spectrum layer.
5
Amazon Redshift Spectrum nodes
scan your Amazon S3 data.
6
7
Amazon Redshift
Spectrum projects,
filters, joins, and
aggregates.
Final aggregations and joins
with local Amazon Redshift
tables done in-cluster.
8
Result is sent back to client.9

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Multiple Redshift Clusters Scaling Performance
Amazon S3
Exabyte-scale object storage
AWS Glue
Data Catalog
Apache Hive Metastore
Amazon
Redshift
Amazon
Redshift
Amazon
Redshift
...
1 2 3 4 N

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift Spectrum – Current Support
File formats
• Avro, CSV, Grok, Ion,
JSON, ORC, Parquet,
RCFile, RegexSerDe,
SequenceFile, TextFile,
and TSV
Compression
• Gzip
• Snappy
• Bz2
• Brotli
Encryption
• SSE with AES256
• SSE KMS with default
key
Column types
• Numeric: bigint, int, smallint, float, double
and decimal
• Char/varchar/string
• Timestamp
• Boolean
• DATE type can be used only as a
partitioning key
Table type
• Non-partitioned table
(s3://mybucket/orders/..)
• Partitioned table
(s3://mybucket/orders/date=YYYY-MM-
DD/..)

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Recently Released Features

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Dense Compute Nodes (DC2)
2x performance as DC1 at the same price
3x more I/O with
Upgrade at no cost
30% better storage utilization
than DC1
“Amazon Redshift’s new DC2 node is giving us a
100 percent performance increase, allowing us to
provide faster insights for our retailers, more cost
effectively, to drive incremental revenue."
NVMe SSD DDR4 memory
Intel E5-2686 v4 (Broadwell)

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Short Query Acceleration
Express Lane for Short Queries
• Machine learning predicts the
runtime of queries
• Short queries are routed to an
express queue
• Resources are dynamically
dedicated to short queries
• Enable it today from your
AWS Management Console
How it works:
Analytics and
BI / Dashboard tools
Amazon
Redshift Machine Learning
Classifier
Machine learning

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Result-Set Caching
Subsecond repeat queries
• Amazon Redshift customers can now serve 35% more queries on average,
using the same compute resources
• Tens of thousands of compute hours are freed up daily to serve the
remaining queries and data ingestion
• Transparent – it just works!
“With Amazon Redshift result caching, 20 percent of our
queries now complete in less than one second,” said
Greg Rokita, Executive Director for Technology, Edmunds

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Commit Enhancements
50% faster data commits for busy clusters
16% faster data ingestion and insertion
Commit Duration Per Transaction for Busy Clusters
Nov Jan Mar
Total Commit Time by Month
ds2.8xlarge, cluster size: 10 and up, us-west-2
Clusters with more than 90 backups a day
p99 p95 p90 p50 Linear (p99)

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Query Performance Improvements
• Faster hash joins
• Improvements to hash algorithm (Jan '18)
• Significant improvement in memory utilization (Feb '18)
• Cache line prefetching to improve join performance (Mar '18)
• Join-intensive workloads like TPC-H and TPC-DS show a performance improvement
ranging from 28% to 2x for several queries
• 64x reduction of memory footprint fleet wide for hash joins and aggregations.
Significant improvement to overall throughput
• Read and write queries can now hop WLM queues

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift Advisor
Advisor provides
automated
recommendations to help
you optimize database
performance and
decrease operating costs.
Shows up to seven
recommendations to help
you optimize your cluster.
Available via the
Amazon Redshift console
at no charge.

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
New Amazon CloudWatch metrics for easy visualization of
cluster performance
• Monitor the performance and
health of your Amazon Redshift
cluster with two new Amazon
CloudWatch metrics, Query
Throughput and Query Duration.
• Query Throughput measures the
average number of queries
completed per second. Query
Duration measures the average
time taken to complete a query.
By observing these metrics, you
can easily determine how your
cluster is performing at any time.

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Redshift Spectrum Enhancements
• Available in 14 AWS Regions
• Added support for processing scalar JSON and ION file formats in S3
• In addition to Parquet, ORC, Avro, CSV, Grok, RCFile, RegexSerDe,
OpenCSV, SequenceFile, TextFile, and TSV
• Support for DATE data type
• Support for IAM role-chaining to assume cross-account roles
• COPY from Parquet, ORC

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Coming Soon: Nested Data Support
• Analyze nested and semi-structured data in Amazon S3 with Spectrum
• Allows easy ETL of nested data in to Redshift using CTAS
• Support for open file formats: Parquet, ORC, JSON and Ion
• Uses dot notation to extend your existing SQL
s3data.clickStream: <<
{ “session_time”: “20171013 14:05:00”,
“clicks”: [ {“page”: “/home”, “referrer”: “”},
{“page”: “/products”, “referrer”: “/home”} ]
},
{ “session_time”: “20171013 14:06:00”,
“clicks”: [ {“page”: “/contact”, “referrer”: “/home”} ]
} >>
SELECT c.page,
COUNT(*) AS count
FROM s3data.clickStream s,
s.clicks c
WHERE s.session_time > ‘2017-10-01 00:00:00’
AND c.referrer = “/home”
GROUP BY c.page;
Example: Find click frequency for links on “/home”:

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Coming Soon: Nested Data Support
Improve query performance by analyzing nested data
OrderID CustomerID OrderTime ShipMode
5 23 10.00 12.50
8 32 1.00 5.60
OrdersWithItems
ItemID Quantity Price
23 10.00 12.50
16 1.00 1.99
32 1.00 5.60
24 5.00 26.50
OrderItems
OrderID ItemID Quantity Price
5 23 10.00 12.50
8 32 1.00 5.60
5 16 1.00 1.99
8 24 5.00 26.50
OrderID CustomerID OrderTime ShipMode
5 23 10.00 12.50
8 32 1.00 5.60
Orders
OrderItems
To improve query
performance, the
new Orders table
includes the
OrdersWithItems as
a nested column,
eliminating join
processing

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Big Data Blog—Amazon Redshift
Amazon Redshift Engineering’s Advanced Table Design Playbook
https://aws.amazon.com/blogs/big-data/amazon-redshift-engineerings-advanced-table-design-
playbook-preamble-prerequisites-and-prioritization/
- Zach Christopherson
Top 10 Performance Tuning Techniques for Amazon Redshift
https://aws.amazon.com/blogs/big-data/top-10-performance-tuning-techniques-for-amazon-redshift/
- Ian Meyers and Zach Christopherson
10 Best Practices for Amazon Redshift Spectrum
https://aws.amazon.com/blogs/big-data/10-best-practices-for-amazon-redshift-spectrum/
- Po Hong and Peter Dalton

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Find out more: https://aws.amazon.com/redshift/
Try Amazon Redshift
Get help with your Proof-of-Concept
Read Amazon Redshift blog articles:
https://aws.amazon.com/redshift/blog-posts/
Get Started with Amazon Redshift
Amazon
Redshift

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Submit Session Feedback
1. Tap the Schedule icon.
2. Select the session you attended.
3. Tap Session Evaluation to submit your
feedback.

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!