Data Orchestration Summit www.alluxio.io/data-orchestration-summit-2019 November 7, 2019 Presto on Alluxio Hands-On Lab Speakers: Bin Fan, Alluxio Zac Blanco, Alluxio Kamil Bajda-Pawlikowski, Starburst, Presto Company Martin Traverso, Presto Software Foundation For more Alluxio events: https://www.alluxio.io/events/

1. DATA ORCHESTRATION SUMMIT
2019
Presto on Alluxio Hands-On Lab
Bin Fan, Zac Blanco (Alluxio)
Kamil Bajda-Pawlikowski (Starburst)
MartinTraverso (Presto Software Foundation)

2. DATA ORCHESTRATION
SUMMIT
2019
Agenda
• (10 min) Overview: Presto on Alluxio
• (30 min) Lab: Running Presto on Alluxio
• (10 min) Q&A

3. DATA ORCHESTRATION
SUMMIT
2019
Overview: Presto on Alluxio
Architecture,Workflow, Benefit

4. How Presto Works with Alluxio
Presto
Hive
Metastore
location=s3://bucket/table
Read/Write
Metadata
Read/Write
Data
Presto
Alluxio
Mounted to Alluxio
Hive
Metastore
location=alluxio:///table
Read/Write
Metadata Read/Write
Data
4

5. § Distributed Data Caching on Demand
• Lower query latency
• More consistent performance
• More concurrency
• Less data transfer
§ Deeper Presto Alluxio Integra?on
• New Alluxio catalog service
• New Alluxio transforma?on service
Why Presto on Alluxio
Now available as Developer Preview in v2.1
5

6. How to Use Alluxio in Presto CLI
> CREATE TABLE alluxio_table (id varchar)
WITH (external_location = 'alluxio:///table');
> SELECT * FROM alluxio_table
Create A Table on Alluxio
Read A Table from Alluxio
6

7. Deployment Setup for High Data Locality
Presto
Coordinator
Alluxio
Master
Presto
Worker
Alluxio
Worker
Presto
Worker
Alluxio
Worker
Presto
Worker
Alluxio
Worker
Hive
Metastore
Metadata Ops
location=alluxio:///table
Mounted to Alluxio
7

8. Common Use Cases
Accelerate query performance
as cloud storage caching
Alluxio
Spark
AlluxioAlluxio
Spark
Alluxio
SparkPresto
Read more
AWS S3 + Alluxio + Presto = ❤ The Ryte Use Case
On-premise satellite
compute clusters across data centers
Satellite Presto Cluster
Alluxio
SparkHive
Main Hadoop Cluster
Presto
Read more Using Alluxio in JD.com
Zero-copy burst workloads in
hybrid cloud environments
Hive
Alluxio
Hive
Alluxio
Hive
Alluxio
Presto
Alluxio
8

9. DATA ORCHESTRATION
SUMMIT
2019
Hands-on Lab
• Create a table stored on Alluxio
• Query the table from Presto
• Mount buckets to Alluxio
• Manage Alluxio cache capacity

10. Presto/Alluxio AMI Sandbox
Presto
Alluxio
Mounted to Alluxio
Hive
Metastore
§ Architecture:
• Single EC2 Instance
• Running Presto/HMS/Alluxio/S3
§ Follow instructions
https://tinyurl.com/y2g9z77q
§ Passphrase: “PrestoAlluxio”
10

11. Recap of the Lab
Presto
Alluxio
Mounted to Alluxio
Hive
Metastore
§ Create a table in Presto CLI by simply
pointing file locations to Alluxio
§ Query latency to read tables from S3
can be slow with large variance
§ Reading tables repeatedly benefits
from data transparently cached in
Alluxio, with faster & more consistent
query latency
§ To rerun this hands-on lab:
https://www.alluxio.io/products/aws/a
lluxio-presto-sandbox-aws/
11

12. DATA ORCHESTRATION
SUMMIT
2019
Q&A

13. § Check out more tutorials https://www.alluxio.io/presto/
§ Additional Reads:
• Starburst Presto + Alluxio = better together
https://www.starburstdata.com/technical-blog/starburst-presto-alluxio-better-together/
• Top 5 performance tips running Presto with Alluxio
https://www.alluxio.io/blog/top-5-performance-tuning-tips-for-running-presto-on-alluxio-1
• Presto + Alluxio + Hive Metastore on your Laptop in 10 min
https://www.alluxio.io/blog/tutorial-presto-alluxio-hive-metastore-on-your-laptop-in-10-
min/
13
Next Step