1.
Support Presto
as a feature of SaaS
Presto Conference Tokyo 2020
November 20th, 2020
Satoru Kamikaseda
Staff Technical Support Engineer, Treasure Data

2.
© 2020 Treasure Data
上加世田 暁（Kamikaseda Satoru）
Background
● Rakuten - Database Administrator (2009/04 ~ )
● Treasure Data - Technical Support Engineer (2016/04 ~ )
Etc…
● Junior Football club
● Foot Golf

3.
© 2020 Treasure Data
Topics in this Presentation
● About Treasure Data & Support team
● Customer Inquiries
● How to support
● Frequently struggle points
● Proactive approaches
● Future ambitions

4.
© 2020 Treasure Data
About Treasure Data
& Support team

5.
© 2020 Treasure Data
About Treasure Data

6.
© 2020 Treasure Data
About Treasure Data

7.
© 2020 Treasure Data
About Support Team
● Head count
○ Manager 1
○ Japan 7
○ USA 2
○ Canada 1
○ UK 1
○ Uganda 1
● Many components
● Focussing on Presto in this session

8.
© 2020 Treasure Data
Customer Inquiries

9.
© 2020 Treasure Data
Customer Inquiries
● Total num of inquiries
○ Around 650 / Month
○ 170 / Week

10.
© 2020 Treasure Data
Customer Inquiries - Percentage 2020
● by inquiry category
○ Data Processing 26.64%
■ Presto
■ Hive
■ General SQL
■ Etc..
○ Workflow
○ Export
○ Import
○ Etc...

11.
© 2020 Treasure Data
● Ratio of query engine
Customer Inquiries - Percentage 2020
● Ratio of inquiry

12.
© 2020 Treasure Data
Customer Inquiries - Types 2020
● Job Investigation - 38.46%
○ The reason of Job Failure, Result, etc...
● SQL Help - 36.11%
○ Explain SQL Syntax, Functions, Advices…
● Notification - 11.32%
○ Proactive Support
■ Incident/Job failure notification,
Query tune advice, Etc….
● Performance Issue - 11.11%
○ Query execution duration issue

13.
© 2020 Treasure Data
● Cases that are difficult to resolve with support alone
○ Cases the cause cannot be identified
○ An error that's first time
○ Buggy behavior
● Aiming for 15% or less
● Roughly achieve around 8%
Customer Inquiries - Escalation Rate

14.
© 2020 Treasure Data
How to support

15.
© 2020 Treasure Data
● Accurate catch-up of the situation
● Check the actual things
● Deep investigation
● Sorting out the situation
● Answer/Report it
How to support

16.
© 2020 Treasure Data
How to support - First of all
● Accurate catch-up of the situation
○ Free format inquiry form
○ Communication is quite important
What’s
happening!?
The query results
are wrong!
Job is slow!
What’s SQL?
How to write?

17.
© 2020 Treasure Data
● Check the actual things
(sql, log, etc....)
How to support - Fact check

18.
© 2020 Treasure Data
● Check the actual things
(sql, log, etc....)
How to support - Fact check

19.
© 2020 Treasure Data
● Check cluster status (DATADOG)
○ Memory, Internal Metrics (Driver, Splits, Tasks), Coordinator, Worker, Storage, etc….
How to support - Perspective

20.
© 2020 Treasure Data
● Processing Cost Comparison (Splunk)
○ Elapsed, Splits, Total Bytes/Rows, Peak Memory, etc...
How to support - In-depth analyses(1)

21.
© 2020 Treasure Data
● Job Timeline (Splunk)
○ Job Concurrency, Memory Limitation
How to support - In-depth analyses(2)

22.
© 2020 Treasure Data
● Job Timeline (Splunk)
○ Job Concurrency, Memory Limitation
How to support - In-depth analyses(2)

23.
© 2020 Treasure Data
● Investigate as a Workflow (Splunk)
○ A single query has a small delay, but when they accumulate,
it becomes a big delay.
How to support - Multifaceted approach

24.
© 2020 Treasure Data
● Sorting out the situation
or escalate to engineering team
● Answer/Report it
○ Make a concise and understandable report
How to support

25.
© 2020 Treasure Data
Frequently struggle points

26.
© 2020 Treasure Data
Frequently struggle points
● Syntax error
● Memory exceeded
○ Join order
○ Efficient use of partitions
○ Optimal Filtering
● Inefficient query
○ Multiple scans to the same table(s)
○ Improper use of CTE (Common Table Expression, WITH Statement)

27.
© 2020 Treasure Data
Proactive approaches

28.
© 2020 Treasure Data
● Find high cost queries
○ Memory
○ Splits
○ Frequency
○ Errors
○ Others
Proactive approaches

29.
© 2020 Treasure Data
Proactive approaches
● How get things done
○ Make a benefit for the customer
■ If no benefit (motivation), nobody will get action
○ Concrete advices
■ Solutions, not just problems, are essential
○ Best communication method
■ From Support? Customer Success?
■ By mail? Slack? Call? Meeting?

30.
© 2020 Treasure Data
Future ambitions

31.
© 2020 Treasure Data
● Resource analysis automation
○ Automatic analysis and reporting of various factors
● Query tuning systemization
○ Detect inefficient queries and suggest specific tuning
points to executors
● Performance validness monitoring
○ “Performance” is an indeterminate measure
○ However, want to embody it from the log and detect
performance problems
Future ambitions

32.
© 2020 Treasure Data
Thank You!