The companies Videology and Zendesk both had the same problem—how to rearchitect their big data processing platforms to scale to meet growing demand, while at the same time improving performance, availability, and cost structure? Videology provides a converged advertising solution that is screen-agnostic, ensuring unduplicated reach with the right frequency cadence to achieve guaranteed results. To achieve this, their big data platform ingests, processes, and analyzes a variety of logs. In this talk, Videology discusses how they migrated our platform to use Cloudera on a mix of m4 and r3 instances using the Amazon EBS Streaming Optimized HDD (st1) volume type. Zendesk provides a cloud-based customer support platform that allows quicker and easier interaction between businesses and customers. To deliver this experience, Zendesk runs a large Elasticsearch, Logstash, Kibana stack. The talk also discusses how Zendesk rearchitected their deployment to use m4s and also leverage the EBS Streaming Optimized HDD (st1) volume type. Tips for success will be shared throughout.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
David Stein, Business Development EBS
November 30, 2016
Case Study: How Zendesk and
Videology Modernized Their Big
Data Platforms on Amazon EBS
STG311

2. What to Expect from the Session
• How to architect big data processing platforms to scale to meet
growing demand while improving performance, availability, and cost
with Amazon EBS
• Learn how about new ST1 and SC1 Throughput Optimized EBS
volumes designed for big data workloads
• Overview of how Zendesk runs a large ELK (Elasticsearch,
Logstach, Kibana) on Amazon EC2 and EBS for their cloud-based
customer support platform
• Overview of how Videology runs a Hadoop architecture on EC2 and
EBS to ingest, process, and analyze logs for their converged
advertising solution

3. Amazon EFS
File
Amazon EBS
Amazon EC2
Instance Store
Block
Amazon S3 Amazon Glacier
Object
AWS storage is a platform
Data Transfer
AWS Direct
Connect
ISV
Connectors
Amazon
Kinesis
Firehose
AWS Storage
Gateway
Amazon S3
Transfer
Acceleration
AWS
Snowball
Amazon
CloudFront
Internet/VPN

4. EBS volume types
Hard disk drive
(HDD)
Solid state drive
(SSD)

5. EBS volume types
General Purpose
SSD
gp2
Provisioned IOPS
SSD
io1
Throughput Optimized
HDD
st1
Cold
HDD
sc1
SSD HDD

6. EBS volume types: throughput
Throughput
Optimized HDD
st1
Baseline: 40 MB/s per TB up to 500 MB/s
Capacity: 500 GB to 16 TB
Burst: 250 MB/s per TB up to 500 MB/s
Ideal for large-block, high-throughput sequential workloads

7. Cold HDD
sc1
EBS volume types: throughput
Baseline: 12 MB/s per TB up to 192 MB/s
Capacity: 500 GB to 16 TB
Burst: 80 MB/s per TB up to 250 MB/s
Ideal for sequential throughput workloads such as logging and backup

8. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kyle House, David Bernstein, Zendesk
November 30, 2016
Case Study: How Zendesk Modernized
Their Big Data Platforms on Amazon EBS
Inside Our New ELK Deployment

9. Zendesk builds software for better customer relationships. It empowers
organizations to improve customer engagement and better understand
their customers. More than 87,000 paid customer accounts in over 150
countries and territories use Zendesk products. Based in San Francisco.

10. What to Expect from the Session
• Discuss storage redesign, utilizing
new Amazon EBS volumes
• Talk through design choices
• Explain benefits of new storage
• model
• Cost benefits of “rightsizing” storage

11. ELK at Zendesk
Distributed database
Log ingestion/parsing
Beautiful visualizations

15. The Problem
- Operational headaches
- Encryption
- Data retention
- Cost too high

18. The Investigation
- User access patterns
- Performance requirements
- New EBS volume types

21. The Proposal
- Full usage of EBS with new volume types
- Create a tiered storage model
- Optimize instance types; decouple instances from
storage

22. Tiered storage
Hot (0-7 days)
General Purpose
SSD (gp2)
Warm (8-30 days)
Throughput
Optimized HDD (st1)
Cold (31-60 days) Cold HDD (sc1)

32. Topology
VPN
gateway
3 x m4.large
esclient/esmaster
Proxy
Bastion
3 x m4.large
esclient/esmaster
gp2 roots
8 x c4.large
logindexers
8 x c4.large
logindexers
gp2 roots
gp2 roots
gp2 roots
gp2 roots +
11G (hot)
st1
35G (warm)
sc1
80g (cold)
10 x r3.2large
esdata
10 x r3.2large
esdata
gp2 roots +
11G (hot)
st1
35G (warm)
sc1
80g (cold)
Availability Zone
Availability Zone

34. Sparkleformation

38. The Result
- Reduced operating costs by 50%
- Increased data retention 3x
- Predictable scaling model
• Storage allocation detached from instance count
- Increased data transport reliability
- Reduced operational overhead
- Increased cluster stability

39. 49% Reduction 79% Reduction

40. Recommendations
- Identify data usage model before you build
- Find places where performance matters, and where
cost can be optimized
- Reduce over-provisioned storage/IOPS
- Utilize AWS managed services whenever possible

41. Thank you!
Up next in this session:
Videology

42. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Videology
Paul Frederiksen – Principal DevOps Engineer
David Ortiz – Senior Software Engineer
Videology Big Data Team
November 30, 2016

43. On the Rocky Road to EBS
Videology’s Journey to EBS-backed Big Data

44. What to Expect from the Session
• Intro to Videology
• Challenges
• Road to EBS-backed cluster
• Happy engineers

45. Videology overview
Founded:
2007 by Scott Ferber, co-founder of Advertising.com, which sold to
AOL Time Warner in 2004 for $497 Million
Corporate
Headquarters:
New York, NY
Operations:
• Operating in 28 Global Markets
• Key Offices – New York, Baltimore, Toronto, London, Singapore
& Sydney
Employees: Approximately 380
Investors
NEA, Comcast Ventures, Harbourvest, Catalyst Investors,
Pinnacle Ventures, Valhalla Venture
Customers:
4,500 Active Users including Brand marketers, agencies, trading
desks, media companies, MVPD’s
Ecosystem
Integrations:
Open platform with 2200+ ecosystem integrations, including 1000+
media companies, 40 data providers, all major 3rd party
verification providers, and dozens of technology partners across
the media ecosystem
Recent Client
Wins:
Videology provides a
converged advertising solution
that is screen-agnostic,
ensuring unduplicated reach
with the right frequency
cadence to achieve
guaranteed results.
45

46. Industry accolades…
Videology was named Best Digital Video Ad Platform by Cynopsis Media at their
2015 Model D Awards.
“ ”
Videology was able to show that their platform drove brand lift that was on average 6X
higher than Nielsen's norms.
“ ”
Videology has the most sophisticated media optimizer to analyze the right
allocation of TV and online video to optimize reach and campaign cost.
“ ”

47. Hadoop overview
NameNode
ResourceManager
Gateway
DataNode
NodeManager

48. Where does big data processing fit in?

49. Original production
Instance
Type
Qty Role vCPU RAM
(GB)
Storage
(GB)
m3.xlarge 1 Jumpbox 4 15 80
m3.xlarge 1 Cloudera
Manager
4 15 80
m3.2xlarge 2 NN/RM 8 30 160
cc2.8xlarge 1 Service Master 32 60 3,200
cc2.8xlarge 30 Worker 32 60 3,200

50. I’ve got 99 problems and Hadoop is a few of them
Reliability
Scalability
Distcp
CPU to Memory Ratio

51. 2015
Q2
2016
Q3
2016
Q4 and
beyond
Engaged Cloudera
for EBS support
Gave up on EBS
and tested D2s
New EBS to
the rescue!
Take advantage of
new hardware

52. CC2.8XL M4.10XLD2.8XL
Old
Not enough disk
Expensive
NirvanaLots of disk!
Not enough memory
Expensive

53. D2.8xl prototype
Instance
Type
Qty Role vCPU RAM
(GB)
Storage
(GB)
r3.large 1 Jumpbox 2 15.25 32
r3.large 1 Cloudera
Manager
2 15.25 32
r3.xlarge 2 NN/RM 8 30 160
r3.2xlarge 2 Service Master 8 61 160
d2.8xl 10 Worker 36 244 48,000

54. M4.10xlarge w/ sc1 prototype
Instance
Type
Qty Role vCPU RAM
(GB)
Storage
(GB)
r3.large 1 Jumpbox 2 15.25 32
r3.large 1 Cloudera
Manager
2 15.25 32
r3.xlarge 2 NN/RM 8 30 160
r3.2xlarge 2 Service Master 8 61 160
m4.10xlarge 18 Worker 40 160 4,000

55. M4.10xlarge w/ st1 prototype
Instance
Type
Qty Role vCPU RAM
(GB)
Storage
(GB)
r3.large 1 Jumpbox 2 15.25 32
r3.large 1 Cloudera Manager 2 15.25 32
r3.xlarge 2 NN/RM 8 30 160
r3.2xlarge 2 Service Master 8 61 160
m4.10xlarg
e
18 Worker 40 160 8,000

56. Problems no more!
• No more rebuilding Nodes
• 1 critical incident since switch vs. 5 in the year prior to release
• Get to play with kids instead of babysitting cluster

57. Engineering benefits - capacity
No longer restricted by
memory, we now have
resources to pursue other
tools to improve our reliability
and speed:
• Spark
• HBase
• Flafka
• Offloading processing from
Amazon Redshift to CDH
More resilient to log volume
increases
Can expand storage as
requirements changes

58. Financial benefits
$0.00
$5,000.00
$10,000.00
$15,000.00
$20,000.00
$25,000.00
$30,000.00
Total Cost Cost by Utilization
Cc2 M4
$0.00
$0.01
$0.02
$0.03
$0.04
$0.05
$0.06
$0.07
$0.08
$0.09
$0.10
Cost to Process 1000 Requests
Cc2 M4

59. Thank you!
Questions?

60. Remember to complete
your evaluations!