Content lake architecture can evolve the media workflow by providing efficiency from content security all the way to value-added services, such as machine learning and content monetization. In this session, technical leaders from 21st Century Fox, Warner Bros., and Astro Malaysia discuss the migration of their petabyte-scale video libraries (production and distribution archives) to the cloud in order to increase the customer reach and value of their media archives. Discover some of the lessons learned, the TCO analysis around various different storage tiers, the challenges and best practices from 10s of petabytes ingest, storage, and value-added compute at scale.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hollywood’s Cloud-Based Content Lakes:
Modernized Media Archives
M A E 2 0 3
Mike Davis
Storage Business Development
AWS
David Sugg
VP Content Arch & Engineering
Warner Bros. Entertainment
Aik-Chong Phuah
CTO
Astro Malaysia

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Focusing on the consumer
requires agile infrastructure
New business opportunities are
rapidly emerging in the media
market
The ability to leverage your growing
asset collection is handicapped by
insufficient metadata and federated
search
Stagnating tape infrastructure is not
keeping pace with business agility

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
This creates a widening
Value Gap, where you don’t
have the agility to respond
to new opportunities, and IT
costs continue to rise
unpredictably.

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS allows studios,
broadcasters, and sports
leagues to close the Value
Gap by building automated
hybrid workflows that
improve agility, reduce cycle
times, and reduce costs.

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Global media firms with Asset Management workloads on AWS
Digital Asset Management & Supply Chain
Content Creation &
Post Production
Distribution
(OTT, Broadcast, Publish)
Machine Learning & Analytics

7. Offline tape
migration
Egress
optimization
Proprietary
conversion
Metadata
export
Asset QC &
registry
Index/
Analyze
Proxy
generation
Amazon
Rekognition
Amazon
Transcribe
Amazon
Comprehend
DAM/CMS modernization
Metadata import
Well architected
Security, access, DR
Lifecycle and cost
optimization
Export Normalization & Augmentation Import & Operationalize
Records Access
Curation Workflow
Import/Quarantine
Discovery
Migration requires methodical planning

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aik-Chong Phuah
CTO
Astro Malaysia

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Knowing & serving our customer better
920k 33k hours
Connected
boxes
On Demand
Content
2M
Downloads
Astro Go
20M
Monthly UV
Digital IPs
76%
Household
5.6mil homes
3.5hrs viewing
per day
16M Weekly
Listeners
Radio
24M
H1 2018
On Demand
Content downloads
1.5M Registered
Customers
Home shopping
eCommerce

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Production Storage
Site 1
Tape Archive 2
Site 2
MAM
Inter-site Fiber
Mirroring
(Secondary
Site with
similar setup
as Site 1)
Media Asset Management
(MAM)
Tape Archive 1
Production
Storage
Media Ingest
& Quality
Check
Edit &
Compliance
High level architecture – CurrentIncomingMediaSource
AWS Direct
Connect,
Singapore
Back Up Link to Singapore (1Gbps)
Amazon
S3
Amazon
EC2
AWS
Elemental
Amazon
CloudFront
AWS Services (Singapore region)
Primary Link (1Gbps)

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Current architecture
• Costly to maintain
and to continue to
expand
• Hard to keep up to
date with new LTO
formats
• Infra was initially
designed for SD
before HD and UHD
• Archives fully utilized
(20PB unique content)
• Inability to meet
content growth rate
of 60K hours (2PB) per
year
• Manual tape handling
risk due to offline
data tape storage
UnscalableHigh InvestmentLegacy System
• Aging LTO library (since 2006)
• Frequent issues last 12
months:
• 8x tape snaps
• 12x drive failures
• 6x robotic breakdowns
• Inability to defragment tape
due to high usage of drives
• Unable to commit to
content publishing time
• Unpredictable
turnaround time of
content retrieval -
15mins to 24hrs retrieval
SLA breach
Dual Site Ops
• Conventional broadcast
workflow
• Driven by MAM for
media processing,
playout, & streaming
• 1Gbps inter-site fiber
link
Media Archive
• Archived & mirrored between
both sites managed by MAM
• SL8500 LTO library each per
site
• Interdependent on-premise
servers, online stores, &
databases
• 15% content not mirrored due
to legacy processesChallenges

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
New architecture with AWS
• Both LTO archives will be migrated to
centralized Amazon Glacier Vault and retired
• Scalable and robust infrastructure, predictable
cost model to support our growth
• Reduced overall total cost of ownership
• Moved from capex-intensive approach
to “pay-per-use”
Cost Optimization
Scalability
Efficiency
Experience
• Predictable retrieval: standard,
bulk & expedite
• Improved viewer experience,
and time to market
Centralized archive and
database
• Unified archive for all content
enabling both linear broadcast &
VOD (DTH and OTT)
• Single database and MAM
Fully redundant DX
• 20Gbps fully redundant Direct
Connect to AWS POP in Kuala
Lumpur relieving bandwidth over
inter-site fiber link
• Redundant DC to AWS Singapore
• Simplified architecture, improved
robustness and SLA
• Active productions can be buffered
to S3 content lake for fast retrieval
while pending for archiving in
Glacier
• Other processes taking place in
parallel

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
High level architecture – New
AWS DX,
Singapore
Back Up Link to Singapore (1Gbps)
Production Storage
Site 1
AWS DX POP
Kuala Lumpur Site 2
Direct Connect
~20Gbps
MAM
Inter-site Fiber
Mirroring
(Secondary Site
with similar setup
as Site 1)
Media Asset Management
(MAM)
Production
Storage
Media Ingest
& Quality
Check
Tape Archive 2Tape Archive 1
Edit &
Compliance
IncomingMediaSource
10Gbps
(temp)
10Gbps
(temp)
S3EC2 ElementalCloudFront
AWS Services (Singapore region)
Amazon
Glacier
Back Up Link to Singapore (1Gbps)
Direct Connect
~20Gbps
Migration Storage Migration Storage

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
TCO analysis
Direct Connect
Costs
Migration
Costs
Parallel run
Costs
Workflow
Development
AWS
Base Costs
20% savings over a 5-year period
moving from on-premise tape archive to Amazon Glacier

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
In-the-Works
Cloud Automation
Cloud Playout
Content Censorship
Machine Translation
Cloud Orchestration
Cloud Editing
Speech
Transcription
Metadata Enrichment
Next Up

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
David Sugg
VP Content Arch & Engineering
Warner Bros. Entertainment

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Warner Bros. is the industry leader across key businesses
#1 OR #2 WW BOX OFFICE
9 OF LAST 10 YEARS
$4B+ WW BOX OFFICE
7 OF THE LAST 8 YEARS
THEATRICAL
#1 TV STUDIO
FOR LAST 9 SEASONS
ONE OF THE LARGEST LICENSORS
OF US PROGRAMMING
TELEVISION
ONLY STUDIO WITH
SCALED GAMES PUBLISHING
OVER $1 BILLION IN
2017 REVENUE
GAMES

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
With some of the world’s greatest franchises
Classic Animation Horror

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The 100-year
challenge
Studios have a different perspective on the
preservation of intellectual property.
We strive to preserve our content for hundreds
of years.
• What does it take?
• Where’s the “Superman Crystal”?
© 2018, Warner Bros. Entertainment, Inc. or its affiliates. All rights reserved.

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The 100-year challenge: Threats
The Digital Archive
Threat Matrix

22. “Fixity, in the preservation sense,
means the assurance that a digital file
has remained unchanged, i.e. fixed.”
Bailey, 2014
© 2018, Warner Bros. Entertainment, Inc. or its affiliates. All rights reserved.

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Fixity is often managed via
checksums
• Can be applied at ingest if
the fixity is known for the
files, throughout the
lifecycle via regular fixity
checks and during any event
such as a file move or copy.
Bit rot, media failure, and the fixity problem
© 2018, Warner Bros. Entertainment, Inc. or its affiliates. All rights reserved.

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Bit rot, media failure,
and the fixity problem
• Every bit is important
• How do we know bits are safe?
• Current standards require regular
checks
• No user accessible processes to expose
the fixitiy status of an object
© 2018, Warner Bros. Entertainment, Inc. or its affiliates. All rights reserved.

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do we manage this today?
T10K and
LTO8 tape
libraries
Redundant,
geographically
dispersed sites
Replication
managed at
storage level

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do we manage this today?
Challenges with using the cloud:Challenges with current solution:
• Elasticity
• Of compute
• Of storage
• Capital costs
• Time to value
• Volume of data
• Perceived costs
• Perceived security issues

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How do we manage this today?
Internet
Colo Location 1
Vendor 1
System 2
Vendor/System n
AccessControl
File/MetadataTransfer
High Speed
Working Storage
T10k/LTO8
Tape Libraries
DAM/Orchestration
Colo Location 2
AccessControl
High Speed
Working Storage
T10k/LTO8
Tape Libraries
DAM/Orchestration
High Speed
Private Interconnect
File/MetadataTransfer

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What might the future look like?
Internet
Cloud Provider
Vendor 1
System 2
Vendor/System N
AccessControl
File/MetadataTransfer
DAM/Orchestration
Colo Location 1
AccessControl
High Speed
Working Storage
T10k/LTO8
Tape Libraries
DAM/Orchestration
File/MetadataTransfer
Working Storage GlacierEBS EFS
/
EC2Lambda
RDS
EC2Lambda
RDS
Direct Connect
WBContentServicesLayer
IngestAPIOrderAPIOutputAPIDataAPI

29. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Mike Davis, AWS, mikdav@amazon.com
Aik-Chong Phuah, Astro Malaysia, pacphuah@astro.com.my
David Sugg, Warner Bros., David.sugg@warnerbros.com

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.