With the power of machine learning, the Sky News “Royal Wedding: Who’s Who” app – built in collaboration with AWS, GrayMeta, and UI Centric – enabled viewers to watch real-time updates of celebrity wedding guests as they entered St. George Chapel on May 19. This exciting Keynote will cover how GrayMeta leveraged Amazon Rekognition for real-time identification of guests and tagging with related information. You will also learn how Sky was able to launch this unique experience in less than four months using AWS Elemental Media Services for live video capture and multiscreen viewing optimization as well as Amazon CloudFront to unify the content for rapid distribution to viewers.

1. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
The Royal Wedding
Who’s Who Experience
Josh Wiggins
Chief Commercial Officer
GrayMeta
Enhancing the Viewer Experience
with Machine Learning:

2. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Facebook calculated that 2.8 million people in Britain and
America alone had written status updates about the royal
wedding in the 24 hours before the service began.
2 Billion Viewers
in 180 Countries
2011
William & Kate
2bn

3. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
2018 Wedding by the numbers…
16,000 glasses of champagne
1 million handfuls of confetti
8,000 street parties across the nation
1.4 million cucumber sandwiches
330 miles of bunting
1.5 billion tissues
Harry & Meghan
SOURCE: https://bridebook.co.uk/article/the-royal-wedding-by-the-numbers

4. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
2018 Wedding by the numbers…
Lots more global viewers
Harry & Meghan
SOURCE: https://bridebook.co.uk/article/the-royal-wedding-by-the-numbers

5. Goal
Facial
recognition
combined
with
additional
metadata –
Bios, News,
Fashion,
Emotions.
QC tools
built in
Mobile-first experience
Provide a way for the viewers to know who was who as they arrived at the
wedding as the event was streaming on a mobile device
Post event On Demand offering

6. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL

7. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Expectations
Project requirements
Goal focused Avoiding scope creep
Timelines

8. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Considerations
Event location Live Camera / Encoding Weather
Guest List Clothing / AppearanceViewer Experience

9. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Time wasn’t on our side
MAR APR MAY
19 May –
Royal
Wedding
Day
Analysis
Design & Build
Test & Learn
ML Training
FEB

10. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Camera to Consumer Timeline
60 seconds

11. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Workflow Components
Content
Acquisition
Content
Encoding
Metadata
Creation
(Machine
Learning)
Metadata
Curation
Content
Streaming
Social Tie In Analytics
Consumer
App w/
Smart
Overlay

12. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
GrayMeta Platform
Manually enter or import bios for each
guest in advance of the event.
Sky News operators use existing
guest images/video to train the ML
in advance of the event.
Anticipated
or confirmed
wedding
guest list
Images/Video of
each guest sourced
and uploaded to
GrayMeta Platform
Test ML facial
recognition and output.
is Lord Byron
is Lady Hampshire
Pre-event workflow

13. Knowing what we were in for

14. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Live workflow during the event
GrayMeta Platform
Manual tagging of unknown faces by Sky
News operators and automatically associate
with bios with the ability to correct and or edit
any of the automatically created metadata.
Facial recognition based on custom model for
Sky News. ML trained using images of people
from confirmed or anticipated guest list (to be
provided by Sky News).
Automatic association of bios & news
items with matched wedding guests.
Live Source
Feed from BBC
Encoder at
Sky/Convert to IP
Chunk
Video into
8 sec clips
Sky News
Video Player
(VOD)
CDNCloud
Storage
Content
Packaging
Unknown
Faces
Automatic
Metadata Creation

15. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
LIVE
curation
experience

17. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Resources
Solution Engineering
Machine Learning Models
Social Connectors
User Experience
Back End Dev
Real-time Curation
User Experience
Front End Dev
Solution Engineering
Cloud Storage
Media Encoding
Rekognition (AI – ML)
Content Delivery
Network
Overall Project Management
Editorial Researchers
Security
Data Privacy

18. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
The data
Training set Names & Bios
Images from stock and other
sources with various angles
and emotions
Collection and distribution of
metadata to appear aside
the end-user experience

19. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Hurdles
Changing
Appearance
Recent changes in
appearance in contrast to
the training data set
Live Curation
QA names in real-time
Test & Learn
Testing of training data sets,
live curation and camera
positions to optimize solution

20. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Optimal Solution
GrayMeta Platform
Manual tagging of unknown faces by Sky
News operators and automatically associate
with bios with the ability to correct and or edit
any of the automatically created metadata.
Facial recognition based on custom model for
Sky News. ML trained using images of people
from confirmed or anticipated guest list (to be
provided by Sky News).
Automatic association of bios & news
items with matched wedding guests.
Live Source
Feed from BBC
Encoder at
Sky/Convert to IP
Chunk
Video into
8 sec clips
Sky News
Video Player
(VOD)
CDNCloud
Storage
Content
Packaging
Unknown
Faces
Automatic
Metadata Creation

21. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
Mobile End User Experience

22. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
End Result Video

23. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
From Chapel to the Cloud

24. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
In the Cloud

25. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
From Cloud to You

26. ©2018 GrayMeta. All rights reserved. PROPRIETARY & CONFIDENTIAL
3 month deployment window
Coordination and communication
Optimization of existing technology for this use case
Training data key
Human curation / validation

27. Thank You