NBCUniversal, a worldwide mass media corporation, was looking for a more affordable and easier way to manage their database solution that hosts their extensive online digital assets. With Datavail’s assistance, NBCUniversal made the move from MongoDB 3.6 to MongoDB Atlas on AWS. In this presentation, learn how making this move enabled the entertainment titan to reduce overhead and labor costs associated with managing its database environment.

1. Why NBC Universal Migrated to MongoDB Atlas
Charleste King,
MongoDB Practice Lead, Datavail
Trinh Nguyen,
Sr. Director of Technology, NBCUniversal

2. MongoDB Practice Lead, Datavail
Charleste King
 Over 20 years in the IT industry
 Expertise in software development to data
analysis, architecture, and administration
 Industry experience includes aerospace,
agriculture, medicine, education, and more.
 Specializes in developing solutions for
unique client problems ranging from multi-
level upgrades to forecasting problem areas
to tuning and performance monitoring
 Has a Master of Science from Georgia
Institute of Technology

3. Datavail Overview
Databases, Analytics, and
Application Development
We are data
specialists
15+ years delivering
data services
Projects, Managed
Services and staffing
500+ customers
7 years average retention
 All major platforms including
MongoDB and MongoDB Atlas
 AWS, Azure and Google
Cloud
 Comprehensive development
& operational services
 US & Global delivery models
 Health checks & performance
tuning
 Upgrades
Database
Operations

4. Sr. Director of Technology,
NBCUniversal
Trinh Nguyen
 12 years with E!’s Interactive Technology team
 Oversees three teams: Software engineers, Web
Developers, & DevOps
 Expertise in Lambda, EC2, ECS, Redis,
GraphQL, Chef to build Spring-based
applications as well as React apps with NodeJS.
 Works primarily in Java/Javascript, using
MongoDB and MySQL for data storage
 Migrated E! to the cloud completely in March
2017 - first Cable brand at NBCU to move its
entire infrastructure to the cloud
 Has a dog named Professor, a son named
Maverick, and a wife who let him come to this
conference
 Bachelor’s in Computer Engineering from the
University of California, Irvine

5. www.nbcuniversal.com
E! is the only global, multi-platform
brand for all things pop culture. The
network is currently available to 91
million cable and satellite subscribers
in the U.S and 161 countries globally.
E! is a network of NBCUniversal Entertainment &
Lifestyle Group, a division of NBCUniversal, one of the
world's leading media and entertainment companies
NBC Universal
E! Entertainment

6. Agenda
• NBC Universal – Business drivers & technical
needs
• Top pain points driving move to MongoDB Atlas
• Technical needs to be overcome before
migration
• Migration
• Post migration observations and benefits
• Lessons learned

7. The Journey to Atlas

8. NBCUniversal Business Drivers For Atlas Migration
 Ability to build digital products and features quickly
 Highly available and secure database
 Streamline operational costs – Licensing and Infrastructure
 Self Service provisioning and management
 Intuitive metrics and visualizations

9. Technical Needs for E! Digital to Migrate to Atlas
Cost and Ease of Management
 Ongoing/Repeated tasks:
 Daily health checks
 Storage issues – constantly checking storage
 Manual deletion of snapshots
 Upgrading/Patching
 Performance tuning

10. E! Digital – Prior Architecture

11. NBCUniversal
Top Pain Points Driving Move to MongoDB Atlas

12. Top Pain Points Driving Move to
MongoDB Atlas
Provisioning Instances
Access to Ops Manager
Downtime / Patching
Storage / Data Retention
Alerts Lagged in Getting to
Right Individuals and
Teams
Self Service
Access Atlas Portal from Anywhere
Auto patching and one click upgrades w/o
downtime
Storage Auto Expand and Automated Backups
Streamlined and Targeted Alerts
Prior Challenge Benefits of Atlas Migration

13. NBCUniversal
Technical Challenges
Engineering Prerequisites

14. In our network we have a proxy appliance, which
all connections to the internet must route through.
Normally this is fine with web browsing, however
when trying to connect from our desktops to
mongo atlas instances connection is denied. Is
there a way to setup connectivity to include the
proxy settings?
Arbi Begian,
DevOps Lead

15. First Challenge
Developers were not able to
connect from our corporate office
to Mongo Atlas since our
corporate network egress is
restricted to go out through a
proxy firewall device, which
blocks all ports except for HTTP
and HTTPS.

16. Connectivity to Atlas is established using the
MongoDB wire protocol, which is not HTTP-
based. Unless the proxy appliance that you
mention has explicit support for the wire protocol or
port forwarding, then it is unlikely that the
appliance will function in the manner you intend.
Mongo Support

17. How E! Online Overcame Connectivity Restriction
To overcome connectivity limitation from our corporate network to
hosted Mongo Atlas we had to reconfigure our network and routing
setup:
 Instead of using corporate internet egress to connect to Atlas we decided to route
through our AWS VPC.
 By VPC peering the two networks, E! was able to establish communication to
MongoDB clusters within the network .
 To abide with our AWS environment security we had to peer according to each of our
specific VPCs (Dev/Shared/Staging/Prod).
 We reconfigured our internal routing to pass the MongoDB communication internally
rather than going through the public interface.

18. /etc/hosts
#Using Dev or Shared Open VPN
172.22.64.XXX dev-pub-shard-00-XX-abcde.mongodb.net
172.22.64.XX dev-pub-shard-00-XX-abcde.mongodb.net
172.22.64.XX dev-pub-shard-00-XX-abcde.mongodb.net
#MongoDB Atlas Properties
### Dev env Atlas ###
atlasConnectionUrl=mongodb://webclient:USER@dev-pub-shard-00-XX-
abcde.mongodb.net:27017,
dev-pub-shard-00-XX-abcde.mongodb.net:27017,
dev-pub-shard-00-XX-
abcde.mongodb.net:27017/{database}?authSource=admin&retryWrites=true
&ssl=true&replicaSet=dev-pub-shard-0
atlasBudConnectionUrl=mongodb://
atlasEnable=true
Developer Setup

19. VPC Peering Developers’ communication issues
were solved since we did not have
to go through the proxy device.
By going this route it also made
our connection to the Atlas cluster
much faster, even faster than
accessing our EC2 instances that
were running MongoDB
Enterprise.
Unexpected
Performance Gains

20. Setting up Your Subnets
 When we setup our subnets we tried to keep the address spaces
similar
 Dev-Test subnet before: 172.21.68.0
 Atlas Dev-Test Cluster after: 172.22.68.0
 During Cluster Setup we did not use default subnet, this has to be
explicitly defined if you want to use your own
 Easy/Helps know your database IPs when scanning, setting up
firewall rules, inspecting logs, etc.

21. Route 53  We ran into problems configuring SRV within
Route53
 SRV service call in Atlas already has logic to
figure out which is node is primary
 Keeping the DNS in sync wasn’t worth it just
for cleaner/shorter names when new nodes
were brought up for maintenance or under
load (cluster nodes increasing/decreasing)
 In our case we weren’t able to leverage
Route53 to abstract the cluster’s configuration
parameters to be stored in DNS but many of
you will be able to leverage SRV and TXT
records as a solution
Solution and Learnings
 Long Mongo hostnames
 Setting up Simple to read
CNAMES
 Namespace “mongodb.net”
Challenge

22. Updating Application Drivers
 Almost all of our applications have a centralized back-end service called
“commons client interface” (CCI).
 Previously CCI was configured with Spring Data MongoDB Core »
2.0.7.RELEASE, packaged with mongodb-driver-async v3.5.0.
 Upgraded to 3.6.1 and leveraged the retryWrites=true parameter.
 Cutover timing.

23. Migration

24. Why we used
Mongomirror Luckily, MongoDB provides
mongomirror which works in this
situation
Solution
If you have VPC Peering with
MongoDB 3.6, Live Migration
requires extra network
configuration that we wanted to
avoid.
Challenge

25. How We Leveraged mongomirror to Help with the
Migration
 Helps to perform live migration without downtime.
 mongomirror is a utility for migrating data from an existing MongoDB
replica set to a MongoDB Atlas replica set
 Does not require to shut down existing replica sets or applications.
 Drawback: it does not import the user/role data, thus we had do
manually create the users in MongoDB Atlas after the migration of the
data.

26. 7 Steps to Use Mongomirror
1. Download mongomirror on secondary node
2. Configuration changes and prerequisites in place for mongomirror to work
3. Setup MongoDB User in the AWS and run getParameter command
4. Setup MongoDB User in the target MongoDB Atlas Cluster with Atlas
admin role.
5. Add IP(s) to whitelist
6. Mongomirror sync and tailing of OpLog
7. Switch application and stop mongomirror

27. use admin
db.createUser(
{
user: "AtlasMigrationUser",
pwd: "passwordxxxxx",
roles: [ "clusterMonitor", "readAnyDatabase" ]
}
)
We used a command similar to the following:

28. mongomirror --host "rs-pub-
dev/mongo.dev.eol.inbcu.com:27017" --username
"datavail_user" --password "passwordxxxxx" --
authenticationDatabase "admin" --destination "dev-
pub-shard-0/dev-pub-shard-00-00-
uwwl2.mongodb.net:27017,dev-pub-shard-00-01-
uwwl2.mongodb.net:27017,dev-pub-shard-00-02-
uwwl2.mongodb.net:27017" --destinationUsername
"admin" --destinationPassword "passwordxxxxxxxx”
Step 6
6. Started mongomirror process. We executed using a statement that looks like the following:

29. mongomirror

30. Post Migration
Observation and
Benefits

31. Benefits of Mongo
Hosted Cloud
Eliminated: EC2 instances
reserved to run
OpsManager and its
replica set.
Additional Gains

32. Benefits of Mongo
Hosted Cloud
Productivity: no double
VPN needed to access
Atlas since it is publicly
accessible.
Additional Gains

33. Benefits of Mongo
Hosted Cloud
Secure: 2FA required to
access Atlas.
Additional Gains

34. Benefits of Mongo
Hosted Cloud
Self Service: Developers
can spin up clusters for
project/prototype-based
microservice work.
Additional Gains

35. Benefits of Mongo
Hosted Cloud
Roles & Permissions are
Transparent and don’t
necessarily need a DBA for
setup.
Additional Gains

36. Benefits of Mongo
Hosted Cloud
Maintenance patching
requires no downtime and
easy one-click.
Additional Gains

37. Lessons Learned
 If you’re part of a large corporation
that uses a proxy… be ready for:
 Know about the port restrictions
 Or have a plan on who can
accommodate your testing/migration
and how
 If you are running in the cloud, VPC
peering helps, but test routing
 Allocate time in your migration plan
to update your applications’ drivers
Talk openly about your
gains to the
organization
 Rapid Prototyping
 Decreased Downtime
 Accessibility
 Alerting
 Storage/Backup
 Autopatching

38. Q&A
Trinh Nguyen
Trinh.Nguyen@nbcuni.com
Charleste King
Charleste.King@datavail.com

39. Fill out the
session evaluation
form for a chance
to win our
Datavail
headphones!

40. Stop by our booth for a
chance to win: Datavailopoly
Our GRAND PRIZE:
Oculus Go Standalone Virtual
Reality Headset - 64GB
OR