SlideShare una empresa de Scribd logo
1 de 31
Descargar para leer sin conexión
Content Delivery
At Aviary
MongoDb User Group 11/19/13
Aviary
Photo-Editing
SDK & Apps
Fully-Baked UI
Configurable, High-Quality Tools
Over 5,000 Partners
Over 50 Million Monthly Users
Over 4 Billion Photos Edited
iOS, Android, Web, Windows, Server
J
Who Are We?
Nir

Jack

Lead Serverside

Director of

Engineer

Engineering

Likes:
●
●
●

Automated deployment
Big-O notation
Brainteasers

Hates:
●

Cilantro

Likes:
●
●
●

Parallelizing processes
DRY code
Seltzer

Hates:
●

Food after the sell-by date
Content
Effects

Frames

Stickers

Messages

J
The “Fat Tiny” Situation
We want to dynamically deliver the Fat Tiny
stickers to our users

How can we do that?
J
The CDS
Aviary’s Content Delivery System
Version 1
● Static files on S3
● Excess data sent
● Data stored in MySQL
● New features meant new code

N
The “Fat Tiny” Situation
We want to display the Fat Tiny sticker pack only
to Picstitch users in Japan who use iOS 7

How can we do that?
N
CDS V2
Version 2 Overview
Stack
Global
Users

Akamai

AWS Cloud Formation
AWS Elastic LoadBalancer (ELB)
Node.js
AWS Elastic Cloud Computing (EC2)
Ubuntu
Bash
MongoDb
JSON
JSON Schema

Content
Delivery
Network

Load Balancer

API
Servers
Database
Cluster
Management
System

N
Why MongoDB?
● Great with Node (JSON-based)
● Schema-less is easy to change/query
● Read-heavy
● Relatively small data set

N
How It Works
A Behind-the-Scenes Look
Delivered Types
● Manifest JSON
○ Content Set
○ Content Versions

● Content JSON
○ Content Metadata
○ All Assets

N
The “Fat Tiny” Situation
We want to manage the Fat Tiny sticker pack
as a single entity, but we want to deliver it to
each device in its own
optimal format.
How can we do that?
N
Response Formatting Model
Content Entry

Response Formats

Responses

JSON document describing
content item

JSON documents defining mappings
from entry to responses

Actual JSON responses
delivered to devices

J
Content Formats
● JSON schema

{
"type":"object",
"properties":{
"metadata": {

● Added properties

"type": "object",
"properties":{
"displayName": {
"type": "string",

● Custom types

}
}
},

● Validates entries

"icon": {
"type": "object",
"customType": "image"
"properties":{
"path": {
"type": "string",
"required": true,

J
Response Formats
● JSON schema

{
"type":"object",
"properties":{
"identifier": {

● dataKey property

"type": "string",
"dataKey": "identifier"
},

● Defines response

"name": {
"type": "string",
"dataKey": "metadata.displayName"
},

structure

"iconImagePath": {
"type": "string",
"dataKey": "icon.path-100"

● Maps content

},
"items": {
"type": "array",
"dataKey": "items"

J
Content Deployment
1. Insert/Update CMS Entry
"identifier": "com.aviary.stickers.234fe"

2. Find Response Formats
"type":"object",

"id": "com.aviary.stickers.234fe",

"properties":{

"metadata": {
"displayName": "Hats"

"name": "Hats",

"id": {
"type": "string",

},

"dataKey": "identifier"

"icon": {
"path": "cds/hats/icon.png"

"iconImagePath": "cds/hats/icon100.png"
"stickers": [
{

},

"identifier": "1"

"name": {

"path-100": "cds/hats/icon100.png"

"type": "string",

},

"imageUrl": "cds/hats/1.png"

"dataKey": "metadata.displayName"

"items": [

},

{

"type": "string",

"imageUrl": "cds/hats/1.png"
}

}
],

"iconImagePath": {

"identifier": "1"

]

3. Generate+Insert Responses

"versionKey": "e4532fd342"

"dataKey": "icon.path-100"
},
"stickers": {
"type": "array",
"dataKey": "items"

J
Manifests
"stickers": [
{
"id": "com.aviary.stickers.234fe",
"versionKey": "e4532fd342"
},
{
"id": "com.aviary.stickers.fed34",
"versionKey": "c54532343d"
}
],
"frames": [
{
"id": "com.aviary.frames.25435",
"versionKey": "fd4324323"
}
]

J
Manifest Deployment
Using Aggregate to Find the Newest Versions

Manifest with Correct Version Keys

{
$match:{

"stickers": [
{

formatId:{$in:formatIds},

"id": "com.aviary.stickers.234fe",

identifier:{$in:identifiers}

"versionKey": "e4532fd342"

}
}

},
],

{
$sort: { _id: -1 }

"effects": [
{

},

"id": "com.aviary.effects.25435",

{

"versionKey": "fd4324323"

$group: {
}

_id: "$identifier",
versionKey:{$first:"$versionKey"}

]

}
}

J
Scopes and Targeting
Deployed Manifests Have Scopes

End Users Have Scope Parameters

Manifest 1
"targetingScope": {
"apiKey": "abc",
"country": ["JP"]
},

/manifest?

"formattingScope": {
"platform": "ios",
"minOsVersion": "7.0.0"
}

apiKey=abc&
country=JP&
language=ja&
platform=ios&

Manifest 2

osVersion=7.2.0

"targetingScope": {
"apiKey": "def",
},
"formattingScope": {
"platform": "android",
"minOsVersion": "6.0.0"
}

N
API Servers
Scope parameters are converted into queries
Manifest 1
"targetingScope": {
db.manifest.find({

"apiKey": "abc",

"apiKey": {$in: ["abc", null]},
/manifest?

country=JP&

"formattingScope": {

"platform": {$in: ["ios", null]},
7002000

"platform": "ios",

"minOsVersion": {$lte: 7002000}

language=ja&

osVersion=7.2.0

},

"language": {$in: ["ja", null]},

apiKey=abc&

platform=ios&

"country": ["JP"]

"country": {$in: ["JP", null]},

"minOsVersion": "7.0.0"

}).sort({

}

"apiKey": -1,

Manifest 2

"language": -1,

"targetingScope": {

"country": -1,

"apiKey": "abc",

"minOsVersion": -1,

},

"platform": -1,

"formattingScope": {

"_id": -1

"platform": "ios",

}).limit(1)

"minOsVersion": "6.0.0"
}

N
Versioned Content
Received Manifests Contain VersionKeys
"stickers": [
{
"id": "com.aviary.stickers.234fe",
"versionKey": "e4532fd342"
},
{
"id": "com.aviary.stickers.fed34",
"versionKey": "c54532343d"

db.content.findOne({
/content?
versionKey=e4532fd342

}

"versionKey": “e4532fd342”
});

],
"frames": [
{
"id": "com.aviary.frames.25435",
"versionKey": "fd4324323"
}
]

N
Response Caching
db.manifests.find({

/manifest?

"apiKey": {$in: ["abc", null]},

apiKey=abc&

"country": {$in: ["JP", null]},

country=JP&

"language": {$in: ["ja", null]},

language=ja&

"platform": {$in: ["ios", null]},

platform=ios&

"minOsVersion": {$gte: 7002000}

osVersion=7.2.0

}).sort({
…,
"_id": -1
}).limit(1)

db.cachedManifests.findOne({
"url": "/manifest?apiKey=abc&country=JP&language=ja&osVersion=7.2.0&platform=ios"
})

N
PAULA
The CDS Management Console
Auto-generated UI

J
Other Mongo Usage
● PAULA permissions in user objects
users collection
{
"name": "nir",
"email": "nir@aviary.com",
"permissions": [
"content",
"dev",
"admin",
"partying"
]
}

● Integration tests interact with schemaless db willy nilly
N
Conclusion
The Takeaway
The Facts
● Built and deployed in 3 months
● Very few struggles with MongoDB
● Seamless management
● Graceful scaling from 0 to over 20M MAUs
● Happy serverside engineers
J
The Future
● Targeted Translations
● Granular User Targeting
● PAULA for the masses

N
Questions?
Comments also welcome
nir@aviary.com

jack@aviary.com

…and by the way, WE’RE HIRING!

Más contenido relacionado

Similar a MongoDB and Content Delivery at Aviary by Nir Zicherman and Jack Sisson

Architecting for change: LinkedIn's new data ecosystem
Architecting for change: LinkedIn's new data ecosystemArchitecting for change: LinkedIn's new data ecosystem
Architecting for change: LinkedIn's new data ecosystem
Yael Garten
 
iPhone/iPad Development with Titanium
iPhone/iPad Development with TitaniumiPhone/iPad Development with Titanium
iPhone/iPad Development with Titanium
Axway Appcelerator
 
Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점
Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점
Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점
Jeado Ko
 

Similar a MongoDB and Content Delivery at Aviary by Nir Zicherman and Jack Sisson (20)

RICOH THETA x IoT Developers Contest : Cloud API Seminar (2nd installation)
RICOH THETA x IoT Developers Contest : Cloud API Seminar (2nd installation)RICOH THETA x IoT Developers Contest : Cloud API Seminar (2nd installation)
RICOH THETA x IoT Developers Contest : Cloud API Seminar (2nd installation)
 
Engineering the New LinkedIn Profile
Engineering the New LinkedIn ProfileEngineering the New LinkedIn Profile
Engineering the New LinkedIn Profile
 
Exploring Google (Cloud) APIs with Python & JavaScript
Exploring Google (Cloud) APIs with Python & JavaScriptExploring Google (Cloud) APIs with Python & JavaScript
Exploring Google (Cloud) APIs with Python & JavaScript
 
Build 2017 - B8002 - Introducing Adaptive Cards
Build 2017 - B8002 - Introducing Adaptive CardsBuild 2017 - B8002 - Introducing Adaptive Cards
Build 2017 - B8002 - Introducing Adaptive Cards
 
Getting started with titanium
Getting started with titaniumGetting started with titanium
Getting started with titanium
 
Using Google (Cloud) APIs
Using Google (Cloud) APIsUsing Google (Cloud) APIs
Using Google (Cloud) APIs
 
The Rise of NoSQL
The Rise of NoSQLThe Rise of NoSQL
The Rise of NoSQL
 
Architecting for change: LinkedIn's new data ecosystem
Architecting for change: LinkedIn's new data ecosystemArchitecting for change: LinkedIn's new data ecosystem
Architecting for change: LinkedIn's new data ecosystem
 
Strata 2016 - Architecting for Change: LinkedIn's new data ecosystem
Strata 2016 - Architecting for Change: LinkedIn's new data ecosystemStrata 2016 - Architecting for Change: LinkedIn's new data ecosystem
Strata 2016 - Architecting for Change: LinkedIn's new data ecosystem
 
Windows 8 Pure Imagination - 2012-11-24 - Getting your HTML5 game Windows 8 r...
Windows 8 Pure Imagination - 2012-11-24 - Getting your HTML5 game Windows 8 r...Windows 8 Pure Imagination - 2012-11-24 - Getting your HTML5 game Windows 8 r...
Windows 8 Pure Imagination - 2012-11-24 - Getting your HTML5 game Windows 8 r...
 
Getting started with Appcelerator Titanium
Getting started with Appcelerator TitaniumGetting started with Appcelerator Titanium
Getting started with Appcelerator Titanium
 
Microsoft Graph: Connect to essential data every app needs
Microsoft Graph: Connect to essential data every app needsMicrosoft Graph: Connect to essential data every app needs
Microsoft Graph: Connect to essential data every app needs
 
Microsoft Graph: Connect to essential data every app needs
Microsoft Graph: Connect to essential data every app needsMicrosoft Graph: Connect to essential data every app needs
Microsoft Graph: Connect to essential data every app needs
 
Building Your First App with MongoDB Stitch
Building Your First App with MongoDB StitchBuilding Your First App with MongoDB Stitch
Building Your First App with MongoDB Stitch
 
iPhone/iPad Development with Titanium
iPhone/iPad Development with TitaniumiPhone/iPad Development with Titanium
iPhone/iPad Development with Titanium
 
Appcelerator iPhone/iPad Dev Con 2010 San Diego, CA
Appcelerator iPhone/iPad Dev Con 2010 San Diego, CAAppcelerator iPhone/iPad Dev Con 2010 San Diego, CA
Appcelerator iPhone/iPad Dev Con 2010 San Diego, CA
 
Power your apps with Gmail, Google Drive, Calendar, Sheets, Slides & more
Power your apps with Gmail, Google Drive, Calendar, Sheets, Slides & morePower your apps with Gmail, Google Drive, Calendar, Sheets, Slides & more
Power your apps with Gmail, Google Drive, Calendar, Sheets, Slides & more
 
Fixing Gaps. Strengthening the Chromium platform for content blocking
Fixing Gaps. Strengthening the Chromium platform for content blockingFixing Gaps. Strengthening the Chromium platform for content blocking
Fixing Gaps. Strengthening the Chromium platform for content blocking
 
Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점
Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점
Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점
 
Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점
Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점 Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점
Angular를 활용한 웹 프론트단 개발과 2.0에서 달라진점
 

Más de Hakka Labs

Más de Hakka Labs (20)

Always Valid Inference (Ramesh Johari, Stanford)
Always Valid Inference (Ramesh Johari, Stanford)Always Valid Inference (Ramesh Johari, Stanford)
Always Valid Inference (Ramesh Johari, Stanford)
 
DataEngConf SF16 - High cardinality time series search
DataEngConf SF16 - High cardinality time series searchDataEngConf SF16 - High cardinality time series search
DataEngConf SF16 - High cardinality time series search
 
DataEngConf SF16 - Data Asserts: Defensive Data Science
DataEngConf SF16 - Data Asserts: Defensive Data ScienceDataEngConf SF16 - Data Asserts: Defensive Data Science
DataEngConf SF16 - Data Asserts: Defensive Data Science
 
DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data
DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast DataDatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data
DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data
 
DataEngConf SF16 - Recommendations at Instacart
DataEngConf SF16 - Recommendations at InstacartDataEngConf SF16 - Recommendations at Instacart
DataEngConf SF16 - Recommendations at Instacart
 
DataEngConf SF16 - Running simulations at scale
DataEngConf SF16 - Running simulations at scaleDataEngConf SF16 - Running simulations at scale
DataEngConf SF16 - Running simulations at scale
 
DataEngConf SF16 - Deriving Meaning from Wearable Sensor Data
DataEngConf SF16 - Deriving Meaning from Wearable Sensor DataDataEngConf SF16 - Deriving Meaning from Wearable Sensor Data
DataEngConf SF16 - Deriving Meaning from Wearable Sensor Data
 
DataEngConf SF16 - Collecting and Moving Data at Scale
DataEngConf SF16 - Collecting and Moving Data at Scale DataEngConf SF16 - Collecting and Moving Data at Scale
DataEngConf SF16 - Collecting and Moving Data at Scale
 
DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQ
DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQDataEngConf SF16 - BYOMQ: Why We [re]Built IronMQ
DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQ
 
DataEngConf SF16 - Unifying Real Time and Historical Analytics with the Lambd...
DataEngConf SF16 - Unifying Real Time and Historical Analytics with the Lambd...DataEngConf SF16 - Unifying Real Time and Historical Analytics with the Lambd...
DataEngConf SF16 - Unifying Real Time and Historical Analytics with the Lambd...
 
DataEngConf SF16 - Three lessons learned from building a production machine l...
DataEngConf SF16 - Three lessons learned from building a production machine l...DataEngConf SF16 - Three lessons learned from building a production machine l...
DataEngConf SF16 - Three lessons learned from building a production machine l...
 
DataEngConf SF16 - Scalable and Reliable Logging at Pinterest
DataEngConf SF16 - Scalable and Reliable Logging at PinterestDataEngConf SF16 - Scalable and Reliable Logging at Pinterest
DataEngConf SF16 - Scalable and Reliable Logging at Pinterest
 
DataEngConf SF16 - Bridging the gap between data science and data engineering
DataEngConf SF16 - Bridging the gap between data science and data engineeringDataEngConf SF16 - Bridging the gap between data science and data engineering
DataEngConf SF16 - Bridging the gap between data science and data engineering
 
DataEngConf SF16 - Multi-temporal Data Structures
DataEngConf SF16 - Multi-temporal Data StructuresDataEngConf SF16 - Multi-temporal Data Structures
DataEngConf SF16 - Multi-temporal Data Structures
 
DataEngConf SF16 - Entity Resolution in Data Pipelines Using Spark
DataEngConf SF16 - Entity Resolution in Data Pipelines Using SparkDataEngConf SF16 - Entity Resolution in Data Pipelines Using Spark
DataEngConf SF16 - Entity Resolution in Data Pipelines Using Spark
 
DataEngConf SF16 - Beginning with Ourselves
DataEngConf SF16 - Beginning with OurselvesDataEngConf SF16 - Beginning with Ourselves
DataEngConf SF16 - Beginning with Ourselves
 
DataEngConf SF16 - Routing Billions of Analytics Events with High Deliverability
DataEngConf SF16 - Routing Billions of Analytics Events with High DeliverabilityDataEngConf SF16 - Routing Billions of Analytics Events with High Deliverability
DataEngConf SF16 - Routing Billions of Analytics Events with High Deliverability
 
DataEngConf SF16 - Tales from the other side - What a hiring manager wish you...
DataEngConf SF16 - Tales from the other side - What a hiring manager wish you...DataEngConf SF16 - Tales from the other side - What a hiring manager wish you...
DataEngConf SF16 - Tales from the other side - What a hiring manager wish you...
 
DataEngConf SF16 - Methods for Content Relevance at LinkedIn
DataEngConf SF16 - Methods for Content Relevance at LinkedInDataEngConf SF16 - Methods for Content Relevance at LinkedIn
DataEngConf SF16 - Methods for Content Relevance at LinkedIn
 
DataEngConf SF16 - Spark SQL Workshop
DataEngConf SF16 - Spark SQL WorkshopDataEngConf SF16 - Spark SQL Workshop
DataEngConf SF16 - Spark SQL Workshop
 

Último

Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 

Último (20)

Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 

MongoDB and Content Delivery at Aviary by Nir Zicherman and Jack Sisson

  • 2. Aviary Photo-Editing SDK & Apps Fully-Baked UI Configurable, High-Quality Tools Over 5,000 Partners Over 50 Million Monthly Users Over 4 Billion Photos Edited iOS, Android, Web, Windows, Server J
  • 3. Who Are We? Nir Jack Lead Serverside Director of Engineer Engineering Likes: ● ● ● Automated deployment Big-O notation Brainteasers Hates: ● Cilantro Likes: ● ● ● Parallelizing processes DRY code Seltzer Hates: ● Food after the sell-by date
  • 5. The “Fat Tiny” Situation We want to dynamically deliver the Fat Tiny stickers to our users How can we do that? J
  • 6. The CDS Aviary’s Content Delivery System
  • 7. Version 1 ● Static files on S3 ● Excess data sent ● Data stored in MySQL ● New features meant new code N
  • 8. The “Fat Tiny” Situation We want to display the Fat Tiny sticker pack only to Picstitch users in Japan who use iOS 7 How can we do that? N
  • 9. CDS V2 Version 2 Overview
  • 10. Stack Global Users Akamai AWS Cloud Formation AWS Elastic LoadBalancer (ELB) Node.js AWS Elastic Cloud Computing (EC2) Ubuntu Bash MongoDb JSON JSON Schema Content Delivery Network Load Balancer API Servers Database Cluster Management System N
  • 11. Why MongoDB? ● Great with Node (JSON-based) ● Schema-less is easy to change/query ● Read-heavy ● Relatively small data set N
  • 12. How It Works A Behind-the-Scenes Look
  • 13. Delivered Types ● Manifest JSON ○ Content Set ○ Content Versions ● Content JSON ○ Content Metadata ○ All Assets N
  • 14. The “Fat Tiny” Situation We want to manage the Fat Tiny sticker pack as a single entity, but we want to deliver it to each device in its own optimal format. How can we do that? N
  • 15. Response Formatting Model Content Entry Response Formats Responses JSON document describing content item JSON documents defining mappings from entry to responses Actual JSON responses delivered to devices J
  • 16. Content Formats ● JSON schema { "type":"object", "properties":{ "metadata": { ● Added properties "type": "object", "properties":{ "displayName": { "type": "string", ● Custom types } } }, ● Validates entries "icon": { "type": "object", "customType": "image" "properties":{ "path": { "type": "string", "required": true, J
  • 17. Response Formats ● JSON schema { "type":"object", "properties":{ "identifier": { ● dataKey property "type": "string", "dataKey": "identifier" }, ● Defines response "name": { "type": "string", "dataKey": "metadata.displayName" }, structure "iconImagePath": { "type": "string", "dataKey": "icon.path-100" ● Maps content }, "items": { "type": "array", "dataKey": "items" J
  • 18. Content Deployment 1. Insert/Update CMS Entry "identifier": "com.aviary.stickers.234fe" 2. Find Response Formats "type":"object", "id": "com.aviary.stickers.234fe", "properties":{ "metadata": { "displayName": "Hats" "name": "Hats", "id": { "type": "string", }, "dataKey": "identifier" "icon": { "path": "cds/hats/icon.png" "iconImagePath": "cds/hats/icon100.png" "stickers": [ { }, "identifier": "1" "name": { "path-100": "cds/hats/icon100.png" "type": "string", }, "imageUrl": "cds/hats/1.png" "dataKey": "metadata.displayName" "items": [ }, { "type": "string", "imageUrl": "cds/hats/1.png" } } ], "iconImagePath": { "identifier": "1" ] 3. Generate+Insert Responses "versionKey": "e4532fd342" "dataKey": "icon.path-100" }, "stickers": { "type": "array", "dataKey": "items" J
  • 19. Manifests "stickers": [ { "id": "com.aviary.stickers.234fe", "versionKey": "e4532fd342" }, { "id": "com.aviary.stickers.fed34", "versionKey": "c54532343d" } ], "frames": [ { "id": "com.aviary.frames.25435", "versionKey": "fd4324323" } ] J
  • 20. Manifest Deployment Using Aggregate to Find the Newest Versions Manifest with Correct Version Keys { $match:{ "stickers": [ { formatId:{$in:formatIds}, "id": "com.aviary.stickers.234fe", identifier:{$in:identifiers} "versionKey": "e4532fd342" } } }, ], { $sort: { _id: -1 } "effects": [ { }, "id": "com.aviary.effects.25435", { "versionKey": "fd4324323" $group: { } _id: "$identifier", versionKey:{$first:"$versionKey"} ] } } J
  • 21. Scopes and Targeting Deployed Manifests Have Scopes End Users Have Scope Parameters Manifest 1 "targetingScope": { "apiKey": "abc", "country": ["JP"] }, /manifest? "formattingScope": { "platform": "ios", "minOsVersion": "7.0.0" } apiKey=abc& country=JP& language=ja& platform=ios& Manifest 2 osVersion=7.2.0 "targetingScope": { "apiKey": "def", }, "formattingScope": { "platform": "android", "minOsVersion": "6.0.0" } N
  • 22. API Servers Scope parameters are converted into queries Manifest 1 "targetingScope": { db.manifest.find({ "apiKey": "abc", "apiKey": {$in: ["abc", null]}, /manifest? country=JP& "formattingScope": { "platform": {$in: ["ios", null]}, 7002000 "platform": "ios", "minOsVersion": {$lte: 7002000} language=ja& osVersion=7.2.0 }, "language": {$in: ["ja", null]}, apiKey=abc& platform=ios& "country": ["JP"] "country": {$in: ["JP", null]}, "minOsVersion": "7.0.0" }).sort({ } "apiKey": -1, Manifest 2 "language": -1, "targetingScope": { "country": -1, "apiKey": "abc", "minOsVersion": -1, }, "platform": -1, "formattingScope": { "_id": -1 "platform": "ios", }).limit(1) "minOsVersion": "6.0.0" } N
  • 23. Versioned Content Received Manifests Contain VersionKeys "stickers": [ { "id": "com.aviary.stickers.234fe", "versionKey": "e4532fd342" }, { "id": "com.aviary.stickers.fed34", "versionKey": "c54532343d" db.content.findOne({ /content? versionKey=e4532fd342 } "versionKey": “e4532fd342” }); ], "frames": [ { "id": "com.aviary.frames.25435", "versionKey": "fd4324323" } ] N
  • 24. Response Caching db.manifests.find({ /manifest? "apiKey": {$in: ["abc", null]}, apiKey=abc& "country": {$in: ["JP", null]}, country=JP& "language": {$in: ["ja", null]}, language=ja& "platform": {$in: ["ios", null]}, platform=ios& "minOsVersion": {$gte: 7002000} osVersion=7.2.0 }).sort({ …, "_id": -1 }).limit(1) db.cachedManifests.findOne({ "url": "/manifest?apiKey=abc&country=JP&language=ja&osVersion=7.2.0&platform=ios" }) N
  • 27. Other Mongo Usage ● PAULA permissions in user objects users collection { "name": "nir", "email": "nir@aviary.com", "permissions": [ "content", "dev", "admin", "partying" ] } ● Integration tests interact with schemaless db willy nilly N
  • 29. The Facts ● Built and deployed in 3 months ● Very few struggles with MongoDB ● Seamless management ● Graceful scaling from 0 to over 20M MAUs ● Happy serverside engineers J
  • 30. The Future ● Targeted Translations ● Granular User Targeting ● PAULA for the masses N