Aviary's customizable SDK powers cross-platform photo editing for over 6,500 partners and over 70 million monthly active users across the globe. Some of our notable partners include Walgreens, Squarespace, Yahoo Mail, Flickr, Photobucket, and Wix. At Aviary, we use node.js for several mission-critical projects in production and have seen extremely positive results. In this talk, we will discuss how we approach some common situations that developers deploying node.js projects will likely need to tackle. We will walk you through our routing mechanism, our automated deployment system, some of our custom middleware, and our testing philosophy.
2. Aviary
Photo-Editing
SDK & Apps
Fully-Baked UI
Configurable, High-Quality Tools
Over 6,500 Partners
Over 70 Million Monthly Users
Over 6 Billion Photos Edited
iOS, Android, Web, Server
J
3. Who Are We?
Nir
Jack
Lead Serverside
Director of
Engineer
Engineering
Likes:
●
●
●
Automated deployment
Big-O notation
Brainteasers
Hates:
●
Cilantro
Likes:
●
●
●
Parallelizing processes
DRY code
Seltzer
Hates:
●
Food after the sell-by date
5. How Do We Use Node?
● In Production:
○
○
○
○
○
● Future:
Analytics Dashboard
○ Server-side Rendering API
Content Delivery (CDS) ○ Automated billing
Public Website
Receipts Collection
Monitoring Tools
6. Why Do We Use Node?
●
●
●
●
●
●
●
●
Extremely fast and lightweight
Easy to iterate on
Common language for client and server
JSON
Cross Platform
npm
express module
Active Community
14. The One-To-Many Problem
Android expects responses to look like this:
iOS 1.0 expects responses to look like this:
iOS 2.0 expects responses to look like this:
15. Response Formatting - The Model
Content Entry
Response Formats
Responses
JSON document describing
content item
JSON documents defining mappings
from entry to responses
Actual JSON responses
delivered to devices
17. Code Sample (Dumbed Down)
var formattedResponse = {};
for (var propName in responseFormat) {
var val = contentEntry[responseFormat[propName].dataKey];
for (var transformation in responseFormat[propName].transformations) {
val = transformationModule[transformation](val);
}
formattedResponse[propName] = val;
}
return formattedResponse;
19. Image Rendering
Challenge: Use our existing image rendering .NET/C++
process from node server
Solution:
require(‘child_process’).spawn(‘renderer.exe’)
Benefits: Easy IPC, asynchronous workflow
20. Code Sample
var spawn = require(‘child_process’).spawn;
var renderer = spawn(‘renderer.exe’, [‘-i’, ‘inputImage.jpg’, … ]);
// read text
renderer.stderr.setEncoding(‘utf8’);
renderer.stderr.on(‘data’, function (data) { json += data; });
// or binary data
renderer.stdout.on(‘data’, function (data) { buffers.push(data); });
renderer.on(‘close’, function (code, signal) {
// respond to exit code, signal (e.g. ‘SIGTERM’), process output
var diagnostics = JSON.parse(json);
var img = Buffer.concat(buffers);
});
24. Automated Deployment: Overview
Git
S3
2) Jenkins
polls for repo
changes
1) Code is
pushed to
master
3) Code is zipped
and uploaded to S3
Jenkins
4) Get a list of
live servers in
this group
AWS API
5) SSH into each
server and run
the bootstrap script
25. Automated Deployment: Bootstrap
5) SSH into each
server and run
the bootstrap script
#!/bin/bash
ZIP_LOCATION="s3://aviary/projectX/deployment.zip";
cd ~/projectX;
sudo apt-get -y -q install nodejs@0.10.0;
sudo apt-get -y -q install s3cmd;
sudo npm install -g forever@0.10.8;
Goals of the bootstrap.sh:
1. Ensure all dependencies are
installed
2. Download and extract project
3. Ensure HTTP traffic is routed to the
proper port
4. Keep the old version of the project
live until the moment the new one is
ready to go live
# Missing step: create s3 configuration file
s3cmd -c /usr/local/s3cfg.config get "$ZIP_LOCATION" theCds.zip;
unzip -q -o deployment.zip
iptables -t nat -A PREROUTING -p tcp --dport 80 -j
REDIRECT --to-ports 8142;
forever stopall;
forever start server.js;
27. Lessons Learned (1)
● Integration tests!
● Watch out for node and npm updates
○ Hardcode the node version you’re using
○ If you’re using package.json, version everything
● Node.js + MongoDb are a great couple
● Make sure you understand hoisting
28. Lessons Learned (2)
● Always callback in async functions
● Always return after a callback
● Node doesn’t always run the same on all platforms
● Use middleware only when necessary
● Always store dates as Unix Timestamps
○ Timezones are a pain in your future
● Throwing unhandled errors will crash your process
29. Conclusion
Today, our production node servers:
● serve dynamic content to 20MM people (soon 70MM)
● power our website: aviary.com
● log real-time receipt data for every in-app purchase
● allow us to analyze hundreds of millions of events daily
● power quick scripts and one-off internal tools