Jaakko Manninen's presentation on Node.js Lesson Learned at SC5 Meetup

1. Node.js at Rocket Pack
Lessons learned; Patterns to know and love
Jaakko Manninen
Rocket Pack / Disney Interactive
Wednesday, May 15, 13

2. Rocket Pack
• Makes awesome HTML5 games and the tools to
build them
• Engine,Tool chain, Services 100% JS
• Acquired by Disney in 2011
• Started using Node.js in 2010, version 0.1
• Full stack, cloud hosted setup
• Very early days for HTML5 games: first toolchain
Wednesday, May 15, 13

3. What is Node.js
• JavaScript engine with IO library & APIs
• Google’sV8 + libuv + core APIs
• V8 is Chrome’s very fast JS engine
• One thread of execution
• Asynch, non-blocking, event driven IO
• Great for servers, real-time messaging, but
also for web and standalone apps
Wednesday, May 15, 13

4. Road to Node
• First as HTTP front end for Scala backend
• Parse request, deliver to AMQP message broker
• Games ran engine code in Rhino/JVM
• Ran it in Node and soon ported everything
to Node
• V8 is very fast :)
• Most packages we have used, we have had to patch :(
• Today package quality improving but still poor :/
Wednesday, May 15, 13

5. Rocket Pack version 1
• Cloud: Scalable, stateless, modular
• Top to bottom JavaScript, all the way from
MongoDB to the client side runtime engine
• Messaging to tie the elements together
• First RabbitMQ & node-amqp
• Then wrote our own Messaging API
Wednesday, May 15, 13

6. Messaging
• Wrote load-balancing, p2p messaging API
using github.com/einaros/ws
• Careful about fanout style exchanges, where processing capability is
limited to one core only
• Central discovery: if you have a multicast group, think about node-mdns
• Socket.IO messages quickly routed to
services by FE
• Sticky lb; no Store yet; no engine.io yet
• Use engine.io with eg. a Redis Store
Wednesday, May 15, 13

7. Processing woes
• An evolving platform needs to run data migrations
• Memory limited to 1.2GB (fixed only in 2012)
• Use cluster to task workers on other cores
• Or discover nodes in cluster (mdns, roll-your-
own)
Wednesday, May 15, 13

8. Lessons part 1
• One-thread-only is your friend in scaling
• Many packages are buggy and you will have
to patch them (but submit a pull request!)
• Use domain to capture out-of-stack errors
• Where did that socket hang up come from? domain will tell you
• Domain itself is experimental
• Never throw - just return errors
• Never use uncaughtException
Wednesday, May 15, 13

9. Part 2 - Local Tools
• Brought dev tools to the dev’s machine
• Windows support; offline support
• Plugins: declare and depend on named
interfaces
• Excellent testability, modularity, composability
• CommonJS, npm, package.json everywhere
Wednesday, May 15, 13

10. • npm is an awesome package manager
• Large dependency trees may cause rare issues
• It doesn’t have to be JavaScript
• Tens of thousands of packages
• Private repository recommended
• Our repo mirrors npm; has our private packages
• Not trivial to set up, but worth it
Wednesday, May 15, 13

11. Wednesday, May 15, 13

12. browserify
• github.com/substack/node-browserify
wraps your CommonJS modules in a
bundle loadable in the browser
• connect + browserify = JIT compilation
• Manage your site/app’s dependencies via
npm
• Most Node APIs available
• Even our UI is a browserified npm plugin
Wednesday, May 15, 13

13. when.js
• github.com/cujojs/when is a lightweight CommonJS
Promises/A+ library with other async goodies
(iterators etc)
• We now use Promises everywhere
• used to use github.com/caolan/async
• Bubbling up errors: handle errors in one place
• Also wraps Node functions in Promises
• nodefn.call(fs.readFile,‘/etc/passwd’).then()
Wednesday, May 15, 13

14. return longTask()
.then(function(result) {
// do stuff with result
})
.otherwise(function(error) {})
longTask(function(err, result) {
if (err) return err;
// do stuff with result
})
Without Promises:
Bubble up Promises:
Wednesday, May 15, 13

15. vfs
• github.com/c9/vfs is aVirtual File System
abstraction that works over WS (with msgpack),
HTTP and locally in Node
• brings the same FS API to the whole stack; lets us
access the fs the same way everywhere; browser
to server
• uses github.com/c9/smith, an RPC agent system
that gives us streams and processes in the browser
• output logs from long-running processes
• EMFILE: file watching by interval not by fd; github.com/isaacs/graceful-fs
Wednesday, May 15, 13

16. Lessons part 2
• Always use jshint in your editor
• Forget callbacks, use and return Promises
• Use browserify and npm to assemble your
software
• Design around one thread per process and
partition around ~3500m/s type of issues
• Build with grunt; test with mocha and
jsdom
• jsdom is fully fledged DOM implementation
Wednesday, May 15, 13

17. Native applications
• github.com/rogerwang/node-webkit lets
you build native apps with Node.js and
WebKit
• Linux, Mac,Windows all supported
• Even WebGL is supported
• TPOLM demo time! (wrapped in node-webkit)
• http://tpolm.com/demos/inedible_candy
Wednesday, May 15, 13

18. Thanks!
jaakko@rocketpack.fi
jaakko.manninen@gmail.com
github.com/rocketpack
@kschzt
Wednesday, May 15, 13