An introduction to Timbits, a widget development framework built on Node.js

1. Project Timbit
Diving into a new web development paradigm
August 17, 2011 - Presented By Edward De Groot, Director Of The Postmedia Digital Innovation Team

2. Introduction
✤ Postmedia’s Digital Innovation Team is finally up and running
✤ We are not about building new frameworks for CMSs but...
✤ Need a platform to quickly build new prototypes and functionality
✤ Want to assist the Professional Services team in expediting development, testing and
deployment of widgets and templates
✤ Need something to keep us busy and out of trouble
✤ Want to contribute to the web development community
✤ You are encouraged to comment and/or ask questions

3. Widget #1
The I/O Problem
Widget #2
Widget #5
Widget #3
✤ Widgets are typically rendered
synchronously, one at a time
✤ A few slow widgets can cause Widget #4
significant delay for the page
✤ (Too) many widgets, fast or slow, will
delay the page Widget #6 Widget #7 Widget #8

4. The Digital
Media Problem
✤ Most digital media sites are
(overly) content heavy
✤ Most incorporate (too much)
content from many different
sources on a single page
✤ e.g. canada.com home page
✤ Built utilizing over 100 widgets.
✤ 80+ feeds from our CMS alone
✤ It’s not alone.

5. Possible Solutions
✤ Utilize caching wherever and whenever possible
✤ Already do that via Akamai, MemCached, etc.
(That’s why canada.com, as big as it is, still renders on the server in < 1s on average)
✤ Reduce number of widgets / page size
✤ Ideal, but we will always be somewhat content heavy
✤ Render widgets in parallel!
✤ Both obvious and promising...

6. ESI to the Rescue
Widget #1 Widget #2 Widget #3 Widget #4 Widget #5 Widget #6 Widget #7 Widget #8
✤ Edge Side Includes (ESI), available via our CDN (Akamai) can help
✤ ESI will allow us to render most widgets in parallel
✤ Postmedia DM released support for ESI / Remote Widgets this week!
✤ (We don’t have any remote widgets.... yet)

7. Better, but not Great.
✤ The majority of widgets render data from remote data sources
(Databases, REST APIs, etc.)
✤ I/O calls introduce significant latency
✤ Therefore, most widgets, under load, spend most of their time waiting for data
✤ In concurrency-based, multi-threaded applications with significant I/IO, high loads
will result in most threads being blocked. The result?
✤ High CPU/memory utilization (significant context switching)
✤ Low throughput (lot’s of blocking)
✤ High request queuing

8. The Main Event
✤ Event-Driven architecture solves the I/O latency issues.
✤ Much better at handling high number of concurrent users
✤ Much better memory efficiency under high loads
✤ No dead-locking, as there are no locks, no blocking calls.
✤ As such, a widget can continue to service new requests while waiting
on I/O for other requests.

9. Node.js
✤ An event-driven server-side JavaScript environment based on V8. It is
intended for writing scalable network programs such as web servers.
✤ Event-Driven libraries are available for other languages (Ruby,
Python etc.) so why Node.js?
✤ In Node.js, event-driven is not an after thought. Less likely to accidentally create
blocking calls, particularly 3rd party packages/libraries
✤ The merging of websites and native applications is inevitable. HTML5, CSS3 and
JavaScript will (finally) enable and empower that platform. (my opinion)
✤ The lines between Client, Cloud, and Server will continue to erode. JavaScript
enables our code to become portable. (more on that later)

10. CoffeeScript
✤ A programming language that transcompiles to JavaScript.
✤ Adds syntactic sugar inspired by Ruby, Python and Haskell
✤ Adds more sophisticated features like array comprehension and
pattern matching.
✤ Compiles predictably to JavaScript, programs can be written with less
code with no effect on runtime performance
✤ (ED loves Ruby, hates JavaScript, and drinks a lot of Coffee.)

11. The Results...?

12. Magic!
✤ Pantry - A JSON/XML resource caching library
✤ Timbits - Widget framework on Express and CoffeeScript
✤ kitkat - Kontinuous integrated testing koffee application template
✤ connect-esi - Edge Side Includes tag processor for the connect framework
✤ Samples - Including an HTML5 based Story widget with multi-device rendering support
(Browser, Tablet, Phone)
✤ All of these projects are open source!

13. Pantry
A JSON/XML resource caching library
✤ Multiple ‘simultaneous’ requests for the same resource will only
generate a single request which will fulfil all others
✤ Optimistic caching - Assumes recently expired items are still ‘Good’
✤ Whenever feasible, all cache refreshes happen in the background
✤ Automatically parses/converts JSON and XML feeds into JavaScript
objects
✤ Fully configurable (shelfLife, maxLife, capacity etc.)

14. Timbits
Widget framework on Express and CoffeeScript
✤ Built on top of some maturing Node.js packages
✤ Connect.js - Middleware framework for node.js
✤ Express.js - Sinatra inspired web development framework
✤ CoffeeKup - HTML templating engine based on CoffeeScript
✤ Makes it easy to build an API consuming, data rendering widget.
✤ By default, it utilizes Pantry for all data consumption
✤ [DEMO]

15. kitkat
Kontinuous integrated testing koffee application template
✤ An automated testing framework for node.js and CoffeeScript
✤ Will automatically run tests when the project changes
✤ Integrates with Growl
✤ Convention over configuration
✤ Includes spec generation (single, all, and automatic)
✤ [DEMO]

16. connect-esi
Edge Side Includes tag processor for the connect
✤ Main purpose (today) is to assist in testing widgets
✤ Only basic ESI support so far
✤ esi:include (including nested)
✤ variables such as $(HTTP _COOKIE), $(QUERY_STRING) etc.
✤ [DEMO]

17. Examples
✤ Objective #1 - Practical proof of concept widget for Timbits
✤ Objective #2 - Display the beauty of CoffeeKup templating language
✤ Objective #3 - Attempting to prove that a single widget template can
be utilized for multiple designs and across multiple browsers/devices
✤ Generic HTML5 utilizing unobtrusive CSS and JavaScript (no
inline styles or code
✤ This still needs a little more work
✤ [DEMO]

18. Going Forward
✤ You can begin building widgets using Timbits today
✤ Moving towards stable, production ready v 0.2 of all libraries
✤ Planning new features, such as
✤ Better error handling and documentation (all)
✤ Scaffolding (timbits)
✤ Optional client side rendering (timbits)
✤ Support for Redis (Pantry)
✤ Windows Development Support (all)