A talk I gave at the August meeting of the Melbourne RORO group on scaling a rails application. It's a quick overview of what I think you need to do early in the life of a rails app to be ready for scale further down the road. Read http://jrb.tumblr.com/post/30570014929/scaling-rails-at-melbourne-roro to add more context to the slides.

1. SCALING RAILS
A topic no longer as amusingly controversial as when I first started speaking on
the topic, but Twitter does still have the odd lolz (ha ha, scala!)

2. @johnbarton
http://goodfil.ms
http://whoisjohnbarton.com

3. Why me?
Because shut up, that’s why.

4. Because the Envato
Marketplaces are really big in
traffic and workload...

5. ... and I helped make the
computer machines do that
At a point in time where people thought Rails couldn’t do that because Twitter
crashed a lot and that’s all people thought of when they said “Big Rails”

6. 50 Million Dynamic Requests a
week when I left
or 80ish requests a second

7. Before that I did loads of
performance work at
MyCareer.com.au

8. After that I’m building ROFL
scale social network thingo.
You know, for movies.
Go sign up at http://goodfil.ms, so I don’t get rusty at web scale.

9. There is a risk of me veering into
my grand theory of software
development and the universe
and everything.

10. What does “Scaling Rails”
mean?

11. Predictably serving more
customers (with your rails app)
next week/month/year than you
have right now

12. It’s a game of tradeoffs
Cost vs. Performance vs. Reliability vs. Flexibility

13. “Complex systems that work
evolve from simple systems that
work”

14. Growing a stack for scale is a
similar exercise in managing
complexity as a growing
codebase and you can use the
same mental models

15. Coupling is the #1 enemy and
beating it is an elaborate game
of divide and conquer

16. Anyway, so some concrete
things about Rails
I can save the hand-wavy stuff for a devops meetup

17. Step #0
Measure everything you can
afford to measure
I would hope that goes without saying.
New Relic and Scout are my favourite tools for that.

18. Step #0.5
Actually check up on what you
are measuring
And learn how to interpret that data correctly (read books).

19. Step #1
YAGNI
You Aint Gonna Need it
(with caveats - you need a wee safety buffer)

20. Know your end game
Based on what kind of app you are working on.
Social Network, Online News, Ecommerce all have different work loads and
optimal stacks. There is no one size fits all stack at the pointy end.

21. Know your techniques
You need to know how to get yourself out of trouble when your metrics say
that you are in it. I will give a reading list for that the end.

22. Then you fake it till you make it

23. Deploy to the cloud
Probably Rackspace or Amazon,
Heroku if you are lazy and rich

24. Avoid vendor lock in for the full
stack
But it can be OK for smaller independent components. You will want to move
hosting companies in your future. Probably more than once.

25. JB’s Golden Mini Stack
one “magic vendor cloud load balancer” + 2 app servers + one datastore +
frequent backups

26. Rent the high memory
instances for app servers
it’s almost invariably the limiting factor

27. Stick with SQL
Rails tooling is heavily biased that way and you will get the most stuff for free.

28. IT DOES NOT MATTER IF YOU
CHOOSE MYSQL OR
POSTGRESQL BOTH WORK
FINE AND BOTH SCALE

29. Use DB backed queues...
delayed_job or roll your own
DBAs will roll their eyes at this

30. ... for the short to mid term
it is a simpler “whole stack” at the expense of doing things right

31. Put job queue workers on your
app slices...
...in direct proportion to the amount of jobs your web processes can make. If
each “app” box can do all of the ruby work for N number of requests, you can
keep adding boxes until your database explodes.

32. Then build a brand spanking
“job system”
With Redis and whatever else magic job shit you want, and dedicated machines
to working off that queue, and probably split your codebase too.

33. Caching is your get out of jail
free card

34. Be careful, it can hide genuine
performance problems
and once caching runs out... you are well and truly cooked

35. YMMV
“I hate to advocate drugs, alcohol, violence, or insanity to anyone, but they’ve
always worked for me”

36. BOOKS
•Building Scalable Web Sites (only if totally new to this)
•Release it!: Design and Deploy Production-Ready Software
•Scalability Rules: 50 Principles for Scaling Web Sites
•High Performance MySql
•PostgreSQL 9.0 High Performance
•PostgreSQL 9 Admin Cookbook

37. INTERWEBS
•http://akfpartners.com/techblog/
•http://highscalability.com/
•http://codeascraft.etsy.com/
•http://instagram-engineering.tumblr.com/
•http://www.mysqlperformanceblog.com/
•http://blog.newrelic.com/category/performance-tech-tips/

38. Thanks!
http://goodfil.ms ● http://whoisjohnbarton.com ● @johnbarton