This document discusses lessons learned from building the Wakoopa software. It outlines how the company scaled its infrastructure from one server to support 200 million users. Key strategies included splitting the server into staging, production and database roles, using background job queues, optimizing database queries, archiving old hourly usage data, and plans to implement MySQL partitioning and other technologies to improve performance.

2. Lessons Learned
while building Wakoopa

3. Menno van der Sman
Developer at Wakoopa
since May 2007

4. Wakoopa is about
software

6. PC Mac

7. 200
million

8. Does it scale?

9. Scaling the front is easy

10. Scaling the back is
harder

11. Limits spawn creativity

12. One server
at Railsmachine
sliced into staging, production and db
MySQL 4.1
Apache + Mongrel

13. Background queue

14. Lots of tools already
BackgroundRB, DelayedJob, Daemons etc

15. Rolling our own
database based
using script/runner
processing multiple items in one loop
one cache update per loop

16. Smart queries
UPDATE developers SET active_seconds = active_seconds + ...
INSERT INTO hourly_usage (...) VALUES (...) ON DUPLICATE KEY UPDATE
active_seconds = active_seconds + ..., idle_seconds = idle_seconds + ...
Allows for concurrency

17. Size = rowsize x rows
Make rowsize smaller

18. sometimes `id`
shouldn’t be the
Primary Key

19. Enter
composite_primary_keys

20. Overrides default PK
class HourlyUsage < ActiveRecord::Base
set_primary_keys :user_id, :software_id, :used_at
...
end
Original idea by Dr Nic
Enhanced for Rail 2.1 by Darrin Holst
http://compositekeys.rubyforge.org/

21. Archive old data
that’s not frequently used

22. HourlyUsage
to
HourlyUsageStorage
SELECT ... FROM hourly_usage WHERE used_at < '...' INTO OUTFILE '/path/to/file'
LOAD DATA INFILE '/path/to/file' REPLACE INTO TABLE hourly_usage_storage
avoids long locks

23. Upcoming
improvements

24. MySQL 5.1
Partitioning
CREATE TABLE hourly_usage_storage ( ... )
PARTITION BY RANGE (software_id) (
PARTITION p0 VALUES LESS THAN (...),
PARTITION p1 VALUES LESS THAN (...),
PARTITION p2 VALUES LESS THAN (...),
PARTITION p3 VALUES LESS THAN (...),
....
PARTITION pn VALUES LESS THAN MAXVALUE
);
Makes ‘WHERE software_id = ...’ very fast
Available in MySQL 5.1

25. And many more plans
Designing a new infrastructure
MySQL 5.1
Memcached
Sphinx
Rails 2.1
Amazon EC2

26. Announcing a new blog
Inside Wakoopa
http://inside.wakoopa.com

27. Questions?
or suggestions?
Looking for C++ and frontend awesomeness
http://wakoopa.com/jobs