1. DBSlayer
Introducing DBSlayer
http://code.nytimes.com/projects/dbslayer
Derek Gottfrid
The New York Times

2. Problem:
Provide Web Apps Access to the Database

3. Alternate Problem:
Avoid evil horrible doctor show.

4. Solution
JSON + HTTP + MYSQL

5. Simple Example
select ci.* from City ci, Country c where
c.name = 'United States' and c.code =
ci.CountryCode order by Population desc

6. HTTP Request
http://localhost:9090/db?%7B%22SQL%22:
%22select%20ci.*%20from%20City%20ci,%
20Country%20c%20where%20c.name%20
=%20'United%20States'%20and%20c.code
%20=%20ci.CountryCode%20order%20by%
20Population%20desc%20limit%2010;%22
%7D

7. HTTP Response
HTTP/1.0 200 OK
Date: Wed, 27 Jun 2007 20:55:59 GMT
Server: dbslayer/server beta-9
Connection: Close
Content-type: text/plain; charset=utf-8
Content-Length: 746

8. JSON Payload
{quot;RESULTquot; : {quot;TYPESquot; : [quot;MYSQL_TYPE_LONGquot; , quot;MYSQL_TYPE_STRINGquot; ,
quot;MYSQL_TYPE_STRINGquot; , quot;MYSQL_TYPE_STRINGquot; , quot;MYSQL_TYPE_LONGquot;] ,
quot;HEADERquot; : [quot;IDquot; , quot;Namequot; , quot;CountryCodequot; , quot;Districtquot; , quot;Populationquot;] ,
quot;ROWSquot; : [[3793 , quot;New Yorkquot; , quot;USAquot; , quot;New Yorkquot; , 8008278] ,
[3794 , quot;Los Angelesquot; , quot;USAquot; , quot;Californiaquot; , 3694820] ,
[3795 , quot;Chicagoquot; , quot;USAquot; , quot;Illinoisquot; , 2896016] ,
[3796 , quot;Houstonquot; , quot;USAquot; , quot;Texasquot; , 1953631] ,
[3797 , quot;Philadelphiaquot; , quot;USAquot; , quot;Pennsylvaniaquot; , 1517550] ,
[3798 , quot;Phoenixquot; , quot;USAquot; , quot;Arizonaquot; , 1321045] ,
[3799 , quot;San Diegoquot; , quot;USAquot; , quot;Californiaquot; , 1223400] ,
[3800 , quot;Dallasquot; , quot;USAquot; , quot;Texasquot; , 1188580] ,
[3801 , quot;San Antonioquot; , quot;USAquot; , quot;Texasquot; , 1144646] ,
[3802 , quot;Detroitquot; , quot;USAquot; , quot;Michiganquot; , 951270]]}}

9. Python Sample Client
import urllib,urllib2,cjson
db = lambda q: cjson.decode( urllib2.urlopen(
“http://localhost:9090/db?
”+urllib.quote('{“SQL ”:”%s”}' % q)).read())

10. Other Solutions
MySQLProxy
●
SQLRelay
●
mysqli
●

11. Requirements
Independent
●
Scalable
●
Flexible
●
Simple
●

12. Disclaimer
Works for us.

13. Problems Not Solved
Back end scripts –
●
bulk import/export
Large Data Retrievals
●
– cursors
Native Binds /
●
Prepared Statements
Enterprise Buzzword
●
Compliance
A Sober Lindsay
●

14. DBSlayer is the middle-man
Connection Pooling
●
Multi-threaded
●
Round robin distribution of queries
●
Automatic fail over to alternate database
●
Easy configuration
●
Statistics / Logging
●
Tiny code base
●

15. Design Considerations

16. Flexiblity
Loose coupling of front and middle
●
easy upgrade
–
abstract location of db – makes re-config a
–
snap
Language independent
●
JSON easily shared across languages
–
Common place to add additional features
●

17. Scalability
Round robin across slaves
●
Automatic fail over for slaves
●
Easily add more slaves
●
HTTP plays nice with load balancers
●
JSON is very cachable
●
Stateless is required for effective scaling
●

18. Simplicity
HTTP and JSON - everyone knows them
●
Transparent protocol – capture, interpret
●
Lots of tools to work with HTTP
●
Works with your browser
●
HTTP server base – easy to deploy new
●
endpoints

19. Browser Testing

20. Stats

21. Pretty Pictures

24. View Source
Built on top of MySQL C API and Apache
●
Runtime Libraries (APR)
Why APR?
●
Platform agnostic
–
Memory Pools
–
Networking / Threading Abstractions
–
Proven
–

25. 3 Parts
server – handles network, HTTP parsing,
●
threading, logging
db – handle mysql access, type mapping,
●
connection failover
util – json de/serializer
●

26. Future / Your Chance
improve HTTP caching layer
● ●
complaince partition / shard
●
more database support
●
backends (APR new language
●
dbd) bindings
alternate serializer
●
more stats /
●
XML logging
security
●
scriptable filters
●

27. Other
(tighten up them ....)

28. Another amazing talk...
Rails Under The Knife
Jacob Harris
July 26th – 2:35pm
Portland 255

29. BoF
Amazon S3/EC2 computing against large
●
datasets – 7:30 F150
Future Newspapers – 8:30 E143-144
●

30. NYTIMES.COM
open.nytimes.com – developers blog
●
code.nytimes.com – code repository
●

31. Free Stuff
Visit the NYTimes Booth

32. The End
Questions: derek@nytimes.com