NoSQL vs. SQL: practical patterns (and antipatterns to avoid) in real projects. Talk at NoSQL Day 2011.

1. [Anti]Patterns with NoSQL
Luca Bonmassar

2. NoSQL vs. SQL
NOSQL DAY ’11

3. NoSQL / NoAgenda
No specific NoSQL technologies (document, key/value, ...)
No specific NoSQL db features (sharding, replication, ...)
No specific ...
...
NOSQL DAY ’11

4. Does it solve my problem?
NOSQL DAY ’11

5. Does it solve my problem?
NOSQL DAY ’11

6. Does it solve my problem?
But what is your problem?
NOSQL DAY ’11

7. Does it solve my problem?
But what is your problem?
NoSQL fragmentation (like NoCloud, NoJava,
NoMicrosoft, ...)
NOSQL DAY ’11

8. [Anti]Patterns
The “dynamic” website
Data mapping
The Alien
The binary Alien
The SQLQueue
Overnormalized
Unpredictable tomorrow
NOSQL DAY ’11

9. Who am I?
Passionate about
technology
Favorite topics: Cloud,
Virtualization, NoSQL
Gamification fan! (or fun?)
NOSQL DAY ’11

10. The “dynamic” website

11. The “dynamic” website
NOSQL DAY ’11

12. The “dynamic” website
I have a website
NOSQL DAY ’11

13. The “dynamic” website
I have a website
Dynamic contents everywhere
NOSQL DAY ’11

14. The “dynamic” website
I have a website
Dynamic contents everywhere
High DB load
NOSQL DAY ’11

15. The “dynamic” website
NOSQL DAY ’11

16. The “dynamic” website
NOSQL DAY ’11

17. The “dynamic” website
SELECT username SELECT COUNT(###) WHERE ...
FROM users WHERE
sessionid = ###
SELECT username
FROM users WHERE
sessionid = ###
SELECT COUNT("submissions"."puzzle_id")
FROM "submissions" WHERE (puzzle_id IS
NOT NULL) AND (user_id IS NOT NULL) SELECT COUNT(###) WHERE ... SELECT COUNT(###) WHERE ...
GROUP BY puzzle_id ...
SELECT users.*, count(distinct puzzle_id) FROM
submissions JOIN users on submissions.user_id SELECT * FROM submissions
= users.id where submissions.current_state = WHERE ...
SELECT avatar.* FROM users
"succeeded" ... WHERE ...
NOSQL DAY ’11

18. The “dynamic” website
SELECT username SELECT COUNT(###) WHERE ...
FROM users WHERE
sessionid = ###
SELECT username
FROM users WHERE
sessionid = ###
SELECT COUNT("submissions"."puzzle_id")
FROM "submissions" WHERE (puzzle_id IS
NOT NULL) AND (user_id IS NOT NULL) SELECT COUNT(###) WHERE ... SELECT COUNT(###) WHERE ...
GROUP BY puzzle_id ...
SELECT users.*, count(distinct puzzle_id) FROM
submissions JOIN users on submissions.user_id SELECT * FROM submissions
= users.id where submissions.current_state = WHERE ...
SELECT avatar.* FROM users
"succeeded" ... WHERE ...
NOSQL DAY ’11

19. The “dynamic” website
NOSQL DAY ’11

20. The “dynamic” website
SELECT username SELECT COUNT(###) WHERE ...
FROM users WHERE
sessionid = ###
SELECT username
FROM users WHERE
sessionid = ###
SELECT COUNT("submissions"."puzzle_id")
FROM "submissions" WHERE (puzzle_id IS
NOT NULL) AND (user_id IS NOT NULL) SELECT COUNT(###) WHERE ... SELECT COUNT(###) WHERE ...
GROUP BY puzzle_id ...
SELECT users.*, count(distinct puzzle_id) FROM
submissions JOIN users on submissions.user_id SELECT * FROM submissions
= users.id where submissions.current_state = WHERE ...
SELECT avatar.* FROM users
"succeeded" ... WHERE ...
NOSQL DAY ’11

21. The “dynamic” website
Is it that dynamic?
Are there any data
structures that fit better
than relational?
NOSQL DAY ’11

22. Cache, Cache, Cache
Use Ad-Hoc Caching (e.g. Rails Caching)
SQL Table as cache
NOSQL DAY ’11

23. SQL Table as Cache
When you use SQL Table as
Cache, a kitten somewhere
dies
Use Memcache,
MemcacheDB, Redis
NOSQL DAY ’11

24. Better tools?
NOSQL DAY ’11

25. Rethink data structures
SELECT COUNT(*) ...
SELECT COUNT(*) ...
SELECT COUNT(*) ...
submitted++
passed++
failed++
NOSQL DAY ’11

26. Rethink data structures
SELECT *, count(distinct ...)
FROM submissions JOIN
users on ... where
submissions.current_state
= "succeeded" ...
List(5)
List.add / List.replace
NOSQL DAY ’11

27. Data mapping

28. Data mapping
An original idea: we need a newsfeed!
Users can comment on feed items
Users can reply to comments
NOSQL DAY ’11

29. What the customer is looking for
NOSQL DAY ’11

30. What the developer is thinking about
NOSQL DAY ’11

31. But how to ...

32. Eureka!
The item has a feed_id and
a parent_id
You have now a navigable
SQL data structure!
NOSQL DAY ’11

33. Data mapping - alternatives
Do not try this at home! (or any office)
A document db can help
{ feed : 42, user : ‘luca.bonmassar’, data : ‘living the
dream!’, comments : [{user : ‘jonny’ : ‘make sense!’ }] }
NOSQL DAY ’11

34. The Alien

35. The Alien
A type of object stored in a relational database
No (or weak) relations with any other table
Stored in the db because the db == persistency
NOSQL DAY ’11

36. User Settings
User preferences
user_id, data
Data as BLOB or TEXT
NOSQL DAY ’11

37. The Alien
The Alien is a key/value entity
Use a key/value storage to store it
NOSQL DAY ’11

38. The binary Alien

39. The binary Alien
Like the Alien, but the payload is pure binary data
Common solution to store “small” images
Even worse: Base64 encoded binary
NOSQL DAY ’11

40. The binary Alien
Move the binary Alien to a binary content provider
S3 or Filesystem are good ones
Let the webserver access/serve them directly
NOSQL DAY ’11

41. The SQLQueue

42. The SQLQueue
Distributed components need to exchange data
Producers / Consumers
Backlog of work to be completed
SQL database (== persistency) as persistent queue
NOSQL DAY ’11

43. Mail delivery
NOSQL DAY ’11

44. Mail delivery
NOSQL DAY ’11

45. Mail delivery
NOSQL DAY ’11

46. Mail delivery
NOSQL DAY ’11

47. The SQLQueue
No “relations”, like the Alien
Simulating a Queue using AUTO_INCREMENT ids and
transactions
NOSQL DAY ’11

48. SQLQueue
In some countries,
SQLQueue is considered a
crime against software
Use message queues
(Amazon SQS, MemcacheQ,
StormMQ, RabbitMQ, ... )
NOSQL DAY ’11

49. Overnormalized

50. Overnormalized
NOSQL DAY ’11

51. Overnormalized
The process of organizing data to minimize redundancy
NOSQL DAY ’11

52. Overnormalized
The process of organizing data to minimize redundancy
A larger schema is broken into smaller ones
NOSQL DAY ’11

53. Overnormalized
The process of organizing data to minimize redundancy
A larger schema is broken into smaller ones
user_id, email
NOSQL DAY ’11

54. Overnormalized
The process of organizing data to minimize redundancy
A larger schema is broken into smaller ones
user_id, email
user_id, phone_num
NOSQL DAY ’11

55. Overnormalized
The process of organizing data to minimize redundancy
A larger schema is broken into smaller ones
user_id, email
user_id, phone_num
user_id, badge
NOSQL DAY ’11

56. Overnormalized
PRO:
reduce redundancy
less overhead
each table scale separately
NOSQL DAY ’11

57. Overnormalized
Cons:
NOSQL DAY ’11

58. Overnormalized
Cons:
... JOIN... JOIN... JOIN... JOIN... JOIN... JOIN... JOIN...
NOSQL DAY ’11

59. Overnormalized
Cache the normalized data
Denormalize / keep a replica of the denormalized view
Use document db or key/value storage for the replica
NOSQL DAY ’11

60. Unpredictable tomorrow

61. Unpredictable tomorrow
You are now part of a new Agile(TM) project
You are Agile(TM), so:
No complete specs
No complete use cases
NOSQL DAY ’11

62. How many wonderful things around me
A mobile App w/ internet backend
“Simple” use cases
User can log in
User can update their location
User can get all the many wonderful things they have
around themselves
NOSQL DAY ’11

63. How many wonderful things around me
Data Model:
User
Places
Aliens!!! Aliens!!! Aliens!!!
NOSQL DAY ’11

64. ...but sooner or later...
NOSQL DAY ’11

65. How many wonderful things around me
Users can send/receive friendship invitations
Users can import FB friends, Twitter followers
Users can interact with messages, pokes
Users can check-in into places
Users can share their checkins with friends
...
NOSQL DAY ’11

66. Unpredictable tomorrow
No silver bullets
Mix technologies
E.g. relational databases to handle relations
Migrations are painful, but always an option
NOSQL DAY ’11

67. One more thing

68. Recap
Does it make sense with the relational paradigm?
Do I need a persistent storage or a relational database?
NOSQL DAY ’11

69. Thanks! Any Questions?
NOSQL DAY ’11

70. Contacts
luca@coderloop.com
linkedin.com/in/lucabonmassar
twitter.com/lucabonmassar
joind.in/talk/view/2939
NOSQL DAY ’11