2. Session Code: DAT451
Designing Role-Based Database
Systems to Achieve Unlimited
Database Scalability
Fernando G. Guerrero
CEO
Solid Quality Mentors
2

3. What forced me to prepare this session
5

4. The "unpredictable workload" case
Millions of copies of a win app around the
world
All of them waiting for file safety, before using
them
At undefined times
Unlimited number of potential files to check
Unpredictable level of workload at any date and
time
Unpredictable geographical distribution of workload
Potentially, millions of requests per hour
Can we solve it?
6

5. The "unreasonable scalability" case
The system already reached its limits
They need to support workload 50 times higher
Scaling up is not an option
Scaling out is not a simple task
They need to support over 15K transactions per
second, with textual based data
Can we solve it?
7

6. Introductions
Fernando G. Guerrero
CEO of Solid Quality Global LLC, USA
President of the SolidQ subsidiaries around the World
fguerrero@solidq.com
Microsoft Regional Director for Spain since 2004
SQL Server MVP from year 2000 till 2007
Usual suspect at many international conferences
8

7. The SolidQ recipe
4 SQL Server MVPs old
friends speaking at a
conference in Palm Springs
A pleasant dinner at a nice
restaurant
2 bottles of my favorite
Spanish wine
1 bottle of a great single
Malt Whisky
Lots of honest and trustful
9
work for 6 years

8. What are you going to see and hear today
10

9. This session is about…
Lesson learned from “unreasonable” and
amazing real systems from real enterprise
customers
Based on actual work performed by our Solid
Quality mentors
Learning to think out of the box when designing
DB systems
Thinking first, acting later
11

10. This session is NOT about…
Code
Demos
Obscure trace flags and registry hacks
Any particular commercial product or service
offering
Description about standard scalability
techniques
12

11. 13

12. First… Listen
14

13. Then, try to understand what
their needs really are…
And what they really want
15

14. A database system is just one
more tool to help achieving
business goals
16

15. Your technical common sense is
your most powerful tool
17

16. If you don’t need to buy
anything to implemnent a
solution, it does not mean that
this is the cheapest solution
18

17. Role based DB system design
A single box might be sufficient
Different tasks might imply different roles
Roles might be incompatible
You’ll never know until you measure it
19

18. The unpredictable workload case
20

19. Role based DB system design
A single box might be sufficient
Different tasks might imply different roles
Roles might be incompatible
You’ll never know until you measure it
21

20. Massive attack
22

21. The unpredictable workload case
23

22. A common sense powered queue
Multi-layered server roles:
The Reader
The Writer
The Master Root
“Magic array” of virtual server names
Self updated definitions
Proprietary advanced hash-based checking
Local vs remote storage
24

23. The Magic Array
25

24. The Reader role
Organized as a Readers tree with a Reader Root
at the top
Verifies signature of requested file
Replies to client with simple response:
Safe
Unsafe
Unknown
If unknown, checks parent Reader for further
information
Receives updates from parent Reader
26

25. The Writer role
Works as a Reader as well
Organized as a flexible array of Writers
Sends signature to the Master Root
and sends file asynchronously if necessary
Replies immediately to client with simple
response:
Safe
Unsafe
Unknown
Receives updates from Distributor
27

26. The Master Root
Stores new files and signatures
Calls the qualification engine to qualify new
files
Distributes updates to root Reader
Distributes updates to array of Writers
28

27. Massive dynamic deployment
29

28. The unreasonable scalability case
30

29. Scalability needs Jan 2005
Workload vs Capacity
Workload Initial C Improved C
300 300
90 100
90
10
31

30. 32
Jan-05
10
90
300
Feb-05
Mar-05
Apr-05
May-05
Jun-05
Jul-05
Workload
Aug-05
Sep-05
Oct-05
Nov-05
Initial C
Dec-05
90
200
300
Jan-06
Feb-06
Workload vs Capacity
Mar-06
Improved C
Apr-06
Scalability needs Aug 2006
May-06
Jun-06
Jul-06
90
300
600
Aug-06

31. Side note:
is there such thing as a write-
only transaction?
or…
how much writing activity do
you really expect?
33

32. Scalability needs Dec 2008
Workload vs Capacity
Workload Initial C Improved C Role Based C
4000
3000
2000
1500
600 600 600 600
200 300 500
90
10 90 90 90 90 90
Apr-05
Apr-06
Apr-07
Apr-08
Jan-06
Jan-05
Jan-07
Jan-08
Jul-05
Oct-05
Jul-06
Oct-06
Jul-07
Oct-07
Jul-08
Oct-08
34

33. Scalability needs beyond 2009
Workload vs Capacity
Workload Initial C Improved C Role Based C
16000
12000
8000
6000
4000
3000
2000
1500
90
10 200
90 600600
300 500
90 90 600
90 600
90 600
90 600
90
Sep-05
Sep-06
Sep-07
Sep-08
Sep-09
Sep-10
Jan-05
Jan-06
Jan-07
Jan-08
Jan-09
Jan-10
May-10
May-05
May-06
May-07
May-08
May-09
35

34. The unreasonable scalability case
36

35. Role-based architecture
37

36. Detailed role assignment
38

37. Technology alone cannot beat
what an experienced engineer
can design on the back of an
envelope
39

38. 40

39. Resources for IT Professionals
www.microsoft.com/teched
Tech·Talks Tech·Ed Bloggers
Live Simulcasts Virtual Labs
http://microsoft.com/technet
Evaluation licenses, pre-released
products, and MORE!

40. Related Content
Breakout Sessions (session codes and titles)
Interactive Theater Sessions (session codes and titles)
Hands-on Labs (session codes and titles)
Hands-on Labs (session codes and titles)

41. Track Resources
Resource 1
Resource 2
Resource 3
Resource 4

42. 1 Year
Subscription!
Complete an
evaluation on
CommNet and
enter to win!

43. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should
not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
46