Developer Data Modeling Mistakes: From Postgres to NoSQL
Tech days 2011 - database design patterns for keeping your database applications available and performing well - charley hanania
1. Charley Hanania :: QS2 AG – Quality Software Solutions
B.Sc (Computing), MCP
, MCDBA, MCITP
,MCTS, MCT
, Microsoft MVP: SQL Server
Principal Consultant - Senior Database Specialist
Database Design Patterns for
Keeping Database Applications
Available and Performing Well.
2. About Charley Hanania
Now:
Microsoft MVP: SQL Server
Database Consultant at QS2 AG
Formerly:
Production Product Owner of MS SQL Server Platform at UBS Investment Bank
Consultant
DB Team Leader at Other Banks
ITIL v3 Certified
SQL Server Certified since 1998
On SQL Server since 1995
Version 4 on OS/2
IT Professional since 1992
PASS
Chapter Leader – Switzerland
Regional Mentor – Europe
24 Hours of PASS Committee Member
Event Speaker
3. Presentation Summary
DBA's will give you reasons and methods for achieving database availability
for your application, but not many can broach the concept of keeping parts
of your application running during an outage from the development
perspective through partitioning your application database logically
into levels of criticality.
Adding this logical Layer to your database design and development allows
you to bring your core application functionality online faster and
more simply during failures and in disaster scenarios decide which
areas of your application should come online first to maintain basic
application operations. It also allows you to place your data better to
cater for space and performance for an overall better performing system.
This presentation will drill into this conceptually as well as provide some
examples you can take back to the workplace to incorporate into your next
evolutionary changes.
5. Design Patterns by any Other Name…
Industry “norms” in using databases
Logically Partitioning your DB for Clarity
Logically Partitioning your DB for Performance
Logically Partitioning your DB for Availability
Appendixes (time permitting):
Partitioned Tables and Views
Asynchronous Queues
Agenda
7. Design Patterns are template-like methodologies.
Focussed on particular goals.
Architect-level focussed, but with follow-on
implementation considerations.
Design Patterns by any Other Name…
16. Logically Partitioning your DB for Performance
Performance-related issues:
Disk I/O is the one most critical element to improving
performance for data intensive workloads.
When data objects are created in the default filegroup
they share space and allocate more as needed.
Extent fragmentation occurs as data from objects with
differing “personalities” are persisted.
Hotspots can occur on system pages that track the
allocation and use of pages in the file.
Read-ahead is affected as the data you’re retrieving is
not laid out contiguously.
User Schema Separation is done, now to look at the next layer down.
17. Logically Partitioning your DB for Performance
Resolutions:
Creating additional files and filegroups helps to isolate
where the objects are written to and lowers
fragmentation.
System pages are spread out per file, lowering contention.
With Trace Flag 1117 and correct sizing of your data
files, “Round-Robin” writes are possible, giving higher
throughput on insert intensive workloads.*
Performance Critical data can be placed on LUNS/Disk
Arrays that have more spindles (eg RAID 1+0 mirrored)
or on mirrored Solid State Drives. Conversely, archive
data can be placed on inexpensive (eg. slower) disks.
*Note: Recommendation valid for systems that have >1 CPU/Core and >1 Disk Spindle
otherwise zero or detrimental performance may be experienced. Test First!
18. Step 2 - Filegroup Separation
Separating out objects into filegroups for isolation,
placing them according to granular performance
needs.
19. Quick Recap – So Far
1. Don’t put user defined objects in the Primary Filegroup
i. Allows the Primary Filegroup to load faster
2. Schema Separate user defined objects
i. For clarity of purpose and operational use
ii. Try not to use dbo schema (good design/operational practice)
iii. DBA’s will schematize objects in a similar fashion to what was in
the Adventureworks sample. You should look to map your
schema’s around your application’s core functionality / use cases
/ criticality
3. Create Filegroups for each Schema
i. Multiple if you want to partition historical data horizontally etc
ii. Name them in a fashion that allows easy understanding of what
is in them, or how they should be used.
21. Logically Partitioning your DB for Availability
Major Points to availability when focussed on Logical
Partitioning
Time taken to open the database on restart, to start
taking connections.
Decreasing the size of the backup to restore through
differential backups of read/write filegroups.
Partial Database availability: a file/disk can go offline
without it affected unrelated objects in the same
database
If architected/implemented correctly, most subsystems
can run unaffected until non peak period for outage
Management & Operations staff can decide which
filegroups & subsystems to bring online first when
recovering from disaster
.
22. Step 3 – Partial Database Availability
What happens when a file or disk goes offline? How are
other running subsystems affected?
26. Step 5 – Service Broker Queues
Writing messages to an asynchronous queue rather
than directly to a table
27. Links and Resources
AdventureWorks Database Diagram
http://www.microsoft.com/downloads/details.aspx?FamilyID=0f6e0bcf-a1b5-4760-8d79-
67970f93d5ff&DisplayLang=en
Whitepaper: “Partial Database Availability” (Danny T
ambs, May 2007)
http://technet.microsoft.com
Whitepaper: “Partitioned T
able and Index Strategies Using SQL Server 2008” (Ron
T
almage, March 2009)
http://technet.microsoft.com
Kimberly Tripp’s Scripts for Creating and Partitioning SalesDB
www.sqlskills.com
Allen White’s Service Broker Session at the PASS Summit 2011
www.sqlpass.org
28. Please help us make TechDays even better
by Evaluating this Session. Thank you!
Give us your feedback!
[Tambs]
While the database is in a state of partial availability, the filegroups that remain online can support queries. However, queries that depend on data that resides in filegroups that are offline return error 8653:
Msg 8653, Level 16, State 1, Line 3
The query processor is unable to produce a plan for the table or view 'X' because the table resides in a filegroup which is not online.