SQLBits VI Presentation on SQL Server Fast Track Data Warehousing Concepts

1. 1© Copyright 2009 EMC Corporation. All rights reserved.
Fast Track Foundations:
Serious about Sequential I/O
James Rowland-Jones
&
Allan Mitchell

2. 2© Copyright 2009 EMC Corporation. All rights reserved.
WTF IS FAST TRACK?
Introduction

3. 3© Copyright 2009 EMC Corporation. All rights reserved.
Blend of Hardware & Software

4. 4© Copyright 2009 EMC Corporation. All rights reserved.
Predictable Performance
Balanced
Configurations
– CPU
– Memory
– Network
– Storage

5. 5© Copyright 2009 EMC Corporation. All rights reserved.
Targeted to the Data Warehouse Workload
Sequential Scanning of Data
Leads to fewer disks

6. 6© Copyright 2009 EMC Corporation. All rights reserved.
Prescriptive
Best Practices
Recommendations
You don’t get to
pick and choose!

7. 7© Copyright 2009 EMC Corporation. All rights reserved.
WHY SQL 2008 ONLY?

8. 8© Copyright 2009 EMC Corporation. All rights reserved.
SQL Server 2008 Enterprise...
...was the Data Warehousing release...
Compression
Star Joins
Few Outer Rows Parallelism
Partitioned Table Parallelism
Resource Governor
Improved Read Ahead

9. 9© Copyright 2009 EMC Corporation. All rights reserved.
SQL 2008 Settings
Startup Parameters
– T1117
– E
Resource Governor
– Reduce Max memory per session
Page Compression ON!

10. 10© Copyright 2009 EMC Corporation. All rights reserved.
Windows 2008 Settings
Multipath I/O Feature is required
Lock Pages In Memory Enabled

11. 11© Copyright 2009 EMC Corporation. All rights reserved.
SEQUENTIAL VS RANDOM I/O
DEMO TIME:

12. 12© Copyright 2009 EMC Corporation. All rights reserved.
Counters
Performance Counter
 Avg. Disk Bytes / Read
 Avg. Disk Sec / Read
 Disk Read Bytes / Sec
 Disk Reads / Sec
Meaning
 Size of I/Os
 Latency “Seek Time”
 Throughput – MB/s
 I/Os per second
 sys.dm_db_index_physical_file_stats
 sys.dm_io_virtual_file_stats
 sys.dm_io_pending_io_requests
DMVs

13. 13© Copyright 2009 EMC Corporation. All rights reserved.
Time to Take the Red Pill...

14. 14© Copyright 2009 EMC Corporation. All rights reserved.

15. 15© Copyright 2009 EMC Corporation. All rights reserved.
DB DESIGN CONSIDERATIONS
Fast Track Thinking

16. 16© Copyright 2009 EMC Corporation. All rights reserved.
Indexing
Think Index Light
 NC Indexes aimed at low volume row lookups
 Indexes are Expensive to maintain

17. 17© Copyright 2009 EMC Corporation. All rights reserved.
Aggregation
Think Aggregation Light
 Introduces Updates (fragmentation)
 Removing aggs reduces maintenance

18. 18© Copyright 2009 EMC Corporation. All rights reserved.
Partitioning
Partition Qualification
Data Management
P1 P4P3P2P
2

19. 19© Copyright 2009 EMC Corporation. All rights reserved.
RG1 RG2
RG3 RG4
RG5 RG6
RG7 RG8
RG9
RG10
Conceptual View of Fast Track Hardware
1
3
2
4
5
7
6
8
H
H
4GB
4GB
4GB
4GB
4GB
4GB
4GB
RAID 1
LUN 01
LUN 02
LUN 03
LUN 04
LUN 17
(Logs)
LUN 05
LUN 07
LUN 06
LUN 08
SPARE
SPASPB
FC1
FC2
FC3
FC4
FibreSwitch
LUN 09
LUN 11
LUN 10
LUN 12
LUN 18
(Logs)
LUN 13
LUN 15
LUN 14
LUN 16
SPARE
SPCSPD
FC5
FC6
FC7
FC8
8 Core 32 GB RAM
Server
4GB

20. 20© Copyright 2009 EMC Corporation. All rights reserved.
RG1 RG2
RG3 RG4
RG5 RG6
RG7 RG8
RG9
RG10
Performance View of Fast Track Hardware
1
3
2
4
5
7
6
8
H
H
LUN 01
LUN 02
LUN 03
LUN 04
LUN 17
(Logs)
LUN 05
LUN 07
LUN 06
LUN 08
SPASPB
FC1
FC2
FC3
FC4
FibreSwitch
LUN 09
LUN 11
LUN 10
LUN 12
LUN 18
(Logs)
LUN 13
LUN 15
LUN 14
LUN 16
SPCSPD
FC5
FC6
FC7
FC8
200 mb/s per CPU core is
central to Fast Track
Architecture
32GB
RAM800mb/s
800mb/s
400mb/s
per port
1 LUN =
125
MB/S
SP Port
650MB/s
FCPort
400MB/S

21. 21© Copyright 2009 EMC Corporation. All rights reserved.
Performance Summary
Component Throughput
8 CPU Cores 1600 MB/S
4 HBA Ports (2 adapters) 1600 MB/S
8 FC Ports (4 per array) 3200 MB/S
4 Storage Processors (2 per array) 2600 MB/S
16 LUNS (Data Only) 2000 MB/S

22. 22© Copyright 2009 EMC Corporation. All rights reserved.
Capacity Summary
16x 300GB Data Disks Raw (formatted) 4352 GB
4x 300GB Log Disks Raw (formatted) 1088 GB
LESS RAID 1
8x RAID 1 Raid Groups for Data 2176 GB
2x RAID 1 Raid Groups for Logs 544 GB
LESS TEMPDB SPACE
Tempdb Data files 576 GB
User Data Uncompressed Available 1600 GB
Data Compression Ratio 2.5:1 4000 GB

23. 23© Copyright 2009 EMC Corporation. All rights reserved.
Database Layout
Fi1 Fi2 Fi3 Fi4 Fi5 Fi6 Fi7
Fi9 Fi10 Fi11 Fi12 Fi13 Fi14 Fi15
Fi1 Fi2 Fi3 Fi4 Fi5 Fi6 Fi7
Fi1 Fi2 Fi3 Fi4 Fi5 Fi6 Fi7
Fi9 Fi10 Fi11 Fi12 Fi13 Fi14 Fi15
Fi9 Fi10 Fi11 Fi12 Fi13 Fi14 Fi15 tempLog
Live Log
Stage Log
Live
FG1
Stage
FG1
Tempdb
Primary
Fi8
Fi8
Fi8
Fi16
Fi16
Fi16
Fi0
Fi0
PrimaryPrimary

24. 24© Copyright 2009 EMC Corporation. All rights reserved.
MANAGING FRAGMENTATION
The arch enemy of Sequential I/O

25. 25© Copyright 2009 EMC Corporation. All rights reserved.
Fragmentation
File System Extent Index

26. 26© Copyright 2009 EMC Corporation. All rights reserved.
File System Fragmentation
Pre-Allocate your Database files
Avoid O/S de-fraggers
They know not what they do...

27. 27© Copyright 2009 EMC Corporation. All rights reserved.
Extent Fragmentation
Occurs Courtesy of
Concurrent DML
Updates
Deletes
Follow the guidelines!
Otherwise you’ll be re-loading the tables....

28. 28© Copyright 2009 EMC Corporation. All rights reserved.
Index Fragmentation
 Don’t use ALTER INDEX REORGANIZE
 Do one of the following
– ALTER INDEX REBUILD ...
SORT_IN_TEMPDB = TRUE (MAXDOP 1)
– INSERT ... SELECT into a new table

29. 29© Copyright 2009 EMC Corporation. All rights reserved.
File Groups
Isolate Volatile Data
 Create a separate Filegroup
 One file per LUN to spread the I/O
Isolate Small / Dimension Tables
Stage Data in a Separate DB or File Group

30. 30© Copyright 2009 EMC Corporation. All rights reserved.
DATA LOADING FOR
SEQUENTIAL I/O
Over to Mitchell...

31. where Allan and Darren live