SlideShare una empresa de Scribd logo
1 de 15
Partitioning Design
For Performance and Maintainability



                                      Martin Cairns
                             http://sqlbyparts.com
             http://twitter.com/MartinCairnsSQL
Who am I?
Partitioning Defined

Partitioning (noun)

1. the action or state of dividing or being
   divided into parts
2. a structure dividing a space into two parts



                  Source: http://oxforddictionaries.com/definition/english/partition
Types of Partitioning

• Horizontal
  • Partition Table by Rows
• Vertical
  • Partition Table by Columns
• File Group
  • Partition Tables by File Group
Horizontal Partitioning

 Divides the rows into small sets by
  boundaries
 Year   PK        Order ID   Product ID   Qty    Cost   Line XML
 2010   1         1          1            5      1.5    <Line…
 2010   2         1          1            7      2      <Line…

 2011   1000000   20000      345          1000   5      <Line…
 2011   1000001   20000      347          3000   12     <Line…

 2012   2000000   40000      705          8000   3      <Line…
 2012   2000001   40001      706          3000   7      <Line…
Vertical Partitioning

 Divides columns from one table into multiple
  tables
 Year   PK        Order ID   …   Year   PK        Line XML
 2010   1         1          …   2010   1         <Line…
 2010   2         1          …   2010   2         <Line…
 2011   1000000   345        …   2011   1000000   <Line…
 2011   1000001   347        …   2011   1000001   <Line…
 2012   2000000   705        …   2012   2000000   <Line…
 2012   2000001   706        …   2012   2000001   <Line…
File Group Partitioning

 File group partitioning is separating the storage of tables
  and indexes onto separate database files
 During a Primary File Group restore the whole database
  will be offline
 Separate the system & user tables to allow the quickest
  restore time
 Since the partitioned database is made up of smaller
  parts it is easier to manage the storage location of the
  files
Using Table Partitioning

 Supports Horizontal Partitioning
 Partition Function defines boundaries based
  on a single column
 Partition Scheme defines the File Group
 Inserts automatically supported
 All partitions share the same definition as the
  table (indexes, columns, fill factor etc)
Table Level Limitations

 Only ~200 steps for each statistics across the
  whole table
 Online Index rebuilds are for the whole table
  not individual partitions
 Fill Factor
 Lock settings (Row, Page, Escalation, etc)
 Indexes are defined for the whole table rather
  than partitions
Using View Partitioning

 View Partitioning allows you to overcome
  Table Level Constraints
 Check Constraints used to define partitions
 Combine all tables with a UNION ALL View
 Can be used together with Table Partitioning
 Trigger required to allow INSERT with Identity
  column
 Allows more complex partitioning schemes
Candidates for Partitioning

 Large vs. Small Tables
 Replicated vs. Non-Replicated Tables
 Normal vs. BLOB columns
 Write Heavy Current data vs. Heavily
  Read Historic data
 Read  Write vs. Read Only Tables
Partial Database Availability

 Also referred to as Piecemeal Restore
 Full Recovery Model is Required
 Individual File & File Groups Restore
 Only the Primary File Group is required to
  restore a database
 Allows restoring a subset of a correctly
  partitioned database for quicker recovery
  from a disaster
Tipping Point  Why is it Table Scanning

 As the number of rows in a table increase the
  depth of the B+Tree increases
 The number of Page Reads for a lookup is
  equal to the depth of the B+Tree
 The Tipping Point is the point where a Full
  Table Scan requires less Page Reads than
  the lookups
What Does Partition Give Us

 Choices
   Using different Tiers of Storage
   Allows quicker recovery strategies
   Allows partial restores of the database
 Performance
   Allows reduction of overhead of backups
   Allows better query plans due to more
    accurate statistics on large tables
Demos

 Moving Tables to File Groups
 Partial Restore + Using Partial Restore to
  Initialise Replication
 Moving Partition Between Tables For Current
  vs. Historic
 Show how View Partitioning can present a
  single view of all tables & still support
  partition elimination

Más contenido relacionado

Destacado

History n Working Of Laser Printer
History n Working Of Laser PrinterHistory n Working Of Laser Printer
History n Working Of Laser Printer
Awais Alam
 
7 state model in Operating System
7 state model in Operating System7 state model in Operating System
7 state model in Operating System
Mohammad jawad khan
 
Usability issues in google chrome & its solutions.
Usability issues in google chrome & its solutions.Usability issues in google chrome & its solutions.
Usability issues in google chrome & its solutions.
Mohammad jawad khan
 
Printer Presentation - H.Toor
Printer Presentation - H.ToorPrinter Presentation - H.Toor
Printer Presentation - H.Toor
hst77
 
Concept of hashing
Concept of hashingConcept of hashing
Concept of hashing
Rafi Dar
 

Destacado (20)

History n Working Of Laser Printer
History n Working Of Laser PrinterHistory n Working Of Laser Printer
History n Working Of Laser Printer
 
7 state model in Operating System
7 state model in Operating System7 state model in Operating System
7 state model in Operating System
 
Oracle Insert Statements for DBAs and Developers
Oracle Insert Statements for DBAs and DevelopersOracle Insert Statements for DBAs and Developers
Oracle Insert Statements for DBAs and Developers
 
Usability issues in google chrome & its solutions.
Usability issues in google chrome & its solutions.Usability issues in google chrome & its solutions.
Usability issues in google chrome & its solutions.
 
Laser printers
Laser printersLaser printers
Laser printers
 
Printers
 Printers Printers
Printers
 
Hashing
HashingHashing
Hashing
 
Printer Presentation - H.Toor
Printer Presentation - H.ToorPrinter Presentation - H.Toor
Printer Presentation - H.Toor
 
Probing
ProbingProbing
Probing
 
11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patil11. Hashing - Data Structures using C++ by Varsha Patil
11. Hashing - Data Structures using C++ by Varsha Patil
 
PRINTERS
PRINTERSPRINTERS
PRINTERS
 
Concept of hashing
Concept of hashingConcept of hashing
Concept of hashing
 
Deterministic Finite Automata
Deterministic Finite AutomataDeterministic Finite Automata
Deterministic Finite Automata
 
7. Tree - Data Structures using C++ by Varsha Patil
7. Tree - Data Structures using C++ by Varsha Patil7. Tree - Data Structures using C++ by Varsha Patil
7. Tree - Data Structures using C++ by Varsha Patil
 
Hashing Techniques in Data Structures Part2
Hashing Techniques in Data Structures Part2Hashing Techniques in Data Structures Part2
Hashing Techniques in Data Structures Part2
 
6. Linked list - Data Structures using C++ by Varsha Patil
6. Linked list - Data Structures using C++ by Varsha Patil6. Linked list - Data Structures using C++ by Varsha Patil
6. Linked list - Data Structures using C++ by Varsha Patil
 
NFA or Non deterministic finite automata
NFA or Non deterministic finite automataNFA or Non deterministic finite automata
NFA or Non deterministic finite automata
 
kerberos
kerberoskerberos
kerberos
 
Types of printer
Types of printerTypes of printer
Types of printer
 
Finite Automata
Finite AutomataFinite Automata
Finite Automata
 

Similar a Partitioning Design

EstlGd.DataPartitioning
EstlGd.DataPartitioningEstlGd.DataPartitioning
EstlGd.DataPartitioning
Daren Bieniek
 
Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…
Aaron Shilo
 
2 extreme performance - smaller is better
2   extreme performance - smaller is better2   extreme performance - smaller is better
2 extreme performance - smaller is better
sqlserver.co.il
 

Similar a Partitioning Design (20)

EstlGd.DataPartitioning
EstlGd.DataPartitioningEstlGd.DataPartitioning
EstlGd.DataPartitioning
 
Teched03 Index Maint Tony Bain
Teched03 Index Maint Tony BainTeched03 Index Maint Tony Bain
Teched03 Index Maint Tony Bain
 
Sql server lesson7
Sql server lesson7Sql server lesson7
Sql server lesson7
 
SQL Server 2016 novelties
SQL Server 2016 noveltiesSQL Server 2016 novelties
SQL Server 2016 novelties
 
Tuning and Optimizing U-SQL Queries (SQLPASS 2016)
Tuning and Optimizing U-SQL Queries (SQLPASS 2016)Tuning and Optimizing U-SQL Queries (SQLPASS 2016)
Tuning and Optimizing U-SQL Queries (SQLPASS 2016)
 
Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…
 
Optimizing Data Accessin Sq Lserver2005
Optimizing Data Accessin Sq Lserver2005Optimizing Data Accessin Sq Lserver2005
Optimizing Data Accessin Sq Lserver2005
 
Database Performance
Database PerformanceDatabase Performance
Database Performance
 
Partitioning kendralittle
Partitioning kendralittlePartitioning kendralittle
Partitioning kendralittle
 
Pptofdistributeddb
PptofdistributeddbPptofdistributeddb
Pptofdistributeddb
 
Windows azure table storage – deep dive
Windows azure table storage – deep diveWindows azure table storage – deep dive
Windows azure table storage – deep dive
 
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...
Best Practices and Performance Tuning of U-SQL in Azure Data Lake (SQL Konfer...
 
MS-SQL SERVER ARCHITECTURE
MS-SQL SERVER ARCHITECTUREMS-SQL SERVER ARCHITECTURE
MS-SQL SERVER ARCHITECTURE
 
初探AWS 平台上的 NoSQL 雲端資料庫服務
初探AWS 平台上的 NoSQL 雲端資料庫服務初探AWS 平台上的 NoSQL 雲端資料庫服務
初探AWS 平台上的 NoSQL 雲端資料庫服務
 
Data warehouse physical design
Data warehouse physical designData warehouse physical design
Data warehouse physical design
 
OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...
OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...
OVERVIEW OF NEW SQL,COMPARING SQL,NOSQL AND NEWSQL,B.Vinithamani,II-M.sc(Comp...
 
Achievements
AchievementsAchievements
Achievements
 
Ms sql server architecture
Ms sql server architectureMs sql server architecture
Ms sql server architecture
 
2 extreme performance - smaller is better
2   extreme performance - smaller is better2   extreme performance - smaller is better
2 extreme performance - smaller is better
 
R2 roadshows
R2 roadshowsR2 roadshows
R2 roadshows
 

Último

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Último (20)

Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 

Partitioning Design

  • 1. Partitioning Design For Performance and Maintainability Martin Cairns http://sqlbyparts.com http://twitter.com/MartinCairnsSQL
  • 3. Partitioning Defined Partitioning (noun) 1. the action or state of dividing or being divided into parts 2. a structure dividing a space into two parts Source: http://oxforddictionaries.com/definition/english/partition
  • 4. Types of Partitioning • Horizontal • Partition Table by Rows • Vertical • Partition Table by Columns • File Group • Partition Tables by File Group
  • 5. Horizontal Partitioning  Divides the rows into small sets by boundaries Year PK Order ID Product ID Qty Cost Line XML 2010 1 1 1 5 1.5 <Line… 2010 2 1 1 7 2 <Line… 2011 1000000 20000 345 1000 5 <Line… 2011 1000001 20000 347 3000 12 <Line… 2012 2000000 40000 705 8000 3 <Line… 2012 2000001 40001 706 3000 7 <Line…
  • 6. Vertical Partitioning  Divides columns from one table into multiple tables Year PK Order ID … Year PK Line XML 2010 1 1 … 2010 1 <Line… 2010 2 1 … 2010 2 <Line… 2011 1000000 345 … 2011 1000000 <Line… 2011 1000001 347 … 2011 1000001 <Line… 2012 2000000 705 … 2012 2000000 <Line… 2012 2000001 706 … 2012 2000001 <Line…
  • 7. File Group Partitioning  File group partitioning is separating the storage of tables and indexes onto separate database files  During a Primary File Group restore the whole database will be offline  Separate the system & user tables to allow the quickest restore time  Since the partitioned database is made up of smaller parts it is easier to manage the storage location of the files
  • 8. Using Table Partitioning  Supports Horizontal Partitioning  Partition Function defines boundaries based on a single column  Partition Scheme defines the File Group  Inserts automatically supported  All partitions share the same definition as the table (indexes, columns, fill factor etc)
  • 9. Table Level Limitations  Only ~200 steps for each statistics across the whole table  Online Index rebuilds are for the whole table not individual partitions  Fill Factor  Lock settings (Row, Page, Escalation, etc)  Indexes are defined for the whole table rather than partitions
  • 10. Using View Partitioning  View Partitioning allows you to overcome Table Level Constraints  Check Constraints used to define partitions  Combine all tables with a UNION ALL View  Can be used together with Table Partitioning  Trigger required to allow INSERT with Identity column  Allows more complex partitioning schemes
  • 11. Candidates for Partitioning  Large vs. Small Tables  Replicated vs. Non-Replicated Tables  Normal vs. BLOB columns  Write Heavy Current data vs. Heavily Read Historic data  Read Write vs. Read Only Tables
  • 12. Partial Database Availability  Also referred to as Piecemeal Restore  Full Recovery Model is Required  Individual File & File Groups Restore  Only the Primary File Group is required to restore a database  Allows restoring a subset of a correctly partitioned database for quicker recovery from a disaster
  • 13. Tipping Point Why is it Table Scanning  As the number of rows in a table increase the depth of the B+Tree increases  The number of Page Reads for a lookup is equal to the depth of the B+Tree  The Tipping Point is the point where a Full Table Scan requires less Page Reads than the lookups
  • 14. What Does Partition Give Us  Choices  Using different Tiers of Storage  Allows quicker recovery strategies  Allows partial restores of the database  Performance  Allows reduction of overhead of backups  Allows better query plans due to more accurate statistics on large tables
  • 15. Demos  Moving Tables to File Groups  Partial Restore + Using Partial Restore to Initialise Replication  Moving Partition Between Tables For Current vs. Historic  Show how View Partitioning can present a single view of all tables & still support partition elimination

Notas del editor

  1. Today I will be talking about partitioning which tends to be thought about after the lack of it has caused a problem.
  2. SQL Server MCM one of roughly 100 in the worldBI Architect/Consultant with 15 years experience of designing large scale and complex database systems
  3. A slightly different definition of partitioning than the marketing releases from MS
  4. Moving out BLOB columns removes SQL 2008 restrictions on Online Index RebuildsInfrequently access columns add overhead to queries which don’t require them, another good reason to avoid SELECT *
  5. It’s best not to separate Tables and their indexes as both will have to restored to use the tables
  6. Fill Factors waste space in the buffer pool and are only required for the active part of a tableRow Locks might be wasteful on the static historic data but required for active data
  7. Grouping the smaller tables away from the large allows a quicker recovery from data lossInitialise Replication from a backupOnline Index Rebuilds in SQL Server 2008 don’t allow LOBsRead-Only file group is the only guaranteed way to make a table read-only, Column Stores will be Read/Write at some point
  8. A single file from a file group can be restored while the database is online as long as it’s not from the Primary File GroupThe transaction log from the earliest file LSN to the current LSN will need to be restored to return the database into a consistent state
  9. Mention Kimberly Trip’s blog on the Tipping Point
  10. Today we will be covering the choices which should be considered when designing a new database and when deal with an existing database which has got new requirements like a higher uptime or scaling to larger volumes of data