Sql server backup internals
•Download as PPTX, PDF•
1 like•881 views
Once the ‘Backup Database’ command executed, SQL Server automatically does few ‘Checkpoint’ to reduce the recovery time and also it makes sure that at point of command execution there is no dirty pages in the buffer pool. After that SQL Server creates at least three workers as ‘Controller’, ‘Stream Reader’ and ‘Stream Writer’ to read and buffer the data asynchronously into the buffer area (Out of buffer pool) and write the buffers into the backup device.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Sql server backup internals
Similar to Sql server backup internals (20)
More from Hamid J. Fard
More from Hamid J. Fard (9)
Recently uploaded
Recently uploaded (20)
Sql server backup internals
- 2. About Me Hamid J. Fard I am SQL Server Data Platform Expert with more than 9 years’ of professional experience, I am currently Microsoft Certified Master: SQL Server 2008, Microsoft Certified Solutions Master: Charter-Data Platform, Microsoft Data Platform MVP and CIW Database Design Specialist. After a few years of being a production database administrator I jumped into the role of Data Platform Expert. Being a consultant allows me to work directly with customers to help solve questions regarding database issues for SQL Server.
- 3. How SQL Server BackupWorks! 1- Checkpoint (Few times) 2- Start Buffering Data (Async I/O) 1- Writing Buffered Data 2- Exchange the Buffers
- 4. SQL Server Backup Buffer Exchange Buffer Buffer Buffer Buffer Buffer Buffer Buffer Buffer Buffer Buffer Buffer Buffer Buffer Buffer ReadThread WriteThread
- 5. SQL Server Backup Options • BufferCount Specifies the total number of I/O buffers to be used for the backup operation.You can specify any positive integer; however, large numbers of buffers might cause “out of memory” errors because of inadequate virtual address space in the Sqlservr.exe process. • MaxTransferSize Specifies the largest unit of transfer in bytes to be used between SQL Server and the backup media. The possible values are multiples of 65536 bytes (64 KB) ranging up to 4194304 bytes (4 MB).
- 6. How Many Concurrent BackupWorks! Controller Worker #1 Read Stream Worker #2 Write Stream Worker #3 Buffer Buffer Buffer Buffer Buffer Buffer Buffer DATABASE BACKUPCOMMAND
- 7. SQL Server BackupWorkers andTasks Backup Database Tempdata to disk = 'C:tempbackup1.bak', disk = 'C:tempbackup2.bak', disk = 'C:tempbackup3.bak',disk = 'C:tempbackup4.bak' With Init, Format, Buffercount =8 , Maxtransfersize=65536; Stream Reader Thread Stream Writer Threads Controller Thread
- 8. What SQL Server Backup is Doing! WARNING: This trace flags should be used under the guidance of Microsoft SQL Server support.They are used in this post for discussion purposes only and may not be supported in future versions. DBCCTRACEON (3004 , 3605 , -1); DatabaseTempData Starting up database 'TempData'. Backup: Media open Backup: Media ready to backup Backup: Clearing differential bitmaps Backup: Bitmaps cleared BackupDatabase:Checkpoint done BackupManager::SyncWithLog: SEReplLSN : 0 (0x00000000:00000000:0000) BackupManager::SyncWithLog: Start LSN : 282000000023200037 (0x0000011a:000000e8:0025) Backup: Scanning allocation bitmaps Backup: Done with allocation bitmaps BackupDatabase:Work estimates done Backup: Leading metadata section done Backup:Copying data Backup: DBReaderCount = 2 ….. BackupDatabase: Database files done Backup:Copying log BackupDatabase : Log files done Backup:Trailing config done Backup: MBC done BackupDatabase:Writing history records Database backed up. Database:TempData, creation date(time): 2015/10/20(20:05:41), pages dumped: 1553, first LSN: … Writing backup history records BACKUP DATABASE successfully processed 338 pages in 0.087 seconds (30.267 MB/sec). BackupDatabase: Finished
Editor's Notes
- Once the ‘Backup Database’ command executed, SQL Server automatically does few ‘Checkpoint’ to reduce the recovery time and also it makes sure that at point of command execution there is no dirty pages in the buffer pool. After that SQL Server creates at least three workers as ‘Controller’, ‘Stream Reader’ and ‘Stream Writer’ to read and buffer the data asynchronously into the buffer area (Out of buffer pool) and write the buffers into the backup device. Be aware that ‘Stream Reader’ thread is one per volume and ‘Stream Writer’ thread is one per backup device.
- As mentioned before, SQL Server backup process creates buffer for backup purposes. The ‘Stream Reader’ thread loads the data into buffer area asynchronously and the ‘Stream Writer’ thread writes the data into the backup device, once the buffer data been written into the backup device the buffer goes under ‘Stream Reader’ queue to be filled up and later on goes to ‘Stream Writer’ queue to be emptied. The count of buffer are vary and the buffer size can be from 64kb to 4MB. For example, based on the above figure our backup buffer can be either minimum 384KB or maximum 24MB.
- To get the buffer count value, you can use the following formula: BufferCount = #ofBackupDevices x 3 + #ofBackupDevices + (2 x #ofVolumeInvolved)
- remaining virtual address space must be considered for concurrent backup operations. For example on a 32 bit system you get into the ‘memory to leave’ discussion but on 64 bit it is not a ‘memory to leave’ discussion, therefore how much you shrink the buffer pool memory to support allocations outside the buffer pool is considered. The direct memory consumers to consider are: The stream threads (one per output target) are threads so the stacks are allocated by the operating system outside the buffer pool. (x86 = .5MB, x86 WOW = .75MB, x64 = 2MB, …) The backup buffers are larger than 8K so they are also allocated outside the buffer pool. So to find how many concurrent backup can work together, we need to do some calculation as following:- NumOfTotalWorkers: #ofSchedulers x 255 - 200 NumOfBackupWorkers: 2 + #OfVolumeInvolved + #OfBackupDevices Concurrent Backups = NumOfTotalWorkers / NumOfBackupWorkers For example: NumOfTotalWorkers = 4 x 255 – 200 = 820 NumOfBackupWorkers = 2 + 2 + 4 = 8 Concurrent Backups = 820 / 8 = 102
- With use of the following script you can capture the backup internal threads and their last wait type as well. while (1=1) begin Insert Into TasksTB select R.command, T.task_address, T.session_id,T.pending_io_byte_average,T.pending_io_count,T.task_state,W.memory_object_address, W.last_wait_type,W.status,W.state, TH.stack_bytes_committed,TH.status [thread status],sysdatetime() as [timestamp] --into taskstb from sys.dm_os_tasks T Inner Join sys.dm_exec_requests R On R.session_id = T.session_id AND R.command = 'BACKUP DATABASE' Inner Join sys.dm_os_workers W on W.worker_address = T.worker_address Inner Join Sys.dm_os_threads TH on TH.worker_address = W.worker_address End Run the above script in a session and open another session and run the following script once the backup operation is done. SELECT distinct [command],[task_address] ,[session_id] ,[last_wait_type],memory_object_address,timestamp FROM [TempData].[dbo].[TasksTB] where timestamp = '2016-04-16 16:48:52.2217219' order by last_wait_type Make sure that you filter it by one of the timestamp values to see how many threads were running during backup operation.
- The following is SQL Server error log output using trace flag 3004 to collect extended information. Trace flag 3605 is also required to force the output into the error log. I highlighted the normal messages to help show the phase breakdowns better. Looking at the detailed messages you can see what is doing on. For example halting the full text crawls happens before you see the starting up database message. Using the timings and the associated message here you can take a standard restore sequence and determine the areas of performance to focus on.