1. Introduction To Debugging ASP.NET applications with
WinDBG and Dump Analysis
AHMET MİTHAT BOSTANCI
MICROSOFT, GTSC
IIS/WEB DEVELOPMENT SUPPORT
AHMET.BOSTANCI@MICROSOFT.COM
2. SCOPE
This is NOT a full dump analysis workshop.
The goal is to introduce the dump analysis as one of
the ASP.NET 2.0 web application debugging technics
and introduce the tools that help to achieve this task.
3. AGENDA
ASP.NET Debugging Technics
What is WinDBG
When are WinDBG and dump analysis useful?
What are dump files?
How to collect dump?
Basic commands
Toolbox & References
4. ASP.NET Debugging
Old School Debugging
Using Response.Write
Mostly not useful in production
Not useful in hang scenario
Usful in development
Useful in production if there is a repro
Looks easy but not useful
Debugging in VS.NET
Good for debugging in development
Useful in production with remote debugging / attaching to w3wp.exe
Not useful in most production env. issues.
Debugging with WinDBG
Live debugging / attaching to w3wp.exe
Offline debugging: Collecting dumps of w3wp.exe and analysing in
WinDBG
5. What is WinDBG
A GUI Debugger with commands.
Part of Debugging Tools For Windows
User mode and kernel mode debugging.
Its commands are mainly useful in unmanaged
debugging.
Extensions are sets of commands to ease debugging.
SOS and PSSCOR2 extensions for .NET debugging.
6. Debugging with WinDBG
Offline debugging
Collecting dump files with specific tools for different scenarios
Analysing in WinDBG.
Live debugging
Attaching a process
Using breakpoints, catching exceptions, dumping the process
out to a file, etc... when necessary.
7. Debugging Scenarios
Slow performance
Web site is responsive but slow.
Hang
Web site is unresponsive.
Crash
Exceptions
8. Exceptions and ASP.NET
Exceptions are important objects in debugging.
First Chance exceptions
Too many exceptions may cause high CPU.
Second Chance
Exceptions which are not handled
Causes crash.
First chance exceptions may cause w3wp.exe
crashes, if it is thrown in a background thread, such
as finalizer threads.
9. When do we need WinDBG?
Hang / Performance Problems – Manuel
Web site is running slow / performance issues.
Web site is leaking memory - ends with performance
problems, hang or crash.
Web site is running with high memory but not not
leaking.
Web site is running with 100% CPU.
Web site is running with high CPU or CPU usage spikes.
Web site is completely hung - Does not response any
(dynamic) requests.
10. When do we need WinDBG?
Crash
Web site or a specific page / method is throwing
exceptions but not crashing the w3wp.exe process,
first chance exceptions.
Web site is crashing - w3wp.exe is crashing. Second
chance exceptions.
11. When do we NOT need WinDBG?
When there is an easier way for debugging, such as a
specific page / method is not functioning as expected
In this case, debugging in VS.NET with a repro code might be
easier.
«Page cannot be displayed» type issues.
In this case, troubleshooting IIS or network side is the correct
approach.
12. Dump
A dump is a snapshot of a process (user mode) or a
snapshot of all of the memory in kernel mode.
Usermode dump: Process specific, such as w3wp.exe
Kernel mode dump: Snapshot of the kernel memory
Complete memory dump: Snapshot of the whole
memory including kernel and user mode.
13. Types of Dump
Minidump
Contains only selected parts of the memory used by the
process.
Fulldump
Contains the full copy of the process memory. Size of the
fulldump is somewhat equal to the virtual memory used by the
process.
This is the dump type we are interested in most of the times.
14. What a dump contains
All the things in the memory of the process.
Information about the objects in heaps, threads,
information about the callstack, stack memory, etc...
Limited information about the dead objects which
are not garbage collected yet.
15. What a dump does NOT contain
Other processes' information.
What was the CPU or memory usage in time. It is
only a snapshot.
Also: Minidump is the smallest in size and does not
contain some useful information such as stack
objects, local variables, etc...
16. Tools for collecting dumps
ADPlus
Part of Debugging Tools For Windows
Command line
adplus -hang -pn w3wp.exe
adplus -crash –iis
Debug Diagnostic Tool
GUI based
Easy to configure and use
ProcDump
Command line
Specific for high CPU usage but can be used for collecting manuel dumps as well
procdump.exe -c 80 -s 30 -n 5 -ma NNNN >procdump.log
Task manager
Can be used to collect manuel dump
Right click and create userdump
Only on Win 7 and Win 2008
17. 32bit vs 64bit
If the process is 32bit then the dump should be taken
by the 32bit version of the related tools. The dump
should be opened with the WinDBG’s 32bit version.
Same rule applies to 64bit process.
18. Which dump in which scenario?
Web site is crashing / throwing exceptions
Crash rule with Debug Diagnostic Tool, ADPlus
Web site is slow, hanging, using high memory
Manuel dump(s) with Debug Diagnostic Tool, ADPlus.
Web site is using high CPU or CPU is spiking
Manuel dump(s) with Debug Diagnostic Tool, ADPlus.
ProcDump for CPU spikes
Web site is leaking memory
Manuel hang dumps.
Memory leak rule created in Debug Diagnostic Tool.
Full IIS dumps
Manuel dump
Collects dumps for all of w3wp.exe, all of dllhost.exe and inetinfo.exe
process.
19. How many dump files should I collect?
Dump is a snapshot of the memory and does not
contain the process history.
Most of the crash scenarios
One crash dump is enough to start debugging and find the
reason of the crash.
Performance, hang
Consecutive dump files of the same process.
For example taking three dumps of w3wp.exe, each are taken
one minute apart.
20. Symbol Files
Footprint information of functions in EXE or DLL
files
Necessary to see useful information in dumps, such
as callstacks or variables.
21. Private vs Public symbols
Private symbols contain local variables, structure
type information, functions, source line no., etc...
Public symbols are the stripped version of the private
symbols.
22. Symbols and WinDBG
Symbol server
http://msdl.microsoft.com/download/symbols
Setting symbol server in WinDBG
srv*c:symbolspublic*http://msdl.microsoft.com/download/
symbols
Symbols are downloaded to the local folder when
needed.
23. Commands in WinDBG
WinDBG commands
Mostly used in unmanaged debugging.
Extensions
DLL files
Command sets for specific debugging requirements
24. Extensions
Copied to the WinDBG installation directory
.NET related extensions
SOS
Comes with .NET framework
PSSCOR2
Superset of SOS
.NET 2.0 specific extension
http://www.microsoft.com/downloads/details.aspx?FamilyID=5c
068e9f-ebfe-48a5-8b2f-0ad6ab454ad4&displayLang=en
Extract and copy to the WinDBG installation folder.
Load from WinDBG:
.load psscor2
25. WinDBG and ASP.NET 2.0 dump analysis
Open dump file
Check and set symbol server
Load the PSSCOR2 extenstion
.load psscor2
26. Some WinDBG commands
Vertarget
Shows information about the system on which you are debugging
Lm
Displays loaded module information
~
Lists all OS threads
~Ns
Switch to thread #N
K
Shows unmanaged call stack of the thread switched.
.hh
Opens help file
.reload /f
Forces to reload the symbols.
!runaway
Shows the thread’s CPU time
27. Some PSSCOR2 commands
!DumpAllExceptions (!dae)
Shows managed exceptions thrown.
!DumpDataTables
Shows data tables objects in memory. Useful when debugging memory issues.
!DumpColumnNames (!dcn)
Prints out the column names for a data table given.
!ASPXPages
Shows ASPX pages running on all threads.
!threadpool
Shows ASP.NET thread pool information including the requests in the queue and the current CPU usage of OS.
!FindDebugTrue
Shows the web applications running in debug mode.
!FindDebugModules
Shows the modules built in debug mode.
!SaveAllModules (!sam)
Saves the all modules in the process. Useful when using the reflector to get the code from dump file.
!eeversion
Prints the .NET framework version. Useful when checking the fixes or service packs installed.
!help
!help <command>
28. Toolbox
Debugging Tools For Windows (WinDBG and ADPlus)
http://www.microsoft.com/whdc/DevTools/Debugging/default.msp
x
Debug Diagnostic Tool v1.1
http://www.microsoft.com/downloads/details.aspx?FamilyID=28B
D5941-C458-46F1-B24D-F60151D875A3&displaylang=en
ProcDump
http://technet.microsoft.com/en-us/sysinternals/dd996900.aspx
PSSCOR2
http://www.microsoft.com/downloads/details.aspx?familyid=5C068
E9F-EBFE-48A5-8B2F-0AD6AB454AD4&displaylang=en