2. Agenda
Production Troubleshooting Issues
WinDbg – a brief introduction
Main Agenda:
○ Part 1 - IntelliTrace
○ Part 2 - System Center APM
3. Me (raviokade at hotmail)
I have been programming since
I like to program in
My favorite programming topic is Data, more
data, big data
4.
5. Production Issue Types
Environmental issues
Connectivity, Hardware issues
High CPU, Memory issues
Typically managed through alerts
Application issues
Related to incorrect application logic
Can lead to incorrect application
behavior, “hang” or termination
6. Troubleshooting Application
Issues
Eventlogs, Perfmon can be of some help
But Application logs are most appropriate.
Requires proper logging to be in place.
Sometimes you have to change to a more
verbose level.
Generates tons of data
Slows down the application
Useless if lots of multi-threading is going on
Parameter values are not appropriate for
logging
7. What do you do when logs don’t
work ?
Try to reproduce! Near impossible.
QA = Production: rarely!
Takes time!
Add more logging and redeploy
Not practical in large organizations
8. Agenda
Production Troubleshooting Issues
WinDbg – a brief introduction
Main Agenda:
○ Part 1 - IntelliTrace
○ Part 2 - System Center APM
9. WinDbg
Available as a silent collector or as an
interactive debugger
Generates dump files
Dump files contain information about the
final state of the application
Used for critical scenarios where
application tends to terminate or hang
10. WinDbg issues
Nice tool .. If you know how to use it!
Need time to diagnose. Not for those
bleeding issues!
Not appropriate for all types of issues.
11. Agenda
Production Troubleshooting Issues
WinDbg – a brief introduction
Main Agenda:
○ Part 1 - IntelliTrace
○ Part 2 - System Center APM
12. Intellitrace
Code is instrumented dynamically
Records program execution flow
Historical debugging
Records program data
Method Parameters
Return values
1st class debugging experience in Visual
Studio
13. Intellitrace
Low overhead for collection (depends on
collection plan)
IntelliTrace supports debugging VB and
C# applications that use.NET version
2.0, 3.0, 3.5, or 4
Available only with Visual Studio
Ultimate edition
15. IntelliTrace Collector Basics
Collector can be deployed with xcopy
You can collect data without installing
Visual Studio.
The collector lets you capture details
about exceptions, threads, Web
requests, modules, and other system
information.
The collector saves this information as a
recording (.iTrace file) that you can open
in Visual Studio Ultimate.
16. IntelliTrace Collection steps
Download and expand the IntelliTrace
collector cab file
Collecting the Trace:
Web Application (need PowerShell):
Start-
IntelliTraceCollection <ApplicationPool> <CollectionPlan> <F
ullOutputPath>
Stop-IntelliTraceCollection <AppPoolName>
Exe Application:
IntellitraceSC.exe launch
/cp: <CollectionPlan> /f: <OutputFile> <ExecutableFile>
17. Open IntelliTrace recording in
Visual Studio Ultimate
Copy the recording (.iTrace file) to a
computer that has Visual Studio
Ultimate.
Start Visual Studio Ultimate. Use
the File menu to find and open the
recording.
18. IntelliTrace DEMO
Tracing an EXE Application
Tracing a Web Application
Customizing Trace
Collection Plan
Image: http://www.crowcanyon.com/Pick-a-demo.html
19. Agenda
Production Troubleshooting Issues
WinDbg – a brief introduction
Main Agenda:
○ Part 1 - IntelliTrace
○ Part 2 - System Center APM
20. System Center APM
Code is instrumented dynamically
Records program execution flow
Records program data (Parameters and
Return values)
Low overhead for collection (depends on
configuration)
21. System Center APM
Code is instrumented dynamically
Records program execution flow
Records program data (Parameters and
Return values)
Low overhead for collection (depends on
configuration)
22. How is it different from
Intellitrace?
Always-on monitoring
Performance monitoring
Remote administration/configuration
Correlation with performance counters
Multi-server call chains
23. System Center .Net Monitoring
setup
System Center System
System Center Application Center
Agent Server DB
Monitored Web
Application System
Center
Operations
Monitor
24. System Center APM Demo
Monitoring a Web Application
View Alerts
View Stack Trace and Parameter
Values
Customizing What is collected
28. System Center APM – Advanced
features
Client side monitoring
System Center Advisor
Collected data automated clean-up
29. When to use what
Scenario WinDbg System Center IntelliTrace
APM
Memory leak Yes Limited No
Exceptions Limited Yes Yes
Performance No Yes No
Deadlocks, Hang Yes No Limited
Running threads and stack Yes No Yes
Performance impact on target Low/Medium Low/Medium Medium/High
application when debugging (configurable) (configurable)
Correlation across tiers No Yes No
spread across servers
Application types All Web All
Target audience Developers Ops (Primary), Developers
(High skill) Developers
Always On ? No Yes No
Cost Free $$ Low
30. References
System Center Application Monitoring
Tutorial
https://cmg.vlabcenter.com/default.aspx?moduleid=7a804b17-0025-4309-957d-
a21c2e121e2b
Intellitrace Tutorial
http://msdn.microsoft.com/en-
us/vs2010trainingcourse_debuggingwithintellitrace.aspx