Discussion of diagnostic/troubleshooting options with the Azure diagnostic agent in Cloud Services.

1. Inside Azure Diagnostics

3. 17
COLUMBUS, OH OCTOBER 17, 2014 CLOUDDEVELOP.ORG

4. 1. The need for diagnostic data in
cloud applications
2. Data we can monitor
3. Using the Azure Diagnostic
Agent
4. Real-world guidance for
troubleshooting Azure apps

5. node.js
C#
Java
Agile
- vs -
Waterfall

6. Diagnostics Data / Telemetry

7. Scenario

8. You’re kidding? Right?

9. Scenario
o
o
o
o
o
o
We have a problem

10. Resolution
1. Enable Azure diagnostics
– Set key performance
counters
2. Add logging statements
around key functionality
– Especially external services
3. Test, test, test
4. Analyze
5. Fix it
Scenario
o
o
o
o
o
o

11. o
o
o
o
o
o
o
o

12. worker roles
web roles

13. worker roles
web roles
Diagnostic Data – 4x

14. 

17. Diagnostic Item Table Name Blob Container Name
Windows Event Logs WADWindowsEventLogsTable
Performance Counters WADPerformanceCountersTable
Trace Log Statements WADLogsTable
Azure Diagnostic Infrastructure
Logs
WADDiagnosticInfrastructureLogs
Custom Logs
(i.e. log4net, NLog, etc.)
<custom>
IIS Logs WADDirectoriesTable* wad-iis-logfiles
IIS Failed Request Logs WADDirectoriesTable* wad-iis-failedreqlogfiles
Crash Dumps WADDirectoriesTable*
* Location of the blob log file is specified in the Container field and name of the blob in the RelativePath field. The
AbsolutePath field contains the name of the file as it existed on the role instance.

18. 1. Role starts
2. Diagnostic monitor agent starts
3. Diagnostics configured
4. Data buffered locally
5. Data transferred to storage
o

20. o Trace logs
o IIS logs
o Infrastructure logs
o No transfer
o OnStart()
o Overrides default
o diagnostics.wadcfg
o Root of worker or
bin of web

21. public override bool OnStart()
{
// Create the DiagnosticMonitorConfiguration object to use for configuring the monitoring agent.
DiagnosticMonitorConfiguration config = DiagnosticMonitor.GetDefaultInitialConfiguration();
// Performance Counter configuration
config.PerformanceCounters.DataSources.Add(new PerformanceCounterConfiguration
{
CounterSpecifier = @"Processor(_Total)% Processor Time",
SampleRate = TimeSpan.FromSeconds(30)
});
config.PerformanceCounters.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
// Log configuration
config.Logs.ScheduledTransferLogLevelFilter = LogLevel.Information;
config.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
// Event Log configuration
config.WindowsEventLog.DataSources.Add("Application!*");
config.WindowsEventLog.DataSources.Add("System!*");
config.WindowsEventLog.ScheduledTransferLogLevelFilter = LogLevel.Warning;
config.WindowsEventLog.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
// Start the diagnostic monitor with the new configuration
DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", config);
return base.OnStart();
}
Impacts local agent only!

22. Deployment ID

23. Declarative Configuration
using Visual Studio

24. 1. wad-control-container
2. Imperative code
3. Declarative configuration
4. Default configuration

25. o
o
o
o
o
o

26. Instruct WAD to transfer specific data sources to
storage
Overwrites current diagnostic configuration
http://mcollier.net/DiagOnDemand

27. Additional host-level data – not DiagnosticAgent.exe

28. o
o
o
o
o

29. Query Azure Diagnostic
Data

30. o
o Vital information
o
o
o
o Day-to-day operational data
o
o
o
o

31. Process for Azure hosted solutions is not that different
from traditional, on-premises solutions.
Compute node
resource usage
Windows Event
logs
Database queries
response times
Application
specific exceptions
Database
connection & cmd
failures
Microsoft Azure
Storage Analytics

32. o
o
o
o
o
o

33. o
o
o
o
o
o
o
o
o

34. o
o
o
o

35. o http://bit.ly/1eomek9
o http://bit.ly/1mVHN3u
o http://bit.ly/1k1YkjI
o http://bit.ly/Q33mkU
o
http://bit.ly/1qp4omC

36. o
o
o
o
o
o
www.JustAzure.com

37. Questions?