2015 02 24 lmtv baselining

© 2015, The Technology
Firm
www.thetechfirm.com
Baselining
Getting Started
From The: Throughput and Latency Class
Tony Fortunato,
Sr Network Performance Specialist

© 2015, The Technology Firm WWW.THETECHFIRM.COM
Why Baseline??
 I’m surprised how many times I see people instructed to ‘create a baseline’ with no
real purpose or goal in mind.
 This makes your job difficult and may turn into one of those never ending requests.
 Here’s some examples of why you may want to create a baseline:
• Educational exercise;
• to learn more about your application or protocols
• Understand how your application behaves for future reference or tuning
• Measure application performance to properly predict network requirements
• Document impact of network or application changes
• To create an effective troubleshooting methodology
• Determine application behavior to properly configure network equipment
(security)
 Do not proceed without a without a clear goal or target!
 Same applies to general tool use!!
2

© 2015, The Technology Firm WWW.THETECHFIRM.COM 3
Bandwidth Exercise
Here’s the scenario:
 Copy a file to and from your server and record how long it takes.
• 12 MB file on a 10 Mb Ethernet
• 120 MB file on a 100 Mb Ethernet
• 1,200 MB file on a 1 Gb Ethernet
• 67.5 MB on 54 Mb WiFi
• Theoretically, all these scenarios should take approximately 10 seconds.
How did you do? What if it took you 11 vs 55 seconds?
• Basic formula is bits/8 x 10 seconds which is strictly theoritical
 How would you measure the network load you created during this test?
o Using SNMP from your switch port?
o Span or mirror your port to a Protocol Analyzer or other types of testers or probes?
o Use Microsoft’s Perfmon or other OS utilities (if available)?
o Does it matter if the server is local, remote?
o What about protocols?
o LAN vs WiFi
o Disk vs RAM

Common Baselines
• Baselining is often a forgotten art and most people don’t baseline for the following
reasons:
o When should I start?
o What should a baseline look like?
o What if someone actually reads it and wants more information?
o Why bother? Everything changes so fast around here.
o I don’t know how to baseline.
• Baselines should be:
o Clearly defined.
o For example PC bootup baseline, Login baseline, Application login Baseline, throughput
or Network Upgrade Baseline.
• As long as your goal is clear and the methodology is documented your baseline
is correct.
• If you have performed a baseline correctly, you will typically find problems to fix along
the way or settings to tune.

Baselining Options
The first step to a successful baseline is to determine what you want to baseline.
Examples:
Network
• Record the load, protocol distribution and other network characteristics
Network Equipment
• Log equipment specific details such as temperature, port errors
Workstations
• Document protocols, application dependencies
Measurement point and methodology must be documented so you can properly
duplicate or retest

Synthetic Transaction Performance
• Several products on the market use “synthetic” transactions.
• Synthetic transactions are modeled after popular applications. i.e. ftp, Telnet, HTTP,
etc… These synthetic transactions are transmitted across a network to estimate
transaction response time.
• Results only as accurate as the synthetic transaction and test PC’s.
• In many cases multiple workstations can be simulated.
• The accuracy of the synthetic transaction versus performing the same transaction [I.e.
ftp] comes into question since the test nodes may not properly represent the
production nodes specifications or load.
• Ensure that these tests are not executed solely from RAM to include realistic disk
latency

Protocol Analyzer - Performance Measurement
• Data is gathered using existing traffic.
• Results are what is really happening on the wire.
• Does not require the introduction of additional traffic onto the wire.
• Takes into account production server/client loads, network configuration and any
undocumented variables.
• Analysis points must be selected carefully to provide accurate results.
• This approach is the most labor intensive and requires a high level of protocol
analysis understanding.
• Try to produce reports from your trace files to better explain and understand your
findings.

Network Configuration Example
Client Switch
Switch
Switch
Switch
Server
Where would you go for your capture packets?
B
D
CA X

Review
• There is no single correct means of measuring network performance.
• Not only must the proper tool be selected for the job, but the proper methodology
must be used along with the tool.
• In some cases you may want to try to document “in what direction is the main
source of latency?”
• Document your methodology as well as your findings to promote consistency.
• Always re-measure after any changes.
• In many cases, you should take more than one measurement.
• i.e. I try to take 5, drop the high, low and average the three.
• Automating or documenting tasks will ensure each test is performed identically.

Where Do You Start?
• When you baseline or troubleshoot where do you start?
• It is quite common for a help desk person to ping a server with the application that a
client may be complaining about.
• Is a ping the same as running the application?
• Do you dismiss the issue as a local issue since no one else has complained … YET!!!
• Do you start troubleshooting near the server, on the Gig backbone, or near the client?
• I typically start investigating problems and baselines from the clients’ perspective.

Application Analysis Steps
• Baseline the application.
• Review the trace files.
• If required, predict the application performance.
• Measure and compare the application performance in production.
• Troubleshoot the application if necessary.

Baseline the Application
• The purpose of this step is to collect all of the traffic that crosses the network from the
start of the application to the end of the application. This traffic will be divided up into
individual tasks for analysis.
• It is imperative that every step and the EXACT configuration and location of all
equipment is documented properly.
• This step directly impacts the remaining process, so be thorough.
• Sample of items to document:
o Segment Name.
o Switch Identifier, Slot and Card.
o Note if mirror ports or hubs are used.
o Time/Date.
o Source Station IP, MAC Address, Protocols loaded, Operating System Version, Application
Version.
o Script of what the operator will do in detail. Screen shots are optimal.

Data Collection And Intercept options
Capturing the data can be done many different ways, but again, the methodology and tool
must be documented.
You can use the following methods to create a baseline:
Netflow, sflow, etc..
SNMP
Server or pc specific stats
Packet capture application installed
• On server
• On client
• On device on network path
Packet intercept options
• Span, or similar commands
• Hub – 10/100 HD connections
• TAP

• Set up Analyzer to capture all traffic to and from the test workstation.
• The Hub configuration is only applicable for Half-Duplex configurations and when you
want to observe application behavior or protocols
• Full Duplex configurations require a full-duplex tap or mirror/span port.
14
HUB
SwitchTest Workstation
Protocol Analyzer
Capturing the Application Traffic – HUB Example
SERVER

• Setup a filter to capture all traffic to and from the MAC address of the workstation.
• Why the MAC?
Filter On Specific Relevant Data

• Create a worksheet that outlines a script of tasks to be performed.
• This will ensure consistency.
• Provide columns for starting and ending frame numbers.
16
Create a Script
Description Start
Frame
End
Frame
Launch Application via Icon [app.exe].
Enter Login Name [Joe Smith]
Enter Login Password [don’t document]
Select Account Query from Main Screen
Query Account 1234
Wireshark

Record Frame Numbers
• Before each task is performed, note the frame number on the analyzer.
• After the task is complete note the frame number.
• Creating a packet bookmark – like a ping between tasks is always helpful
• Repeat this process for all items on your script.
• Make sure the person running the application waits for your signal before they
move on to the next task.
• After the testing is complete, go through the capture file with the worksheet.
• Try to resist the impulse to start analyzing data since it will distract you.
Description Start Frame End Frame
Launch Application via Icon [app.exe]. 1 111
Enter Login Name [Joe Smith] 112 121
Enter Login Password [don’t document] 122 144
Select Account Query from Main Screen 145 222
Query Account 1234 223 332

Reporting Your Results
• Now that you have gathered and calculated the traffic per task, prepare a chart with
your findings.
• The only other variable to include is how many clients will be using this application.
• This is important because you may be asked to predict load based on a number of
clients that will be accessing this application.
Description Start Frame End Frame Bytes
Launch Application via Icon [app.exe]. 1 111 120,000
Enter Login Name [Joe Smith] 112 121 10,000
Enter Login Password [don’t document] 122 144 10,120
Select Account Query from Main Screen 145 222 60,000
Query Account 1234 223 332 132,022

Trace File Review
• Analyze each task associated with the application to determine the processing
time consumed by the client and each of its dependencies.
• Note if any additional servers, protocols or nodes are required to complete any tasks.
• You may want to graphically display these results.
Application X
0
10000
20000
30000
40000
50000
60000
70000
3:32:17
3:33:26
3:33:37
3:33:48
3:33:59
3:34:10
3:34:21
3:34:32
3:34:43
3:35:47
3:36:00
3:36:11
3:36:25
3:36:37
3:36:48
3:37:01
Time
Bits/Sec
Login Query
Excel Chart

Reporting Your Results
• Sample chart comparing various protocols and applications copying the same file.
File Copy Comparison
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
9,000,000
10,000,000
1 2 3 4 5 6 7 8 9 10 11 12
Seconds
bits
smb ip copy smb ipx copy smb nb copy http copy ftp copy

Bootup Configuration Baseline
Goal:
• To record the behavior of a PC 'on the wire' when it is powered on. The client does
not to be present since no login is required.
• Any dependencies, protocols and load will be documented.
Methodology:
• Place your protocol analyzer either on the same hub as the workstation or configure
some form of port monitoring.
• Be careful if you set up a MAC/DLC filter based on the target workstation's hardware
address. You may miss some packets, like some DHCP replies.. If DHCP is used
and you wish to use a MAC filter, include the bootp/DHCP protocol.
• After powering the workstation on ensure that all background services are loaded by
simply observing the hard drive light. When the hard drive light 'settles down', and the
frame counters stop, you're done.

Simple Baseline Example
 Simply go to a folder that is located on your network drive and search for something
common. This was a baseline that I did for a client who complained about
performance when they searched for documents.
 In my example, I started a capture and navigated to my articles archive and searched
for the word Cisco.
 The goal of this exercise was to record which protocols are used and how much traffic
is generated.
 This type of test allows us to understand how remote users may experience this same
task which they perform regularly.
22

Test Configuration
23
Search
Articles on O:
drive folder
for the word
Cisco
Capture filter with
server MAC address
10.44.10.94
10.44.10.171

Sample Results
24

Protocol Breakdown
25
Note TCP port 445 was used for this test

Document Reported Issues
26
Nothing wrong with an opinion as long as you remember that’s all it is

Packet Slicing
 Since we are not examining the payload, we can packet slice, or only save the first
128 Bytes
 Slicing the trace file makes the file size much smaller which allows Wireshark, or
other tools to process it much quicker.
 This will provide all the addressing and enough of the protocol for this exercise
 I don’t know of a way to slice a trace file from within Wireshark after I’ve captured it
and simply use the editcap utility that Wireshark provides
 "Program FilesWiresharkeditcap.exe" -s 128 "search for cisco for mapped
drive.pcapng" "search for cisco for mapped drive sliced.pcapng"
 In this example, the trace file went from approximately 600 MB to 98 MB
27

Conclusions, Questions, or Recommendations
 Can a different protocol be used instead of SMB
 Do the results change if you chose a subfolder and perform a similar search
 What effect does IPV6 have
 Since this department does this operation often, can they create an application to
make this more efficient
 Would remote clients require internet access with significant symmetrical bandwidth
28

© 2015, The Technology
Firm
www.thetechfirm.com
Baselining
Getting Started
Thanks for watching
Tony Fortunato,
Sr Network Performance Specialist

2015 02 24 lmtv baselining

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (17)

Similar to 2015 02 24 lmtv baselining

Similar to 2015 02 24 lmtv baselining (20)

Recently uploaded

Recently uploaded (20)

2015 02 24 lmtv baselining

Editor's Notes