Firewall Testing Methodology

Rethink Firewall Testing

A Methodology to measure the performance, security, and stability of firewalls under realistic conditions

www.breakingpoint.com
© 2005 - 2010. BreakingPoint Systems, Inc. All rights reserved. The BreakingPoint logo is a trademark of BreakingPoint Systems, Inc. 1
All other trademarks are the property of their respective owners.


Table of Contents
Introduction .................................................................................................................................................................................................................... 3

Baseline Application Traffic Test: Maximum Connections .............................................................................................................................. 6

Baseline Application Traffic Test: Throughput .................................................................................................................................................... 20

Baseline Attack Mitigation: SYN Flood .................................................................................................................................................................. 31

Baseline Attack Mitigation: Malicious Traffic ....................................................................................................................................................... 40

Application Traffic with SYN Flood ......................................................................................................................................................................... 52

Application Traffic with Malicious Traffic .............................................................................................................................................................. 65

Application Traffic with Malicious Traffic and SYN Flood................................................................................................................................ 78

Jumbo Frames ................................................................................................................................................................................................................ 89

IP, UDP, and TCP Fuzzing ............................................................................................................................................................................................. 99

Concurrency Test ........................................................................................................................................................................................................... 107

About BreakingPoint ................................................................................................................................................................................................... 120

Introduction
A firewall is a network device that continues to grow in importance every year. Obviously, organizations install firewalls in order to block
unauthorized access to the corporate network. At the same time as blocking unauthorized traffic, a firewall allows authorized traffic to



enter the network on certain configured ports, such as port 80 for the Web server or port 143 for IMAP. Depending on how the firewall is
configured, different ports will be open depending on the requirements and servers running within the network. These configurations
can lead to serious performance and security issues if not tested properly prior to deployment. Measuring the performance, security and
stability of a firewall using realistic traffic, load and security attacks is the only way to verify whether the firewall is preventing unwanted
traffic, while adhering to rules established to allow permissible traffic. This Resiliency Methodology describes how to perform the required
tests to ensure that a firewall performs as expected.

Traditionally, firewall testing was done using RFC 3511 - Benchmarking Methodology for Measuring Firewall Performance. More specifically,
section 5.1 “IP Throughput of RFC 3511”, focuses on determining the throughput and forwarding rate for unicast IP packets sent at a
constant rate and packet size. While stateless UDP traffic performance is valuable in determining the raw packet forwarding performance of
the engine, it simply is no longer applicable to real world deployments.

The BreakingPoint Firewall Resiliency Methodology is designed to evaluate firewalls and identify the performance characteristics of these
devices as they operate in a production environment. Since vendor-supplied datasheet specifications often reflect “best case” scenarios that
do not reflect real-world performance, this Resiliency Methodology is designed to accurately emulate the production environment in which
the firewall will be deployed. By fully understanding a firewall’s true performance, a network security manager can effectively decide which
vender or firewall to use in their network, the appropriate device placement, and when it is necessary to upgrade existing equipment.

The test environment should emulate the deployment environment as closely as possible. Devices connected directly to the device under
test (DUT) may affect packet loss, latency, and data integrity. If it is not feasible to recreate the deployment environment, it is recommended
that the BreakingPoint Storm CTM™ be directly connected to the firewall. All devices being evaluated must use the same test environment
to ensure comparable results.

Each firewall contains a different set of features. However, most firewalls allow rules to be created to allow or disallow traffic to flow to a
certain segment of the network. Also, the firewall will allow for the creation of two or more zones: LAN, and DMZ. The LAN is usually where
workstations will reside and the DMZ is where the servers will reside. This allows the ability to lock down the LAN segment of the network
and permit incoming connections to the DMZ network segment. As firewalls are used on a LAN segment of the network, DHCP and NAT are
supported. Some firewall vendors do provide support for VPNs and the ability for the device to use a virus checker (checking viruses is more
of a Unified Threat Management function). These are some of the more common features that firewalls support.

This Resiliency Methodology includes:

Baseline Application Traffic: Maximum Connections
Determine the number of connections per second that the firewall is able to handle. This will validate the performance of the firewall
when sending only good traffic with an “Allow All” policy. The TCP setup time will be analyzed to determine how a greater number of TCP
connections per second affects the time it takes to establish the TCP connection.

Baseline Application Traffic: Throughput
Determine the throughput that the firewall is able to handle to establish overall bandwidth supported. This validates the throughput
performance of the firewall when sending only good traffic with an “Allow All” policy.

Baseline Attack Mitigation: SYN Flood
Determine a baseline measurement for how the firewall performs when only handling a malicious SYN flood. Once a baseline has
been established, it will be compared with the results from the tests that blend together both application and malicious traffic. The number



of attempted sessions for the SYN Flood will be determined as well as the number of attempted sessions for the SYN Flood that were
blocked by the firewall.

Baseline Attack Mitigation: Malicious Traffic
Determine the ability of the firewall to remain stable while vulnerabilities, worms, and backdoors are transmitted through it. To
perform this test, the BreakingPoint Storm CTM™ will be configured to use an Attack Series that includes high-risk vulnerabilities, worms,
and backdoors. Some firewalls have Intrusion Prevention System (IPS) functionality and this will block some of the attacks. If the firewall has
IPS functionality, the number of attacks blocked by the firewall will be determined as well as the number of attacks that were able to get
through the firewall.

Application Traffic with SYN Flood
This test determines the ability of the firewall to handle both application traffic and a SYN Flood. The results will be compared to
both the Throughput Test and the SYN Flood Test. The ability of the firewall to detect and mitigate a SYN flood will be determined as well as
the ability of the firewall to forward application traffic while a SYN flood is taking place. The effect on the application traffic’s throughput,
latency, time-to-open, and time-to-close will be analyzed to determine the SYN flood’s effect.

Application Traffic with Malicious Traffic
This test determines the ability of the firewall to handle both application and malicious traffic. The results will be compared to both
the Throughput Test and the SYN Flood Test. The firewall’s ability to detect and mitigate a SYN flood will be determined. Also, the effect of
security traffic on the application traffic’s throughput, latency, time-to-open, and time-to-close will be analyzed. The firewall’s performance
will also be analyzed to determine the performance difference from the baseline test to the blended test performed. Finally, the firewall’s
ability to detect and mitigate the same number of attacks as it did in the SYN Flood Test will be tested.

Application Traffic with Malicious Traffic and SYN Flood
This test determines the ability of the firewall to handle application traffic, a SYN flood, and malicious traffic. The results will be
compared to both the Throughput Test and the SYN Flood Test. The firewall’s ability to detect and mitigate a SYN flood will be determined.
Also, the effect of the malicious traffic on the application traffic’s throughput, latency, time-to-open, and time-to-close will be analyzed.
Finally, the firewall’s ability to detect and mitigate the same number of attacks as in the previous Security tests will be tested.

Jumbo Frames
This test uses the Throughput test, except the Maximum Segment Size (MMS) parameter will be increased. The MTU size of the port
will be verified and increased if necessary. This test will determine if the firewall is able to perform better, worse, or the same when handling
jumbo frames. These results will be compared to the results from the Throughput Test.

IP, UDP, and TCP Fuzzing
The BreakingPoint Storm CTM™ will be configured to use the Stack Scrambler component. This test component has the ability to
send malformed IP, UDP, TCP and Ethernet packets (produced by a fuzzing technique) to the firewall. The fuzzing technique will modify a
part of the packet (checksum, protocol options, etc.) to generate the corrupt data. The firewall’s ability to handle malformed packets will
be determined. Take notice if the firewall crashes during the test, as this would indicate that the firewall is not able to handle the packets.
Also, analyze the effects the malformed packets had on the application traffic and determine if the firewall’s attack detection and mitigation
capabilities were affected.

Concurrency Simulation
This test will utilize the IP, UDP, and TCP Fuzzing Test, the Application Traffic with Malicious Traffic and SYN Flood Test. This test will
verify the effect all these different elements have on the firewall while running at the same time. The results will be analyzed to determine



the effect of the continuous operation on the application traffic’s throughput, latency, time-to-open, and time-to-close.

Baseline Application Traffic Test: Maximum Connections



RFC:
• RFC 793 – Transmission Control Protocol

Overview:
The specifications from the firewall data sheet will be used to determine if the firewall meets or exceeds the stated capacity. To determine
the capabilities, a Session Sender test component will be used to push the firewall beyond its stated limits. The Session Sender will be
configured to overload the firewall’s TCP connection rate to determine the maximum connection rate.

Objective:
To evaluate the firewall’s ability to create and maintain TCP sessions at a high rate.

Setup:



1. Open your favorite Web Browser and connect to the BreakingPoint Storm CTM™. Once the page has loaded, click Start
BreakingPoint Systems Control Center.

2. Login to the BreakingPoint Storm CTM™ by entering your Login ID and Password and clicking Login.



3. Once logged in, reserve the required ports to run the test.

4. Next, select Control Center Network Neighborhood.



5. Under the Network Neighborhoods heading, click the Create a new network neighborhood button in the lower right-
hand corner.

6. In the Give the new network neighborhood a name box, enter “Firewall Tests” as the name. Click OK.



7. Notice four Interface tabs are available for configuration. Only two are required for the tests. The first Interface tab
should be selected. Click the X button to delete this interface. When prompted about removing the interface, click Yes.
The remaining interfaces will be renamed. Repeat this process until only two interfaces remain.

8. With Interface 1 selected, configure the Network IP Address, Netmask, Gateway IP Address, Router IP Address, and finally,
the Minimum IP Address and the Maximum IP Address. Click Apply Changes.



9. Select the Interface 2 tab. Configure the Network IP Address, Netmask, and the Gateway IP Address. Using the Type drop-
down menu, select Host. Finally, configure Minimum IP Address and the Maximum IP Address. Click Apply Changes
once completed. Click Save Network.

10. Now that the Network Neighborhood has been created, the test can be configured. Select Test  New Test.

11. Under Test Quick Steps, click Select the DUT/Network.



12. In the Choose a device under test and network neighborhood window under the Device Under Test(s) section,
verify that BreakingPoint Default is selected. Under Network Neighborhood(s), verify that the newly created Network
Neighborhood is selected. Click Accept.

13. When prompted about switching Network Neighborhoods because the new one has fewer interfaces, click Yes.

14. Under the Test Quick Steps, select Add a Test Component.



15. Select Session Sender (L4) from the Select a component type window.

16. Under the Information tab, enter “Maximum Connections” as the name and click Apply Changes.

17. Select the Interfaces tab and verify that only Interface 1 Client and Interface 2 Server are enabled.



18. Select the Parameters tab. Several parameters will need to be changed in this section. The first parameter that needs
to be changed is the Distribution type. In the Segment Size Distribution section, use the Distribution type drop-down
menu and select Constant. Also, change the Minimum segment size to 512 and click Apply Changes.

19. Next, update the TCP Session Duration (segments) value to 4 and click Apply Changes.

20. Update the Data Rate value to 900 and click Apply Changes.



21. In the Session Ramp Distribution, several parameters will be changed. First, using the Ramp Up Behavior drop-down
menu, select Full Open + Data + Close. Next, change the Ramp Up Seconds to 20, Steady-State Seconds to 120, and
Ramp Down Seconds to 20. To update some of these parameters, scrolling will be required. Click Apply Changes when
complete.

22. Update the Maximum Simultaneous Sessions to 200% of the stated maximum. In this case, the firewall states a
maximum of 1,000,000 sessions, so a value of 2,000,000 is entered. For the Maximum Sessions Per Second to 160% of the
stated maximum sessions per second. A value of 40,000 is entered, as the firewall’s stated maximum sessions per second
is 25,000. Both these parameters are in the Session Configuration section. Click Apply Changes.

23. The configuration of the test is complete. Before continuing, the test component needs to be saved as a Preset due to
it being used in several other tests in this journal. Saving the test component as a preset allows for quicker and easier
configuration later on. To save as a preset, right-click on the test component and select Save Component as a Preset
from the menu.



24. Enter Maximum Connections as the name of the preset and click Save.

25. If desired, enter a description for the test under the Test Information section.

26. Verify that the Test Status has a green checkmark. If it does not have a green checkmark, click Test Status and make the
required changes.

27. Under Test Quick Steps, select Save and Run.



28. When prompted for a name to Save Test As, enter “Maximum Connections” and click Save.

While the test is running, it is possible to view real-time statistics. On the Summary tab it is possible to view the TCP Connection Rate, the
total number of TCP connections in the Cumulative TCP Connections section, and the overall bandwidth used.



29. To view more information about TCP connections, select the TCP tab. This view displays a basic TCP state diagram and a
line graph of the TCP Connections per Second.

When the test is completed, a window appears stating that the test criteria completed successfully.

30. Click View the report.



31. Expand the Test Results for Maximum Connections folder and the Detail folder. Select the TCP Concurrent Connections
result view. A graph and a table will be displayed. Using both items, determine the maximum sessions the DUT is able to
handle.

32. Select the TCP Connection Rate result view. A graph and a table will be displayed. Using both, determine the maximum
new sessions per second the DUT is able to handle. Then determine the maximum sessions per second during the
steady-state the DUT is able to handle. During the steady-state, sessions are actively being opened and closed.

The DUT used in this test was able to handle just under 630,000 Connections and about 30,000 Connections per second. These results are
required for the next test.

Other tests can also be performed. The following are some examples that can be run:

• Vary the TCP Segment size
• Change the Distribution type to random
• Change the TCP Session Duration (segments)
• Increase the test time for a longer test
• If HAR is going to be used, test how it affects traffic



Baseline Application Traffic Test: Throughput
RFC:
• RFC 768 – User Datagram Protocol
• RFC 791 – Internet Protocol

Overview:
A test setup very similar to the previous one will be used. A BreakingPointÔ Application Simulator test component will be used to generate
approximately 80% of the effective session capacity of the firewall as determined in the previous test, while trying to maximize throughput.

Objective:
To evaluate the firewall’s ability to forward a wide variety of application traffic and the overall rate that it is able to do so.

Setup:




4. Select Test  Open Recent Tests  Maximum Connections.

5. Click Save Test As.



6. Enter Maximum Throughput as the name and click Save.

7. Click Application Simulator to change the component type. When prompted about changing the components type,
select Yes. Next, change the name to Maximum Throughput and click Apply Changes.



8. Select the Presents tab and select Enterprise Apps. Click Apply Changes.

9. Select the Parameters tab. Several parameters will need to be changed. The first one that needs to be changed is the
Minimum data rate to 900. Click Apply Changes.

10. Next, parameters in the Session Ramp Distribution section need to be updated. Change the Ramp Up Seconds to 20,
Steady-State Seconds to 120, and Ramp Down Seconds to 20. Scrolling down will be required to change some of the
parameters. Click Apply Changes once all changes have been completed.



11. 11. In the Session Configuration section, two parameters will need to be changed. The first parameter that needs
to be changed is the Maximum Simultaneous Sessions. Take 10% of the total number of connections from the first test
and use this value. The next parameter that needs to be changed is the Maximum Sessions per Second. Take 10% of the
total number of connections per second from the first test. Click Apply Changes.

12. 12. If desired, change the test Description by clicking Edit Description under Test Information.

13. 13. Before running the test, the test component needs to be saved as a preset for use in later tests (saving as a
preset allows for quicker and easier configuration). Right-click on the test component and select Save Component As
Preset from the list.



14. When prompted for a name to Save Preset As, enter Maximum Throughput.

15. Verify that the Test Status has a green checkmark. If it does not have a green checkmark, click Test Status and make the
required changes.




The Summary tab will be initially displayed. This tab provides you with a great deal of information while the test is running. View the
different categories for different results that vary from overall Bandwidth to different TCP metrics.

17. 17. Select the Application tab. This tab provides details for each of the different Applications that are being
transmitted through the firewall. It is possible to use the drop-down menus to select different protocols.



18. Once the test has completed, a window will be displayed stating that the test completed successfully. Click Close.

19. Click View the report. Detailed results are displayed in a browser window.

20. Expand Test Results for Maximum Throughput and select App Bytes Transmitted. A byte count that each protocol
transmitted is displayed.



21. Expand the Details folder and select TCP Setup Time. The shorter the TCP Setup Time the better as the DUT is able to
quickly handle the requests and continue operating as expected.

22. Select TCP Response Time. When the TCP Response Time is short, the DUT is better able to quickly respond to requests
and continue operating.

23. Select TCP Close Time. When the TCP Response Time is short, the DUT is better able close out the current connection
quickly and to free up resources to open a new connection.



24. Detail folder. Select the Frame Data Rate and determine the maximum transmit and receive frame rate using the graph
and the table.

25. To determine how each protocol was handled by the firewall, five different results will be viewed. Under the Detail folder,
expand and analyze the results of the following: App Concurrent Flows: by protocol, App Throughput: by protocol, App
Transaction Rates: by protocol, and App Failures: by protocol.

Other test variations of this test can be run. The following are a couple examples:

• Step both Maximum Simultaneous Sessions and Maximum Sessions per Second by 10% till 80% has been reached
• Use different presets, such as the Service Provider App
• Increase the duration of the test time



Baseline Attack Mitigation: SYN Flood
RFC:
• RFC 4987 – TCP SYN Flooding Attacks and Common Mitigations

Overview:
A SYN Flood is when a client starts a TCP connection but never sends an ACK and keeps trying to initiate a TCP connection. This is harmful to
a firewall, as it has to provide resources to the TCP connection requests, but hopefully has the ability to detect and prevent the SYN Flood. A
Session Sender test component will be used to create a SYN Flood to attack the firewall.

Objective:
To evaluate the firewall’s ability to detect and mitigate a SYN flood.

Setup:




4. Select Test  Open Recent Tests  Maximum Connections.

5. Click Save Test As because this test is basically a repeat of the previous test with only minor changes.



6. Enter Syn Flood as the name and click Save.

7. The Information tab should already be selected. Change the name of the test component to Syn Flood and click Apply
Changes.

8. Select the Parameters tab. Several parameters will be changed in this section. Change TCP Sessions Duration (segments)
to 0. Click Apply Changes.



9. In the Data Rate section, change the Minimum data rate to 100 and click Apply Changes.

10. Next, in the Session Ramp Distribution section, use the Ramp Up Behavior drop-down menu and select SYN Only.
Change Ramp Up Seconds to 120, Steady-State Seconds to 0, and Ramp Down Seconds to 0. Scrolling down will be
required to update some of the parameters. Click Apply Changes.

11. Finally, in the Session Configuration section, verify that Maximum Simultaneous Sessions is set to 2,000,000. Change
Maximum Sessions Per Second to 45000. Click Apply Changes.



12. If desired, change the test Description under Test Information section.

13. Verify that the Test Status has a green checkmark. If it does not, click Test Status and make the needed changes.

14. Before running the test, the test component needs to be saved as a preset for use in later tests (saving as a preset allows
for quicker and easier configuration). Right-click on the test component and select Save Component As Preset from the
list.

15. When prompted for a name to save the preset as, enter SYN Flood and click Save.



16. Finally, under Test Quick Steps, select Save and Run.

Under the Summary tab, it is possible to determine how the firewall is handling the SYN Flood attack. Under TCP Connection Rate under
Client there should only be a value for Attempted. For Cumulative TCP Connections a value should only be present for Client Attempted.
The Bandwidth for RX should be very low, if not 0.

17. Select the TCP tab. No Successful connections should be present; this is another way of verifying that the firewall is
successfully handling the SYN Flood attack.



18. When the test finishes, a new window appears stating the test failed. This is expected as no connections were
successfully made. Click Close.

19. Click View the Report.

20. Expand Test Results for SYN Flood and select TCP Summary. Verify that Client attempted is 2,000,000. Both Client
established and Server established are 0. This means that the firewall was able to successfully handle the SYN Flood.



Other test variations can also be run. The following are a couple of variations:

• Increase the test length for a longer SYN Attack



Baseline Attack Mitigation: Malicious Traffic
RFC:

Overview:
It is important to evaluate how malicious traffic will affect the performance of a firewall even if it does not have built-in IPS functionality. A
Security test component will be used in this test. Five default attack series are available to use, but during this test, only Strike Level 3 will be
used. Strike Level 3 includes all high-risk vulnerabilities, worms, and backdoors.

Objective:
To evaluate the firewall’s ability to detect and mitigate vulnerabilities, worms, and backdoors.

Setup:



3. 3. Once logged in, reserve the required ports to run the test.

4. Select Test  New Test.




6. In the Choose a device under test and network neighborhood window in the Device Under Test(s) section, verify that
BreakingPoint Default is selected. Under Network Neighborhood(s), verify that the Network Neighborhood created during
the first test is selected. Click Accept.




8. Under Test Quick Steps, select Add a Test Component.

9. In the Select a component type dialog box, click Security.

10. The Information tab should be selected. Change the Name of the component to Security Strike and click Apply
Changes.




12. Next, select the Presets tab and select Security Level 3. Click Apply Changes.

13. Select the Parameters tab. If static attacks are desired set the Random Seed to any integer value other than 0. If changes
are made, click Apply Changes.



14. Under Test Quick Steps, select Define Test Criteria.

15. Select one of the Test Criteria and then click Disable all default criteria for this component.

16. Click the Add a new test criteria button.



17. Under Define Test Criteria, enter a Name, Description, Fail Description, and use the Statistic drop-down menu to select
Security Strike.Destination Gateway ARP Response. Click Create Criteria.

18. Repeat the previous two steps, except select Security Strike.Source Gateway ARP Response in the Statistic drop-
down menu.



19. Once both have been added, click Close.

20. If desired, enter a test Description under Test Information.

21. The configuration of the test is complete. Before continuing, the test component needs to be saved as a Preset due to
it being used in several other tests in this journal. Saving the test component as a preset allows for quicker and easier
configuration. To save as a preset, right-click on the test component and select Save Component as a Preset.

22. Enter Malicious Traffic as a name and click Save.



23. Verify the Test Status has a green checkmark. If it does not have a green checkmark, click Test Status and make the
required changes.


25. Enter Malicious Traffic as the name of the test and click Save.

26. Select the Attacks tab. No rules are present on the firewall, therefore most of the attacks should pass through the
firewall.



27. Since the default test criteria were changed to ignore malicious traffic transmitted through the DUT, the test passes as
expected. Click Close.

28. Click View the report. More detailed results are displayed in a Web browser.

29. Expand Test Results for Security Strike and select Strike Results. Verify the total number of attacks blocked by the
firewall and the total number allowed to pass through the firewall.



Other test variations can also be run including:

• Increase the test length for a longer a Malicious Traffic Attack
• Change the Security Threat Level



Application Traffic with SYN Flood
RFC:

Overview:
Since tests for application performance and a SYN Flood have already been configured and saved as presets, they will be used in this test.
Two test components will be used during this test, an Application Simulator and a Session Sender component.

Objective:
To combine application traffic with SYN flood traffic and compare the results against the results from the Throughput Test and the SYN
Flood Test.

Setup:




4. Select Test  New Test.




6. In the Choose a device under test and network neighborhood window, in the Device Under Test(s) section, verify
that BreakingPoint Default is selected. Under Network Neighborhood(s), verify that the Network Neighborhood created
during the first test is selected. Click Accept.

7. When prompted about switching Network Neighborhoods because the new one has fewer interfaces, select Yes.



8. Under Test Quick Steps, select Add a Test Component.

9. In the Select a component type window, click Application Simulator (L7).

10. The Information tab should automatically be selected. Enter Generic Traffic for the name of the test component. Click
Apply Changes.




12. 12. Next, select the Presets tab and select Maximum Throughput. Click Apply Changes.

13. 13. Under Test Quick Steps, select Add a Test Component.



14. In the Select a component type window, select the Session Sender (L4).

15. Select the Information tab and change the name to SYN Flood. Click Apply Changes.

16. Select the Presents tab and select SYN Flood from the list. Click Apply Changes.



17. If desired, edit the test Description under the Test Information section.

18. Next, verify that Test Status has a green checkmark next to it. If it does not, click Test Status and make the required
changes.

19. Finally, under Quick Test Steps, select Save and Run.

20. When prompted for a name to Save Test As, enter Application Traffic with SYN Flood. Click Save.



The Summary tab is visible and provides a great deal of information about the current running test and results. The Summary tab provides
information about the Application Flows to TCP connections and metrics to the overall bandwidth currently being used.

21. Detailed results about each protocol can be viewed under the Application tab. Use the drop-down menus to display
results from different protocols.



22. Once the test completes, a new window appears showing that the test failed. This is expected, as the firewall should
block a majority of the protocols being transmitted. Also, the SYN flood could be causing some of the legitimate
application traffic to be classified as bad. With having the traffic classified as bad could cause some of the failed
application transactions. Click Close to continue.

23. Select View the report. More detailed results are displayed in a Web browser.

24. To determine the ability of the firewall to handle a SYN flood while also processing legitimate traffic, expand Test Results
for SYN Flood and select TCP Summary. Verify that no clients were able to establish a connection and that no server
established a connection. Also, view the firewall’s state table and verify that the number of established connections on
the BreakingPoint Storm CTM™ matches that of the firewall’s state table. When you have finished viewing these results,
for easier navigation, minimize Test Results for SYN Flood.



25. Expand Test Results for Generic Traffic and select TCP Setup Time. The quicker the setup times the better, as the firewall
is able to react and respond to the incoming request. Determine the effect the SYN flood had on the TCP setup time of
the application traffic.

26. Select TCP Response Time. Just as with TCP Setup Time, the quicker the response times the better. Determine the effect
the SYN flood had on the TCP response time of the application traffic.



27. Next, select TCP Close Time. The quicker the firewall is able to close the TCP connection the quicker it frees up those
resources and can use them to start a new connection. Determine the effect the SYN flood had on the TCP close time of
the application traffic.

28. Select Frame Latency and determine how the SYN flood affected the latency of the application traffic.



29. Expand both the Detail folder and the App Throughput: by protocol folder. Select the first item, App Throughput:
protocol aol and determine if any traffic was able to pass through the firewall. View the entire list to determine how each
protocol was handled. The only protocols that should have been allowed are DNS, FTP, HTTP and SMTP.

30. Repeat the previous step with App Transaction Rates: by protocol, App Response Time: by protocol, and App Failures:
by protocol. Determine if transmitting blended traffic had an effect on any of the protocols.

31. Compare all of the collected results from the current test with the baseline tests to determine any differences.

32. If any test variations were run with either the Baseline Application Traffic Test: Throughput or the Baseline Attack
Mitigation: SYN Flood, be sure to run those variations on this test too.



Application Traffic with Malicious Traffic
RFC:

Overview:
Since tests for application performance and malicious traffic have already been configured and saved as presets, they will be used in this test.
Two test components will be used during this test, an Application Simulator and a Security component.

Objective:
To combine application traffic with malicious traffic and to compare the results with the results of the security test.

Setup:




4. Select Test New Test.

5. Under the Test Quick Steps, click Select the DUT/Network.



6. In the Choose a device under test and network neighborhood window in the Device Under Test(s) section, verify
that BreakingPoint Default is selected. Under Network Neighborhood(s), verify that the Network Neighborhood created
during the first test is selected. Click Accept.





9. In the Select a component type window, click Application Smulator (L7).

10. The Information tab should automatically be selected. Enter Generic Traffic for the name of the test component. Click
Apply Changes.




12. Next, choose the Presets tab and select Maximum Throughput. Click Apply Changes.

13. Again, under the Test Quick Steps, select Add a Test Component.

14. 14. From the Select a component type, select the Security component.



15. Under the Information tab, enter Malicious Traffic as the name and click Apply Changes.

16. Select the Presets tab and select the Malicious Traffic option. Click Apply Changes.

17. If desired, enter a test Description under the Test Information section.

18. Verify that Test Status has a green checkmark next to it. If it does not have a green checkmark, click Test Status and
make the required changes.




20. When prompted for a name, enter Application Traffic with Malicious Traffic. Click Save.





22. Select the Attacks tab. This tab provides real-time information of how the firewall is performing with the malicious traffic.
As can be seen in the image below, some attacks have been allowed.



23. When the test ends, a window appears saying the test failed. Click Close.

24. Select View the report. More detailed results are displayed in the browser.

25. Expand Test Results for Malicious Traffic and select Strike Results. Determine how well the DUT was able to handle the
different strikes and maintain blocking them while still transmitting regular traffic. Once completed, collapse Test results
for Malicious Traffic.



26. Expand Test Results for Generic Traffic and select TCP Setup Time. The quicker a firewall is able to react and setup the
TCP connection the better. Determine the effect the malicious traffic had on the TCP Setup Time.

27. Next, select TCP Response Time. Again, the quicker the firewall is able to respond to the incoming connection the
better, as the connection can be established quicker.

28. Select TCP Close Time. The ability of the firewall to quickly terminate a connection allows the firewall to quickly free
those resources for a new connection or another process.



29. Select Frame Latency and determine the effect malicious traffic had on the overall latency.

30. Next, expand both the Details folder and the App Throughput: by protocol folder. Select the first item, App
Throughput: protocol aol and determine if any traffic was able to pass through the firewall. View the entire list to
determine how each protocol was handled. The only protocols that should have been allowed are DNS, FTP, HTTP and
SMTP.



31. Repeat the previous step with App Transaction Rates: by protocol, App Response Time: by protocol, and App Failures: by
protocol. Determine if transmitting blended traffic had an effect on any of the protocols.

32. Finally, select Frame Data Rate and determine how the malicious traffic affects the data rate.


34. If any test variations were run with either the Baseline Application Traffic Test: Throughput or the Baseline Attack
Mitigation: Malicious Traffic, make sure to run those variations on this test too.



Application Traffic with Malicious Traffic and SYN Flood
RFC:

Overview:
Since tests for application performance, malicious traffic, and a SYN Flood have already been configured and saved as presets, they will be
used in this test. Three test components will be used during this test; an Application Simulator, a Security component, and a Session Sender
component. This test will determine the ability of the firewall to handle malicious traffic while also having to deal with a SYN Flood and
allowing good traffic to pass through.

Objective:
To concurrently send application traffic with SYN flood and malicious traffic to the firewall, and compare the results of this test against the
results of the baseline tests.

Setup:




4. Select Test  Open Recent Tests  Application Traffic with SYN Flood. Using this test as a starting point will accelerate
the configuration process because most of the test has already been configured.

5. In the lower left corner, click Save Test As.



6. A dialog box appears asking for a name to save the test as. Enter App Traffic SYN Flood Malicious Traffic and click Save.


8. From the Select a component type, select the Security component.



9. Under the Information tab, enter Malicious Traffic as the name and click Apply Changes.

10. Select the Presets tab and select the Malicious Traffic option. Click Apply Changes.

11. Notice the Test Status has an exclamation point next to it. This is due to having oversubscribed the ports. The Generic
Traffic component is configured to transmit 900 Mbps, SYN Flood is configured to transmit 100 Mbps and Malicious
Traffic is configured to transmit 5 Mbps for a total of 1005 Mbps. Select the Generic Traffic test component and then
select the Parameters tab. In the Data Rate section, change the Minimum data rate to 895 and click Apply Changes.



12. Make sure the Test Status now contains a green checkmark. If not, click Test Status and make the required changes to
continue.

13. Change the test Description if desired under the Test Information section.

14. Under Test Quick Steps, click Save and Run.



information about the Application Flows to TCP connections and metrics, to the overall bandwidth currently being used.




16. Select the Attacks tab. This provides a real-time look in on how the firewall is performing with the malicious traffic. As
can be seen from the image below, some of the attacks are being allowed to pass through the firewall.

17. When the test ends, a new window appears stating that the test criteria failed. Click Close to continue.

18. Click View the report. Detailed results are displayed in a browser window.



19. Expand Test Results for SYN Flood and select TCP Summary. Verify that no TCP connections were established. Collapse
Test Results for SYN Flood.

20. Expand Test Results for Malicious Traffic and select Strike Results. Determine how well the firewall was able to block and
not allow different strikes to pass through. Again, collapse Test Result for Malicious Traffic.

21. Expand Test Results for Generic Traffic and select TCP Setup Time. The quicker a firewall is able to react and setup the
TCP connection the better. Determine the effect the malicious traffic had on the TCP Setup Time. As can be quickly seen,
the TCP setup time has been affected and increased in duration.



22. Next, select TCP Response Time. Again, the quicker the firewall is able to respond to the incoming connection the
better because the connection can be established quicker. As can be quickly seen, the time for TCP response time has
increased.

23. Select TCP Close Time. The ability of the firewall to quickly terminate a connection allows the firewall to quickly free
those resources. The TCP close time has also increased compared to the baseline tests.

24. Select Frame Latency and determine the effect malicious traffic and the SYN flood had on the overall latency.



25. Next, expand both the Details folder and the App Throughput: by protocol folder. Select the first item, App
Throughput: protocol aol and determine if any traffic was able to pass through the firewall. View the entire list to
determine how each protocol was handled. The only protocols that should have been allowed are DNS, FTP, HTTP and
SMTP.

26. Repeat the previous step with App Transaction Rates: by protocol, App Response Time: by protocol, and App
Failures: by protocol. Determine if transmitting blended traffic had an effect on any of the protocols.

27. Finally, select Frame Data Rate and determine how the malicious traffic and SYN Flood affects the data rate.


29. If any test variations were run with either the Baseline Application Traffic Test: Throughput, the Baseline Attack
Mitigation: Malicious Traffic or Baseline Attack Mitigation: SYN Flood, make sure to run those variations on this test
too.



Jumbo Frames
RFC:
• RFC 894– A Standard for the Transmission of IP Datagrams over Ethernet

Overview:
The Throughput test will be used as a starting point in this test. Once the test is opened, the Maximum Segment size will be changed to
4,000 to send jumbo frames.

Objective:
To analyze how the firewall handles jumbo frames.

Setup:




4. Select Test  Open Recent Tests  Maximum Throughput. Using this test as a starting point accelerates the
configuration process because most of the test has already been configured.

5. In the lower left corner, click Save Test As.



6. A dialog box appears asking for a name to save the test as. Enter Jumbo Frames and click Save.

7. Select the Parameters tab and under the TCP Configuration section change the Maximum Segment Size (MSS) to a value
greater than 1500 but less than 9142. In this example a 4000-byte packet was used. Once the changes have been
completed click Apply Changes.

8. Next, select Control Center  Device Status.



9. When prompted about saving the test due to changes, click Yes.

10. Right-click on a reserved port and select Configure Port.

11. Verify that the MTU is large enough and click Close. If needed increase the MTU size and click Apply. Repeat this process
for the other reserved port too.



12. To return to the test configuration select Test  Open Recent  Jumbo Frames.

13. Under the Test Information section, edit the test Description.

14. Verify that the Test Status has a green checkmark. If it does not contain a green checkmark click Test Status and make
the required changes.



15. Under Test Quick Steps, click Save and Run.




16. When the test ends, a new window appears stating either the test passed or failed. Click Close to continue.

17. Click View the report. A Webpage containing more detailed results is displayed.

18. Expand Test Results for Maximum Throughput and select App Bytes Transmitted. A byte count that each protocol
transmitted is displayed.



19. Expand the Details folder and select TCP Setup Time. The shorter the TCP Setup Time the better as the DUT is able to
quickly handle the requests and continue operating as expected.

20. Select TCP Response Time. Again, the shorter the TCP Response Time the better as the DUT is able to quickly respond to
requests and continue operating.



21. Expand the Detail folder. Select the Frame Data Rate and determine the maximum transmit and receive rate using the
graph and the table.

22. To determine how each protocol was handled by the firewall five different results will be shown. Under the Detail folder,
expand and analyze the results of the following: App Concurrent Flows: by protocol, App Throughput: by protocol, App
Transaction Rates: by protocol, and App Failures: by protocol.

23. Using the results from the current test and the results from the Maximum Throughput test determine if the firewall
performed better, worse, or the same when handling jumbo frames.

Other test variations can also be run. The following are some test variation examples:

• Test several different sizes of Jumbo Frames, specifically making sure to test the 9,000-byte frame.
• Increase the test duration



IP, UDP, and TCP Fuzzing
RFC:

Overview:
The Maximum Throughput test will be used as a starting point and a Stack Scrambler component will be used too. The Stack Scrambler
tests the integrity of different protocols by sending malformed IP, UDP, TCP, and Ethernet packets to the firewall. The fuzzing technique will
modify only a single part of the packet to generate corrupt data.

Objective:
To send fuzzed traffic through the firewall and determine how it affects the firewall and the other protocols.

Setup:


Firewall Testing Methodology

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (10)

Similar to Firewall Testing Methodology

Similar to Firewall Testing Methodology (20)

More from Ixia

More from Ixia (20)

Recently uploaded

Recently uploaded (20)

Firewall Testing Methodology