The performance of wireless ad hoc networks is impacted significantly by the way TCP reacts to lost packets. TCP was designed specifically for wired, reliable networks; thus, any packet loss is attributed to congestion in the network. This assumption does not hold in wireless networks as most packet loss is due to link failure. In our research we analyzed several implementations of TCP, including TCP Vegas, TCP Feedback, and SACK TCP, by measuring throughput, retransmissions, and duplicate acknowledgements through simulation with ns-2. We discovered that TCP throughput is related to the number of hops in the path, and thus depends on the performance of the underlying routing protocol, which was DSR in our research.
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
A Performance Comparison of TCP Protocols
1. A Performance Comparison of TCP Protocols
over Mobile Ad Hoc Wireless Networks
Michelle Berger, Servio Lima, Alexandros Manoussakis, Juan Pulgarin, and Benicio Sanchez
Information Networking Institute
Carnegie Mellon University
Pittsburgh, PA 15213
May 14, 2000
Abstract
The performance of wireless ad hoc networks is impacted significantly by the way TCP reacts to lost
packets. TCP was designed specifically for wired, reliable networks; thus, any packet loss is attributed to
congestion in the network. This assumption does not hold in wireless networks as most packet loss is due
to link failure. In our research we analyzed several implementations of TCP, including TCP Vegas, TCP
Feedback, and SACK TCP, by measuring throughput, retransmissions, and duplicate acknowledgements
through simulation with ns-2. We discovered that TCP throughput is related to the number of hops in the
path, and thus depends on the performance of the underlying routing protocol, which was DSR in our
research.
1 Introduction
The recent growth of mobile computers has led to the increased need for wireless ad hoc networks. A
wireless ad hoc network is a network in which a group of mobile computers communicate without the need
for a fixed infrastructure through the use of infrared or radio communications. TCP suffers poor
performance in wireless networks because of losses and corruption caused by errors inherent to a wireless
medium, as well as its inability to distinguish between link failure and congestion, which causes it to suffer
from low throughput when nodes move.
The focus of our research is to analyze several modifications made to TCP that could improve its
performance in mobile ad hoc networks. The TCP implementations we studied include TCP Vegas, TCP
Feedback, and SACK TCP. TCP Vegas attempts to improve on TCP Reno with increased granularity of
the timers, and thus a more efficient retransmission mechanism. TCP Feedback is able to distinguish
between packets lost from congestion and those lost from link failures by informing the sender when a link
fails. SACK TCP provides the sender with a better picture of the receiver’s buffers through the use of
selective acknowledgements, allowing for fewer retransmissions and duplicate acknowledgements.
We analyzed each of the above protocols in terms of retransmissions, duplicate acknowledgements, and
throughput. The data was collected through simulation, using DSR [Johnson1] as the routing protocol. As
TCP Reno is one of the most common implementations of TCP, we used it as the standard against which to
compare each of the other protocols. Similar research has been performed with TCP Reno and Explicit
Loss Feedback Notification using DSR in [Holland1], but we are not aware of any analysis of TCP Vegas
and SACK TCP using DSR.
2 TCP Protocols Analyzed
2.1 TCP Vegas
TCP Vegas [Brakmo1, Danzig1, Ahn1] is an implementation of TCP based on TCP Reno with
modifications in the sending side that try to achieve a more efficient use of available bandwidth. It was
developed at the University of Arizona, and the authors claim that it achieves 40% to 70% better
throughput with one-fifth to one-half the losses compared to TCP Reno. Vegas features a new
retransmission mechanism, a congestion avoidance mechanism, and a modified Slow Start.
1
2. Dave says this next paragraph implies that normal retransmissions have to wait for 3 duplicate acks, but
ignores the timer timing out.
The feature of Vegas that wireless ad hoc networks in particular could benefit from is the more efficient
retransmission mechanism. It reads the system clock for each segment and thus calculates a more accurate
RTT time. This mechanism allows it to measure a timeout with higher precision and retransmit without
having to wait for three duplicate ACKS or the less precise, and thus longer, retransmission timer to time
out. The authors of TCP Vegas found the average timeout interval of Reno to be more than three times
higher the correct value that would be calculated with an accurate clock. This inaccuracy in Reno's timers
introduces significant delays to retransmissions, lowering overall throughout. [Brakmo1, Ahn1]
Vegas also adds another improvement; when a non-duplicate ACK is received and it is the first or second
after a retransmission, Vegas checks if the time interval since the segment was sent is larger than the
timeout value. If it is, it retransmits the segment without having to wait for a duplicate ACK to arrive.
2.2 TCP Feedback (TCP-F)
The purpose of TCP-F [Chandran1] is to distinguish between packet loss due to congestion and packet loss
due to route failures. It does so using a feedback mechanism to signal the sender when a route failure
occurs and when a route is re-established. In order to accomplish this, TCP-F changes the way a Mobile
Host reacts to a packet loss.
Assume a TCP connection is established and the source Mobile Host (henceforth referred to as sender) is
sending packets to the destination Mobile Host. As soon as the network layer at any intermediate node
detects the disruption of a route it sends a Route Failure Notification (RFN) to the sender. Each
intermediate node that receives the RFN packet invalidates the route. If the intermediate node knows an
alternate route to the destination, this alternate route can be used and the RFN discarded.
Upon receipt of the RFN, the sender stops sending new packets or retransmissions, freezes its timers, and
saves the window size and the values of all the state variables regarding this connection. It then starts a
Route Failure Timer (RFT), which limits the time a sender waits for the re-establishment of a certain route.
The sender remains in this snooze state until a Route Re-establishment Notification (RRN) is received, or
until the RFT expires.
When an intermediate node learns an alternate route to the destination through a routing update, it sends a
RRN to the source. Any node that receives the RRN forwards it to the source. When the RRN is received
by the source, it first restores the saved state (timers, state variables, and window size). It then stops and
resets the RFT, and resumes transmission of new and retransmitted packets according to the restored
window and timeout values.
2.3 SACK TCP
The objective of SACK TCP [Mathis1] is to alleviate TCP's poor performance when multiple packets are
lost from one window of data. SACK TCP defines TCP's selective acknowledgement option, which allows
the receiver to inform the sender which packets have been received, thereby giving the sender a better
picture of the receiver's buffer. The sender can then retransmit only the missing data segments. Without
SACK, the sender is forced to either wait a roundtrip time to find out about each lost packet, or to
retransmit segments which have already been correctly received, generating needless retransmissions. It is
possible for the receiver to send up to four SACK blocks in each segment, although if the timestamp option
is used for RTTM, this is reduced to three SACK blocks.
The receiver must follow these rules if it chooses to send a SACK option:
1. The first SACK block must specify the block of data containing the segment which triggered this
ACK, unless that segment advanced the Acknowledgement Number field in the header.
2. Include as many distinct SACK blocks as possible in the SACK option.
3. The SACK option should be filled out by repeating the most recently reported SACK blocks. After the
first SACK block, the following SACK blocks may be listed in arbitrary order.
2
3. The redundancy of blocks in the SACK option increases the robustness in the presence of lost ACKs; if one
ACK is lost, the next ACK will contain overlapping information.
3 Methodology
The results obtained are based on simulations run using the ns-2 simulator with wireless extensions from
the Monarch project in CMU.
Each simulation consisted of an ad hoc network with 50 wireless nodes. Each of the nodes moved in a
1500m x 300m area, so as to create different spacings between the transmitting stations. The movement
was described accordingly through the random waypoint mobility model. The simulations were run for 90
seconds each. Seven different pause times were used to generate the scenarios, ranging from 0 to 90
second pause times, evenly spaced in fifteen second intervals. A total of thirteen scenarios were generated
for each of the pause times. Each of the TCP implementations was simulated with each of these scenarios,
giving a total of 91 runs per TCP implementation, and a grand total of 364 simulations.
For each of the protocols, a one-way TCP connection from one a node to another one was established for
the duration of the simulation. There was no restriction on the number of hops between them. Both nodes
were randomly selected and were used as the two connecting nodes through all of the simulations. DSR
was used as the routing protocol throughout the simulations.
No additional traffic was generated in the simulation in order to isolate the performance of TCP in a
wireless network from other factors that could affect it, such as packet collisions or congestion due to other
connections.
Three metrics were used to evaluate the four protocols, including:
• TCP throughput
• Number of duplicate acknowledgements
• Number of retransmissions
TCP throughput was defined as the average arrival rate of non-duplicate packets at the receiver. We
calculated this by taking the total number of non-duplicate data bits received by the receiver and dividing
by the total simulation time, which was 90 seconds in our case. None of our simulations had a network
partition between the sender and receiver that would affect our calculation of throughput.
4 Results
Tables 1 and 2 show the results of our simulations.
TCP Implementation Average Throughput (Kbps)
Standard Deviation of
Throughput (Kbps)
Feedback 336.9840 277.9139
Reno 361.2201 268.8416
Sack 366.7259 266.6461
Vegas 394.3716 268.6351
Table 1: Throughput results
TCP
Implementation
Average Number
of
Retransmissions
Standard Deviation
of Retransmissions
Average Number of
Duplicate
Acknowledgements
Standard Deviation
of Duplicate
Acknowledgements
Feedback 54.76 54.07 92.98 92.19
Reno 25.44 36.27 67.52 78.27
Sack 25.95 34.46 75.62 88.39
Vegas 12.55 12.95 26.67 25.71
Table 2: Retransmissions and Duplicate Acknowledgements results
3
4. The previous tables show Vegas as the TCP protocol with the greater throughput and least amount of
retransmissions duplicate acknowledgements.
The graph in Figure 2 shows the sequence number of the packets sent versus time for each version of TCP.
This graph shows only one run of the simulator, but is a good representation of our overall results. TCP
Vegas reaches the highest sequence number in this run, followed by TCP Feedback, SACK TCP, and Reno.
SACK TCP and Reno have almost exactly the same data points. SACK TCP and Reno both enter an
exponential backoff phase around 62 seconds into the simulation, and never recover. This was a common
occurrence in our simulations, and is explained in the next section.
Figure 1: Sequence Number versus Time for one run
4.1 TCP Over DSR
DSR is an on demand routing protocol that initiates route discoveries when there is a request from a higher
layer. The different versions of TCP sent packets at different times due to different timers, therefore,
each version of TCP had different routes. This led to discrepancies such as the above exponential
backoff phases. Each version of TCP was very dependent on the performance of DSR. If a route becomes
invalid, it might happen that TCP would enter exponential backoff and never recover since DSR only
attempted to find routes when TCP attempted to send packets. The fewer packets sent, the less DSR
attempted to find a new route. So once TCP entered exponential backoff, there was little chance for
recovery.
Figure 2: Sequence Number versus Time for one run (luck graph)
Figure 2 shows an anomaly that we noticed while running our simulations. Reno and Vegas seem to take
off, while SACK does not. This happened in several of our simulation runs, but in each one different
implementations of TCP had a very high throughput. It is caused by the fact that each of the TCP
simulations causes DSR to find different routes because they each send at different times. It just so
happens that in the above graph, Vegas and Reno both have a path at 44 and 58 seconds respectively that is
still current, while SACK has a stale route at around 45 seconds, enters exponential backoff, and never
recovers. This “luck” factor has a large impact on our results.?? It is very difficult to compare each
of the TCP implementations when each of them has a different set of routes for the same scenario.
4.2 TCP Vegas
We expected Vegas to have better throughput than Reno because of its more aggressive transmission
mechanism. Duplicate acknowledgements were expected to decrease, and retransmissions to increase
slightly.
Vegas is the clear winner in all areas, throughput, retransmissions, and duplicate acknowledgements.
With an on demand routing protocol, sending more often results in fewer stale routes. This explains
why Vegas has fewer retransmissions and duplicate acknowledgements. If the sender is using a stale route,
it must retransmit the data later on because the original data sent with the stale route will not reach the
receiver. Vegas has more precise (??) timers, which caused it to attempt to send packets more often
than the other protocols, which increased its chances of having DSR find a route. This is the reason
Vegas has such a high throughput compared to the other protocols. We refer the reader to [Holland1]
for a discussion of the proliferation of stale routes in DSR when using cached routes.
4.3 SACK TCP
SACK TCP was expected to perform very well. Duplicate acknowledgements and retransmissions were
expected to decrease as the sender would have a better picture of the receiver’s buffer, which would allow
throughput to increase.
4
5. SACK TCP performed fairly well in our simulations. Its throughput is a little better than that of Reno’s,
which was expected. SACK’s throughput should increase over that of Reno’s because it should not
retransmit the packets specified in the selective acknowledgement option, while Reno will. The use of
the selective acknowledgement option is the only difference between the two protocols
Need to explain why dup acks goes up so much here.
SACK TCP also introduces a small overhead in the acknowledgement packets with the 40 byte TCP
options header. This did not produce any noticeable differences in the throughput in our simulations, as the
SACK header was not used very often.
4.4 Feedback
As Feedback was designed for wireless networks, we expected it to perform better than Reno. Throughput
was expected to improve because upon reconnection, Feedback doesn’t use the Slow Start algorithm, but
instead sets its timers, state variables, and window sizes to the values saved before the route failure.
Duplicate acknowledgements and retransmissions were also expected to decrease.
Feedback had the lowest average throughputs of the protocols studied.
4.5 Effect of Number of Hops on Throughput
Intuition says that throughput should decrease as pause time increases, but we did not find this to be the
case. Our throughput was very constant across pause times, as shown in figure 3. The more important
factor was the minimum number of hops between the sender and receiver. The higher this number, the
lower TCP’s throughput was. The graph in figure 4 shows that there is an exponential relationship between
throughput and the average minimum number of hops between the sender and receiver. This average
minimum number of hops was determined with the use of the scenario files. This average is a weighted
average determined by the amount of time the minimum number of hops was valid. ?? needs
rewording
Figure 3: Throughput versus Pause Time
Figure 4: Throughput versus Average Minimum Path Length
According to [Gerla1], hidden terminal losses in CSMA, which become very substantial for higher
longer paths, cause the loss of TCP ACKs with consequent timeouts and major throughput
degradation.
4.6 Explanation of High Standard Deviation
We ran thirteen different scenarios with seven pause times in each to try to reduce our standard deviation.
Increasing the number of runs did not seem to have any affect. We believe this is due to the fact that we
did not limit the number of hops between the sender and receiver; this value was also highly varied in the
simulation runs. As we discussed above, throughput is very dependent on this value, and since we did not
limit the number of hops, it caused our throughput to vary greatly within the runs. There were times when
TCP would enter its exponential backoff phase and never recover, making the throughput very low.
5 Related Work
Recently, some researchers have considered the performance of TCP in multi-hop networks. [Gerla1,
Chandran1] In [Gerla1], Gerla et al. investigated the impact of the MAC protocol on performance of TCP
in multi-hop networks. Chandran et al. [Chandran1] proposed the TCP Feedback (TCP-F) protocol, which
uses explicit feedback in the form of route failure and re-establishment control packets. In [Holland1],
Holland and Vaidya investigated the effects of mobility on TCP performance in mobile ad hoc networks
5
6. using a 30 node network model with TCP Reno as a base protocol for comparison with the Explicit Link
Failure Notification (ELFN) technique.
6 Conclusion and Future Work
We have shown that TCP throughput is highly dependent on the path length in wireless ad hoc networks. It
is also related to how often the TPC sender attempts to send packets. The more often this occurs, the better
chance DSR will have of not having a stale route because the previous attempt to send found a route which
is not yet stale.
Our research did not limit the number of hops between the sender and receiver. If this was limited to three
hops or less, a more accurate comparison of TCP protocols could be obtained because there would be fewer
packet losses due to hidden terminal losses. According to [Gerla1], hidden terminal losses in CSMA,
which become very substantial for higher longer paths, cause the loss of TCP ACKs with consequent
timeouts and major throughput degradation.
We expected SACK TCP to perform better than it did. But our simulations did not include an error model.
SACK TCP should perform better in the case of dropped packets due to errors in the wireless transmission.
Future work should be done in this area.
Another area for future work is in the network congestion that is introduced by the protocols when their
timers are more precise as in the case of TCP Vegas. Even though Vegas had a higher throughput in our
simulations due to its timers, it is still important to determine if there is an increase in the congestion of the
ad hoc network.
6
7. References
[Ahn1] Jong Suk Ahn, Peter B.Danzig, Zhen Liu, and Limin Yan. Evaluation of TCP Vegas:
Emulation and Experiment. In Proceedings of SIGCOMM '95, 1995.
[Brakmo1] Lawrence S.Brakmo, William S.O’Malley, and L.L.Peterson. TCP vegas: New
techniques for congestion detection and avoidance. In Proceedings of ACM SIGCOMM
'94, pages 24-35, May, 1994.
[Chandran1] K.Chandran, S.Raghunathan, S.Venkatesan, and R.Prakash. A feedback based scheme for
improving TCP performance in ad-hoc wireless networks. In Proceedings of
International Conference on Distributed Computing Systems. 1998.
[Danzig1] Peter B.Danzig, Zehn Liu, and Limin Yan. An Evaluation of TCP Vegas by Live
Emulation. Proceedings of ACM SIGMetrics `95, 1995.
[Gerla1] M.Gerla, K.Tang, and R.Bagrodia. TCP performance in wireless multi-hop networks. In
Proceedings of IEEE WMCSA’99, February 1999.
[Holland1] Gavin Holland and Nitin Vaidya. Analysis of TCP Performance over Mobile Ad Hoc
Networks. Fifth Annual International Conference on Mobile Computing and Networking
(MOBICOM). August 1999
[Johnson1] David B. Johnson and David A. Maltz. Dynamic source routing in ad hoc wireless
networks. In Mobile Computing, edited by Tomasz Imielinski and Hank Korth, chapter
5, pages 153 – 181. Kluwer Academic Publishers, 1996.
[Mathis1] M.Mathis, J.Mahdavi, S.Floyd, and A.Romanow. TCP Selective Acknowledgment
Options. RFC 2018, October, 1996.
7