North American Network Operators Group
Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: "Does TCP Need an Overhaul?" (internetevolution, via slashdot)

  • From: Kevin Day
  • Date: Sat Apr 05 06:55:07 2008


On Apr 4, 2008, at 8:51 PM, Paul Vixie wrote:

What is really necessary is to detect just the flows that need to slow
down, and selectively discard just one packet at the right time, but
not more, per TCP cycle. Discarding too many will cause a flow to
stall -- we see this when Web access takes forever.
...

i wouldn't want to get in an argument with somebody who was smart and savvy
enough to invent packet switching during the year i entered kindergarden,
but, somebody told me once that keeping information on every flow was *not*
"inexpensive." should somebody tell dr. roberts?

I don't claim to understand this area more than Dr. Roberts either, but to paraphrase George Santayana:

"Those who do not understand SACK are condemned to re-implement it." (or fight it)

Years ago I worked on a project that was supposed to allow significant over-subscription of bandwidth to unmodified clients/servers by tracking the full state of all TCP traffic going through it. When the output interface started filling up, we'd do tricks like delaying the packets and selectively dropping packets "fairly" so that any one flow wasn't penalized too harshly unless it was monopolizing the bandwidth. To sum up many months of work that went nowhere:

As long as you didn't drop more packets than SACK could handle (generally 2 packets in-flight) dropping packets is pretty ineffective at causing TCP to slow down. As long as the packets are making it there quickly, and SACK retransmits happen fast enough to keep the window full... You aren't slowing TCP down much. Of course if you're intercepting TCP packets you could disable SACK, but that strikes me as worse off than doing nothing.

If you are dropping enough packets to stall SACK, you're dropping a lot of packets. With a somewhat standard 32K window and 1500 byte packets, to lose 3 non-contiguous packets inside a 32K window you're talking about 13% packet loss within one window. I would be very annoyed if I were on a connection that did this regularly. You get very little granularity when it comes to influencing a SACK flow - too little loss and SACK handles it without skipping a beat. Too much loss and you're severely affecting the connection.

You've also got fast retransmit, New Reno, BIC/CUBIC, as well as host parameter caching to limit the affect of packet loss on recovery time. I don't doubt that someone else could do a better job than I did in this field, but I'd be really curious to know how much of an effect a intermediary router can have on a TCP flow with SACK that doesn't cause more packet loss than anyone would put up with for interactive sessions.

The biggest thing we learned was that end user perceived speed is something completely different from flow speed. Prioritizing UDP/53 and TCP setup packets had a bigger impact than anything else we did, from a user's perspective. If either of those got delayed/dropped, pages would appear to stall while loading, and the delay between a click and visible results could greatly increase. This annoyed users far more than a slow download.

Mark UDP/53 and tcpflags(syn) packets as high priority. If you wanted to get really fancy and can track TCP state, prioritize the first 2k of client->server and server->client of HTTP to allow the request and reply headers to pass uninterrupted. Those made our client happier than anything else we did, at far far less cost.

-- Kevin