[Bloat] First draft of complete "Bufferbloat And You" enclosed.

[Justin McCann]

On Tue, Feb 8, 2011 at 1:18 PM, Eric Raymond <esr at thyrsus.com> wrote:
>...
> The underlying image in my just-so stories about roads and parking lots
> is that packet flow coming in smooth on the upstream side of a buffwer
> gets turned into a buffer fill, followed by a burst of packets as it
> overflows, followed by more data coming into the buffer, followed by
> overflow...repeat.

I think this image isn't quite right for wired networks, but happens a
lot in wireless networks.

>...
> I don't understand "ack clocking". Alas, my grasp of networking becomes
> sketchy below the level of socket APIs. I know what's in a TCP packet,
> roughly, but have no precise feel for what happens with bits on the wire.

The first figure in the paper Bill Sommerfeld linked to is the best
explanation; here's another decent picture of it:

   http://sd.wareonearth.com/woe/Briefings/tcptune/sld038.htm

I can explain in more detail, but I figure it will be less precise
than what's in the paper. in the analogy, the parking lot has a
metered-rate exit--- cars coming in from ten different entrances at
different rates can *never* exit faster than the fixed outgoing rate.
They can exit slower than the metered rate, but not faster, so any
"clumping" has to be caused by something other than the size of the
parking lot (at least at this link).

Considering Richard's email, I think the VJ paper assumes that the
scheduling in the OS and hardware are not bursty. That is, that the OS
doesn't send clumps of packets to the interface hardware to transmit,
so the hardware buffer doesn't oscillate between full and empty. If
the OS *does* send in bursts due to interrupt latency, scheduling, bus
contention, or a weird application, then you have a different nasty
problem. That sort of clumpy/flighty/bursty behavior is problematic in
general, but I think bufferbloat is only an indirect cause (glad to be
corrected here).

For example, if you're trying to transmit on a wireless link, you may
have to wait a while before you can use it (noise, contention,
scheduling). If you have a buffer full of packets, you'll jam them out
at a much higher rate than you were doing just moments ago. That
really screws with ACK clocking/pacing, and leads to wild swings in
what TCP sends out--- slides 48 and 49 from that link
(http://sd.wareonearth.com/woe/Briefings/tcptune/sld048.htm) are
examples of what we don't want.

   Justin

[1] This skips all sorts of details about varying packet sizes and so
on, but you get the idea.