[Codel] sprout

[Dave Taht]

On Wed, Jul 10, 2013 at 1:46 PM, Jim Gettys <jg at freedesktop.org> wrote:
>
>
>
> On Wed, Jul 10, 2013 at 4:41 PM, Keith Winstein <keithw at mit.edu> wrote:
>>
>> On Wed, Jul 10, 2013 at 4:38 PM, Jim Gettys <jg at freedesktop.org> wrote:
>>>
>>> Did you compare with fq_codel?  None of us (Van included) advocate CoDel
>>> by itself.
>>
>>
>> Hi Jim,
>>
>> In our evaluation there's only one flow over the queue (in each direction)
>> -- the videoconference. So, yes.
>>
>> In the upcoming paper where we try and come up with the best end-to-end
>> schemes for multiuser traffic, we compare against Cubic-over-sfqCoDel
>> (http://www.pollere.net/Txtdocs/sfqcodel.cc).

One of my problems with keith's paper is that it is playing against
traces that assume the background load will itself not mutate based on
available bandwidth.

Love the tracing idea tho...

>
> Great.
>
> I sure wish we could get together on *fq_codel variants.  Kathy thinks that
> there is no difference between sfq_codel and fq_codel, and Dave thinks there
> is a difference.

I documented the differences between codels and fq_codels at some
point on some web page somewhere.

The original ns2 codel code as published in the magazine contained an
error and should not be used at all.

linux codel uses a more drastic fall-off curve than ns2 codel.

The ns2_codel patch I have for linux uses much "tighter" constraints
on the interval and in the range of RTTs I was testing at the time
(sub 20ms) did a lot better than linux codel. It is based on the
"experimental" codel code on the pollere web site, but excludes the
floating point hack at highest rates.

The nfq_codel patch, based on that, tries to use a compromise between
SFQ and DRR, providing single packet SFQ-like service within a smaller
DRR-like quantum (usually 300 these days) across the flows.

Results of nfq_codel vs fq_codel have been rather inconclusive, and I
decided I needed tests that covered the entire range of useful stuff,
like web, voip, and video conferencing, which is a lot of work for
which we have no funding. There are some hints that nfq_codel does
better against voip, in particular, but competes more slowly against
reverse traffic than the DRR approach in fq_codel.

the ns2 sfqcodel uses the std ns2 model of codel, and something very
close to nfq_codel. It has been unclear from reading the code if it
actually does the "new flow" idea right relative to the
smaller-than-mtu quantum.

BUT in summary:

In all cases the differences between *fq_codels are pretty subtle, but
rather dramatic compared to std drop tail. I'm satisfied at this point
that drop tail should die, and that some form of fq_codel be made the
default in linux and then we can get around to arguing other points
with data at scale. I am unsure if enough of the core linux network
devs agree - and there are issues with handling multi-queue devices in
particular remaining, and I am unfond of TSO/GSO/GRO as it stands (but
eric's got some fixes planned). It would be great to see a patch
obsoleting pfifo_fast hit mainline linux...

In particular web traffic "cuts through" other sustained flows like
butter, and is almost entirely based on the RTT (as are things like
dns).

My primary nfq_codel deployment with a couple dozen users is pretty
darn successful compared to what was running before. It is really
astonishing how well it works even against multiple DASH flows and
torrent.

and in any case, what I'm mostly concerned about these days are dozens
of flows in slow start on a given web page vs 1-3 uploads/downloads
(and lately, videoconferencing), AND the behavior on wifi access
points, which as keith points out, really needs per user queues in
order to even begin to think about performing as well as it did a
decade ago, prior to N, and has a dozen other "easy" fixes that will
take many man-months to develop and test even on a single
architecture.

I am reluctant to re-join the simulation campaign using ns2, as the
deployed TCPs differ rather radically from the models. I prefer
measuring reality, which unfortunately is sloppy, buggy, and hard to
make repeatable. It does appear that Eric's new netem patches are
going to make it possible to run more accurate longer RTT (100ms) live
network simulations at higher rates than I ever could before. (THX
ERIC)

The results I get in the lab with the current (prepatched) version of
netem are pretty dismal...

http://results.lab.taht.net/delay/rtt_fair_50_netem_10000_fq_codel-100mbit-2.svg

and unusable at gigabit

http://results.lab.taht.net/delay/rtt_fair_50_netem_10000_fq_codel.svg

/me goes back to building patched kernels

>
> Sigh....
>                               - Jim
>
>>
>>
>> Cheers,
>> Keith
>
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html