[Bloat] Flow queuing performance (was: Re: [tsvwg] New Version Notification for draft-baker-tsvwg-aqm-recommendation-00.txt)

[Bloat] Flow queuing performance (was: Re: [tsvwg] New Version Notification for draft-baker-tsvwg-aqm-recommendation-00.txt)

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 2182 bytes --]

To begin with, I tend to use the term 'flow queuing' to distinguish what
we're doing with classic "fair queuing", which is overbound in most
people's minds, and some of what we do is "unfair" by some metrics.

Secondly, an extremely important factoid about why we got so excited about
fq_codel (which is really DRR, in term derived from SFQ, combined with
CoDel along with detecting thin vs. thick flows) is its performance:

If my memory serves me correctly, Eric Dumazet has bench marked fq_codel,
when running on a modern x86 processor with modern NIC's only takes a few
percent of one x86 core at a 10gE rate.  And we get higher total bandwidth
over the link as a result.  Eric, please correct me if I'm wrong, but you
did say I could quote you!

Similarly, Dave Taht reports when using fq_codel on CeroWrt (a 600mhz MIPS
commodity home router), while the cost of running the fq_codel algorithm is
visible in the performance traces, it's well down the list of what
functions in Linux are taking the CPU time on that architecture, with huge
latency benefits.

As a result, at the last Linux Plumber's Conference in August, there was
general consensus that we could replace PFIFO_FAST as the default qdisc in
Linux, awaiting just our confidence that fq_codel (or something very
similar) was mature enough for such a step.

These two results demonstrated that on current general purpose
architectures, an algorithm such as fq_codel is really do-able for the edge
of the Internet.

That is not the same statement as saying you can turn on existing
implementations of various other fair queuing algorithms in existing big
routers or not; just that on general purpose processors we know we can do
with these algorithms immediately.

I think we all need to reset our expectations from the 1990's where we took
away the impression that SFQ/DRR etc were too expensive for deployment:
today's general purpose systems are very different than those we had in
that era, now often dominated by cache misses more than anything else.
 Everything we think we know about performance needs to be revisited in the
light of modern architectures.
             Best regards,
                    - JIm

[-- Attachment #2: Type: text/html, Size: 2377 bytes --]

Re: [Bloat] Flow queuing performance (was: Re: [tsvwg] New Version Notification for draft-baker-tsvwg-aqm-recommendation-00.txt)

[Stephen Hemminger]

On Sun, 17 Mar 2013 15:12:16 -0400
Jim Gettys <jg@freedesktop.org> wrote:

> As a result, at the last Linux Plumber's Conference in August, there was
> general consensus that we could replace PFIFO_FAST as the default qdisc in
> Linux, awaiting just our confidence that fq_codel (or something very
> similar) was mature enough for such a step.

The only thing holding this back is that some user's still want to have priority queueing
of control packets. I (and others) have done some experiments on layering a traditional
three class priority layer on top of fq_codel, but nothing concrete has been submitted
for review.

Re: [Bloat] Flow queuing performance (was: Re: [tsvwg] New Version Notification for draft-baker-tsvwg-aqm-recommendation-00.txt)

[Eric Dumazet]

On Sun, 2013-03-17 at 12:24 -0700, Stephen Hemminger wrote:
> On Sun, 17 Mar 2013 15:12:16 -0400
> Jim Gettys <jg@freedesktop.org> wrote:
> 
> > As a result, at the last Linux Plumber's Conference in August, there was
> > general consensus that we could replace PFIFO_FAST as the default qdisc in
> > Linux, awaiting just our confidence that fq_codel (or something very
> > similar) was mature enough for such a step.
> 
> The only thing holding this back is that some user's still want to have priority queueing
> of control packets. I (and others) have done some experiments on layering a traditional
> three class priority layer on top of fq_codel, but nothing concrete has been submitted
> for review.


I did a patch on top of fq_codel to add 3 bands (same prio
classification than pfifo_fast).

I added a new attribute at that time to enable this 3 bands (and
disabling the external classifier, as it uses same skb->priority field.)

I probably better clone fq_codel, because if we want a pfifo_fast
replacement, it should run without the help of a "tc" command.

Re: [Bloat] Flow queuing performance (was: Re: [tsvwg] New Version Notification for draft-baker-tsvwg-aqm-recommendation-00.txt)

[Mikael Abrahamsson]

On Sun, 17 Mar 2013, Jim Gettys wrote:

> Secondly, an extremely important factoid about why we got so excited 
> about fq_codel (which is really DRR, in term derived from SFQ, combined 
> with CoDel along with detecting thin vs. thick flows) is its 
> performance:

I can imagine! One thought, have tests been done that indicate that 
generally an all-TCP-ACK "flow" (as seen in the egress direction) is still 
considered a "thin" flow? In the ADSL case with 20 meg down and 1 meg up, 
I seem to remember that ACKing 20 megabit/s of traffic uses 30-40% of the 
upstream bandwidth so it's a matter of opinion if this is actually a thick 
flow or not?

I'm trying to understand how codel handles the use case of 100 upload 
torrent TCP sessions (all trying to run max upload speed) and a single tcp 
http/ftp download. This is one of the most common complaints I've seen 
over the years about bufferbloat, uploading destroys download performance.

Another thing that I would like to see tested on Linux is the combination 
of fq_codel and shaping the speed. This is for the use case in ETTH 
selling for instance a 100/10 connection, where the 10 upstream connection 
is achieved by policing the bw in the ETTH access switch (the physical 
port is 100/100. It would give the user a better experience if the CPE 
could shape the traffic outgoing to 10 megabit to avoid the packet loss 
caused by the policing. This is something I would like to put in as an RFQ 
requirement, so it would be good if any AQM standard actually defined this 
mechanism and it was part of the description.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se