Re: [Codel] hardware multiqueue in fq_codel?

[Eric Dumazet <eric.dumazet@gmail.com>]

On Thu, 2013-07-11 at 11:06 -0700, Dave Taht wrote:

> Gotcha. So what I actually did (felix did, in openwrt, actually) was
> just make fq_codel the default qdisc to avoid having to inspect things
> to set the number of queues in mq and mqprio. I see, for example, that
> mq is the default for tg3...
> 
> http://snapon.lab.bufferbloat.net/~cero2/deb/patches/0003-Use-FQ_codel-by-default.patch
> 
> I just added it to htb and hfsc too:
> 
> http://snapon.lab.bufferbloat.net/~cero2/deb/patches/0008-Make-fq_codel-the-default-qdisc-for-htb-and-hfsc.patch
> 
> There's a patch to obsolete pfifo_fast entirely in openwrt, which is a
> tad premature.
> 
> A remaining concern is to what this affects:
> 
> A) people that expect ifconfig X txqueuelen Y to do anything will be
> misled. Perhaps this could be fixed by having the fq_codel default
> limit be txqueuelen rather than the default (and overlarge) limit of
> 10k, but as tons of people are supplying oddball txqueuelens, I tend
> to think just ignoring txqueuelen going forward is more the right
> thing.
> 
> Do you actually get close to 10k packets outstanding in 10GigE under
> any sane circumstances?


10GigE can send 10.000.000 packets per second.

10k is only 1ms of buffering, which is pretty low considering the cpu
able to restart a queue might be blocked ~10 ms in a softirq handler.

Whole point of codel is that number of packets in the queue is
irrelevant. Only sojourn time is.

Now if your host has memory concerns, that's a separate issue, and you
can adjust the qdisc limit, or add a callback from mm to be able to
shrink queue in case of memory pressure, if you deal with non elastic
flows.

> 
> B) people that expect pfifo_fast semantics, for which substituting
> fq_codel behaves oddly in two ways -
> 
> 1) if you are explicitly setting skb->priority for the default
> pfifo_fast 3 bands  and expecting a result, nothing happens - but in
> the general case, people setting skb->priority are trying to get
> better latency in the first place, and I really don't think almost
> anybody will notice.
> 
> 2) if you are using a filter on pfifo_fast that expects 3 bands, and
> end up using fq_codel by default anyway we get DRR-like behavior over
> codel rather than strict prioritization and lose fq_codel's full
> benefits... which is still a win IMHO. I am not fond of being able to
> starve the other two bands....
> 
> 3) trying to explicitly set pfifo_fast via tc doesn't work with this patch.
> 
> 4) ECN processing is enabled by default (but off by default in sysctl)

There is no 'one solution fits every needs'.

codel is _not_ a replacement of pfifo_fast, its a replacement for pfifo.

If you want to replace pfifo_fast, you want PRIO + 3 codel, because
pfifo_fast is really PRIO + 3 pfifo.