[Codel] hardware multiqueue in fq_codel?

[Eric Dumazet]

On Thu, 2013-07-11 at 10:09 -0700, Dave Taht wrote:
> In my default environments (wifi, mainly) the hardware queues have
> very different properties.
> 
> I'm under the impression that in at least a few ethernet devices they
> are essentially the same. That said, in the sch_mq case, an entirely
> separate qdisc is created per hardware queue, and it's always been
> puzzling to me as to how to attempt to use them within a single qdisc
> in the pull-through manner.
> 
> logically, you should be able to take the fq_codel hash index (idx %
> dev->num_tx_queues) and spread out across the hardware queues that
> way, but I have no idea where that info would go (the skb? the flow?)
> or even if it were possible as per the pull through problem...
> 
> (This does not mean that I necessarily think hardware multiqueues are
> a good idea... (certainly the results I get out of 802.11e are
> terrible - but it would be nice to have a unified solution for hw
> multiqueue devices)
> 

We do not have a fixed/unified queue selection.

It can be tweaked by many different things, depending on exact needs.

MQ is not a qdisc per se, it's only a fake one, a demux if you want, so
that each tx queue has a separate qdisc lock.

If you stick one fq_codel at the top of the hierarchy (instead of MQ),
then you loose all the pros of having multiple locks : sending packets
from fq_codel to different queues on hardware makes no sense, since the
single qdisc lock is the bottleneck.

So if you want fq_codel and MQ, to be able to drive 40G links from many
cpus, just use :

ETH=eth0
NQUEUES=16  # or more, check how many tx queues your NIC supports
tc qd del dev $ETH root 2>/dev/null
tc qd add dev $ETH root handle 1: mq
for i in `seq 1 $NQUEUES` 
do
 tc qd add dev $ETH parent 1:$i fq_codel
done

Thats only replaces the default pfifo_fast on each slave qdisc by
fq_codel.