[Cerowrt-devel] fq_codel on netronome's NICs?

[Dave Taht]

Being kind of inspired by all the tricks
https://homes.cs.washington.edu/~arvind/papers/afq.pdf used on the
cavium, I went looking for other smart nics to play with.
https://open-nfp.org/resources/ looked interesting so I pinged them...

from netronome:

"I think it would be feasible to implement fq_codel on the NFP.

The hardware schedulers do not support fq_codel, so the schedulers
would have to be implemented in one of the NFP firmware languages
(e.g. micro-C or micro-code); the NFP hardware rings could be used for
the queueing mechanism.  Practically, this may be one way of making it
work:

The main worker threads could calculate the flow hash in order to
select which ring should be used, and then issue the packet to a
re-ordering thread.
I believe the re-ordering thread can push the packets to the internal
NFP rings instead of the wire.
The scheduler thread could then make the scheduling decision, pop the
packet from the corresponding ring, then send the packet to the
hardware packet schedulers (or drop the packet if performing a
head-drop), and also check the timestamp for the CoDel portion of the
algorithm.
The hardware packet schedulers should then transmit the packet.


In terms of handling any rate-mismatch on the outgoing interface, you
could have another thread monitor the NFP hardware packet scheduler
queue levels.  The scheduler thread can then throttle the packet rate
being sent to the hardware packet schedulers (unless of course it is
okay to tail-drop at the hardware packet scheduler queues).

Finally, if the outgoing interface is not the natural point of
congestion/rate mis-match (e.g. if the outgoing Ethernet interface is
attached to a cable/DLS modem), the NFP hardware does have some
support for rate-limiting the outgoing interface (e.g. limiting a 10
Gigabit Ethernet interface down to 600 Mbps outbound), so as to move
the congestion/rate mis-match point to the NFP, so that fq_codel can
take effect in terms of handling the buffer bloat."

-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619