[Cake] cake at 60gbit

[Pete Heist]

> On Jul 3, 2018, at 11:18 AM, Jonathan Morton <chromatix99 at gmail.com> wrote:
> 
>> On 3 Jul, 2018, at 1:23 am, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
>> 
>> My hunch is that this has something to do with the way mlx5 uses multiple receive queues (and thus multiple CPUs). Which is probably different from veth...
> 
> At this stage I'm pretty confident it has nothing to do with Cake, and everything to do with the Mellanox hardware and driver.  It does strike me that Linux' default handling of multiqueue hardware doesn't map very well to the qdisc interface.

That’s looking highly likely to me too. I’ll stop trying to repro this unless we come up with something new to try.

I wonder if enabling the “lockless qdisc” support (https://lwn.net/Articles/698135/) we discussed late last year would change anything. It should work either way, even with a single qdisc lock, but just speculating, at those speeds with multiple cores competing for a single qdisc lock I wonder if we’re exposing a problem in the driver or somewhere else.