[Cake] Lockup at high speeds

[Toke Høiland-Jørgensen]

Dave Taht <dave.taht at gmail.com> writes:

> On Sat, Jun 2, 2018 at 11:21 AM, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
>> Dave Taht <dave.taht at gmail.com> writes:
>>
>>> How on earth are you getting these speeds? I'm stuck at a pathetic *4*
>>> gbit here on the admittedly ancient 12 core boxes I'd got.
>>> (I'll go rework my veth topology)
>>
>> Well, having actual hardware do the packet forwarding, and only having
>> each machine process each packet once probably helps...
>>
>>> rcu stuff makes me nervous, what happens if that nat stuff is disabled entirely.
>>>
>>> What if you made q.len and sparse/bullk flow counts atomic ops?
>>
>> The whole qdisc dequeue op is running while holding a global lock. The
>> freezes I'm seeing are happening because the other cores get stuck
>> waiting for the cake dequeue operation to complete (and thus release the
>> qdisc lock). Which is why I suspect an infinite loop somewhere...
>
> Am I wrong in remembering that some qdisc stats are multi-core (like
> qlen?), and summed at some other step?

Well, I *think* you are... :)

> If you pin the card or qdisc to a single core what happens? (Besides
> it getting slower)

No idea. Will try that, once i have more time to debug this; in a few
weeks. I'm trying to get someone else to do something about it in the
meantime, as you can maybe tell ;)

-Toke