[Cake] Lockup at high speeds
toke at toke.dk
Sat Jun 2 15:25:35 EDT 2018
Dave Taht <dave.taht at gmail.com> writes:
> On Sat, Jun 2, 2018 at 11:21 AM, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
>> Dave Taht <dave.taht at gmail.com> writes:
>>> How on earth are you getting these speeds? I'm stuck at a pathetic *4*
>>> gbit here on the admittedly ancient 12 core boxes I'd got.
>>> (I'll go rework my veth topology)
>> Well, having actual hardware do the packet forwarding, and only having
>> each machine process each packet once probably helps...
>>> rcu stuff makes me nervous, what happens if that nat stuff is disabled entirely.
>>> What if you made q.len and sparse/bullk flow counts atomic ops?
>> The whole qdisc dequeue op is running while holding a global lock. The
>> freezes I'm seeing are happening because the other cores get stuck
>> waiting for the cake dequeue operation to complete (and thus release the
>> qdisc lock). Which is why I suspect an infinite loop somewhere...
> Am I wrong in remembering that some qdisc stats are multi-core (like
> qlen?), and summed at some other step?
Well, I *think* you are... :)
> If you pin the card or qdisc to a single core what happens? (Besides
> it getting slower)
No idea. Will try that, once i have more time to debug this; in a few
weeks. I'm trying to get someone else to do something about it in the
meantime, as you can maybe tell ;)
More information about the Cake