From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.toke.dk (mail.toke.dk [52.28.52.200]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 663D83BA8E for ; Sat, 2 Jun 2018 14:21:25 -0400 (EDT) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=toke.dk; s=20161023; t=1527963683; bh=TE6VKo781GgmClyGhfJ1RDrnYYSMUsiCK1XfuCiXsJw=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=vqK7PQ2TBCaCjnpdphSRrwfndFjIw3HbJwmRQ2adjpOvNKHCyERznpd38Qy3Hqjat EwY2SOzdcAchrVhEkSssbMTRPys+bp8HjOIX3B6hzmgJwhbd+13YOL/SAi1BK2L/26 uBOc9D0B1cFOGcOq1UvhI7HVNQtebxgdYbkaDxvmCkFEHg0LJefEWC9OgmLu4dwnqj 9VdhyafF31ncV6Ca7mteGFiDz+MLePSXybVxv+5P2+MRz0WoFOK9sz6w9Hf05ZmC3o t/2qbOFYS6+V6CfJgXj4qwWmtvR2MWhaX5w6KZDtA9IE/eRGuEiM6nVZVa/g8tzYV5 7HVCtJnwQU3kg== To: Dave Taht Cc: Jonathan Morton , Cake List In-Reply-To: References: <87muwe9z7j.fsf@toke.dk> <87k1ri9yve.fsf@toke.dk> <877enimhuh.fsf@toke.dk> <9A273CA6-FDF6-485D-B466-8B01D4573BD9@gmail.com> <871sdqmg8l.fsf@toke.dk> Date: Sat, 02 Jun 2018 20:21:22 +0200 X-Clacks-Overhead: GNU Terry Pratchett Message-ID: <87a7sdcan1.fsf@toke.dk> MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Cake] Lockup at high speeds X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jun 2018 18:21:25 -0000 Dave Taht writes: > How on earth are you getting these speeds? I'm stuck at a pathetic *4* > gbit here on the admittedly ancient 12 core boxes I'd got. > (I'll go rework my veth topology) Well, having actual hardware do the packet forwarding, and only having each machine process each packet once probably helps... > rcu stuff makes me nervous, what happens if that nat stuff is disabled entirely. > > What if you made q.len and sparse/bullk flow counts atomic ops? The whole qdisc dequeue op is running while holding a global lock. The freezes I'm seeing are happening because the other cores get stuck waiting for the cake dequeue operation to complete (and thus release the qdisc lock). Which is why I suspect an infinite loop somewhere... > I tend to lean towards an overflow post 40gbit also. An overflow of what? 40 Gbps is 5 GBps, which is the unit that rates are being measured in. So there's no obvious reason to expect that... Which is not to say it couldn't happen, of course... -Toke