From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-x22b.google.com (mail-lb0-x22b.google.com [IPv6:2a00:1450:4010:c04::22b]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id C833821F1F3; Mon, 1 Sep 2014 11:07:02 -0700 (PDT) Received: by mail-lb0-f171.google.com with SMTP id n15so6344693lbi.16 for ; Mon, 01 Sep 2014 11:06:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=i1xaTQ3gulOal2ekC2REmfbu5T6/aWu/EKWXI+PxCGY=; b=SS4k7m9exnTHdLvZZLM85cU68kGZkYMT6d+PHGn92x+8FKSGXUDSRLAIApPlnOZHaH dY2PuyWQSK8lAj/MmU8UKiqoG7xP2uo0Kbq/C+fWcDpyhhK45xa49YcbAsadI0se8+KV u11PDpy1IzU+AEL/r4t9755YoyyA0T20JtOTvA9h/OguZWcQXMg1VGxXa8yHxBCdOiyZ QAQixf+A9AYjqBKmfIAte69HnTd8AEkTIheP9tIHbhqM9O6L4p0VjXCk2mZLtenHJhwp 0RD4lmUttz9EJWUSKBpp6U0xfpdJuq5rmF/1s8VnleKxSZTO5UgGyMR7ItNe3fIYtcR5 SlSw== X-Received: by 10.152.18.166 with SMTP id x6mr30195349lad.1.1409594819750; Mon, 01 Sep 2014 11:06:59 -0700 (PDT) Received: from bass.home.chromatix.fi (87-93-123-167.bb.dnainternet.fi. [87.93.123.167]) by mx.google.com with ESMTPSA id n6sm2035925lbr.38.2014.09.01.11.06.57 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 01 Sep 2014 11:06:58 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: Jonathan Morton In-Reply-To: Date: Mon, 1 Sep 2014 21:06:56 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: References: <87ppfijfjc.fsf@toke.dk> <4FF4917C-1B6D-4D5F-81B6-5FC177F12BFC@gmail.com> <4DA71387-6720-4A2F-B462-2E1295604C21@gmail.com> <0DB9E121-7073-4DE9-B7E2-73A41BCBA1D1@gmail.com> To: Dave Taht X-Mailer: Apple Mail (2.1085) Cc: "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [Bloat] Comcast upped service levels -> WNDR3800 can't cope... X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Sep 2014 18:07:03 -0000 On 1 Sep, 2014, at 8:01 pm, Dave Taht wrote: > On Sun, Aug 31, 2014 at 3:18 AM, Jonathan Morton = wrote: >>=20 >> On 31 Aug, 2014, at 1:30 am, Dave Taht wrote: >>=20 >>> Could I get you to also try HFSC? >>=20 >> Once I got a kernel running that included it, and figured out how to = make it do what I wanted... >>=20 >> ...it seems to be indistinguishable from HTB and FQ in terms of CPU = load. >=20 > If you are feeling really inspired, try cbq. :) One thing I sort of = like about cbq is that it (I think) > (unlike htb presently) operates off an estimated size for the next = packet (which isn't dynamic, sadly), > where the others buffer up an extra packet until they can be = delivered. It's also hilariously opaque to configure, which is probably why nobody = uses it - the RED problem again - and the top link when I Googled for = best practice on it gushes enthusiastically about Linux 2.2! The idea = of manually specifying an "average packet size" in particular feels = intuitively wrong to me. Still, I might be able to try it later on. Most class-based shapers are probably more complex to set up for simple = needs than they need to be. I have to issue three separate 'tc' = invocations for a minimal configuration of each of them, repeating = several items of data between them. They scale up reasonably well to = complex situations, but such uses are relatively rare. > In my quest for absolutely minimal latency I'd love to be rid of that > last extra non-in-the-fq_codel-qdisc packet... either with a "peek" > operation or with a running estimate. I suspect that something like fq_codel which included its own shaper = (with the knobs set sensibly by default) would gain more traction via = ease of use - and might even answer your wish. > It would be cool to be able to program the ethernet hardware itself to > return completion interrupts at a given transmit rate (so you could > program the hardware to be any bandwidth not just 10/100/1000). Some > hardware so far as I know supports this with a "pacing" feature. Is there a summary of hardware features like this anywhere? It'd be = nice to see what us GEM and RTL proles are missing out on. :-) >> Actually, I think most of the CPU load is due to overheads in the = userspace-kernel interface and the device driver, rather than the qdiscs = themselves. >=20 > You will see it bound by the softirq thread, but, what, exactly, > inside that, is kind of unknown. (I presently lack time to build up > profilable kernels on these low end arches. ) When I eventually got RRUL running (on one of the AMD boxes, so the = PowerBook only has to run the server end of netperf), the bandwidth = maxed out at about 300Mbps each way, and the softirq was bouncing around = 60% CPU. I'm pretty sure most of that is shoving stuff across the PCI = bus (even though it's internal to the northbridge), or at least waiting = for it to go there. I'm happy to assume that the rest was mostly = kernel-userspace interface overhead to the netserver instances. But this doesn't really answer the question of why the WNDR has so much = lower a ceiling with shaping than without. The G4 is powerful enough = that the overhead of shaping simply disappears next to the overhead of = shoving data around. Even when I turn up the shaping knob to a value = quite close to the hardware's unshaped capabilities (eg. 400Mbps = one-way), most of the shapers stick to the requested limit like glue, = and even the worst offender is within 10%. I estimate that it's using = only about 500 clocks per packet *unless* it saturates the PCI bus. It's possible, however, that we're not really looking at a CPU = limitation, but a timer problem. The PowerBook is a "proper" desktop = computer with hardware to match (modulo its age). If all the shapers = now depend on the high-resolution timer, how high-resolution is the = WNDR's timer? - Jonathan Morton