From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-14-iad.dyndns.com (mxout-112-iad.mailhop.org [216.146.32.112]) by lists.bufferbloat.net (Postfix) with ESMTP id ACEBD2E0271 for ; Tue, 15 Mar 2011 18:00:54 -0700 (PDT) Received: from scan-12-iad.mailhop.org (scan-12-iad.local [10.150.0.209]) by mail-14-iad.dyndns.com (Postfix) with ESMTP id EBBE444A798 for ; Wed, 16 Mar 2011 01:00:53 +0000 (UTC) X-Spam-Score: 0.0 () X-Mail-Handler: MailHop by DynDNS X-Originating-IP: 70.61.120.58 Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by mail-14-iad.dyndns.com (Postfix) with ESMTP id 9EA6A44A771 for ; Wed, 16 Mar 2011 01:00:53 +0000 (UTC) Received: from uucp by smtp.tuxdriver.com with local-rmail (Exim 4.63) (envelope-from ) id 1Pzf6a-0007kC-3u; Tue, 15 Mar 2011 21:00:52 -0400 Received: from linville-8530p.local (linville-8530p.local [127.0.0.1]) by linville-8530p.local (8.14.4/8.14.4) with ESMTP id p2G0lM58029290; Tue, 15 Mar 2011 20:47:22 -0400 Received: (from linville@localhost) by linville-8530p.local (8.14.4/8.14.4/Submit) id p2G0lMsj029288; Tue, 15 Mar 2011 20:47:22 -0400 Date: Tue, 15 Mar 2011 20:47:22 -0400 From: "John W. Linville" To: Jonathan Morton Message-ID: <20110316004722.GD28663@tuxdriver.com> References: <4D7F4121.40307@freedesktop.org> <20110315175942.GA10064@goldfish> <1300212877.2087.2155.camel@tardy> <20110315183111.GB2542@tuxdriver.com> <29B06777-CC5F-4802-8727-B04F58CDA9E3@gmail.com> <20110315205146.GF2542@tuxdriver.com> <219C7840-ED79-49EA-929D-96C5A6200401@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <219C7840-ED79-49EA-929D-96C5A6200401@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: bloat@lists.bufferbloat.net Subject: Re: [Bloat] Random idea in reaction to all the discussion of TCP flavours - timestamps? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Mar 2011 01:00:55 -0000 On Wed, Mar 16, 2011 at 12:01:41AM +0200, Jonathan Morton wrote: > > On 15 Mar, 2011, at 10:51 pm, John W. Linville wrote: > > >>> If you don't throttle _both_ > >>> the _enqueue_ and the _dequeue_, then you could be keeping a nice, > >>> near-empty tx queue on the host and still have a long, bloated queue > >>> building at the device. > >> > >> Don't devices at least let you query how full their queue is? > > > > I suppose it depends on what you mean? Presumably drivers know that, > > or at least can figure it out. The accuracy of that might depend on > > the exact mechanism, how often the tx rings are replinished, etc. > > > > However, I'm not aware of any API that would let something in the > > stack (e.g. a qdisc) query the device driver for the current device > > queue depth. At least, I don't think Linux has one -- do other > > kernels/stacks provide that? > > I get the impression that eBDP is supposed to work relatively > close to the device driver, rather than in the core network stack. > As such it's not a qdisc, but instead manages a parameter used by > a well-behaved device driver. (The number of well-behaved device > drivers appears to be small at present.) If you count mac80211 as part of the "driver", what is between the qdisc and the "driver"? I wouldn't consider the bottom of the qdisc as the core of the stack. I would really like to see eBDP (or ALT or A* or something similar) implemented in a single place if possible, rather than reimplemented (perhaps poorly) in a series of drivers. I know Felix and others think that 802.11n aggregation makes that impossible, but I'm inclined to think that is still at least partly from a bias towards throughput at the expense of latency -- I could be wrong! :-) Someone suggested that perhaps eBDP/ALT/A*/whatever could be implemented as a library for drivers to call -- that still requires individual driver hacking, but maybe it is reasonable? I'd have to see the code. > So there's a queue in the qdisc, and there's a queue in the hardware, > and eBDP tries to make the latter smaller when possible, allowing the > former (which is potentially much more intelligent) to do more work. So that is a possible implementation -- limit the tx queue length in the driver, similar to manually doing 'ethtool -G'. But on the other hand, you can achieve a similar effect by throttling the input to the driver/hardware tx queue no matter how many hardware tx slots are physically allowed. > Knowing the occupancy of the hardware buffer is useful if the > size of the buffer cannot be changed, because it is then possible > to simply decline to fill the buffer more than a certain amount. > If you can also assume that packets are sent in order of submission, > or by some other easy rule, then you can also infer the time that > the oldest packet has spent there, and use it to tune the future > occupancy limit even if you can't cancel the old packet. Yes, I think we agree. > Cancelling old packets is potentially desirable because it allows > TCPs and applications to retransmit (which they will do anyway) > without fear of exacerbating a wireless congestion collapse. I do > appreciate that not all hardware will support this, however, and it > should be totally unnecessary for wired links. As other said later in the queue, I think trying to reach down into the driver/hardware to cancel an already posted packet would be difficult, slow, etc. John -- John W. Linville Someday the world will need a hero, and you linville@tuxdriver.com might be all we have. Be ready.