From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-21-ewr.dyndns.com (mxout-147-ewr.mailhop.org [216.146.33.147]) by lists.bufferbloat.net (Postfix) with ESMTP id 29B172E0354 for ; Fri, 4 Feb 2011 10:25:04 -0800 (PST) Received: from scan-22-ewr.mailhop.org (scan-22-ewr.local [10.0.141.244]) by mail-21-ewr.dyndns.com (Postfix) with ESMTP id 79258963 for ; Fri, 4 Feb 2011 18:25:03 +0000 (UTC) X-Spam-Score: 0.0 () X-Mail-Handler: MailHop by DynDNS X-Originating-IP: 76.96.62.32 Received: from qmta03.westchester.pa.mail.comcast.net (qmta03.westchester.pa.mail.comcast.net [76.96.62.32]) by mail-21-ewr.dyndns.com (Postfix) with ESMTP id 8DA4E5666 for ; Fri, 4 Feb 2011 18:24:59 +0000 (UTC) Received: from omta19.westchester.pa.mail.comcast.net ([76.96.62.98]) by qmta03.westchester.pa.mail.comcast.net with comcast id 3tqF1g00627AodY53uR04Q; Fri, 04 Feb 2011 18:25:00 +0000 Received: from [192.168.1.119] ([98.229.99.32]) by omta19.westchester.pa.mail.comcast.net with comcast id 3uQz1g00P0hvpMe3fuQzUs; Fri, 04 Feb 2011 18:25:00 +0000 Message-ID: <4D4C447A.6070307@freedesktop.org> Date: Fri, 04 Feb 2011 13:24:58 -0500 From: Jim Gettys Organization: Bell Labs User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101208 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: Juliusz Chroboczek References: <87k4hgdte9.fsf@trurl.pps.jussieu.fr> <4D4C05A3.3000901@freedesktop.org> <871v3nemck.fsf_-_@trurl.pps.jussieu.fr> In-Reply-To: <871v3nemck.fsf_-_@trurl.pps.jussieu.fr> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: bloat@lists.bufferbloat.net Subject: Re: [Bloat] Buffer bloat at the sender [was: About Stochastic Fair Blue (SFB)] X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Feb 2011 18:25:05 -0000 On 02/04/2011 12:33 PM, Juliusz Chroboczek wrote: >> Juliusz, have you thought about the host case at all? > >> One of the places we're getting insane buffering is in the operating >> systems themselves (e.g. the experiment I did with a 100Mbps >> switch). > > Yes. You have three to four layers of buffering: > > (1) the device driver's buffer; > (2) the packet scheduler's buffer; > (3) TCP's buffer; > (4) the application's buffer. There are a few more than the four you identify: 5) sometimes device drivers have some private buffers, independent of the device itself; for example the Marvell driver for the wireless module used on OLPC has such an internal buffer; dwmw2 used it to simplify his locking problems with USB. 6) the device themselves may have buffers. Again, on the OLPC wireless module (which does a form of mesh routing internally, without needing host support), there are 4 packet buffers out in the device (which is itself an ARM processor with hundreds of kilobytes of code). I also hypothesise that there could be busses that support multiple outstanding transactions, that could add yet more buffering. I have no idea if this hypothesis is true. > > It will come as no surprise to the readers of this list that (1) and (2) > are usually too large. For example, (1) the ath9k driver has a buffer > of 200 packets; and (2) the default scheduler queue is 1000 packets (!). > >> My intuition is that we have to do AQM in hosts, not just routers. > > Hmm... I would argue that the sending host is somewhat easier than the > intermediate router. In the sender, the driver/packet scheduler can > apply backpressure to the transport layer, to cause it to slow down > without the need for the lengthy feedback loop that dropping/delaying > a packet in an intermediate router has to rely on [1]. Yup. I was noting, however, with 1-4 above, we today have a major problem here in practice. And I don't know if there is any way to signal back pressure to UDP based protocols; I've never worked on one. > > Unfortunately, at least under Linux, most drivers do not apply > backpressure correctly. Markus Kittenberger has recently determined [2] > that among b43-legacy, ath5k, ath9k and madwifi, only the former two do > the right thing. Yes, the host side can and should apply backpressure correctly, and it may be an easier case. It sounds like we have work to that area to do. > > --Juliusz > > [1] Now why did we give up on source quench again? > [2] http://article.gmane.org/gmane.network.olsr.user/4264 >