From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qy0-f171.google.com (mail-qy0-f171.google.com [209.85.216.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id E4E292021D9 for ; Tue, 20 Sep 2011 09:14:17 -0700 (PDT) Received: by qyk29 with SMTP id 29so15521346qyk.16 for ; Tue, 20 Sep 2011 09:14:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=M26jEECYfKSb+hzlbu6cLQ7p6WRXcHrFlIsaZkFsGX0=; b=pVLgB1vi1ThaRwbxw4/iSuZh1Z0+ql88nqIVzpPUdxMWOnOK9xvgFvQGv5i0tXiowW eUB/yeAncz4fZf9chU6VBhFFFET6evXCc90vmqAu8oHczngt8D+HrLz6BymDaUd0otvc u6qdevvOyYYiXQ2Bh43NXYLW9IwZ8V9Ns3eDM= Received: by 10.224.33.138 with SMTP id h10mr860993qad.64.1316535256776; Tue, 20 Sep 2011 09:14:16 -0700 (PDT) Received: from [172.30.45.80] (c-98-229-130-122.hsd1.ma.comcast.net. [98.229.130.122]) by mx.google.com with ESMTPS id v6sm1956172qaz.7.2011.09.20.09.14.14 (version=SSLv3 cipher=OTHER); Tue, 20 Sep 2011 09:14:15 -0700 (PDT) Sender: Jim Gettys Message-ID: <4E78BBD5.9080705@freedesktop.org> Date: Tue, 20 Sep 2011 12:14:13 -0400 From: Jim Gettys Organization: Bell Labs User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:6.0.2) Gecko/20110906 Thunderbird/6.0.2 MIME-Version: 1.0 To: "Collier-Brown, David (LNG-CAN)" References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: davecb@spamcop.net, bloat@lists.bufferbloat.net Subject: Re: [Bloat] Hmmn, this is in part a capacity planning/management problem. X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Sep 2011 16:14:18 -0000 On 09/14/2011 11:17 AM, Collier-Brown, David (LNG-CAN) wrote: > I was reading the latest article in LWN, and commented there, but part > of the comment may be relevant to the list... > > -- reply to mlankhorst (subscriber, #52260) -- > Changing the subject slightly, there's a subtle, underlying problem in > that when developing products and protocols, we tend to work with what's > easy, not what's important. > > We work with the bandwidth/delay product because it's what we needed in > the short run, and we probably couldn't predict we'd need something more > at the time. We work with buffer sizes because that's dead easy. > > What we need instead is to work in the delay, latency and/or service > time of the various components. It's easy to deal with performance > problems that are stated in time units and are fixed by varying the > times things take. It's insanely hard to deal with performance problems > when all we know is a volume in bytes. It's a bit like measuring the > performance of large versus small cargo containers when you don't know > if they're on a truck, a train or a ship! You are exactly correct, and I certainly said so at the LPC wireless meeting and on the Linux networking mailing lists recently. Bytes or packets are not useful metrics on wireless at all, and naive even on wired ethernet (due to contention and switches doing flow control). Bytes are not interchangeable: a broadcast/multicast packet in 802.11 may cost as much as 2 orders of magnitude more than a unicast packet, and how fast you can transmit any packet depends on the current environment, which is also dynamically changing. My naive view (before August) was that if you couldn't send a packet, you should drop the rate; but this is usually wrong in practice; you are going to take less time to transmit a packet (and it may be more likely to get through since noise gets a bigger shot at a lower rate to damage a packet) at a higher rate despite multiple transmission attempts if you are able to get most packets through intact. 802.11n aggregation, in fact, allows you to send multiple packets in a frame, and tells you which packets got through intact with a bitmask. This then begs the question of retransmitting damaged packets, and not doing massive reordering when there is packet loss. Thankfully, Linux has a module called Minstrel, which is dynamically monitoring the likely cost of sending a packet at various rates, so it can make a much better than random guess what strategy to transmit a packet is best. Andrew McGregor (one of the Minstrel authors) worked with Felix Fietkau (the ath9k driver maintainer) and Dave Taht to greatly improve the aggregation in the driver since the Quebec City IETF (this is part of why things have been quiet, along with vacations). Buffering is now about 1/3 what it was (worst case), and the infinite retry problems cured (we were observing ICMP packets taking 1.6 seconds to get across the air under some circumstances). Our thanks to them. This code is in RC6; I'm not sure it is in debloat-testing yet; I should ping John Linville. Using Minstrel, and estimating contention on the network, we may be able to get to a point of a proper metric and short term prediction on how fast data may flow in an 802.11 network. That is going to be necessary for any AQM to work properly. > > If you expose any time-based metrics or tuneables in your investigation, > please highlight them. Anything that looks like delay or latency would > be seriously cool. > > One needs very little to analyze this class of problems. Knowing the > service time of a packet, the number of packets, and the time between > packets is sufficient to build a tiny little mathematical model of the > thing you measured. From the model you can then predict what happens > when you improve or disimprove the system. More information allows for > more predictive models, of course, and eventually to my mathie friends > becoming completely unintelligible (;-)) Again, a look at Minstrel is a good thing to do. And you have to keep in mind that the time to transmit something is constantly varying. You can't compute a time to transmit something once and expect it to be correct seconds later. You can't fool mother nature; or she bites back. - Jim > > --dave (davecb@spamcop.net) c-b > -- > > As you might guess, I'm a capacity planner, and might be able to help a > bit on the modeling side. Besides, I'm looking for a networking example > for my next book (;-)) > > --dave > > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat