From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pv0-f171.google.com (mail-pv0-f171.google.com [74.125.83.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 08FC6201A6D for ; Thu, 5 May 2011 21:14:16 -0700 (PDT) Received: by pva4 with SMTP id 4so1739588pva.16 for ; Thu, 05 May 2011 21:19:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to:x-mailer; bh=VAYjU4mHFJEj79cIffXuoq3xNW38KN5B3u7PscjrQYA=; b=g9lWUW1cUfzmsDZlGJtMgoDY3gZqiQKq8vnqgtxiI9O1IBA3Y0a7dsh/2sNr5I6mk0 5qnSg7nhtq9FZqGU0EuSgM/5pAC2Iv9qn3kmBreIA2DoJkIJtrrVeo15F95SzbnUA6oe 24DIl9+1v4/SzLAPeWO4KVcQCdOJ9THEqviLQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=joTIpSlcWgw1I+jNqOgUWJMLy61NT5YVPmnL1veZoGJQlfoKlfRe3kEZckIzTY0u41 HccgDYFPlr+sOakxeVjOY4N6ESoiav/4XfX3S7m2jKOpXI9WL0DvQlwjkaFTz8uiBHqs t48ot6Y8hJIpZHYyH+TIoO+DOj9/cUUXl7dgc= Received: by 10.68.64.10 with SMTP id k10mr4232520pbs.348.1304655542060; Thu, 05 May 2011 21:19:02 -0700 (PDT) Received: from [127.0.0.1] (wsip-98-173-193-12.sb.sd.cox.net [98.173.193.12]) by mx.google.com with ESMTPS id f1sm1871948pbm.93.2011.05.05.21.19.00 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 05 May 2011 21:19:01 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Fred Baker In-Reply-To: <4DC2C9D2.8040703@freedesktop.org> Date: Thu, 5 May 2011 21:18:47 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <1EA9A6B3-F1D0-435C-8029-43756D53D8FD@gmail.com> References: <4DB70FDA.6000507@mti-systems.com> <4DC2C9D2.8040703@freedesktop.org> To: Jim Gettys X-Mailer: Apple Mail (2.1084) X-Mailman-Approved-At: Fri, 06 May 2011 06:50:46 -0700 Cc: bloat@lists.bufferbloat.net Subject: Re: [Bloat] Goodput fraction w/ AQM vs bufferbloat X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 May 2011 04:14:17 -0000 There are a couple of ways to approach this, and they depend on your = network model. In general, if you assume that there is one bottleneck, losses occur in = the queue at the bottleneck, and are each retransmitted exactly once = (not necessary, but helps), goodput should approximate 100% regardless = of the queue depth. Why? Because every packet transits the bottleneck = once - if it is dropped at the bottleneck, the retransmission transits = the bottleneck. So you are using exactly the capacity of the bottleneck. the value of a shallow queue is to reduce RTT, not to increase or = decrease goodput. cwnd can become too small, however; if it is possible = to set cwnd to N without increasing queuing delay, and cwnd is less than = N, you're not maximizing throughput. When cwnd grows above N, it merely = increases queuing delay, and therefore bufferbloat. If there are two bottlenecks in series, you have some probability that a = packet transits one bottleneck and doesn't transit the other. In that = case, there is probably an analytical way to describe the behavior, but = it depends on a lot of factors including distributions of competing = traffic. There are a number of other possibilities; imagine that you = drop a packet, there is a sack, you retransmit it, the ack is lost, and = meanwhile there is another loss. You could easily retransmit the = retransmission unnecessarily, which reduces goodput. The list of silly = possibilities goes on for a while, and we have to assume that each has = some probability of happening in the wild. On May 5, 2011, at 9:01 AM, Jim Gettys wrote: > On 04/30/2011 03:18 PM, Richard Scheffenegger wrote: >> I'm curious, has anyone done some simulations to check if the = following qualitative statement holds true, and if, what the = quantitative effect is: >>=20 >> With bufferbloat, the TCP congestion control reaction is unduely = delayed. When it finally happens, the tcp stream is likely facing a = "burst loss" event - multiple consecutive packets get dropped. Worse = yet, the sender with the lowest RTT across the bottleneck will likely = start to retransmit while the (tail-drop) queue is still overflowing. >>=20 >> And a lost retransmission means a major setback in bandwidth (except = for Linux with bulk transfers and SACK enabled), as the standard (RFC = documented) behaviour asks for a RTO (1sec nominally, 200-500 ms = typically) to recover such a lost retransmission... >>=20 >> The second part (more important as an incentive to the ISPs = actually), how does the fraction of goodput vs. throughput change, when = AQM schemes are deployed, and TCP CC reacts in a timely manner? Small = ISPs have to pay for their upstream volume, regardless if that is "real" = work (goodput) or unneccessary retransmissions. >>=20 >> When I was at a small cable ISP in switzerland last week, surely = enough bufferbloat was readily observable (17ms -> 220ms after 30 sec of = a bulk transfer), but at first they had the "not our problem" view, = until I started discussing burst loss / retransmissions / goodput vs = throughput - with the latest point being a real commercial incentive to = them. (They promised to check if AQM would be available in the CPE / = CMTS, and put latency bounds in their tenders going forward). >>=20 > I wish I had a good answer to your very good questions. Simulation = would be interesting though real daa is more convincing. >=20 > I haven't looked in detail at all that many traces to try to get a = feel for how much bandwidth waste there actually is, and more formal = studies like Netalyzr, SamKnows, or the Bismark project would be needed = to quantify the loss on the network as a whole. >=20 > I did spend some time last fall with the traces I've taken. In those, = I've typically been seeing 1-3% packet loss in the main TCP transfers. = On the wireless trace I took, I saw 9% loss, but whether that is = bufferbloat induced loss or not, I don't know (the data is out there for = those who might want to dig). And as you note, the losses are = concentrated in bursts (probably due to the details of Cubic, so I'm = told). >=20 > I've had anecdotal reports (and some first hand experience) with much = higher loss rates, for example from Nick Weaver at ICSI; but I believe = in playing things conservatively with any numbers I quote and I've not = gotten consistent results when I've tried, so I just report what's in = the packet captures I did take. >=20 > A phenomena that could be occurring is that during congestion = avoidance (until TCP loses its cookies entirely and probes for a higher = operating point) that TCP is carefully timing it's packets to keep the = buffers almost exactly full, so that competing flows (in my case, simple = pings) are likely to arrive just when there is no buffer space to accept = them and therefore you see higher losses on them than you would on the = single flow I've been tracing and getting loss statistics from. >=20 > People who want to look into this further would be a great help. > - Jim >=20 >=20 > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat