From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-01-iad.dyndns.com (mxout-110-iad.mailhop.org [216.146.32.110]) by lists.bufferbloat.net (Postfix) with ESMTP id F363A2E0044 for ; Mon, 21 Mar 2011 12:43:42 -0700 (PDT) Received: from scan-02-iad.mailhop.org (scan-02-iad.local [10.150.0.207]) by mail-01-iad.dyndns.com (Postfix) with ESMTP id 864036E0CC for ; Mon, 21 Mar 2011 19:43:43 +0000 (UTC) X-Spam-Score: -1.0 (-) X-Mail-Handler: MailHop by DynDNS X-Originating-IP: 209.85.215.171 Received: from mail-ey0-f171.google.com (mail-ey0-f171.google.com [209.85.215.171]) by mail-01-iad.dyndns.com (Postfix) with ESMTP id 13A3B6DD7F for ; Mon, 21 Mar 2011 19:43:42 +0000 (UTC) Received: by eydd26 with SMTP id d26so1947183eyd.16 for ; Mon, 21 Mar 2011 12:43:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:subject:mime-version:content-type:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to:x-mailer; bh=IbMseWNM5HWnVGCjk3EGSeVYZQS6dBgohSCiYf6c+jI=; b=NfDwKybRV5QcNJ8BSMy9LIvvcesAdFczuRzl53ZwgoGBoGHconGQ1reFbLKlsKXu7m FPz6ogFEB5RrlySUfayvCOJuhIC3fJmpEpI/ve3bGPpqwRVUx0g/8hzNYTO9geZaxMeB 9nvWCv6xsO5Tn230mMQOnm0WeWSJVDIRB6Ht0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=vwo0ekrSDP3OqhSdrb7fUT6D1w7fvYBlvdpfdKbpPT+KLtfMFOW/ApIht74cYTB6cB LKC2x6MilAOxqHVAFmJqxjMNkau/j9h1lgxJqZX5IlBgFJJca1tYMaX7G+AhxDQ9vssp nNgjWLa4aACteeze5I81EQwwJKtch6Xa38rfM= Received: by 10.14.17.75 with SMTP id i51mr1481850eei.107.1300736621101; Mon, 21 Mar 2011 12:43:41 -0700 (PDT) Received: from [192.168.239.42] (xdsl-83-150-84-172.nebulazone.fi [83.150.84.172]) by mx.google.com with ESMTPS id y7sm727612eeh.21.2011.03.21.12.43.40 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 21 Mar 2011 12:43:40 -0700 (PDT) Subject: Re: debloat-testing loadlatency test Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=iso-8859-1 From: Jonathan Morton In-Reply-To: <87wrjsbki1.fsf@cruithne.co.teklibre.org> Date: Mon, 21 Mar 2011 21:43:38 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <92F8B514-76F9-45E5-BB9F-C94BA1CA66B4@gmail.com> References: <87wrjsbki1.fsf@cruithne.co.teklibre.org> To: d@taht.net (=?iso-8859-1?Q?Dave_T=E4ht?=) X-Mailer: Apple Mail (2.1082) Cc: bloat-devel@lists.bufferbloat.net X-BeenThere: bloat-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Developers working on AQM, device drivers, and networking stacks" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Mar 2011 19:43:43 -0000 On 21 Mar, 2011, at 4:45 pm, Dave T=E4ht wrote: > I aborted the test when I got up this morning. Yes, it looks as though it may have got stuck, possibly on a = partly-mitigated race condition I found. A full test run should never = take that long, as there is an override condition which forces a = scenario to complete after about 10 minutes. It would probably be worth re-running this with better propagation = conditions. We've already established that even with high-quality = physical links, the statistics are usually remarkably bad - I'd like to = establish that high smoothness and responsiveness figures are actually = possible. At least you don't seem to have any actual dropped connections. Here's = a run between two Macs on GigE: MinRTT: 0.0ms Scenario 1: 0 uploads, 1 downloads... 107198 KiB/s down, 22.82 Hz = smoothness Scenario 2: 1 uploads, 0 downloads... 105388 KiB/s up, 31.00 Hz = smoothness Scenario 3: 0 uploads, 2 downloads... 110783 KiB/s down, 7.52 Hz = smoothness Scenario 4: 1 uploads, 1 downloads... 50998 KiB/s up, 85859 KiB/s down, = 2.90 Hz smoothness Scenario 5: 2 uploads, 0 downloads... 108230 KiB/s up, 2.73 Hz = smoothness Scenario 6: 0 uploads, 3 downloads... 112491 KiB/s down, 7.29 Hz = smoothness Scenario 7: 1 uploads, 2 downloads... 33296 KiB/s up, 100526 KiB/s down, = 3.11 Hz smoothness Scenario 8: 2 uploads, 1 downloads... 69995 KiB/s up, 66111 KiB/s down, = 2.53 Hz smoothness Scenario 9: 3 uploads, 0 downloads... 102502 KiB/s up, 4.96 Hz = smoothness Scenario 10: 0 uploads, 4 downloads... 111745 KiB/s down, 2.55 Hz = smoothness Scenario 11: 1 uploads, 3 downloads... 21816 KiB/s up, 104424 KiB/s = down, 6.00 Hz smoothness Scenario 12: 2 uploads, 2 downloads... 52828 KiB/s up, 93344 KiB/s down, = 2.96 Hz smoothness Scenario 13: 3 uploads, 1 downloads... 82564 KiB/s up, 52255 KiB/s down, = 6.89 Hz smoothness Scenario 14: 4 uploads, 0 downloads... 110715 KiB/s up, 2.57 Hz = smoothness Scenario 15: 0 uploads, 32 downloads... 0 KiB/s down, 0.00 Hz smoothness Scenario 16: 1 uploads, 31 downloads... 14712 KiB/s up, 0 KiB/s down, = 0.00 Hz smoothness Scenario 17: 16 uploads, 16 downloads... 76252 KiB/s up, 85105 KiB/s = down, 0.86 Hz smoothness Scenario 18: 31 uploads, 1 downloads... 113567 KiB/s up, 5775 KiB/s = down, 2.64 Hz smoothness Scenario 19: 32 uploads, 0 downloads... 107671 KiB/s up, 2.55 Hz = smoothness OVERALL: Upload Capacity: 50450 KiB/s Download Capacity: 0 KiB/s Link Responsiveness: 0 Hz Flow Smoothness: 0 Hz In the above, during scenarios 15 and 16, the G5 (acting as the server) = reported hard shutdowns of several spew() threads, indicating that the = TCP/IP stack had cancelled the connections. The MBP (as client) didn't = hard-shutdown any connections. Shutdown connections are deliberately = recorded as zeroes because they constitute a serious failure of the = network, and the way the stats are combined ensures that this is = reflected in the overall results. It seems that OSX uses a rather aggressive TCP which can actually = saturate GigE even with one connection. The tradeoff is that with = multiple flows in contention, they can be mutually unfair enough for = some flows to be completely starved out by packet losses. When that = happens, the TCP retransmits, but the retransmissions also get lost at a = rather high probability. Eventually, the stack decides the flow is dead = and cancels it. It's not clear whether both ends are using ECN here (the G5 is = restricted to OSX 10.5.x because 10.6 requires an Intel CPU), but either = way it is clearly ineffective because there is no AQM on the GigE ports = (or at least not the one in the G5). ECN requires AQM to function. - Jonathan