From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-f98.google.com (mail-la0-f98.google.com [209.85.215.98]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id C1A4121F29F for ; Mon, 4 May 2015 05:34:41 -0700 (PDT) Received: by labgd6 with SMTP id gd6so2920553lab.2 for ; Mon, 04 May 2015 05:34:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=7lr/oH9TeAwBHFp0hRwwpYhR3ugkFF3/eb1HNrysHgg=; b=Az8uY+Cw9V4H2tFWGvNE5DXpb/mbVA5Fk8bZFkzWV0N689yWqUnls6Ml86ueE5LbD9 OrdCNy9ig+wYzHlpKBsCMF32GXxAE6ta91OUmcwHy4Ec4jPVGMxofMaWi28qt3aHr16S gqztUrQDKZZSnEnbrux5wW0AWXCuCO/qYlmthOwhxzX+MDlbJEjGqpvlNMJxdAenX+C/ 7MtywKruXHBKQbyZP5aAr7CC0q+0GVZRQfDYJW8o0h6Ntz5ZJKq0vAWyB8zzo7RPHmCK cN5sfQ7rJdYSHJTPqG5vgyA75gFMygWqmVr8s4vUnUYwTZp6ylOAq1MKDdb8+m8bc2Io Yp2Q== X-Gm-Message-State: ALoCoQmP3kWJp+fXiKFmk2C72NL3pPKSBogyYM5VwCrlffgopP61zjuadflccVLwnthqIGgfdD3/UUtqFeD+FEEKYBwQsjrufA== X-Received: by 10.194.173.226 with SMTP id bn2mr41736009wjc.148.1430742879100; Mon, 04 May 2015 05:34:39 -0700 (PDT) Received: from mail.la.pnsol.com (eu1sys200aog126.obsmtp.com. [207.126.144.174]) by mx.google.com with SMTPS id bl4sm665873wjb.4.2015.05.04.05.34.38 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 04 May 2015 05:34:39 -0700 (PDT) X-Relaying-Domain: pnsol.com Received: from mail.la.pnsol.com ([89.145.213.110]) (using TLSv1) by eu1sys200aob126.postini.com ([207.126.147.11]) with SMTP ID DSNKVUdnXa7kJY1RaID8RS7pOaVXglfcY8gI@postini.com; Mon, 04 May 2015 12:34:38 UTC Received: from ba6-office.pnsol.com ([172.20.5.199]) by mail.la.pnsol.com with esmtp (Exim 4.76) (envelope-from ) id 1YpFZs-0007AS-8R; Mon, 04 May 2015 13:34:28 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Neil Davies In-Reply-To: <6DD94D01-DA3E-4099-A8CA-99A8C06567BC@gmail.com> Date: Mon, 4 May 2015 13:35:42 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <72DB0260-F0DF-426F-A3F3-ECF5D8AF228F@pnsol.com> <766042D4-0C90-4C77-9033-07B8E436C35B@pnsol.com> <2F4DCB53-1E46-4829-B2F8-F8131664D1FF@pnsol.com> <0F8CB21C-792F-4F95-BC49-BED3DF0A2100@unimore.it> <5D20E7F1-37D0-4CD3-8793-C29149695C97@pnsol.com> <48540B54-B7B0-428C-BA71-0BA5E40C6FB6@gmail.com> <5716B8D7-E2E7-4267-B8F1-9385329222B2@pnsol.com> <6DD94D01-DA3E-4099-A8CA-99A8C06567BC@gmail.com> To: Jonathan Morton X-Mailer: Apple Mail (2.1878.6) Cc: bloat Subject: Re: [Bloat] Detecting bufferbloat from outside a node X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 May 2015 12:35:10 -0000 Jonathan We see the problem as the difference between averages and instantaneous.=20= Network media is never =93average=94 used - it is either =93in-use=94 or = =93idle=94 - what we were seeing (and it was not an ISP but the core of = a public service network here in the UK) was that delay can be =93high=94 = even when the loading is =93low=94 (in the particular 5minute period the = actual offered traffic was <0.01% of the capacity) - it was that the = path under examination happened to be the constraining factor for a bulk = transfer - the induced delay was high enough to place at risk other = real-time applications (as defined by the public service network=92s = users). The reasoning that you seem to be applying below assumes a = time-homogenity that doesn=92t correspond to network traffic patterns = that occur in the engagements we=92ve done over the last 15 years. The = graph I was referring to is the one example that we can publicly discuss = (all the rest are under NDA!). What you are describing - if I=92m understanding it properly - is the = =93busy period=94. I would accept that Network Providers (ISP=92s, = telcos etc) have a problem in that they are relying on the system = becoming idle frequently (the busy periods not accreting into longer and = longer periods of non-idleness). However that is a pattern as well as a = load dependent phenomena.=20 Neil On 4 May 2015, at 13:17, Jonathan Morton wrote: >=20 >> On 4 May, 2015, at 14:39, Neil Davies wrote: >>=20 >>>> Noting that, delay and loss is, of course, a natural consequence of = having a shared medium >>>=20 >>> Not so. Delay and loss are inherent to link oversubscription, not = to contention. Without ECN, delay is traded off against loss by the = size of the buffer; a higher loss rate keeps the queue shorter and thus = the induced delay lower. >>=20 >> Sorry Jonathan - that=92s not what we=92ve observed. We=92ve measured = =93excessive=94 delay on links that are averagely loaded << 0.1% (as = measured over a 15 min period) - I can supply pointers to the graphs for = that.=20 >=20 > Presumably those would involve oversubscription on short timescales, = and a lot of link idle time between those episodes. >=20 > One ISP I know of charges by data volume per month, currently in units = of 75GB (minimum 2 per month, so 150GB). This is on ADSL lines where = the link rate might reasonably be 15Mbps or so in the relevant = direction. At that speed, it would take 100,000 seconds to exhaust the = first two units - which is not much more than 24 hours. There is = therefore roughly a 26-fold mismatch between the peak rate available to = the user and the average rate he must maintain to stay within the data = allowance. >=20 > (I am ignoring small niceties in the calculations here, in favour of = revealing the big picture without too much heavy maths.) >=20 > By your measure, that would mean that the link could only ever be = 3.85% utilised (1/26th) on month-long timescales, and is therefore = undersubscribed. But I can assure you that, during the small percentage = of time that the link is in active use, it will spend some time at 100% = utilisation on RTT timescales, with TCP/IP straining to achieve more = than that. That is link oversubscription which results in high induced = delay. >=20 > More precisely, instantaneous link oversubscription results in either = *increasing* induced delay (as the buffer fills) or lost packets (which = *will* happen if the buffer becomes completely full), while = instantaneous link undersubscription results in either *decreasing* = induced delay (as the buffer drains) or link idle periods. = Long-timescale measures of link utilisation are simply averages of these = instantaneous measures. >=20 >> A single flow can contend the medium just as much as a multiple ones >=20 > I think here, again, we are using wildly different terminology. >=20 > There is no contention for the medium on the dedicated full-duplex = link I described, only for queue space - and given a single flow, it = cannot contend with itself. >=20 > The same goes for a full-duplex shared-access medium (such as DOCSIS = cable) with only one host active. There is no contention for the = medium, because it is always available when that single host requests = it, which it will as soon as it has at least one packet in its queue. = There is a more-or-less fixed latency for medium access, which becomes = part of what you call the structural delay. The rest is down to over- = or under-subscription on short timescales, as above. >=20 > On a half-duplex medium, such as obsolete bus Ethernet or = not-so-obsolete wifi, then there can be some contention for the medium = between forward data and reverse ack packets. But I was *not* talking = about half-duplex. Full-duplex is an important enough subset of the = problem - covering at least ADSL, cable, VDSL, satellite, fibre - on = which most of the important effects can be observed, including the ones = we=92re talking about. >=20 > - Jonathan Morton >=20