From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-f99.google.com (mail-la0-f99.google.com [209.85.215.99]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id A7F8921F209 for ; Tue, 28 Apr 2015 00:17:41 -0700 (PDT) Received: by labgd6 with SMTP id gd6so2701416lab.2 for ; Tue, 28 Apr 2015 00:17:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=vEs+x59CmKyOYCQsLVT9djoZ3LRojtBqy1f9DtHyZiQ=; b=mmMv/Tkj0UJ+NbI2uIkyQHQFnc9aK0mhvql3atseZqsPQqjWiF7G6UodqDGOO4JJSa 8+8woe9t8AlalYE5ubHAw+OcVWHyaYP+/cV8z+3BbuK2ixMr/d4i6jAX57FZmDP7qA05 QpTbJDplmrxgZxRH3ILSRF9l1vQR7bgdi4zJkaBMZcKj64FLnJ3tcMxN6dvc5nIoXkmm 7wXKL78nWZkJvtQbvfAi/c/ud7cNCK4WcqixaUv4qh6cMA80qBt/qtcDFwCuXdhx60sa DnCa1HlvNnRopI9gi9m/yNOQh6K8FHHRiB7iQiGt0sEXMErIB6xXFsytaeNla9bFZ1kt 6+IA== X-Gm-Message-State: ALoCoQk6fwP3kMc6fssZDr3FqS4MLXIRQOtDLOOIi7bXpFytbFMkLo/yTfMUF8cz32QCvHOxyIT0yvmZ7T1KSrLnatC3n44M0Q== X-Received: by 10.194.75.168 with SMTP id d8mr29942356wjw.87.1430205459724; Tue, 28 Apr 2015 00:17:39 -0700 (PDT) Received: from mail.la.pnsol.com (eu1sys200aog112.obsmtp.com. [207.126.144.133]) by mx.google.com with SMTPS id k3sm476910wiz.1.2015.04.28.00.17.38 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 28 Apr 2015 00:17:39 -0700 (PDT) X-Relaying-Domain: pnsol.com Received: from mail.la.pnsol.com ([89.145.213.110]) (using TLSv1) by eu1sys200aob112.postini.com ([207.126.147.11]) with SMTP ID DSNKVT80Eos6xpVx+47AvuVXnXW2m7FRqp4T@postini.com; Tue, 28 Apr 2015 07:17:39 UTC Received: from git.pnsol.com ([172.20.5.238] helo=roam.smtp.pnsol.com) by mail.la.pnsol.com with esmtp (Exim 4.76) (envelope-from ) id 1Ymzlu-0004rz-Az; Tue, 28 Apr 2015 08:17:34 +0100 Received: from [172.20.5.109] by roam.smtp.pnsol.com with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1Ymzlk-0000C9-5l; Tue, 28 Apr 2015 07:17:24 +0000 Content-Type: multipart/alternative; boundary="Apple-Mail=_F940FC40-0216-4C23-A299-EEA1873BC44A" Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Neil Davies In-Reply-To: Date: Tue, 28 Apr 2015 08:17:37 +0100 Message-Id: References: <72DB0260-F0DF-426F-A3F3-ECF5D8AF228F@pnsol.com> <766042D4-0C90-4C77-9033-07B8E436C35B@pnsol.com> To: Jonathan Morton X-Mailer: Apple Mail (2.1878.6) Cc: bloat Subject: Re: [Bloat] Detecting bufferbloat from outside a node X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Apr 2015 07:17:42 -0000 --Apple-Mail=_F940FC40-0216-4C23-A299-EEA1873BC44A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Jonathan The timestamps don't change very quickly - dozens (or more) of packets = can have the same timestamp, so it doesn't give you the appropriate = discrimination power. Timed observations at key points gives you all you = need (actually, appropriately gathered they give you all you can = possibly know - by observation) Neil On 28 Apr 2015, at 00:11, Jonathan Morton wrote: > On 27 Apr 2015 23:31, "Neil Davies" wrote: > > > > Hi Jonathan > > > > On 27 Apr 2015, at 16:25, Jonathan Morton = wrote: > > > >> One thing that might help you here is the TCP Timestamps option. = The timestamps thus produced are opaque, but you can observe them and = measure the time intervals between their production and echo. You should = be able to infer something from that, with care. > >> > >> To determine the difference between loaded and unloaded states, you = may need to observe for an extended period of time. Eventually you'll = observe some sort of bulk flow, even if it's just a software update = cycle. It's not quite so certain that you'll observe an idle state, but = it is sufficient to observe an instance of the link not being completely = saturated, which is likely to occur at least occasionally. > >> > >> - Jonathan Morton > > > > We looked at using TCP timestamps early on in our work. The problem = is that they don't really help extract the fine-grained information = needed. The timestamps can move in very large steps, and the accuracy = (and precision) can vary widely from implementation to implementation. >=20 > Well, that's why you have to treat them as opaque, just like I said. = Ignore whatever meaning the end host producing them might embed in them, = and simply watch which ones get echoed back and when. You only have to = rely on the resolution of your own clocks. >=20 > > The timestamps are there to try and get a gross (if my memory serves = me right ~100ms) approximation to the RTT - not good enough for = reasoning about TCP based interactive/"real time" apps >=20 > On the contrary, these timestamps can indicate much better precision = than that; in particular they indicate an upper bound on the = instantaneous RTT which can be quite tight under favourable = circumstances. On a LAN, you could reliably determine that the RTT was = below 1ms this way. >=20 > Now, what it doesn't give you is a strict lower bound. But you can = often look at what's going on in that TCP stream and determine that = favourable circumstances exist, such that the upper bound RTT estimate = is probably reasonably tight. Or you could observe that the stream is = mostly idle, and thus probably influenced by delayed acks and Nagle's = algorithm, and discount that measurement accordingly. >=20 > - Jonathan Morton >=20 --Apple-Mail=_F940FC40-0216-4C23-A299-EEA1873BC44A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Jonathan

The timestamps don't = change very quickly - dozens (or more) of packets can have the same = timestamp, so it doesn't give you the appropriate discrimination power. = Timed observations at key points gives you all you need (actually, = appropriately gathered they give you all you can possibly know - by = observation)

Neil

On 28 = Apr 2015, at 00:11, Jonathan Morton <chromatix99@gmail.com> = wrote:

On 27 Apr 2015 23:31, "Neil Davies" <neil.davies@pnsol.com> = wrote:
>
> Hi Jonathan
>
> On 27 Apr 2015, at 16:25, Jonathan Morton <chromatix99@gmail.com> = wrote:
>
>> One thing that might help you here is the TCP Timestamps = option. The timestamps thus produced are opaque, but you can observe = them and measure the time intervals between their production and echo. = You should be able to infer something from that, with care.
>>
>> To determine the difference between loaded and unloaded states, = you may need to observe for an extended period of time. Eventually = you'll observe some sort of bulk flow, even if it's just a software = update cycle. It's not quite so certain that you'll observe an idle = state, but it is sufficient to observe an instance of the link not being = completely saturated, which is likely to occur at least = occasionally.
>>
>> - Jonathan Morton
>
> We looked at using TCP timestamps early on in our work. The problem = is that they don't really help extract the fine-grained information = needed. The timestamps can move in very large steps, and the accuracy = (and precision) can vary widely from implementation to = implementation.

Well, that's why you have to treat = them as opaque, just like I said. Ignore whatever meaning the end host = producing them might embed in them, and simply watch which ones get = echoed back and when. You only have to rely on the resolution of your = own clocks.

> The timestamps are there to try and = get a gross (if my memory serves me right ~100ms) approximation to the = RTT - not good enough for reasoning about TCP based interactive/"real = time" apps

On the contrary, these timestamps can = indicate much better precision than that; in particular they indicate an = upper bound on the instantaneous RTT which can be quite tight under = favourable circumstances. On a LAN, you could reliably determine that = the RTT was below 1ms this way.

Now, what it doesn't = give you is a strict lower bound. But you can often look at what's going = on in that TCP stream and determine that favourable circumstances exist, = such that the upper bound RTT estimate is probably reasonably tight. Or = you could observe that the stream is mostly idle, and thus probably = influenced by delayed acks and Nagle's algorithm, and discount that = measurement accordingly.

- Jonathan Morton


= --Apple-Mail=_F940FC40-0216-4C23-A299-EEA1873BC44A--