From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-f98.google.com (mail-la0-f98.google.com [209.85.215.98]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 9076021F22A for ; Tue, 28 Apr 2015 03:22:33 -0700 (PDT) Received: by labmn9 with SMTP id mn9so2852930lab.0 for ; Tue, 28 Apr 2015 03:22:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=1/HLqKLTQA7vy6NyhPNovm+HJ0bMByM0OzJ3DxAq8E4=; b=Rg0IExYVyLPXbZt5I9TGZlaSkj77hh5E1t5UjT75wNdjoCkTAfr8d5hmbXHdUkrYva S7Ha4hjJluD6rqFOHWyl5xfqYXgh6++nzBLYzdSdvF0mSe6cFcUBmeUPrNvQjpMQVdXl hP0t0F6GnGVdKzOkwBKZeo/l7NYAR3PONDtJJNz1bPJ1pyR7bazUBgD9pfWvQ7MU2aun simyTiXTC1JmcGO8Mhx7iMuOv5cc3jzHdothJPw0716U9i7XFmiUpaprjgv/3uUZhG/L uBbSpCn9R9mFC4fp+XqOTvdalXRuKO9bs0yav30GySZhRSZxfhOwjmbEtk5hKwfsnf2o d1lw== X-Gm-Message-State: ALoCoQm/KZyA9SZqqDU0xrYhBZzS/sgwkEzn8x3Lrr5jXUlIf7WHXD+vIhhKWfyfn3ekKy064Ko41bGh42Iu4HFQHck1cKZ1WQ== X-Received: by 10.194.185.229 with SMTP id ff5mr30580515wjc.30.1430216551581; Tue, 28 Apr 2015 03:22:31 -0700 (PDT) Received: from mail.la.pnsol.com (eu1sys200aog124.obsmtp.com. [207.126.144.157]) by mx.google.com with SMTPS id dj3sm549795wib.0.2015.04.28.03.22.30 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 28 Apr 2015 03:22:31 -0700 (PDT) X-Relaying-Domain: pnsol.com Received: from mail.la.pnsol.com ([89.145.213.110]) (using TLSv1) by eu1sys200aob124.postini.com ([207.126.147.11]) with SMTP ID DSNKVT9fZuw7xYAF5i8KMhuQ3hq4MHnT6K+d@postini.com; Tue, 28 Apr 2015 10:22:30 UTC Received: from ba6-office.pnsol.com ([172.20.5.199]) by mail.la.pnsol.com with esmtp (Exim 4.76) (envelope-from ) id 1Yn2en-0005La-P4; Tue, 28 Apr 2015 11:22:25 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Neil Davies In-Reply-To: Date: Tue, 28 Apr 2015 11:23:23 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <2F4DCB53-1E46-4829-B2F8-F8131664D1FF@pnsol.com> References: <72DB0260-F0DF-426F-A3F3-ECF5D8AF228F@pnsol.com> <766042D4-0C90-4C77-9033-07B8E436C35B@pnsol.com> To: Sebastian Moeller X-Mailer: Apple Mail (2.1878.6) Cc: Jonathan Morton , bloat Subject: Re: [Bloat] Detecting bufferbloat from outside a node X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Apr 2015 10:23:03 -0000 On 28 Apr 2015, at 10:58, Sebastian Moeller wrote: > Hi Neil, >=20 >=20 > On Apr 28, 2015, at 09:17 , Neil Davies wrote: >=20 >> Jonathan >>=20 >> The timestamps don't change very quickly - dozens (or more) of = packets can have the same timestamp, so it doesn't give you the = appropriate discrimination power. Timed observations at key points gives = you all you need (actually, appropriately gathered they give you all you = can possibly know - by observation) >=20 > But this has two issues: > 1) =E2=80=9Ctimed observations=E2=80=9D: relatively easy if all nodes = are under your control otherwise hard. I know about the CERN paper, but = they had all nodes under their control, symmetric bandwidth and shipload = of samples, so over the wild internet =E2=80=9Ctimed observations=E2=80=9D= are still hard (and harder as the temporal precision requirement goes = up) =E2=88=86Q (with its improper CDF semantics and G,S and V basis set) has = composition and de-composisition properties - this means that you = don=E2=80=99t need to be able to observe everywhere - even in Lucian=E2=80= =99s case his observation points were limited (certain systems) - the = rest of the analysis is derived using the properies of the =E2=88=86Q = calculus. Lucian also demonstrated how the standard timing observations (which = include issues of clock drift and distributed accuracy) can be resolved = in a practical situation - he reproduced - starting from libpcap = captures on machines - results that CERN guys build specialist h/w with = better than 20ns timing only 5 years before. The good thing about Lucian=E2=80=99s thesis is that it is in the public = domain - but we use the same approach over wide (i.e world) networks and = get same properties (unfortunately that is done in a commercial = context). This all arises because we can perform the appropriate = measurement error analysis, and hence use standard statistical = techniques. >=20 > 2) =E2=80=9Ckey points=E2=80=9D: once you know the key points you = already must have a decent understanding on the effective topology of = the network, which again over the wider internet is much harder than if = one has all nodes under control. Not really - the key points (as a start) are the end ones - and those = you have (reasonable) access to - and even if you don=E2=80=99t have = access to the *actual* end points - you can easily spin up a measurement = point that is very close (in =E2=88=86Q terms) to the ones you are = interested in - AWS and Google Compute are your friends here. >=20 >=20 > I am not sure how Paolo=E2=80=99s =E2=80=9Cno-touching=E2=80=9D = problem fits into the requirements for your deltaQ (meta-)math ;) I see =E2=80=9Cno touching=E2=80=9D as =E2=80=9Cno modification=E2=80=9D = - you can=E2=80=99t deduce information in the absence of data - what you = need to understand is the minimum data requirements to achieve the = measurement outcome - =E2=88=86Q calculus gives you that handle. >=20 > Best Regards > Sebastian >=20 >>=20 >> Neil >>=20 >> On 28 Apr 2015, at 00:11, Jonathan Morton = wrote: >>=20 >>> On 27 Apr 2015 23:31, "Neil Davies" wrote: >>>>=20 >>>> Hi Jonathan >>>>=20 >>>> On 27 Apr 2015, at 16:25, Jonathan Morton = wrote: >>>>=20 >>>>> One thing that might help you here is the TCP Timestamps option. = The timestamps thus produced are opaque, but you can observe them and = measure the time intervals between their production and echo. You should = be able to infer something from that, with care. >>>>>=20 >>>>> To determine the difference between loaded and unloaded states, = you may need to observe for an extended period of time. Eventually = you'll observe some sort of bulk flow, even if it's just a software = update cycle. It's not quite so certain that you'll observe an idle = state, but it is sufficient to observe an instance of the link not being = completely saturated, which is likely to occur at least occasionally. >>>>>=20 >>>>> - Jonathan Morton >>>>=20 >>>> We looked at using TCP timestamps early on in our work. The problem = is that they don't really help extract the fine-grained information = needed. The timestamps can move in very large steps, and the accuracy = (and precision) can vary widely from implementation to implementation. >>>=20 >>> Well, that's why you have to treat them as opaque, just like I said. = Ignore whatever meaning the end host producing them might embed in them, = and simply watch which ones get echoed back and when. You only have to = rely on the resolution of your own clocks. >>>=20 >>>> The timestamps are there to try and get a gross (if my memory = serves me right ~100ms) approximation to the RTT - not good enough for = reasoning about TCP based interactive/"real time" apps >>>=20 >>> On the contrary, these timestamps can indicate much better precision = than that; in particular they indicate an upper bound on the = instantaneous RTT which can be quite tight under favourable = circumstances. On a LAN, you could reliably determine that the RTT was = below 1ms this way. >>>=20 >>> Now, what it doesn't give you is a strict lower bound. But you can = often look at what's going on in that TCP stream and determine that = favourable circumstances exist, such that the upper bound RTT estimate = is probably reasonably tight. Or you could observe that the stream is = mostly idle, and thus probably influenced by delayed acks and Nagle's = algorithm, and discount that measurement accordingly. >>>=20 >>> - Jonathan Morton >>>=20 >>=20 >> _______________________________________________ >> Bloat mailing list >> Bloat@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/bloat >=20