From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com [IPv6:2607:f8b0:4864:20::734]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 96F853B29E for ; Thu, 17 Jun 2021 10:25:04 -0400 (EDT) Received: by mail-qk1-x734.google.com with SMTP id 5so3281538qkc.8 for ; Thu, 17 Jun 2021 07:25:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=forwardingplane-net.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=JFi7twsCfpgAT3KTc/HdO1olpC+hjyfSJB9NS4oQWDk=; b=oUGlRbPeoLNFaFtJjlbX3kBArPp4cxxs87l3u0sCzHNLpM1BcRe3S/BDW22NIgGgRv xerYQ7hi2iwEJJuZY/h09OtCyHo8DGK7evGkOga15JPZlYj7MrhsIGJrzj7EMDzcRX6Y OFAo/WqVqTCZqg6H41qgAFCCob54eBK9uaYEftYgG8IYHmXSynAox4l1L8Ogr7kJfCyL LGnbjPFZ2g6YqGbgQdwjRZG5oY9f6KTYXdYSazm9CXKNH/XvLHmfaI/Jl18+9AK1QS/x FhOnc0zOX88ZesJsh2rE5bHTf6jq1p12vbOkMg6Buqz3B+S+hh32gpmbgZQHZDqlej6A prrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=JFi7twsCfpgAT3KTc/HdO1olpC+hjyfSJB9NS4oQWDk=; b=AkTHVuccDOUyB2uM0D4/t1IzxaPN7sHFvSFA7HpYLx/Rksusd6IGssn+T4itybLusZ 19x6ezrN5SXw5owyjFMuvRJFPlF4HtCvxqmYxUd2HscGOumodpiy4qdb/+jBsV0sYXEN McMLcLvsgJuyiZCGZjj0Jyi/aTpGZLVOIeCJugBw7A6EwipY6LGsmkO8vrNdf2ZA8NSs T+7cmOWHyJI2S4weXvmAeXmQfC4+KoMFqzjPFBrs/xifORz/jhQ9rFxsc7ZsvycDGbwK mqM9VUJvFSKabGFWzCpNu7cU9IVejPuK0DyAA+pSLmF6/PoSOIqre9NrldKinoP7mbJA Dc4g== X-Gm-Message-State: AOAM531wUQsX1i1lvtshVRt4VMDCxzZ7tVOLbUqt1k9WbXQ82oEoITFW z6uNClAq63zNLY5rCGU9ktZuFm3y3JAYrTc0iw3D7w== X-Google-Smtp-Source: ABdhPJxsFyvHMVfg4+bX1qtnmiK0/tWYqE1IpoVi4tFJfR91LZAq53Yl5Rp/mG8ThbtnzkJkfXuZ//f3oDRt+BmVBCg= X-Received: by 2002:a37:a5d6:: with SMTP id o205mr3996573qke.118.1623939903858; Thu, 17 Jun 2021 07:25:03 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Nick Buraglio Date: Thu, 17 Jun 2021 09:24:53 -0500 Message-ID: To: Dave Taht Cc: starlink@lists.bufferbloat.net, Matt Mathis Content-Type: multipart/alternative; boundary="0000000000001b414705c4f6f99d" Subject: Re: [Starlink] plotting all the data X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jun 2021 14:25:04 -0000 --0000000000001b414705c4f6f99d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable This is much more common in the high performance computing and networking space (i.e. perfsonar, TWAMP, and OWAMP). I have also been pushing "gather and store all the data" for ....since I was an engineer working on the Teragrid (which is where I first saw Matt's MTU talk around 2002 or 03, BTW). =3D) High fidelity plots of everything that can be gathered is laborious to curate but is invaluable for so many reasons. Now we just need a way to make it happen everywhere for everyone in a way that's easy. nb On Thu, Jun 17, 2021 at 8:57 AM Dave Taht wrote: > Capturing and plotting *all* the data is often revealing. > > Sometimes plotting the data you are discarding (for what seems like sane > reasons) is quite revealing. Saw this on slashdot this morning, it=E2=80= =99s > good... > > > https://www.newyorker.com/magazine/2021/06/21/when-graphs-are-a-matter-of= -life-and-death > > In the bufferbloat effort I=E2=80=99ve fought time and time again for fol= k to stop > throwing out data above the 95 percentile, and at the very least plot > everything they threw out to find patterns... > > dslreports=E2=80=99 graphing tools, for example, throws out a ton of =E2= =80=9Coutliers" =E2=80=A6 > and the only reason why there is no data past 4 sec here, is that the tes= t > doesn=E2=80=99t run long enough. > > http://www.dslreports.com/speedtest/results/bufferbloat?up=3D1 > > (been trying to get ahold of someone over there to buy their raw data for > years now. They have the biggest - 8 years worth - collection) > > mlabs has a similar data reduction issue that they haven=E2=80=99t got ar= ound to > fixing. > > And more recently we encountered a smoothing problem in wireshark that > made a halt in packet processing look more like a normal tcp cwnd cut=E2= =80=A6. > > _______________________________________________ > Starlink mailing list > Starlink@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/starlink > --0000000000001b414705c4f6f99d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
This is much more common in the high performance computing= and networking space (i.e. perfsonar, TWAMP, and OWAMP). I have also been = pushing "gather and store all the data" for ....since I was an en= gineer working on the Teragrid (which is where I first saw Matt's MTU t= alk around 2002 or 03, BTW).=C2=A0 =3D)=C2=A0
High fidelity plots of ev= erything that can be gathered is laborious to curate but is invaluable for = so many reasons. Now we just need a way to make it happen everywhere for ev= eryone in a way that's easy.=C2=A0

nb


On Thu, Jun 17, 2021 at 8:57 AM Dave Taht <davet@teklibre.net> wrote:
Capturing and plotting *all* the dat= a is often revealing.

Sometimes plotting the data you are discarding (for what seems like sane re= asons) is quite revealing.=C2=A0 Saw this on slashdot this morning, it=E2= =80=99s good...

https://www.= newyorker.com/magazine/2021/06/21/when-graphs-are-a-matter-of-life-and-deat= h

In the bufferbloat effort I=E2=80=99ve fought time and time again for folk = to stop throwing out data above the 95 percentile, and at the very least pl= ot everything they threw out to find patterns...

dslreports=E2=80=99 graphing tools, for example, throws out a ton of =E2=80= =9Coutliers" =E2=80=A6 and the only reason why there is no data past 4= sec here, is that the test doesn=E2=80=99t run long enough.

http://www.dslreports.com/speedtest/re= sults/bufferbloat?up=3D1

(been trying to get ahold of someone over there to buy their raw data for y= ears now. They have the biggest - 8 years worth - collection)

mlabs has a similar data reduction issue that they haven=E2=80=99t got arou= nd to fixing.

And more recently we encountered a smoothing problem in wireshark that made= a halt in packet processing look more like a normal tcp cwnd cut=E2=80=A6.=

_______________________________________________
Starlink mailing list
Starlin= k@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink
--0000000000001b414705c4f6f99d--