From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id F123A3B29E for ; Thu, 17 Jun 2021 11:49:37 -0400 (EDT) Received: by mail-wr1-x433.google.com with SMTP id c5so7319436wrq.9 for ; Thu, 17 Jun 2021 08:49:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=IHuLBNq1dZN5GoXao+cC39aTJEm4XoJbhKIkK3q4m3Y=; b=PltC6uIB6dmu7tfP8ZIvMWkvRgefebSvrg30xwkKf1N3cnfnZPtF9r5Q1doXZNeONR x/gC1yYPxMrSOc/MXCsM7XJE9UzjttVsoc8ld9ySsrGoYBNPinw/88sLgxqDE+Nzjy0D kXxw2UizT07IHe/cWz4UrfaHR3uYeMpk6i4Ai3QizIze96pPBJ/iBgyfYFmi576AYiwu sIPNRzTxGIvJPnWTtmh1ttzbYtRY1HpHjxI4664NDPwMijmmkO8Eocgw1DDBnIA0ApAg PLh9usjknvrAiyPSbCMoc8bEt7P03/cKdCOEf83rV3EpGDYBzN+JaFRicZSKltdYTvbf I44w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=IHuLBNq1dZN5GoXao+cC39aTJEm4XoJbhKIkK3q4m3Y=; b=PC2ar7ne5kYYAgM+N5wgc3egpfPUZo1mgYJdm+9psvaUAdG9CTJQ61X1UvdqL0hmq2 uH9N9IQRmATK0dqKYJokGpwp54lq85V9EIWW67yr5oSGfNF+Dsh/r8wRI606B+XeZoQr h/j7umQb7wkKO/gm9u7D/Mu7Rtlt69kwXpkI+9foOG54sJAJK3Zaew/Tw7pHVW6TPFel M8zcWbfM/EYfr98uR9C4zvyzupL27WwYSrv5L9dHH8i1HVdG784AAHyHMe6x8INCobxU n+gMIL8vXJpb6CghczTrnx7Lb8OEx/yQarm7Rrgxs+NIPH2Yu88/0NPIg/rbDGr6PCyc djUg== X-Gm-Message-State: AOAM533cVdlzWq694IW8eHDalrXaot9wKxaf6C0NDr881fbND2SJoUzH 613DV+wkfKBvbqXAX0TlT59tns4Dn9kg32pf89cc0g== X-Google-Smtp-Source: ABdhPJz7y+qfIFQQT0ZgUMHpjrP4LtpW/+LhJ55UhYuSi0PFcWUVmuFmymAZE0gDsBCvxMtXromjxznFXaSKqQ4XCww= X-Received: by 2002:adf:d1e4:: with SMTP id g4mr6576949wrd.405.1623944976757; Thu, 17 Jun 2021 08:49:36 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matt Mathis Date: Thu, 17 Jun 2021 08:49:22 -0700 Message-ID: To: Nick Buraglio Cc: Dave Taht , starlink@lists.bufferbloat.net Content-Type: multipart/alternative; boundary="0000000000007a241705c4f827f8" X-Mailman-Approved-At: Thu, 17 Jun 2021 11:50:55 -0400 Subject: Re: [Starlink] plotting all the data X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jun 2021 15:49:38 -0000 --0000000000007a241705c4f827f8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Some time recently I read a casual paper (on Medium I think) that made the point that deep diving into outliers and understanding them has led to a half dozen Nobel prizes, because they lead to discoveries of phenomena that nobody else had even noticed. See for instinance the Holmdel Horn https://en.wikipedia.org/wiki/Holmdel_Horn_Antenna To keep sane, I tend to keep outliers and clip them as last as possible, e.g. by choice of graph axis. This way I have the opportunity to notice otherwise hidden patterns. In mlab data we sometimes see outliers that suggest "out of bounds" data rates. e.g. a repeated test that clearly has a max rate of 50 Mb/s or something, and then every so often a one test at 200 Mb/s or higher. My assumption is that these are from software managed shapers that occasionally fail to properly load their configurations. (I admit to not having looked hard enough to prove this hypnosis). Thanks, --MM-- The best way to predict the future is to create it. - Alan Kay We must not tolerate intolerance; however our response must be carefully measured: too strong would be hypocritical and risks spiraling out of control; too weak risks being mistaken for tacit approval. On Thu, Jun 17, 2021 at 7:25 AM Nick Buraglio wrote: > This is much more common in the high performance computing and networking > space (i.e. perfsonar, TWAMP, and OWAMP). I have also been pushing "gathe= r > and store all the data" for ....since I was an engineer working on the > Teragrid (which is where I first saw Matt's MTU talk around 2002 or 03, > BTW). =3D) > High fidelity plots of everything that can be gathered is laborious to > curate but is invaluable for so many reasons. Now we just need a way to > make it happen everywhere for everyone in a way that's easy. > > nb > > > On Thu, Jun 17, 2021 at 8:57 AM Dave Taht wrote: > >> Capturing and plotting *all* the data is often revealing. >> >> Sometimes plotting the data you are discarding (for what seems like sane >> reasons) is quite revealing. Saw this on slashdot this morning, it=E2= =80=99s >> good... >> >> >> https://www.newyorker.com/magazine/2021/06/21/when-graphs-are-a-matter-o= f-life-and-death >> >> In the bufferbloat effort I=E2=80=99ve fought time and time again for fo= lk to >> stop throwing out data above the 95 percentile, and at the very least pl= ot >> everything they threw out to find patterns... >> >> dslreports=E2=80=99 graphing tools, for example, throws out a ton of =E2= =80=9Coutliers" =E2=80=A6 >> and the only reason why there is no data past 4 sec here, is that the te= st >> doesn=E2=80=99t run long enough. >> >> http://www.dslreports.com/speedtest/results/bufferbloat?up=3D1 >> >> (been trying to get ahold of someone over there to buy their raw data fo= r >> years now. They have the biggest - 8 years worth - collection) >> >> mlabs has a similar data reduction issue that they haven=E2=80=99t got a= round to >> fixing. >> >> And more recently we encountered a smoothing problem in wireshark that >> made a halt in packet processing look more like a normal tcp cwnd cut=E2= =80=A6. >> >> _______________________________________________ >> Starlink mailing list >> Starlink@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/starlink >> > --0000000000007a241705c4f827f8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Some time recently I read a casual paper (on Medium I thin= k) that made the point that deep diving into outliers and understanding the= m has led to a half dozen Nobel prizes, because they lead to discoveries of= phenomena=C2=A0that nobody else had even noticed.=C2=A0 See for instinance= the Holmdel=C2=A0Horn=C2=A0https://en.wikipedia.org/wiki/Holmdel_Horn_Antenna
=
To keep sane, I tend to keep outliers and clip them as last = as possible, e.g. by choice of graph axis.=C2=A0 This way I have the opport= unity to notice otherwise hidden patterns.

In mlab= data we sometimes see outliers that suggest "out of bounds" data= rates.=C2=A0 e.g. a repeated=C2=A0test that clearly has a max rate of 50 M= b/s or something, and then every so often a one test at 200 Mb/s or higher.= =C2=A0 =C2=A0My assumption is that these are from software managed shapers = that occasionally fail to properly load their configurations.=C2=A0 =C2=A0 = =C2=A0 (I admit to not having looked hard enough to prove this hypnosis).

Thanks,
--MM--
The best way to predict the future is to create it. = =C2=A0- Alan Kay

We must not tolerate intolerance;
=C2=A0 =C2=A0 =C2=A0 =C2=A0however our response must be carefully meas= ured:=C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 too strong = would be hypocritical and risks spiraling out of control;
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 too weak risks being mistaken for tacit = approval.


On Thu, Jun 17, 2021= at 7:25 AM Nick Buraglio <buraglio@forwardingplane.net> wrote:
This is much more common = in the high performance computing and networking space (i.e. perfsonar, TWA= MP, and OWAMP). I have also been pushing "gather and store all the dat= a" for ....since I was an engineer working on the Teragrid (which is w= here I first saw Matt's MTU talk around 2002 or 03, BTW).=C2=A0 =3D)=C2= =A0
High fidelity plots of everything that can be gathered is laborious= to curate but is invaluable for so many reasons. Now we just need a way to= make it happen everywhere for everyone in a way that's easy.=C2=A0

nb


On Thu, Jun 17, 2021 at 8:57 A= M Dave Taht <dav= et@teklibre.net> wrote:
Capturing and plotting *all* the data is often revealing.
Sometimes plotting the data you are discarding (for what seems like sane re= asons) is quite revealing.=C2=A0 Saw this on slashdot this morning, it=E2= =80=99s good...

https://www.= newyorker.com/magazine/2021/06/21/when-graphs-are-a-matter-of-life-and-deat= h

In the bufferbloat effort I=E2=80=99ve fought time and time again for folk = to stop throwing out data above the 95 percentile, and at the very least pl= ot everything they threw out to find patterns...

dslreports=E2=80=99 graphing tools, for example, throws out a ton of =E2=80= =9Coutliers" =E2=80=A6 and the only reason why there is no data past 4= sec here, is that the test doesn=E2=80=99t run long enough.

http://www.dslreports.com/speedtest/re= sults/bufferbloat?up=3D1

(been trying to get ahold of someone over there to buy their raw data for y= ears now. They have the biggest - 8 years worth - collection)

mlabs has a similar data reduction issue that they haven=E2=80=99t got arou= nd to fixing.

And more recently we encountered a smoothing problem in wireshark that made= a halt in packet processing look more like a normal tcp cwnd cut=E2=80=A6.=

_______________________________________________
Starlink mailing list
Starlin= k@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/starlink
--0000000000007a241705c4f827f8--