[Bloat] Internet Speed Measurement: Current Challenges and Future Recommendations

Fri May 17 03:36:05 EDT 2019

Jason is on the bloat list, nick feamster used to be, but isn't.
Sometimes I wish I could drag in various folk at the FCC, FTC, or some
other regulatory body more often. BITAG. Places like that. Having some
co-ordinated "Here are our comments on your thing" might be better
than the general chaos on the bloat list, and, since google is so
ineffective at indexing mailing lists these days some way to raise it
in the search results.

On Fri, May 17, 2019 at 12:01 AM Jonathan Foulkes
<jf at jonathanfoulkes.com> wrote:
>
> Thanks for sharing Dave.
>
> A good paper, but there are few gaps worthy of mentioning on this list:
>
> Testing when there is an AQM present means the test must adapt to the challenge of smaller cwnd existing for any one stream, therefore it will take many more streams to saturate a line with cwnd = 30 than if the cwnd is allowed to grow to >1,000
> In general, the impact of cwnd relative to saturation and impact on delay was not visited, and yet it’s critical. One of the reasons for spiky delays on high speed lines is the ginormous cwnds hogging the line with their 800ms+ RTT’s
>
> Asymmetry of provisioned upload relative to download, at some point, the ack-stream can be held up by either lack of capacity or bloat in the uplink. So even though a link can deliver 300Mbps down, a bloated uplink of 5mbps might never allow that level to be reached.
> There are ISPs provisioning truly crazy asymmetric service.
>
> They do make a good point about the local network, WiFi specifically being the new bottleneck, which is why we included an iperf instance that can be started on the IQrouter to help run client to server tests that help spot local network capacity limits, typically on WiFi.
>
> Regarding their point about ‘Cross traffic’ impact on measurements, Cake’s per-host / per-target fairness also complicates AQM-enabled testing from client devices. Which is why we make the built-in speed test the arbiter of true line capacity, as it factors for ALL traffic flowing through the router. But, as you mention, that is also a challenge from a CPU resource standpoint on higher speeds.
>
> The biggest gap in this paper is not paying sufficient attention to latency as a critical metric, and one that is controllable by an AQM. Bufferbloat metrics have more impact on end-user experience than +/- 50Mbps on a 100mbps baseline.
> I was rather miffed they do not even mention the DSLreports.com speedtest, or the fast.com test, as those are the two that provide a bufferbloat metric.

I keep hoping government will get it.

I note that I'm in general quite pleased with the overall trendlines
on dslreports (
http://www.dslreports.com/speedtest/results/bufferbloat?up=1 ). A lot
of the bloat on cablemodems disappearing is due to uplink speeds
getting much better, and holding buffer sizes constant, of course,
rather than the docsis 3.1 deployment or our efforts, but I do tend to
look at the long fuzzy blue data on the left of the green 30ms line
and think that's mostly us.

The thing that scares though, is that since that test is exaustively
used by those tuning their links with fq_codel, is that that dataset
is now quite polluted. So what does reality actually look like? On
this trip I encountered network after network essentially in a state
of congestion collapse.

The other big problems with dslreports is that the test does not run
for long enough at this higher rates, and that, without seeing the raw
data (just the inverse of what's being displayed, what dslreports is
throwing out), I worry. It's a cosmic background bufferbloat detector
that it's needed. Still dsl sucks, still there's data at +1 second
(With a cutoff of 4 sec that is an artifact of the test), those really
worry me.

> The industry as whole MUST pay attention and socialize the relevancy of managed latencies as being critical to customer satisfaction and good application performance. And that starts with tests that clearly grade that critical aspect.

One can wish. I think only the bufferbloat issue breaking out
thoroughly into the business press, in places like the economist or
cto or fortune, will make a difference here.

> Cheers,
>
> Jonathan
>
> > On May 15, 2019, at 3:58 AM, Dave Taht <dave.taht at gmail.com> wrote:
> >
> > If it helps any: Nick Feamster and Jason Livingood just published "
> > Internet Speed Measurement: Current Challenges and Future
> > Recommendations " ( https://arxiv.org/pdf/1905.02334.pdf ) a few days
> > ago, and outlines quite a few problems going forward at higher speeds.
> > I do wish the document had pointed out more clearly that router based
> > measurements have problems also, with weaker cpus unable to source
> > enough traffic for an accurate measurement, but I do hope this
> > document has impact, and it's a good read, regardless.
> >
> > Still, somehow getting it right at lower speeds is always on my mind.
> > I'd long ago hoped that DSL devices would adopt BQL, and that
> > cablemodems would also, thus moving packet processing a little higher
> > on the stack so more advanced algorithms like cake could take hold.
> >
> > On Wed, May 15, 2019 at 9:32 AM Sebastian Moeller <moeller0 at gmx.de> wrote:
> >>
> >> Hi All,
> >>
> >>
> >> I believe the following to be relevant to this discussion: https://apenwarr.ca/log/20180808
> >> Where he discusses a similar idea including implementation albeit aimed at lower bandwidth and sans the automatic bandwidth tracking.
> >>
> >>
> >>> On May 15, 2019, at 01:34, David P. Reed <dpreed at deepplum.com> wrote:
> >>>
> >>>
> >>> Ideally, it would need to be self-configuring, though... I.e., something
> >>> like the IQRouter auto-measuring of the upstream bandwidth to tune the
> >>> shaper.
> >>
> >> @Jonathan from your experience how tricky is it to get reliable speedtest endpoints and how reliable are they in practice. And do you do any sanitization, like take another measure immediate if the measured rate differs from the last by more than XX% or something like that?
> >>
> >>
> >>>
> >>> Sure, seems like this is easy to code because there are exactly two ports to measure, they can even be labeled physically "up" and "down" to indicate their function.
> >>
> >> IMHO the real challenge is automated measurements over the internet at Gbps speeds. It is not hard to get some test going (by e.g. tapping into ookla's fast net of confederated measurement endpoints) but getting something where the servers can reliably saturate 1Gbps+ seems somewhat trickier (last time I looked one required a 1Gbps connection to the server to participate in speedtest.net, obviously not really suited for measuring Gbps speeds).
> >> In the EU there exists a mandate for national regulators to establish and/or endorse an anointed "official" speedtests, untended to keep ISP marketing honest, that come with stricter guarantees (e.g. the official German speedtest, breitbandmessung.de will only admit tests if the servers are having sufficient bandwidth reserves to actually saturate the link; the enduser is required to select the speed-tier giving them a strong hint about the required rates I believe).
> >> For my back-burner toy project "per-packet-overhead estimation on arbitrary link technology" I am currently facing the same problem, I need a traffic sink and source that can reliably saturate my link so I can measure maximum achievable goodput, so if anybody in the list has ideas, I am all ears/eyes.
> >>
> >>>
> >>> For reference, the GL.iNet routers are tiny and nicely packaged, and run
> >>> OpenWrt; they do have one with Gbit ports[0], priced around $70. I very
> >>> much doubt it can actually push a gigabit, though, but I haven't had a
> >>> chance to test it. However, losing the WiFi, and getting a slightly
> >>> beefier SoC in there will probably be doable without the price going
> >>> over $100, no?
> >>>
> >>> I assume the WiFi silicon is probably the most costly piece of intellectual property in the system. So yeah. Maybe with the right parts being available, one could aim at $50 or less, without sales channel markup. (Raspberry Pi ARM64 boards don't have GigE, and I think that might be because the GigE interfaces are a bit pricey. However, the ARM64 SoC's available are typically Celeron-class multicore systems. I don't know why there aren't more ARM64 systems on a chip with dual GigE, but I suspect searching for them would turn up some).
> >>
> >> The turris MOX (https://www.turris.cz/en/specification/) might be a decent startimg point as it comes with one Gbethernet port and both a SGMII and a PCIe signals routed on a connector, they also have a 4 and an 8 port switch module, but for our purposes it might be possible to just create a small single Gb ethernet port board to get started.
> >>
> >> Best Regards
> >>        Sebastian
> >>
> >>>
> >>> -Toke
> >>>
> >>> [0] https://www.gl-inet.com/products/gl-ar750s/
> >>> _______________________________________________
> >>> Cerowrt-devel mailing list
> >>> Cerowrt-devel at lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> >>
> >> _______________________________________________
> >> Bloat mailing list
> >> Bloat at lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/bloat
> >
> >
> >
> > --
> >
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-205-9740
> > _______________________________________________
> > Bloat mailing list
> > Bloat at lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>

--

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740