From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp121.ord1d.emailsrvr.com (smtp121.ord1d.emailsrvr.com [184.106.54.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 07AF53B29E for ; Thu, 16 May 2019 18:01:39 -0400 (EDT) Received: from smtp16.relay.ord1d.emailsrvr.com (localhost [127.0.0.1]) by smtp16.relay.ord1d.emailsrvr.com (SMTP Server) with ESMTP id B782340246; Thu, 16 May 2019 18:01:39 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=g001.emailsrvr.com; s=20190322-9u7zjiwi; t=1558044099; bh=ROWwouPT9QmA3EyNwAao4Pnybwm6fN/+9yzW2rRLADA=; h=Subject:From:Date:To:From; b=Nv+x8m6wylPVsJXPJHsr7QkBwPp5K7BOZbvlBlnAoVDIj2Z0qnyPPrcG23J2NizdJ nCl4j0E5Jjwo7L8/6zh2uaqW65aVWsxXt0o2L8DjNkKGFCLeUnDq7/dFaxYdFPLbrP 8tk9vsv7XI/OolT0Hupy07To5cZQb2WXd+DwJMhE= X-Auth-ID: jf@jonathanfoulkes.com Received: by smtp16.relay.ord1d.emailsrvr.com (Authenticated sender: jf-AT-jonathanfoulkes.com) with ESMTPSA id 3222B40091; Thu, 16 May 2019 18:01:39 -0400 (EDT) X-Sender-Id: jf@jonathanfoulkes.com Received: from jonathans-mbp.lan (h26.135.213.151.dynamic.ip.windstream.net [151.213.135.26]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:25 (trex/5.7.12); Thu, 16 May 2019 18:01:39 -0400 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) From: Jonathan Foulkes In-Reply-To: Date: Thu, 16 May 2019 18:01:38 -0400 Cc: Sebastian Moeller , Rich Brown , cerowrt-devel , "David P. Reed" , bloat Content-Transfer-Encoding: quoted-printable Message-Id: <2D23494F-7B4B-4426-AC57-8CFEA993D60B@jonathanfoulkes.com> References: <2936.1557856670@turing-police> <1557859131.759530583@apps.rackspace.com> <1557871532.754117608@apps.rackspace.com> <87lfz81x7b.fsf@toke.dk> <1557876841.69888745@apps.rackspace.com> <25460D05-4F53-4317-9722-2878B160BD7B@gmx.de> To: Dave Taht X-Mailer: Apple Mail (2.3445.104.8) Subject: Re: [Cerowrt-devel] [Bloat] (no subject) X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2019 22:01:40 -0000 Thanks for sharing Dave. A good paper, but there are few gaps worthy of mentioning on this list: Testing when there is an AQM present means the test must adapt to the = challenge of smaller cwnd existing for any one stream, therefore it will = take many more streams to saturate a line with cwnd =3D 30 than if the = cwnd is allowed to grow to >1,000 In general, the impact of cwnd relative to saturation and impact on = delay was not visited, and yet it=E2=80=99s critical. One of the reasons = for spiky delays on high speed lines is the ginormous cwnds hogging the = line with their 800ms+ RTT=E2=80=99s Asymmetry of provisioned upload relative to download, at some point, the = ack-stream can be held up by either lack of capacity or bloat in the = uplink. So even though a link can deliver 300Mbps down, a bloated uplink = of 5mbps might never allow that level to be reached. There are ISPs provisioning truly crazy asymmetric service. They do make a good point about the local network, WiFi specifically = being the new bottleneck, which is why we included an iperf instance = that can be started on the IQrouter to help run client to server tests = that help spot local network capacity limits, typically on WiFi. Regarding their point about =E2=80=98Cross traffic=E2=80=99 impact on = measurements, Cake=E2=80=99s per-host / per-target fairness also = complicates AQM-enabled testing from client devices. Which is why we = make the built-in speed test the arbiter of true line capacity, as it = factors for ALL traffic flowing through the router. But, as you mention, = that is also a challenge from a CPU resource standpoint on higher = speeds. The biggest gap in this paper is not paying sufficient attention to = latency as a critical metric, and one that is controllable by an AQM. = Bufferbloat metrics have more impact on end-user experience than +/- = 50Mbps on a 100mbps baseline.=20 I was rather miffed they do not even mention the DSLreports.com = speedtest, or the fast.com test, as those are the two that provide a = bufferbloat metric. The industry as whole MUST pay attention and socialize the relevancy of = managed latencies as being critical to customer satisfaction and good = application performance. And that starts with tests that clearly grade = that critical aspect.=20 Cheers, Jonathan > On May 15, 2019, at 3:58 AM, Dave Taht wrote: >=20 > If it helps any: Nick Feamster and Jason Livingood just published " > Internet Speed Measurement: Current Challenges and Future > Recommendations " ( https://arxiv.org/pdf/1905.02334.pdf ) a few days > ago, and outlines quite a few problems going forward at higher speeds. > I do wish the document had pointed out more clearly that router based > measurements have problems also, with weaker cpus unable to source > enough traffic for an accurate measurement, but I do hope this > document has impact, and it's a good read, regardless. >=20 > Still, somehow getting it right at lower speeds is always on my mind. > I'd long ago hoped that DSL devices would adopt BQL, and that > cablemodems would also, thus moving packet processing a little higher > on the stack so more advanced algorithms like cake could take hold. >=20 > On Wed, May 15, 2019 at 9:32 AM Sebastian Moeller = wrote: >>=20 >> Hi All, >>=20 >>=20 >> I believe the following to be relevant to this discussion: = https://apenwarr.ca/log/20180808 >> Where he discusses a similar idea including implementation albeit = aimed at lower bandwidth and sans the automatic bandwidth tracking. >>=20 >>=20 >>> On May 15, 2019, at 01:34, David P. Reed = wrote: >>>=20 >>>=20 >>> Ideally, it would need to be self-configuring, though... I.e., = something >>> like the IQRouter auto-measuring of the upstream bandwidth to tune = the >>> shaper. >>=20 >> @Jonathan from your experience how tricky is it to get reliable = speedtest endpoints and how reliable are they in practice. And do you do = any sanitization, like take another measure immediate if the measured = rate differs from the last by more than XX% or something like that? >>=20 >>=20 >>>=20 >>> Sure, seems like this is easy to code because there are exactly two = ports to measure, they can even be labeled physically "up" and "down" to = indicate their function. >>=20 >> IMHO the real challenge is automated measurements over the internet = at Gbps speeds. It is not hard to get some test going (by e.g. tapping = into ookla's fast net of confederated measurement endpoints) but getting = something where the servers can reliably saturate 1Gbps+ seems somewhat = trickier (last time I looked one required a 1Gbps connection to the = server to participate in speedtest.net, obviously not really suited for = measuring Gbps speeds). >> In the EU there exists a mandate for national regulators to establish = and/or endorse an anointed "official" speedtests, untended to keep ISP = marketing honest, that come with stricter guarantees (e.g. the official = German speedtest, breitbandmessung.de will only admit tests if the = servers are having sufficient bandwidth reserves to actually saturate = the link; the enduser is required to select the speed-tier giving them a = strong hint about the required rates I believe). >> For my back-burner toy project "per-packet-overhead estimation on = arbitrary link technology" I am currently facing the same problem, I = need a traffic sink and source that can reliably saturate my link so I = can measure maximum achievable goodput, so if anybody in the list has = ideas, I am all ears/eyes. >>=20 >>>=20 >>> For reference, the GL.iNet routers are tiny and nicely packaged, and = run >>> OpenWrt; they do have one with Gbit ports[0], priced around $70. I = very >>> much doubt it can actually push a gigabit, though, but I haven't had = a >>> chance to test it. However, losing the WiFi, and getting a slightly >>> beefier SoC in there will probably be doable without the price going >>> over $100, no? >>>=20 >>> I assume the WiFi silicon is probably the most costly piece of = intellectual property in the system. So yeah. Maybe with the right parts = being available, one could aim at $50 or less, without sales channel = markup. (Raspberry Pi ARM64 boards don't have GigE, and I think that = might be because the GigE interfaces are a bit pricey. However, the = ARM64 SoC's available are typically Celeron-class multicore systems. I = don't know why there aren't more ARM64 systems on a chip with dual GigE, = but I suspect searching for them would turn up some). >>=20 >> The turris MOX (https://www.turris.cz/en/specification/) might be a = decent startimg point as it comes with one Gbethernet port and both a = SGMII and a PCIe signals routed on a connector, they also have a 4 and = an 8 port switch module, but for our purposes it might be possible to = just create a small single Gb ethernet port board to get started. >>=20 >> Best Regards >> Sebastian >>=20 >>>=20 >>> -Toke >>>=20 >>> [0] https://www.gl-inet.com/products/gl-ar750s/ >>> _______________________________________________ >>> Cerowrt-devel mailing list >>> Cerowrt-devel@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >>=20 >> _______________________________________________ >> Bloat mailing list >> Bloat@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/bloat >=20 >=20 >=20 > --=20 >=20 > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat