[Cake] Advantages to tightly tuning latency

Wed Apr 22 13:42:54 EDT 2020

Having asymmetric gigabit cable modem service (1 Gb/s down) and very short latencies (5 ms.) to many servers of interest that can source 1 Gb/s), I would just comment that I find it very, very useful for "normal" use.

Perhaps my point is this: "normal" isn't a narrow gaussian distribution of performance needs. It's what might be called a time-varying long tailed distribution.

I pay for 1 gb/sec because it is "worth it" to download from, say, github cloning or a docker container image in under 1 second.

To think that isn't valuable is to miss the point that the Internet's performance isn't about isochronous flows or slow FTPs - it's not about throughput. It's about service delay.

And congestion control is about mitigating service delays under load, by eliminating sustained queueing delays that build up due to multiplexed use otherwise.

To talk about one use at a time, and treat an average throughput as the goal metric is to miss the entire point.

A home access connection is frequently multiplexed over unrelated uses. If you are single, live in your own apartment, ... you have a very, very warped idea of real usage.
On Wednesday, April 22, 2020 11:28am, "Luca Muscariello" <muscariello at ieee.org> said:

On Wed, Apr 22, 2020 at 4:48 PM Dave Taht <[ dave.taht at gmail.com ]( mailto:dave.taht at gmail.com )> wrote:On Wed, Apr 22, 2020 at 2:04 AM Luca Muscariello <[ muscariello at ieee.org ]( mailto:muscariello at ieee.org )> wrote:
 >
 >
 >
 > On Wed, Apr 22, 2020 at 12:44 AM Dave Taht <[ dave.taht at gmail.com ]( mailto:dave.taht at gmail.com )> wrote:
 >>
 >> On Tue, Apr 21, 2020 at 3:33 PM Jonathan Morton <[ chromatix99 at gmail.com ]( mailto:chromatix99 at gmail.com )> wrote:
 >> >
 >> > > On 22 Apr, 2020, at 1:25 am, Thibaut <[ hacks at slashdirt.org ]( mailto:hacks at slashdirt.org )> wrote:
 >> > >
 >> > > My curiosity is piqued. Can you elaborate on this? What does [ free.fr ]( http://free.fr ) do?
 >> >
 >> > They're a large French ISP.  They made their own CPE devices, and debloated both them and their network quite a while ago.  In that sense, at least, they're a model for others to follow - but few have.
 >> >
 >> >  - Jonathan Morton
 >>
 >> they are one of the few ISPs that insisted on getting full source code
 >> to their DSL stack, and retained the chops to be able to modify it. I
 >> really admire their revolution v6 product. First introduced in 2010,
 >> it's been continuously updated, did ipv6 at the outset, got fq_codel
 >> when it first came out, and they update the kernel regularly. All
 >> kinds of great features on it, and ecn is enabled by default for those
 >> also (things like samba). over 3 million boxes now I hear....
 >>
 >> with <1ms of delay in the dsl driver, they don't need to shape, they
 >> just run at line rate using three tiers of DRR that look a lot like
 >> cake. They shared their config with me, and before I lost heart for
 >> future internet drafts, I'd stuck it here:
 >>
 >> [ https://github.com/dtaht/bufferbloat-rfcs/blob/master/home_gateway_queue_management/middle.mkd ]( https://github.com/dtaht/bufferbloat-rfcs/blob/master/home_gateway_queue_management/middle.mkd )
 >>
 >> Occasionally they share some data with me. Sometimes I wish I lived in
 >> paris just so I could have good internet! (their fiber offering is
 >> reasonably buffered (not fq_codeled) and the wifi... maybe I can get
 >> them to talk about what they did)
 >>
 >> When [ free.fr ]( http://free.fr ) shipped fq_codel 2 months after we finalized it, I
 >> figured the rest of the world was only months behind. How hard is it
 >> to add 50 lines of BQL oriented code to a DSL firmware?
 >>
 >
 > Free has been using SFQ since 2005 (if I remember well).
 > They announced the wide deployment of SFQ in the [ free.fr ]( http://free.fr ) newsgroup.
 > Wi-Fi in the [ free.fr ]( http://free.fr ) router was not as good though.

 They're working on it. :)

 > In Paris there is a lot of GPON now that is replacing DSL. But there is
 > a nation-wide effort funded by local administrations to get fiber
 > everywhere. There are small towns in the countryside with fiber.
 > Public money has made, and is making that possible.
 > There is still a little of Euro-DOCSIS, but frankly compared to fiber
 > it has no chance to survive.

 I am very, very happy for y'all. Fiber has always been the sanest
 thing. Is there
 a SPF+ gpon card yet I can plug into a convention open source router yet?

 >
 > I currently have 2Gbps/600Mbps access with [ orange.fr ]( http://orange.fr ) and [ free.fr ]( http://free.fr ) has a subscription
 > at 10Gbps GPON. I won't tell you the price because you may feel depressed
 > compared to other countries where prices are much higher.

 I'd emigrate!!!

 > The challenge becomes to keep up with these link rates in software
 > as there is a lot of hardware offloading.

I just meant that these routers tend to use HW offloading 
and kernel qdiscs may be bypassed.

 At this point, I kind of buy the stanford sqrt(bdp) argument. All you
 really need for gigE+ fiber access to work well
 for most modern traffic is a fairly short fifo (say, 20ms). Any form
 of FQ would help but be hardly noticible. I think
 there needs to be work on the hop between the internet and the subscriber...

 Web traffic is dominated by RTT above 40mbit (presently).
 streaming video traffic - is no more than 20Mbit, and your occasional
 big download is a dozen big streams that would
 bounce off a short fifo well.
 gbit access to the home is (admittedly glorious, wonderful!) overkill
 for all present forms of traffic.

 I'm pretty sure if I had gig fiber I could come up with a way to use
 it up (exiting the cloud entirely comes to mind), but
 lacking new applications that demand that much bandwidth...

 I of course, would like to see lola ( [ https://lola.conts.it/ ]( https://lola.conts.it/ ) ) finally
 work, and videoconferencing and game stream with high rates and faster
 (even raw) encoding also has potential to reduce e2e latencies
 enormously at that layer.

 >
 > As soon as 802.11ax becomes the norm, software scheduling will become
 > a challenge.

 Do you mean in fiber or wireless? wireless is really problematic at ANY speed.

I meant that software scheduling becomes a challenge for the same
reason as above. Increase in total throughput of the box
will call for hardware offloading and kernel qdisc may be bypassed.
It is not a challenge per se, it is a challenge because traffic
may not be managed by the kernel.

 at gfiber, the buffering moved to the wifi, and there are other
 problems that really impact achievable bandwidth. When I was last in
 paris, I could "hear" 300+ access points from my apt, and could only
 get 100-200kbit per second out of the wireless n ap I had, unless I
 cheated and stuck my traffic in the VI queue. A friend of mine there,
 couldn't even get wifi across the room! Beacons ate into a lot of the
 available
 bandwidth. Since 5ghz (and soon 6ghz - is 6E a thing in france) is
 shorter range I'm hoping that's got better, but with
 802.11ac and ax peeing on half the wifi spectrum by default, I imagine
 achievable rates in high density locations with many APs will be very
 low... and very jittery... and thus still require good ATF, fq, and
 aqm technologies.

 I have high hopes for OFDMA and DU but thus far haven't found an AP
 doing it. I'm not sure what to do about the beaconing problem except
 offer a free tradein to all my neighbors still emitting G style
 frames....

 And in looking over some preliminary code for the mt76 ax chip, I
 worry about both bad design of the firmware, and
 insufficient resources on-chip to manage well.

 How is the 5G rollout going in france?

Good question. I've just seen a speed test at Gbps on a phone
which can drain your battery in less than 5 minutes. Amazing tech!

 I recently learned that much of japan is... wait for it... wimax.

 >
 > Luca

 -- 
 Make Music, Not War

 Dave Täht
 CTO, TekLibre, LLC
[ http://www.teklibre.com ]( http://www.teklibre.com )
 Tel: 1-831-435-0729
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20200422/108021c2/attachment-0001.html>