From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 958FD3B29E for ; Wed, 22 Apr 2020 11:28:18 -0400 (EDT) Received: by mail-wm1-x32b.google.com with SMTP id x25so2838945wmc.0 for ; Wed, 22 Apr 2020 08:28:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ieee.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HEYcSxCtfV8F2mk4jxQNftDGgQrGGLx9v27bxt2HZmc=; b=OmDslKFJ7/A4fpZCl91PxDp//YbrkeVU6NmBhL/v//HB+NuC4SoGFxgjcW7+YM1aq1 PUzzE7MoUMEpeMUyOyXQRL3P/sQzfLiuaFFnPYTMnhltlb7wwKOas5OYScE5Rq0w0QUD cPPGIEutPSd5uZW0WnoOu+I33eQY092FplbZU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HEYcSxCtfV8F2mk4jxQNftDGgQrGGLx9v27bxt2HZmc=; b=X3ESs+Lf1dIDx31magGxcJyj1f5ITZA4qA33CtsyZwjePnyTdyUmAlWsaDSFnJBnN2 iqteyS5UzSE3N6M8l+8iGSCWTq4GbOv4CAtp9uw/OnOnnIcWcmGWAqYGscuy+znUVlAX UpuiVblpMJpu4YxdUhk6U8tmuAMo5F5dwXeSNWZvQRH5tejhRBfzLVlWhXfjnOM5ViBA 7v+BcdrmAlUknXeUA5cfFmb5pc43ScLzTurKf7M6nT7aDHtVmJf2aetx1D7IJJ0Ou6hm Ip9zevkqD9z/M43A4hHU/7gVzjcecglwGrVtIy3qbWY6hZKco6TYR2bve/muRkL5pCsB 7Nww== X-Gm-Message-State: AGi0PubvBxlZsPr6CBr/XwCLqEw5n0GJRQ3+G6pyiAQRlt4lgJZxDGCM 7r+SJ1Htk75FZt1q6iaFvyYP70+BYxNM4ZG184a1AA== X-Google-Smtp-Source: APiQypI7DqD6JadGb9ZElt4bIFSgANFDTcPt+NG/tNbSXXfShIqRixdwshX8YEVRgm6THZzfbeLf+TB1FsTf1csAiGI= X-Received: by 2002:a1c:c2d6:: with SMTP id s205mr11745031wmf.90.1587569297621; Wed, 22 Apr 2020 08:28:17 -0700 (PDT) MIME-Version: 1.0 References: <75FEC2D9-BFC8-4FA2-A972-D11A823C5528@gmail.com> <603DFF79-D0C0-41BD-A2FB-E40B95A9CBB0@gmail.com> In-Reply-To: From: Luca Muscariello Date: Wed, 22 Apr 2020 17:28:06 +0200 Message-ID: To: Dave Taht Cc: Jonathan Morton , Cake List , Maxime Bizon Content-Type: multipart/alternative; boundary="0000000000000ab5b205a3e2c88d" Subject: Re: [Cake] Advantages to tightly tuning latency X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Apr 2020 15:28:18 -0000 --0000000000000ab5b205a3e2c88d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Apr 22, 2020 at 4:48 PM Dave Taht wrote: > On Wed, Apr 22, 2020 at 2:04 AM Luca Muscariello > wrote: > > > > > > > > On Wed, Apr 22, 2020 at 12:44 AM Dave Taht wrote: > >> > >> On Tue, Apr 21, 2020 at 3:33 PM Jonathan Morton > wrote: > >> > > >> > > On 22 Apr, 2020, at 1:25 am, Thibaut wrote: > >> > > > >> > > My curiosity is piqued. Can you elaborate on this? What does > free.fr do? > >> > > >> > They're a large French ISP. They made their own CPE devices, and > debloated both them and their network quite a while ago. In that sense, = at > least, they're a model for others to follow - but few have. > >> > > >> > - Jonathan Morton > >> > >> they are one of the few ISPs that insisted on getting full source code > >> to their DSL stack, and retained the chops to be able to modify it. I > >> really admire their revolution v6 product. First introduced in 2010, > >> it's been continuously updated, did ipv6 at the outset, got fq_codel > >> when it first came out, and they update the kernel regularly. All > >> kinds of great features on it, and ecn is enabled by default for those > >> also (things like samba). over 3 million boxes now I hear.... > >> > >> with <1ms of delay in the dsl driver, they don't need to shape, they > >> just run at line rate using three tiers of DRR that look a lot like > >> cake. They shared their config with me, and before I lost heart for > >> future internet drafts, I'd stuck it here: > >> > >> > https://github.com/dtaht/bufferbloat-rfcs/blob/master/home_gateway_queue_= management/middle.mkd > >> > >> Occasionally they share some data with me. Sometimes I wish I lived in > >> paris just so I could have good internet! (their fiber offering is > >> reasonably buffered (not fq_codeled) and the wifi... maybe I can get > >> them to talk about what they did) > >> > >> When free.fr shipped fq_codel 2 months after we finalized it, I > >> figured the rest of the world was only months behind. How hard is it > >> to add 50 lines of BQL oriented code to a DSL firmware? > >> > > > > Free has been using SFQ since 2005 (if I remember well). > > They announced the wide deployment of SFQ in the free.fr newsgroup. > > Wi-Fi in the free.fr router was not as good though. > > They're working on it. :) > > > In Paris there is a lot of GPON now that is replacing DSL. But there is > > a nation-wide effort funded by local administrations to get fiber > > everywhere. There are small towns in the countryside with fiber. > > Public money has made, and is making that possible. > > There is still a little of Euro-DOCSIS, but frankly compared to fiber > > it has no chance to survive. > > I am very, very happy for y'all. Fiber has always been the sanest > thing. Is there > a SPF+ gpon card yet I can plug into a convention open source router yet? > > > > > I currently have 2Gbps/600Mbps access with orange.fr and free.fr has a > subscription > > at 10Gbps GPON. I won't tell you the price because you may feel depress= ed > > compared to other countries where prices are much higher. > > I'd emigrate!!! > > > The challenge becomes to keep up with these link rates in software > > as there is a lot of hardware offloading. > I just meant that these routers tend to use HW offloading and kernel qdiscs may be bypassed. > > At this point, I kind of buy the stanford sqrt(bdp) argument. All you > really need for gigE+ fiber access to work well > for most modern traffic is a fairly short fifo (say, 20ms). Any form > of FQ would help but be hardly noticible. I think > there needs to be work on the hop between the internet and the > subscriber... > > Web traffic is dominated by RTT above 40mbit (presently). > streaming video traffic - is no more than 20Mbit, and your occasional > big download is a dozen big streams that would > bounce off a short fifo well. > gbit access to the home is (admittedly glorious, wonderful!) overkill > for all present forms of traffic. > > I'm pretty sure if I had gig fiber I could come up with a way to use > it up (exiting the cloud entirely comes to mind), but > lacking new applications that demand that much bandwidth... > > I of course, would like to see lola ( https://lola.conts.it/ ) finally > work, and videoconferencing and game stream with high rates and faster > (even raw) encoding also has potential to reduce e2e latencies > enormously at that layer. > > > > > As soon as 802.11ax becomes the norm, software scheduling will become > > a challenge. > > Do you mean in fiber or wireless? wireless is really problematic at ANY > speed. > I meant that software scheduling becomes a challenge for the same reason as above. Increase in total throughput of the box will call for hardware offloading and kernel qdisc may be bypassed. It is not a challenge per se, it is a challenge because traffic may not be managed by the kernel. > > at gfiber, the buffering moved to the wifi, and there are other > problems that really impact achievable bandwidth. When I was last in > paris, I could "hear" 300+ access points from my apt, and could only > get 100-200kbit per second out of the wireless n ap I had, unless I > cheated and stuck my traffic in the VI queue. A friend of mine there, > couldn't even get wifi across the room! Beacons ate into a lot of the > available > bandwidth. Since 5ghz (and soon 6ghz - is 6E a thing in france) is > shorter range I'm hoping that's got better, but with > 802.11ac and ax peeing on half the wifi spectrum by default, I imagine > achievable rates in high density locations with many APs will be very > low... and very jittery... and thus still require good ATF, fq, and > aqm technologies. > > I have high hopes for OFDMA and DU but thus far haven't found an AP > doing it. I'm not sure what to do about the beaconing problem except > offer a free tradein to all my neighbors still emitting G style > frames.... > > And in looking over some preliminary code for the mt76 ax chip, I > worry about both bad design of the firmware, and > insufficient resources on-chip to manage well. > > How is the 5G rollout going in france? > Good question. I've just seen a speed test at Gbps on a phone which can drain your battery in less than 5 minutes. Amazing tech! > > I recently learned that much of japan is... wait for it... wimax. > > > > > Luca > > > > -- > Make Music, Not War > > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-435-0729 > --0000000000000ab5b205a3e2c88d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Wed, Apr 22, 2020 at 4:48 PM Dave Taht <= ;dave.taht@gmail.c= om> wrote:
muscariello@ieee.org> wrote: >
>
>
> On Wed, Apr 22, 2020 at 12:44 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> On Tue, Apr 21, 2020 at 3:33 PM Jonathan Morton <chromatix99@gmail.com> = wrote:
>> >
>> > > On 22 Apr, 2020, at 1:25 am, Thibaut <hacks@slashdirt.org> wrote= :
>> > >
>> > > My curiosity is piqued. Can you elaborate on this? What = does free.f= r do?
>> >
>> > They're a large French ISP.=C2=A0 They made their own CPE= devices, and debloated both them and their network quite a while ago.=C2= =A0 In that sense, at least, they're a model for others to follow - but= few have.
>> >
>> >=C2=A0 - Jonathan Morton
>>
>> they are one of the few ISPs that insisted on getting full source = code
>> to their DSL stack, and retained the chops to be able to modify it= . I
>> really admire their revolution v6 product. First introduced in 201= 0,
>> it's been continuously updated, did ipv6 at the outset, got fq= _codel
>> when it first came out, and they update the kernel regularly. All<= br> >> kinds of great features on it, and ecn is enabled by default for t= hose
>> also (things like samba). over 3 million boxes now I hear....
>>
>> with <1ms of delay in the dsl driver, they don't need to sh= ape, they
>> just run at line rate using three tiers of DRR that look a lot lik= e
>> cake. They shared their config with me, and before I lost heart fo= r
>> future internet drafts, I'd stuck it here:
>>
>> https://github.com/dtaht/bufferbloat-rfcs/blob/master/home_gateway_queue= _management/middle.mkd
>>
>> Occasionally they share some data with me. Sometimes I wish I live= d in
>> paris just so I could have good internet! (their fiber offering is=
>> reasonably buffered (not fq_codeled) and the wifi... maybe I can g= et
>> them to talk about what they did)
>>
>> When free.fr shipped fq_codel 2 months after we finalized it, I
>> figured the rest of the world was only months behind. How hard is = it
>> to add 50 lines of BQL oriented code to a DSL firmware?
>>
>
> Free has been using SFQ since 2005 (if I remember well).
> They announced the wide deployment of SFQ in the free.fr newsgroup.
> Wi-Fi in the free.fr router was not as good though.

They're working on it. :)

> In Paris there is a lot of GPON now that is replacing DSL. But there i= s
> a nation-wide effort funded by local administrations to get fiber
> everywhere. There are small towns in the countryside with fiber.
> Public money has made, and is making that possible.
> There is still a little of Euro-DOCSIS, but frankly compared to fiber<= br> > it has no chance to survive.

I am very, very happy for y'all. Fiber has always been the sanest
thing. Is there
a SPF+ gpon card yet I can plug into a convention open source router yet?
>
> I currently have 2Gbps/600Mbps access with orange.fr and free.fr has a subscriptio= n
> at 10Gbps GPON. I won't tell you the price because you may feel de= pressed
> compared to other countries where prices are much higher.

I'd emigrate!!!

> The challenge becomes to keep up with these link rates in software
> as there is a lot of hardware offloading.


I just meant that these routers tend=C2=A0to use HW offloading=C2= =A0
and k= ernel qdiscs may be bypassed.

=C2=A0

At this point, I kind of buy the stanford sqrt(bdp) argument. All you
really need for gigE+ fiber access to work well
for most modern traffic is a fairly short fifo (say, 20ms). Any form
of FQ would help but be hardly noticible. I think
there needs to be work on the hop between the internet and the subscriber..= .

Web traffic is dominated by RTT above 40mbit (presently).
streaming video traffic - is no more than 20Mbit, and your occasional
big download is a dozen big streams that would
bounce off a short fifo well.
gbit access to the home is (admittedly glorious, wonderful!) overkill
for all present forms of traffic.

I'm pretty sure if I had gig fiber I could come up with a way to use it up (exiting the cloud entirely comes to mind), but
lacking new applications that demand that much bandwidth...

I of course, would like to see lola ( https://lola.conts.it/ ) finally
work, and videoconferencing and game stream with high rates and faster
(even raw) encoding also has potential to reduce e2e latencies
enormously at that layer.

>
> As soon as 802.11ax becomes the norm, software scheduling will become<= br> > a challenge.

Do you mean in fiber or wireless? wireless is really problematic at ANY spe= ed.

I meant that software scheduling becomes a chall= enge for the same
reason as above. Increase in total throughput of the box
will call for hard= ware offloading and kernel qdisc may be bypassed.

It is not a challenge per se, it is a = challenge because traffic
may not be managed by the kernel.


=
=C2=A0

at gfiber, the buffering moved to the wifi, and there are other
problems that really impact achievable bandwidth. When I was last in
paris, I could "hear" 300+ access points from my apt, and could o= nly
get 100-200kbit per second out of the wireless n ap I had, unless I
cheated and stuck my traffic in the VI queue. A friend of mine there,
couldn't even get wifi across the room! Beacons ate into a lot of the available
bandwidth. Since 5ghz (and soon 6ghz - is 6E a thing in france) is
shorter range I'm hoping that's got better, but with
802.11ac and ax peeing on half the wifi spectrum by default, I imagine
achievable rates in high density locations with many APs will be very
low... and very jittery... and thus still require good ATF, fq, and
aqm technologies.

I have high hopes for OFDMA and DU but thus far haven't found an AP
doing it. I'm not sure what to do about the beaconing problem except offer a free tradein to all my neighbors still emitting G style
frames....

And in looking over some preliminary code for the mt76 ax chip, I
worry about both bad design of the firmware, and
insufficient resources on-chip to manage well.

How is the 5G rollout going in france?

=
Good question.= I've just seen a speed test at Gbps on a phone
which can drain your battery in = less than 5 minutes. Amazing tech!

=C2=A0

I recently learned that much of japan is... wait for it... wimax.

>
> Luca



--
Make Music, Not War

Dave T=C3=A4ht
CTO, TekLibre, LLC
ht= tp://www.teklibre.com
Tel: 1-831-435-0729
--0000000000000ab5b205a3e2c88d--