From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DCFD93CB42; Thu, 23 Mar 2023 14:23:46 -0400 (EDT) Received: by mail-yb1-xb29.google.com with SMTP id r187so1848528ybr.6; Thu, 23 Mar 2023 11:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679595826; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=IwST1yHGD5Is+wVHLaYC3vzF0IZhK3S9cDZ15IMtD+s=; b=AdJUGHmEOK1K1/5JhORU2NJcKa6BfH027UbBUPBeaAgUvq/5TcfSsrsqGcvN7b/GrY 85EA+IzbMaiUamnZQsuxmJyZobsXFgKJMQM9hcSDKavQdLQvg62Ifi6k1iGPVZeehZhN RQtNIltxKdlG9TijkVMfHa+nlEgQwgTVxNtNMChVG+1AK5/u62n3x87HSyfbRBWnLXU3 fI/ulO7EKcKTI+bwQyaX64Oozqtx5tWDuQPMP6v/c755QPrRDO/YNbdronid93FuA1ct elHwOBZAU2rN2gRnsHRGG5pHiSowEFXVbH9FqJBQDe9u/hAMctj/oLzbolqS24napTG+ CW/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679595826; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=IwST1yHGD5Is+wVHLaYC3vzF0IZhK3S9cDZ15IMtD+s=; b=xV58BTyr6vV62jMxVGVgDpHqKZ0Cbk3/v6E5g7hx8gCi6Ur6OiyHOE5RmRoZbQf4Ec JRS3UU7N5GisDiXCCahIBQzkWBijE+C9dVdest4CR879OeCn0lh+ilTRiEvqmGQvKKOB N9LKSHLKHe8yzmh+09qTWWgFINQkO0lwSKNPOsnUtdENYtx6UNWx8ekcUVlYAK4kBHyi /0I9hWjbZDgEbwDcu+FsJpgyaoxReWR3ez275MC1bJgOVp8pJSMJ8KfppQBb6N2VExyK D2D985G3y6Ro8lVC8oWi80BeNNkBmakxAtDZy86OII4zeoS7YD7pAE/1w+fXjUZXiRkB /y+A== X-Gm-Message-State: AAQBX9f/iq131Gb+58RVlp1eKfZOAizkhW6Bbw0f9nuOD32/T2brsXIj 7HZB34DekPGZ2QzpFrj9cMbVZaOhzRBRQfJAaQs= X-Google-Smtp-Source: AKy350YkZ6OByR5Rwz7yXxlEprpXf8HA/HRD1iycYSUAZjSJoJ4Od+P2rMojn+NsIR6jOSaeZgiEvrsJ6Fd4EQdrcuk= X-Received: by 2002:a05:6902:1083:b0:b67:d295:d614 with SMTP id v3-20020a056902108300b00b67d295d614mr2409313ybu.12.1679595826017; Thu, 23 Mar 2023 11:23:46 -0700 (PDT) MIME-Version: 1.0 References: <20230321001019.GA4531@sunf68.rd.bbc.co.uk> <46A0D5B3-FFAA-40A3-B94C-6D01C99D92EB@gmail.com> In-Reply-To: From: dan Date: Thu, 23 Mar 2023 12:23:35 -0600 Message-ID: To: Sebastian Moeller Cc: Rich Brown , Frantisek Borsik , bloat , libreqos Content-Type: multipart/alternative; boundary="00000000000093799305f79560ae" Subject: Re: [Bloat] [Starlink] Annoyed at 5/1 Mbps... X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2023 18:23:47 -0000 --00000000000093799305f79560ae Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable All TDMA has to keep all clients connected in each scheduling window so it's at a minimum of '(client count x .5RTT) + root->client broadcast'. most of the time it's actually client count x RTT because lots of TDMA tech only talks to one client per frame. ie, all 802.11ac and earlier radios, even most proprietary products. 'ax wifi or more specifically OFDMA based products are the better .5RTT numbers. Active ethernet is not this. it's a 1:1 full duplex link hop to hop. If that's home runs great, but if it's a routed network then we will always be a multiple of the copper ports on each side x hops plus whatever routing delays in the routers/switches. all adds up to more latency but via different paths. I can tell you that putting 10 hardware accelerated routers (I used Mikrotik NP16 in my bend test) adds up to about the same latency as a GPON system. On Tue, Mar 21, 2023 at 1:04=E2=80=AFPM Sebastian Moeller = wrote: > Hi Dan, > > > > On Mar 21, 2023, at 18:22, dan wrote: > > > > GPON is TDMA so the latency is going to be at a minimum the RTT * > connected ONUs, vs DSL which is a fixed ratio/scheduler. > > Assuming no proactive grants... are these a thing in PON or only > in DOCSIS?, but since GPON frames can be shared between ONUs how do you > derive the "RTT * connected ONUs" formula? > > > > Standard GPON deployments are typically well over 1 second to the OLT. > > I read that as millisecond, which would mean 8 GPON frames... for > sending the request, processing and arbitrating all requests, assign > transmit slots and send the transmit maps back to the ONUs, which then > actually need to send the packets... RTT should not be all that noticeabl= e, > at 20 Km the wave propagation of light in fiber would be around > 2*(20000/300000000 * 3/2)*1000 =3D 0.2 milliseconds... (not sure what a > realistic maximum length for a PON tree is, which probably depends on a > number of things anyway, but google says up to 20 Km for GPON)... but tha= t > RTT would be the same for active ethernet... > > > Not that it's bad or anything, but in comparison GPON has very > 'wireless' like best case latency but without the wireless variances. > > All centrally scheduled link layers will have similar challenges = I > guess? > > Kind Regards > Sebastian > > > > > On Tue, Mar 21, 2023 at 6:31=E2=80=AFAM Sebastian Moeller > wrote: > > I have to push back gently on this... > > > > XG(S)-PON is gross 10Gbps (after FEC you are left with around 8,6 Gbps)= , > Noki's proprietary (aka not ITU) @% Gbps PON seems to be abbreviated > 25GS-PON. > > > > Now XGS-PON allows maximally 128 end-nodes in the tree, so: > > 8600/128 =3D 67.18 Mbps/subscriber > > > > unless the ISPs royally screwed up the configuration there should be a > CIR per subscriber of around 60 Mbps. So setting your cake shaper to 50 > Mbps shpuld give you: > > a) 10 times the throughput of the 5/1 Mbps DSL (ignoring overhead > compensation for a change, which likely will be in favor of PON) > > b) decent low latency, round robin delay for full MTU packets between > 128 active nodes would be: > > packet/sec: ((8.6 * 1000^3)/(1500*8)) =3D 716666.666667 > > millisec/packet: 1000 / ((8.6 * 1000^3)/(1500*8)) =3D > 0.00139534883721 > > round-robin delay 128: 128 * 1000 / ((8.6 * 1000^3)/(1500*8)) = =3D > 0.178604651163 milliseconds... > > > > DSL uses a 4KHz clock so 1000/4000 =3D 0.25 millisecond > quantization > > So XGS-PON has at least theoretical potential to deliver lower latency > than DSL, but the details depend on if/how packets are aggregated. HOWEVE= R > the 125=C2=B5sec GPON frames can be shared between different ONUs in upst= ream > and downstream direction... so these are not a hard quantisation but more > the interval between control information required for the access grant > cycle... > > > > c) robustness against RF noise sources and electricity/lightning > > > > So I am not su sure I would prefer the 5/1 (A)DSL over a PON... > > > > That however is orthogonal to me preferring a competent ISP that takes > care of keeping latency under load at bay. > > > > > > > > > On Mar 21, 2023, at 12:26, Rich Brown via Starlink < > starlink@lists.bufferbloat.net> wrote: > > > > > > > > > > > >> On Mar 21, 2023, at 1:21 AM, Frantisek Borsik via Rpm < > rpm@lists.bufferbloat.net> wrote: > > >> > > >> Now, I hope to really piss You off with the following statement :-P > but: > > >> > > >> even sub 5/1 Mbps =E2=80=9Cbroadband=E2=80=9D in Africa with bufferb= loat fixed on as > many hops along the internet journey from a data center to the customers > mobile device (or with just LibreQoS middle box in the ISP=E2=80=99s netw= ork) is > feeling way better than 25Gbps XG-PON. The only time the XG-PON guy could > really feel like a king of the world would be during his speedtest. > > > > > > Nope. Sorry - this doesn't piss me off :-) It's just true. > > > > > > - 7mbps/768kbps DSL with an IQrouter works fine for two simultaneous > Zoom conferences. (Even though no one would think that it's fast.) > > > - I recommend people on a budget drop their ISP speed so they can > afford a router that does SQM > https://forum.openwrt.org/t/so-you-have-500mbps-1gbps-fiber-and-need-a-ro= uter-read-this-first/90305/40 > > > > Even simpler, even on a 100Gbps link nobody stops you from > setting your shaper to 50/10 if that is all your router can deliver (and = I > agree if there are cheaper plans closer to the 50/10 it makes economic > sense to scale down the plan)... > > > > > > > > > > The people that get annoyed are those who just upgraded to 1Gbps > service and still are getting fragged in their games. > > > > > > Rich > > > _______________________________________________ > > > Starlink mailing list > > > Starlink@lists.bufferbloat.net > > > https://lists.bufferbloat.net/listinfo/starlink > > > > --00000000000093799305f79560ae Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
All TDMA has to keep all clients connected in each schedul= ing window so it's at a minimum of '(client count x .5RTT)=C2=A0+ r= oot->client broadcast'.=C2=A0 most of the time it's actually cli= ent count x RTT because lots of TDMA tech only talks to one client per fram= e.=C2=A0 ie, all 802.11ac and earlier radios, even most proprietary product= s.=C2=A0 'ax wifi or more specifically OFDMA based products are the bet= ter .5RTT numbers.

Active ethernet is not this.=C2=A0 it's a 1:1= full duplex link hop to hop.=C2=A0 If that's home runs great, but if i= t's a routed network then we will always be a multiple of the copper po= rts on each side x hops plus whatever routing delays in the routers/switche= s.=C2=A0 all adds up to more latency but via different paths.

I can = tell you that putting 10 hardware accelerated=C2=A0routers (I used Mikrotik= NP16 in my bend test) adds up to about the same latency as a GPON system.<= br>



On Tue, Mar 21, 2023 at 1:04=E2=80=AFPM Sebastian Moeller &= lt;moeller0@gmx.de> wrote:
Hi Dan,


> On Mar 21, 2023, at 18:22, dan <dandenson@gmail.com> wrote:
>
> GPON is TDMA so the latency is going to be at a minimum the RTT * conn= ected ONUs, vs DSL which is a fixed ratio/scheduler.=C2=A0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 Assuming no proactive grants... are these a thi= ng in PON or only in DOCSIS?, but since GPON frames can be shared between O= NUs how do you derive the "RTT * connected ONUs" formula?


> Standard GPON deployments are typically well over 1 second to the OLT.=

=C2=A0 =C2=A0 =C2=A0 =C2=A0 I read that as millisecond, which would mean 8 = GPON frames... for sending the request, processing and arbitrating all requ= ests, assign transmit slots and send the transmit maps back to the ONUs, wh= ich then actually need to send the packets... RTT should not be all that no= ticeable, at 20 Km the wave propagation of light in fiber would be around 2= *(20000/300000000 * 3/2)*1000 =3D 0.2 milliseconds... (not sure what a real= istic maximum length for a PON tree is, which probably depends on a number = of things anyway, but google says up to 20 Km for GPON)... but that RTT wou= ld be the same for active ethernet...

>=C2=A0 Not that it's bad or anything, but in comparison GPON has ve= ry 'wireless' like best case latency but without the wireless varia= nces.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 All centrally scheduled link layers will have s= imilar challenges I guess?

Kind Regards
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Sebastian

>
> On Tue, Mar 21, 2023 at 6:31=E2=80=AFAM Sebastian Moeller <moeller0@gmx.de> wrote= :
> I have to push back gently on this...
>
> XG(S)-PON is gross 10Gbps (after FEC you are left with around 8,6 Gbps= ), Noki's proprietary (aka not ITU) @% Gbps PON seems to be abbreviated= 25GS-PON.
>
> Now XGS-PON allows maximally 128 end-nodes in the tree, so:
> 8600/128 =3D 67.18 Mbps/subscriber
>
> unless the ISPs royally screwed up the configuration there should be a= CIR per subscriber of around 60 Mbps. So setting your cake shaper to 50 Mb= ps shpuld give you:
> a) 10 times the throughput of the 5/1 Mbps DSL (ignoring overhead comp= ensation for a change, which likely will be in favor of PON)
> b) decent low latency, round robin delay for full MTU packets between = 128 active nodes would be:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0packet/sec: ((8.6 * 1000^3)/(1500*8))= =3D 716666.666667
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0millisec/packet: 1000 / ((8.6 * 1000^= 3)/(1500*8)) =3D 0.00139534883721
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0round-robin delay 128: 128 * 1000 / (= (8.6 * 1000^3)/(1500*8)) =3D 0.178604651163 milliseconds...
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0DSL uses a 4KHz clock so 1000/4000 = =3D 0.25 millisecond quantization
> So XGS-PON has at least theoretical potential to deliver lower latency= than DSL, but the details depend on if/how packets are aggregated. HOWEVER= the 125=C2=B5sec GPON frames can be shared between different ONUs in upstr= eam and downstream direction... so these are not a hard quantisation but mo= re the interval between control information required for the access grant c= ycle...
>
> c) robustness against RF noise sources and electricity/lightning
>
> So I am not su sure I would prefer the 5/1 (A)DSL over a PON...
>
> That however is orthogonal to me preferring a competent ISP that takes= care of keeping latency under load at bay.
>
>
>
> > On Mar 21, 2023, at 12:26, Rich Brown via Starlink <starlink@lists.bu= fferbloat.net> wrote:
> >
> >
> >
> >> On Mar 21, 2023, at 1:21 AM, Frantisek Borsik via Rpm <rpm@lists.buffe= rbloat.net> wrote:
> >>
> >> Now, I hope to really piss You off with the following stateme= nt=C2=A0 :-P but:
> >>
> >> even sub 5/1 Mbps =E2=80=9Cbroadband=E2=80=9D in Africa with = bufferbloat fixed on as many hops along the internet journey from a data ce= nter to the customers mobile device (or with just LibreQoS middle box in th= e ISP=E2=80=99s network) is feeling way better than 25Gbps XG-PON. The only= time the XG-PON guy could really feel like a king of the world would be du= ring his speedtest.
> >
> > Nope. Sorry - this doesn't piss me off :-) It's just true= .
> >
> > - 7mbps/768kbps DSL with an IQrouter works fine for two simultane= ous Zoom conferences. (Even though no one would think that it's fast.)<= br> > > - I recommend people on a budget drop their ISP speed so they can= afford a router that does SQM https://forum.openwrt.org/t/so-you-have-5= 00mbps-1gbps-fiber-and-need-a-router-read-this-first/90305/40
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Even simpler, even on a 100Gbps link = nobody stops you from setting your shaper to 50/10 if that is all your rout= er can deliver (and I agree if there are cheaper plans closer to the 50/10 = it makes economic sense to scale down the plan)...
>
>
> >
> > The people that get annoyed are those who just upgraded to 1Gbps = service and still are getting fragged in their games.
> >
> > Rich
> > _______________________________________________
> > Starlink mailing list
> > Starlink@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/st= arlink
>

--00000000000093799305f79560ae--