From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B40CB3CB44 for ; Wed, 1 May 2024 17:12:41 -0400 (EDT) Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-2ae913878b0so5418397a91.2 for ; Wed, 01 May 2024 14:12:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ieee.org; s=google; t=1714597961; x=1715202761; darn=lists.bufferbloat.net; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=ZxiWbDL+HAgdRyrAqMMTgmZCb4BoalldtPZJfvb/jlo=; b=F/7Nv0wOYi10m3rgRikynF82ymqygY9dIjRrSYhuAY23x3bDMl+KPSWK3BFRoPkcj+ ojG9stdUL5PMMIJGOVOOlUEK2sJGqp/amaduBUCGW2D8fY5qvUyVDT5wnWN012fDNSw1 dUjqTImZjJ2cRlDvNNyb0uAiH6EclEsuog4jM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714597961; x=1715202761; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZxiWbDL+HAgdRyrAqMMTgmZCb4BoalldtPZJfvb/jlo=; b=s3PKhh2T+w7qKRS4EZsf96nRodsxtTiR+xyeJ3M9pnSvKmmvC/1NlgrgF3+9b7aavC a4zNIfxwEs48UkhoLpSEa5ykG+Es1XG4c1NPd6cqN9VMM8jJ8auhsnAown+els/UXXUk Dfy/Rw8fWmI1c514fUEg57kAJTLZqNpUmE2AIi3UFGRVdirH7xHckrW546IoJtyfg7Yh 7/jO1FyqMZFkS91l3lmYUr1FqfdQAY7E4OpeYiekZFw5lvjYrZKsTNyS+0tKWrr/w6om C9y4BOokCKZglpseYkDxylrsxG8C7CG5MEJZ/byRm+/LbG9BABol7uS2khpB2XhVntd3 Zyzw== X-Forwarded-Encrypted: i=1; AJvYcCUmOLuN399h6DytoRJNNdVl/XxPiGm3MeHtrFZpjb0f5+xjHHBQmeBciPlqvfYwZ67iMynRU5kR5nEAcpkn+pnoVuc2URDvzDegemH8uf4= X-Gm-Message-State: AOJu0Yz1m5cV1+im2AR6vRKNr9Nv2lYit2mVY9Q0aHESoom6+12ye1kV 37USlKnaIfJHRuRZWHhTYCp94+j3kNZ09DsxT56hjP9UoYCxllpXhYJ2Pe+DRXjQkvWWWpZCUjM 3bg== X-Google-Smtp-Source: AGHT+IHDwotWchIkhujm3XtvSz+slVv7OzIQdFjiuMoN/yQEkqMblLCKAudLyqaSo5mCELvRVo15pw== X-Received: by 2002:a17:90a:d0c2:b0:2b2:a5aa:917b with SMTP id y2-20020a17090ad0c200b002b2a5aa917bmr3287679pjw.3.1714597960461; Wed, 01 May 2024 14:12:40 -0700 (PDT) Received: from smtpclient.apple (dhcp-72-253-194-45.hawaiiantel.net. [72.253.194.45]) by smtp.gmail.com with ESMTPSA id sl15-20020a17090b2e0f00b002b05e390c59sm1782572pjb.27.2024.05.01.14.12.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 May 2024 14:12:40 -0700 (PDT) From: Eugene Y Chang Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_0A00AC7F-ADE7-4EC7-A3CE-3B41C3C7FCC8"; protocol="application/pgp-signature"; micalg=pgp-sha256 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.8\)) Date: Wed, 1 May 2024 11:12:37 -1000 In-Reply-To: <6qop2p3o-351p-788q-q1q2-86sosnq3rn21@ynat.uz> Cc: Eugene Y Chang , Jim Forster , Dave Taht via Starlink , Colin_Higbie To: David Lang References: <438B1BC4-D465-497A-B6BA-700E1D411036@ieee.org> <79C02ABB-B2A6-4B4D-98F4-6540D3F96EBB@ieee.org> <7E918B58-382A-4793-A144-13A7075CA56C@connectivitycap.com> <13rq2389-9012-p95n-s494-q3pp070s497n@ynat.uz> <6qop2p3o-351p-788q-q1q2-86sosnq3rn21@ynat.uz> X-Mailer: Apple Mail (2.3696.120.41.1.8) Subject: Re: [Starlink] =?utf-8?q?It=CA=BCs_the_Latency=2C_FCC?= X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2024 21:12:41 -0000 --Apple-Mail=_0A00AC7F-ADE7-4EC7-A3CE-3B41C3C7FCC8 Content-Type: multipart/alternative; boundary="Apple-Mail=_DE5CFA24-FBB6-4776-9C0B-DF211148F960" --Apple-Mail=_DE5CFA24-FBB6-4776-9C0B-DF211148F960 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Thank you David. Now, shifting the focus a bit. Would a gamer experience some improvement = if they made a change in their router? What needs to be done for a gamer to get tangible improvement? Gene ---------------------------------------------- Eugene Chang IEEE Life Senior Member IEEE Communications Society & Signal Processing Society, Hawaii Chapter Chair IEEE Life Member Affinity Group Hawaii Chair IEEE Entrepreneurship, Mentor eugene.chang@ieee.org m 781-799-0233 (in Honolulu) > On May 1, 2024, at 9:18 AM, David Lang wrote: >=20 > On Wed, 1 May 2024, Eugene Y Chang wrote: >=20 >> Thanks David, >>=20 >>=20 >>> On Apr 30, 2024, at 6:12 PM, David Lang wrote: >>>=20 >>> On Tue, 30 Apr 2024, Eugene Y Chang wrote: >>>=20 >>>> I=E2=80=99m not completely up to speed on the gory details. Please = humor me. I am pretty good on the technical marketing magic. >>>>=20 >>>> What is the minimum configuration of an ISP infrastructure where we = can show an A/B (before and after) test? >>>> It can be a simplified scenario. The simpler, the better. We can = talk through the issues of how minimal is adequate. Of course and ISP = engineer will argue against simplicity. >>>=20 >>> I did not see a very big improvement on a 4/.5 dsl link, but there = was improvement. >>=20 >> Would a user feel the improvement with a 10 minute session: >> shopping on Amazon? >> using Salesforce? >> working with a shared Google doc? >=20 > When it's only a single user, they are unlikely to notice any = difference. >=20 > But if you have one person on zoom, a second downloading something, = and a third on Amazon, it doesn't take much to notice a difference. >=20 >>> if you put openwrt on the customer router and configure cake with = the targeted bandwith at ~80% of line speed, you will usually see a = drastic improvement for just about any connection. >>=20 >> Are you saying some of the benefits can be realized with just = upgrading the subscriber=E2=80=99s router? This makes adoption harder = because the subscriber will lose the ISP=E2=80=99s support for any = connectivity issues. If a demo impresses the subscribers, the ISP still = needs to embrace this change; otherwise the ISP will wash their hands of = any subscriber problems. >=20 > Yes, just upgrading the subscriber's device with cake and configuring = it appropriately largely solves the problem (at the cost of sacraficing = bandwith because cake isn't working directly on the data flowing from = the ISP to the client, and so it has to work indirectly to get the = Internet server to slow down instead and that's a laggy, imperect = work-around. If the ISPs router does active queue management with = fq_codel, then you don't have to do this. >=20 > This is how we know this works, many of use have been doing this for = years (see the bufferbloat mailing list and it's archives_ >=20 >>> If you can put fq_codel on both ends of the link, you can usually = skip capping the bandwidth. >>=20 >> This is good if this means the benefits can be achieved with just the = CPE. This also limits the changes to subscribers that care. >=20 > fq_codel on the ISPs router for downlink, and on the subscribers = router for uplink. >=20 > putting cake on the router on the subscriber's end and tuning it = appropriately can achieve most of the benefit, but is more work to = configure. >=20 >>>=20 >>> unfortunantly, it's not possible to just add this to the ISPs = existing hardware without having the source for the firmware there (and = if they have their queues in ASICs it's impossible to change them. >>=20 >> Is this just an alternative to having the change at the CPE? >> Yes this is harder for routers in the network. >=20 > simple fq_codel on both ends of the bottleneck connection works quite = well without any configuration. Cake adds some additional fairness = capabilities and has a mode to work around the router on the other end = of the bottleneck not doing active queue management >=20 >>> If you can point at the dramatic decrease in latency, with no = bandwidth losses, that Starlink has achieved on existing hardware, that = may help. >>=20 >> This is good to know for the engineers. This adds confusion with the = subscribers. >>=20 >>>=20 >>> There are a number of ISPs around the world that have implemented = active queue management and report very good results from doing so. >>=20 >> Can we get these ISPs to publically report how they have achieved = great latency reduction? >> We can help them get credit for caring about their subscribers. It = would/could be a (short term) competitive advantage. >> Of course their competitors will (might) adopt these changes and = eliminate the advantage, BUT the subscribers will retain glow of the = initial marketing for a much longer time. >=20 > several of them have done so, I think someone else posted a report = from one in this thread. >=20 >>> But showing that their existing hardware can do it when their = upstream vendor doesn't support it is going to be hard. >>=20 >> Is the upstream vendor a network provider or a computing center? >> Getting good latency from the subscriber, through the access network = to the edge computing center and CDNs would be great. The CDNs would = harvest the benefits. The other computing configurations would have make = the change to be competitive. >=20 > I'm talking about the manufacturer of the routers that the ISPs deploy = at the last hop before getting to the subscriber, and the router on the = subscriber end of the link (although most of those are running some = variation of openWRT, so turning it on would not be significant work for = the manufacturer) >=20 >> We wouild have done our part at pushing the next round of adoption. >=20 > Many of us have been pushing this for well over a decade. Getting = Starlink's attention to address their bufferbloat issues is a major = success. >=20 > David Lang >=20 >> Gene >>=20 >>>=20 >>> David Lang >>>=20 >>>>=20 >>>> We will want to show the human visible impact and not debate good = or not so good measurements. If we get the business and community = subscribers on our side, we win. >>>>=20 >>>> Note: >>>> Stage 1 is to show we have a pure software fix (that can work on = their hardware). The fix is =E2=80=9Cso dramatic=E2=80=9D that = subscribers can experience it without debating measurements. >>>> Stage 2 discusses why the ISP should demand that their equipment = vendors add this software. (The software could already be available, but = the ISP doesn=E2=80=99t think it is worth the trouble to enable it.) = Nothing will happen unless we stay engaged. We need to keep the = subscribers engaged, too. >>>>=20 >>>> Should we have a conference call to discuss this? >>>>=20 >>>>=20 >>>> Gene >>>> ---------------------------------------------- >>>> Eugene Chang >>>> IEEE Life Senior Member >>>>=20 >>>>=20 >>>>=20 >>>>> On Apr 30, 2024, at 3:52 PM, Jim Forster = wrote: >>>>>=20 >>>>> Gene, David, >>>>> =E2=80=98m >>>>> Agreed that the technical problem is largely solved with cake & = codel. >>>>>=20 >>>>> Also that demos are good. How to do one for this problem> >>>>>=20 >>>>> =E2=80=94 Jim >>>>>=20 >>>>>> The bandwidth mantra has been used for so long that a technical = discussion cannot unseat the mantra. >>>>>> Some technical parties use the mantra to sell more, faster, = ineffective service. Gullible customers accept that they would be happy = if they could afford even more speed. >>>>>>=20 >>>>>> Shouldn=E2=80=99t we create a demo to show the solution? >>>>>> To show is more effective than to debate. It is impossible to = explain to some people. >>>>>> Has anyone tried to create a demo (to unseat the bandwidth = mantra)? >>>>>> Is an effective demo too complicated to create? >>>>>> I=E2=80=99d be glad to participate in defining a demo and = publicity campaign. >>>>>>=20 >>>>>> Gene >>>>>>=20 >>>>>>=20 >>>>>>> On Apr 30, 2024, at 2:36 PM, David Lang > = >>> wrote: >>>>>>>=20 >>>>>>> On Tue, 30 Apr 2024, Eugene Y Chang via Starlink wrote: >>>>>>>=20 >>>>>>>> I am always surprised how complicated these discussions become. = (Surprised mostly because I forgot the kind of issues this community = care about.) The discussion doesn=E2=80=99t shed light on the following = scenarios. >>>>>>>>=20 >>>>>>>> While watching stream content, activating controls needed to = switch content sometimes (often?) have long pauses. I attribute that to = buffer bloat and high latency. >>>>>>>>=20 >>>>>>>> With a happy household user watching streaming media, a second = user could have terrible shopping experience with Amazon. The = interactive response could be (is often) horrible. (Personally, I would = be doing email and working on a shared doc. The Amazon analogy probably = applies to more people.) >>>>>>>>=20 >>>>>>>> How can we deliver graceful performance to both persons in a = household? >>>>>>>> Is seeking graceful performance too complicated to improve? >>>>>>>> (I said =E2=80=9Cgraceful=E2=80=9D to allow technical = flexibility.) >>>>>>>=20 >>>>>>> it's largely a solved problem from a technical point of view. = fq_codel and cake solve this. >>>>>>>=20 >>>>>>> The solution is just not deployed widely, instead people argue = that more bandwidth is needed instead. --Apple-Mail=_DE5CFA24-FBB6-4776-9C0B-DF211148F960 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Thank= you David.

Now, = shifting the focus a bit. Would a gamer experience some improvement if = they made a change in their router?
What needs to = be done for a gamer to get tangible improvement?

Gene
----------------------------------------------
Eugene Chang
IEEE Life Senior Member
IEEE = Communications Society & Signal Processing Society,    
    Hawaii Chapter Chair
IEEE Life Member = Affinity Group Hawaii Chair
IEEE Entrepreneurship, Mentor
eugene.chang@ieee.org
m 781-799-0233 (in = Honolulu)



On May 1, 2024, at 9:18 AM, David Lang <david@lang.hm> = wrote:

On Wed, 1 May 2024, Eugene Y Chang wrote:

Thanks = David,


On Apr 30, 2024, at 6:12 PM, David Lang <david@lang.hm> wrote:

On Tue, 30 Apr 2024, Eugene Y Chang wrote:

I=E2=80=99m= not completely up to speed on the gory details. Please humor me. I am = pretty good on the technical marketing magic.

What is the minimum configuration of an ISP infrastructure = where we can show an A/B (before and after) test?
It can = be a simplified scenario. The simpler, the better. We can talk through = the issues of how minimal is adequate. Of course and ISP engineer will = argue against simplicity.

I = did not see a very big improvement on a 4/.5 dsl link, but there was = improvement.

Would a user feel = the improvement with a 10 minute session:
shopping on = Amazon?
using Salesforce?
working with a = shared Google doc?

When it's only a single user, they are unlikely to notice any = difference.

But if you = have one person on zoom, a second downloading something, and a third on = Amazon, it doesn't take much to notice a difference.

if you put openwrt on = the customer router and configure cake with the targeted bandwith at = ~80% of line speed, you will usually see a drastic improvement for just = about any connection.

Are you = saying some of the benefits can be realized with just upgrading the = subscriber=E2=80=99s router? This makes adoption harder because the = subscriber will lose the ISP=E2=80=99s support for any connectivity = issues. If a demo impresses the subscribers, the ISP still needs to = embrace this change; otherwise the ISP will wash their hands of any = subscriber problems.

Yes, just upgrading the subscriber's device with cake and = configuring it appropriately largely solves the problem (at the cost of = sacraficing bandwith because cake isn't working directly on the data = flowing from the ISP to the client, and so it has to work indirectly to = get the Internet server to slow down instead and that's a laggy, = imperect work-around. If the ISPs router does active queue management = with fq_codel, then you don't have to do this.

This is how = we know this works, many of use have been doing this for years (see the = bufferbloat mailing list and it's archives_

If you can put fq_codel = on both ends of the link, you can usually skip capping the bandwidth.

This is good if this means the = benefits can be achieved with just the CPE. This also limits the changes = to subscribers that care.

fq_codel on = the ISPs router for downlink, and on the subscribers router for = uplink.

putting cake = on the router on the subscriber's end and tuning it appropriately can = achieve most of the benefit, but is more work to configure.


unfortunantly, it's not possible to just add this to the ISPs = existing hardware without having the source for the firmware there (and = if they have their queues in ASICs it's impossible to change them.

Is this just an alternative to = having the change at the CPE?
Yes this is harder for = routers in the network.

simple = fq_codel on both ends of the bottleneck connection works quite well = without any configuration. Cake adds some additional fairness = capabilities and has a mode to work around the router on the other end = of the bottleneck not doing active queue management

If you can point at the = dramatic decrease in latency, with no bandwidth losses, that Starlink = has achieved on existing hardware, that may help.

This is good to know for the = engineers. This adds confusion with the subscribers.


There are = a number of ISPs around the world that have implemented active queue = management and report very good results from doing so.

Can we get these ISPs to = publically report how they have achieved great latency reduction?
We can help them get credit for caring about their = subscribers. It would/could be a (short term) competitive advantage.
Of course their competitors will (might) adopt these changes = and eliminate the advantage, BUT the subscribers will retain glow of the = initial marketing for a much longer time.

several of = them have done so, I think someone else posted a report from one in this = thread.

But showing that their = existing hardware can do it when their upstream vendor doesn't support = it is going to be hard.

Is the = upstream vendor a network provider or a computing center?
Getting good latency from the subscriber, through the access = network to the edge computing center and CDNs would be great. The CDNs = would harvest the benefits. The other computing configurations would = have make the change to be competitive.

I'm talking = about the manufacturer of the routers that the ISPs deploy at the last = hop before getting to the subscriber, and the router on the subscriber = end of the link (although most of those are running some variation of = openWRT, so turning it on would not be significant work for the = manufacturer)

We = wouild have done our part at pushing the next round of adoption.

Many of us have been pushing this for well over a decade. = Getting Starlink's attention to address their bufferbloat issues is a = major success.

David Lang

Gene


David = Lang

We will want to show the human visible impact and not debate = good or not so good measurements. If we get the business and community = subscribers on our side, we win.

Note:
Stage 1 is to show we have a pure software fix (that can work = on their hardware). The fix is =E2=80=9Cso dramatic=E2=80=9D that = subscribers can experience it without debating measurements.
Stage 2 discusses why the ISP should demand that their = equipment vendors add this software. (The software could already be = available, but the ISP doesn=E2=80=99t think it is worth the trouble to = enable it.) Nothing will happen unless we stay engaged. We need to keep = the subscribers engaged, too.

Should we = have a conference call to discuss this?


Gene
----------------------------------------------
Eugene Chang
IEEE Life Senior Member



On Apr 30, 2024, at 3:52 PM, Jim Forster <jim@connectivitycap.com> wrote:

Gene, David,
=E2=80=98m
Agreed = that the technical problem is largely solved with cake & codel.

Also that demos are good. How to do one for = this problem>

=E2=80=94 Jim

The = bandwidth mantra has been used for so long that a technical discussion = cannot unseat the mantra.
Some technical parties use the = mantra to sell more, faster, ineffective service. Gullible customers = accept that they would be happy if they could afford even more speed.

Shouldn=E2=80=99t we create a demo to show the = solution?
To show is more effective than to debate. It is = impossible to explain to some people.
Has anyone tried to = create a demo (to unseat the bandwidth mantra)?
Is an = effective demo too complicated to create?
I=E2=80=99d be = glad to participate in defining a demo and publicity campaign.

Gene


On Apr 30, 2024, at 2:36 = PM, David Lang <david@lang.hm <mailto:david@lang.hm> = <mailto:david@lang.hm<mailto:david@lang.hm>>> wrote:

On Tue, 30 Apr 2024, Eugene Y Chang via Starlink wrote:

I am = always surprised how complicated these discussions become. (Surprised = mostly because I forgot the kind of issues this community care about.) = The discussion doesn=E2=80=99t shed light on the following scenarios.

While watching stream content, activating = controls needed to switch content sometimes (often?) have long pauses. I = attribute that to buffer bloat and high latency.

With a happy household user watching streaming media, a = second user could have terrible shopping experience with Amazon. The = interactive response could be (is often) horrible. (Personally, I would = be doing email and working on a shared doc. The Amazon analogy probably = applies to more people.)

How can we deliver = graceful performance to both persons in a household?
Is = seeking graceful performance too complicated to improve?
(I = said =E2=80=9Cgraceful=E2=80=9D to allow technical flexibility.)

it's largely a solved problem = from a technical point of view. fq_codel and cake solve this.

The solution is just not deployed widely, = instead people argue that more bandwidth is needed = instead.
<= /blockquote>

= --Apple-Mail=_DE5CFA24-FBB6-4776-9C0B-DF211148F960-- --Apple-Mail=_0A00AC7F-ADE7-4EC7-A3CE-3B41C3C7FCC8 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEERPTGiBqcibajhTSsv0/8FiYdKmAFAmYysEUACgkQv0/8FiYd KmBU9BAAlsuymcFzd7d40Z1qaicwrEbfhUgOB9l7qCp32f+tZ4lYtHVchPgc8pRm ereWnOPes7lg7Xfz9xMhJOb++d0FAc1inzF4ltueEMnuqZob93f8oqzlo8qDM7oU 8XH/Zal1YHUyYyzqSrMlPMIYkcsQs89Tpnd7QcXBUANHL9Y5BdQeJuH9v4dQdfj2 u5KSiWDkM1K4hgcjEHWCQsBeTmXHBaRsP9eKE9Q4q6xDIRAWjVRMTeJxXK1MpnO2 iIUBGjvgnCHtEPc+gSRR7ZrsqdZsjbeb64V631f33WsCGVf9nkvTWXh2iCxWRuic E4P8CKd3tAFKmsFhbQqHPMuOHEQbJK9fPv/BfRfAtAZohMz/Ae+kTlqpdoEIn0Z1 aA1EkbqAZMQ+sv1W0OqG+gOmQPp8ymQ1H2cAPLE2RcirBufJeWkVlCqmA2T/o2JJ AVC2V4tTTJJyZSUzUB/+mccaQ3JAaPdlIV8zCOANGOROLUSdMIOMnB3bWYGqF/YD W6/X2TMr7QD7bHAhkUjXkcFtLDBCxz4hSgf9631gdNn+JNbQ3RBuOKE4NP1TI1m2 6EWhU0OYOzDwW+Vdscj1bA44JokebpLjrMYHacPz/3ThwZuWeUNqt2mRAYGM2GcW lp1kX34w8gC6qD1fBPibtZ9TqwRGz0gjic5mT2FBDoM3UlhFgRs= =P6Ji -----END PGP SIGNATURE----- --Apple-Mail=_0A00AC7F-ADE7-4EC7-A3CE-3B41C3C7FCC8--