From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AED023B29E for ; Thu, 24 Feb 2022 09:23:31 -0500 (EST) Received: by mail-ej1-x633.google.com with SMTP id vz16so4793787ejb.0 for ; Thu, 24 Feb 2022 06:23:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=+I1bofRpvQ6wmFoL2EMX3LvXFqPTxxkfzgSuXT+Tp+U=; b=boAP8yxINk2Tx6Rwn65DVgZdDkVkVrbQT1mR4krQw/DHM8Y3L5ZpMDbyUXRApPmICl W9wVTXDB37cL6LT0p9yoqwdAE6muHa9hslXYsEyjWCO1g2a3kKanNv7fu6zRtZ7HlT7s B4NOydjpYRQXgXpr6FT6CmZqlZWSrFC8SvOJr+ED0ndUlhxW7SlbXILF61QPH4FjcHbk PR+eGlbXlY0pWyBihX387FnQQyw/e0XqAVqKBgZIF7JuuSKY7QgvOJty82mbJ85xHSOx RVCw7Fgyo1lbO6Kcsamdy/v4kRfr8VtePKPPwVJNE9kekiJLX/AMhgSLblFntmlLBEDj 0liA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=+I1bofRpvQ6wmFoL2EMX3LvXFqPTxxkfzgSuXT+Tp+U=; b=t/y0qWkUjLk7XSf/42nWExinwv87UICy4NVeaxnVM1pdTEy3Axq/9mE+2O4vlH3oE0 jwkEQTOCcqkyGFd+WN+gg9n8Pyk+7mwShCFMC0qxBg+PXs+DdbwFKpy70l5lA9NHxUYh sgx+lf0QviBBoOCRq0kLXm/ZuZKIq1daKe5sRzSwzVIXoKEPwgX5gWPA+auZcH24su/t gq9PN/PSJkjctyHaA9p3kOoqr6DjdegSlfIK4BIXOnLVEqifdm24wu6EKyL+hA4/0X6R KjdurbnLIhVuBZIuS7jf/iZixMn7HIMLThUWCTUq/gqly+jTEuSmgCyQEAQdhxy9HkGa o5eA== X-Gm-Message-State: AOAM5322m6kWMGLPFbLQtirq47d99Oy93Ww7lA1ZQRfNtPvSgNLykDeo ztN0IXtD1cCNhcsk5FM68UofVRrHmnIQzOuYT58= X-Google-Smtp-Source: ABdhPJwz+F5gOUAiTxzaUoxwdIwC1g29cizuN9TLxEkjcO0KBt5W6rNF0QIgawjj+ry9Ass2Pdbr8MInR2XQEIHdLbg= X-Received: by 2002:a17:906:b095:b0:6cf:752c:fb88 with SMTP id x21-20020a170906b09500b006cf752cfb88mr2486041ejy.128.1645712610508; Thu, 24 Feb 2022 06:23:30 -0800 (PST) MIME-Version: 1.0 References: <4AFF4380-4AA5-47C5-8BF0-440101A3D788@gmx.de> <3157ED41-1438-4519-9E84-847404BE229D@gmx.de> In-Reply-To: From: Dave Taht Date: Thu, 24 Feb 2022 09:23:17 -0500 Message-ID: To: =?UTF-8?Q?Bj=C3=B8rn_Ivar_Teigen?= Cc: Sebastian Moeller , starlink@lists.bufferbloat.net Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Starlink] FQ for rocket scientists X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Feb 2022 14:23:31 -0000 To bring this back to starlink more coherently, FQ would provide stabler behavior for voip, videoconferencing and gaming traffic, even if the characteristics of the underlying link mutate from 2Mbit to 300Mbit on any interval under 20ms and the underlying buffering kept short. In the now ancient data we had gathered, even the sat-sat switching delay was under 40ms on roughly a 2m interval, which is easily compensated for in a jitter buffer (though I might tune one to adjust more quickly), and I imagine they can switch sats faster in the future. I wish we knew more about the underlying multiplexing of the uplink. We did some high resolution irtt based measurements in the early days, having *two* starlink terminals fairly close together and repeating that L3 measurement with well synced clocks would reveal much (as would analog measurements). My guess is for subscriber density they are more bound by the uplinks than downlinks (please correct me if I'm wrong)! as a downlink transmission could contain data for all terminals all the time, separately decoded.... I hope the dishy provides a GPLS locked ntp clock one day. That leaves the backhaul (skyhaul?) questions still wide open. I would be trying to regulate bandwidth (and buffering) on the ground stations to keep the queues in the sky relatively empty, doing ack-filtering in both directions (because that works well with bursty, blocked, traffic and reduces pressure on packet fifos), but when the laser links go up... my head still explodes. Too many unknown unknowns. On Mon, Feb 14, 2022 at 2:21 PM Bj=C3=B8rn Ivar Teigen wro= te: > > FQ clearly gets us very good behavior, so no argument there. > Also agree the cost of giving up work conservation might be too high, and= I'm not saying we should necessarily do that. I just wanted to point out t= hat insisting on work conservation also comes at a cost (which we are free = to choose to pay of course!). It's a choice between adding some middlebox c= omplexity or accepting the amount of jitter induced by the amount of queuin= g needed to maintain high utilization (which depends on the jitteriness of = the interface working to empty the queue). > > Regards, > Bj=C3=B8rn Ivar > > > On Mon, 14 Feb 2022 at 18:39, Sebastian Moeller wrote: >> >> Regarding work conservation, FQ as so often appears to be a decent solut= ion that while not optimal will also not be pessimal. Which IMHO is as much= as we can hope for unless we want to burden all middleboxes with deducing = relative importance of packets.... >> >> Regards >> Sebastian >> >> >> >> On 14 February 2022 18:52:29 CET, "Bj=C3=B8rn Ivar Teigen" wrote: >>> >>> Sebastian, >>> >>> On Mon, 14 Feb 2022 at 17:15, Sebastian Moeller wrote= : >>>> >>>> Hi Bj=C3=B8rn, >>>> >>>> >>>> I guess I should have started with the obvious. Nice short article! >>> >>> Thank you! >>>> >>>> >>>> > "See the problem? Buffers produce jitter!" >>>> > >>>> > I would rephrase that as "over-sized and under-managed" buffers incr= ease jitter unduly. Interestingly, bound jitter can be converted into stati= c latency by using, wait for it, (delay) buffers and a scheduler. >>>> > >>>> > You are absolutely right. I was hoping someone would spot that! Ther= e is one problem with that approach though. To actually remove the jitter t= he scheduler can no longer be work-conserving (needs delay as you also poin= t out), and that increases TCP ramp-up times (among other things). Can be a= hard sell. I think the benefits outweigh the costs though, so I would do i= t your way. >>>> >>>> But that is what end-points already do, on-line games do this = to equalize internet access quality between players (allowing them to make = matches over larger populations), as do DASH type video streaming applicati= ons, where the isochronous play-out takes the role of the scheduler and the= race-to-fill-the-play-out-buffers serves to keep the buffers filled so the= scheduler never runs "dry". >>> >>> >>> Good points. The application clients and servers can choose to use thei= r resources in a way that is not work-conserving and thus achieve sharing o= f resources without introducing jitter. I would argue it's a different stor= y with the queues "in the network", in routers, switches, access points, et= c. There seems to be an unwritten law that those schedulers must be work co= nserving, presumably to minimize round-trip times, and this makes it harder= to achieve well-behaved sharing. So in network devices configured to forwa= rd packets as quickly as possible, I think it's mostly true to say that buf= fers produce jitter. >>> >>>> Given Pete's data at https://github.com/heistp/l4s-tests I am = cautious to hold my breath... or to put it differently, I am certain they w= ill not deliver on their promises, the bigger question is whether the incre= mental improvement they offer (over the default FIFO) is decent enough to r= etroactively justify the disruption they will have caused... >>> >>> >>> Ohh, that work has been updated a lot since the IETF showdown! Thanks f= or reminding me, I'll have a look at it again. >>> >>> - Bj=C3=B8rn Ivar >>> >>>> >>>> Regards >>>> Sebastian >>>> >>>> >>>> > > >>>> > > >>>> > > On Mon, Feb 14, 2022 at 7:07 AM Bj=C3=B8rn Ivar Teigen wrote: >>>> > >> >>>> > >> Hi everyone, >>>> > >> >>>> > >> I was inspired by the latest Starship presentation to write a pie= ce on network quality in the language of rocket science. The blog can be fo= und here: https://www.domos.no/news-updates/network-quality-for-rocket-scie= ntists >>>> > >> >>>> > >> Cheers, >>>> > >> Bj=C3=B8rn Ivar Teigen >>>> > >> >>>> > >> -- >>>> > >> Bj=C3=B8rn Ivar Teigen >>>> > >> Head of Research >>>> > >> +47 47335952 | bjorn@domos.no | www.domos.no >>>> > >> WiFi Slicing by Domos >>>> > >> _______________________________________________ >>>> > >> Starlink mailing list >>>> > >> Starlink@lists.bufferbloat.net >>>> > >> https://lists.bufferbloat.net/listinfo/starlink >>>> > > >>>> > > >>>> > > >>>> > > -- >>>> > > I tried to build a better future, a few times: >>>> > > https://wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org >>>> > > >>>> > > Dave T=C3=A4ht CEO, TekLibre, LLC >>>> > > _______________________________________________ >>>> > > Starlink mailing list >>>> > > Starlink@lists.bufferbloat.net >>>> > > https://lists.bufferbloat.net/listinfo/starlink >>>> > >>>> > >>>> > >>>> > -- >>>> > Bj=C3=B8rn Ivar Teigen >>>> > Head of Research >>>> > +47 47335952 | bjorn@domos.no | www.domos.no >>>> > WiFi Slicing by Domos >>>> >>> >> -- >> Sent from my Android device with K-9 Mail. Please excuse my brevity. > > > > -- > Bj=C3=B8rn Ivar Teigen > Head of Research > +47 47335952 | bjorn@domos.no | www.domos.no > WiFi Slicing by Domos --=20 I tried to build a better future, a few times: https://wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org Dave T=C3=A4ht CEO, TekLibre, LLC