From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 9F7C23B2A4 for ; Tue, 19 Apr 2022 17:36:19 -0400 (EDT) Received: by mail-wr1-x434.google.com with SMTP id p18so23324613wru.5 for ; Tue, 19 Apr 2022 14:36:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3RxjnkL362vnD6+qxqtoCOy/ES1OUpAABGvqq3zxK4Q=; b=Ild63PMtbpOCE3txOrhqj6q0r/9vKRIwQ4IZhfvcNporNlpTY3sr1GDln4Ax0iiE5g ydsmX7p4l+4vpcy+5R5zfk7AwGomE6GMNWm2oCufnjSK7+WZa4yu/lWYX/CrMiCmwVyP mEMwAuSV35fc0N2T1lIkBOSm5Q+kHrMvAoC+1Ui6j0g21s0Wcr1q4rrzaCEQQqwsF2iN lpqFMjDBCVbTTQkIIAV/iJ0ZC7VEXzfEIz7M7XYeVvnUm9wLuWOXhdOKK3h5Gl10wY3f zAbrdCuqkwWbKLy2Vbm30XSe+4rfh0GWNDUFYcPADCI/wL4Wlp5s294DMKG6eTyk0m3n XQIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3RxjnkL362vnD6+qxqtoCOy/ES1OUpAABGvqq3zxK4Q=; b=12VPSoZD8ym1xHyoaX8sw4jxW9qPb3RgiG+RSzuD3vy90KHFp+nxvgcHIbo85dj34K 9dL5R1qWdE5uGyKS0Xv4zhGyM5W3KEuA4djYzwgTGqYf4XW0sNx/Y9taW2D3kurPEzNs BTXSrfCwXyJYRJ6vU/IBN657S7kSQ77GRCQ8xUC2nhztNcrDLQNK+b5CwNr81kDEDTGs IDVi+QP5/oL0q4vY4/5zW4nKMkUqjkoqGK2TY1Zb5fFwXuVvarcbiquI7vXSUUgYxVbr coNe2JyuNMkGpVQXR8fGuyiFwk/HgobOzKYthOt1Dee6USBSLfU6zT7k9EHE6g8Cln+n 6u6g== X-Gm-Message-State: AOAM5304XoCDeFagxVyDxJSFCQ+TuvttkUtGgH3PrEZhWQ1zfo8ntMqy aRLP3mrXNwGa5WUtJ1soRuvbjF4HnENHmep+vvx87OeldRJzRQ== X-Google-Smtp-Source: ABdhPJwGTE4Q0hGrbIeEuBsJsAQqHL5UQqGH2ASq1XzDjVHXP1eg7hh3F71RFIXqXavgv9+nkRteI8sMnm0ElVXDLBM= X-Received: by 2002:a05:6000:2c5:b0:20a:9675:d26c with SMTP id o5-20020a05600002c500b0020a9675d26cmr8891571wry.185.1650404178393; Tue, 19 Apr 2022 14:36:18 -0700 (PDT) MIME-Version: 1.0 References: <4430DD9F-2556-4D38-8BE2-6609265319AF@ifi.uio.no> <1649778681.721621839@apps.rackspace.com> <0026CF35-46DF-4C0C-8FEE-B5309246C1B7@ifi.uio.no> <08F92DA0-1D59-4E58-A289-3D35103CF78B@gmx.de> <1649955272.49298319@apps.rackspace.com> <1650400809.579413230@apps.rackspace.com> In-Reply-To: <1650400809.579413230@apps.rackspace.com> From: Vint Cerf Date: Tue, 19 Apr 2022 17:36:06 -0400 Message-ID: To: "David P. Reed" Cc: Sebastian Moeller , ecn-sane@lists.bufferbloat.net Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-256; boundary="000000000000d14b5305dd08aa69" Subject: Re: [Ecn-sane] rtt-fairness question X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Apr 2022 21:36:19 -0000 --000000000000d14b5305dd08aa69 Content-Type: multipart/alternative; boundary="000000000000ca821505dd08aa2d" --000000000000ca821505dd08aa2d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable David's last point reminds me of a time-sharing system I once worked on. We adjusted the scheduling so tasks that needed lower latency got priority and we deliberately increased latency for tasks that users assumed would take a while :-))) v On Tue, Apr 19, 2022 at 4:40 PM David P. Reed wrote: > Sebastian - all your thoughts here seem reasonable. > > > > I would point out only two things: > > > > 1) 100 ms. is a magic number for human perception. It's basically the > order of magnitude of humans' ability to respond to unpredictable events > outside the human. That's why it is magic. Now humans can actually percei= ve > intervals much, much shorter (depending on how we pay attention), but > usually it is by comparing two events' time ordering. We can even > synchronize to external, predictable events with finer resolution (as in > Jazz improv or just good chamber music playing). A century of careful > scientific research supports this, niot just one experiment. Which is why > one should take it seriously as a useful target. (the fact that one can > achieve it across the planet with digital signalling networks makes it a > desirable goal for anything interactive between a human and any entity, b= e > it computer or human). If one can do better, of course, that's great. I > like that from my home computer I can get lots of places in under 8 msec > (15 msec RTT). > > > > 2) given that a particular heavily utilized link might be shared for path= s > where the light-speed-in-fiber round trip for active flows varies by an > order of magnitude, why does one try to make fair RTT (as opposed to all > other possible metrics on each flow) among flows. It doesn't make any sen= se > to me why. Going back to human interaction times, it makes sense to me th= at > you might want to be unfair so that most flows get faster than 200 ms. RT= T, > for example, penalizing those who are really close to each other anyway. > > If the RTT is already low because congestion has been controlled, you > can't make it lower. Basically, the ideal queue state is < 1 packet in th= e > bottleneck outbound queues, no matter what the RTT through that queue is. > > > > > > > > On Thursday, April 14, 2022 5:25pm, "Sebastian Moeller" > said: > > > Just indulge me here for a few crazy ideas ;) > > > > > On Apr 14, 2022, at 18:54, David P. Reed wrote: > > > > > > Am I to assume, then, that routers need not pay any attention to RTT = to > > achieve RTT-fairness? > > > > Part of RTT-bias seems caused by the simple fact that tight control > loops work > > better than sloppy ones ;) > > > > There seem to be three ways to try to remedy that to some degree: > > 1) the daft one: > > define a reference RTT (larger than typically encountered) and have all > TCPs > > respond as if encountering that delay -> until the path RTT exceeds tha= t > > reference TCP things should be reasonably fair > > > > 2) the flows communicate with the bottleneck honestly: > > if flows would communicate their RTT to the bottleneck the bottleneck > could > > partition its resources such that signaling (mark/drop) and puffer size > is > > bespoke per-flow. In theory that can work, but relies on either the RTT > > information being non-gameably linked to the protocol's operation* or > everybody > > being fully veridical and honest > > *) think a protocol that will only work if the best estimate of the RTT > is > > communicated between the two sides continuously > > > > 3) the router being verbose: > > If routers communicate the fill-state of their queue (global or per-flo= w > does not > > matter all that much) flows in theory can do a better job at not puttin= g > way too > > much data in flight remedying the cost of drops/marks that affects high > RTT flows > > more than the shorter ones. (The router has little incentive to lie > here, if it > > wanted to punish a flow it would be easier to simply drop its packets > and be done > > with). > > > > > > IMHO 3, while theoretically the least effective of the three is the onl= y > one that > > has a reasonable chance of being employed... or rather is already > deployed in the > > form of ECN (with mild effects). > > > > > How does a server or client (at the endpoint) adjust RTT so that it i= s > fair? > > > > See 1) above, but who in their right mind would actually implement > something like > > that (TCP Prague did that, but IMHO never in earnest but just to > "address" the > > L4S bullet point RTT-bias reduction). > > > > > Now RTT, technically, is just the sum of the instantaneous queue > lengths in > > bytes along the path and the reverse path, plus a fixed wire-level > delay. And > > routers along any path do not have correlated queue sizes. > > > > > > It seems to me that RTT adjustment requires collective real-time > cooperation > > among all-or-most future users of that path. The path is partially > shared by many > > servers and many users, none of whom directly speak to each other. > > > > > > And routers have very limited memory compared to their > throughput-RTdelay > > product. So calculating the RTT using spin bits and UIDs for packets > seems a bit > > much to expect all routers to do. > > > > If posed like this, I guess the better question is, what can/should > routers be > > expected to do here: either equitably share their queues or share queue > > inequitably such that throughput is equitable. From a pure router point > of the > > view the first seems "fairest", but as fq_codel and cake show, within > reason > > equitable capacity sharing is possible (so not perfectly and not for > every > > possible RTT spread). > > > > > > > > So, what process measures the cross-interactions among all the users > of all > > the paths, and what control-loop (presumably stable and TCP-compatible) > actually > > converges to RTT fairness IRL. > > > > Theoretically nothing, in reality on a home link FQ+competent AQM goes = a > long way > > in that direction. > > > > > > > > > > Today, the basis of congestion control in the Internet is that each > router is > > a controller of all endpoint flows that share a link, and each router i= s > free to > > do whatever it takes to reduce its queue length to near zero as an > average on all > > timescales larger than about 1/10 of a second (a magic number that is > directly > > derived from measured human brain time resolution). > > > > The typical applies, be suspicious of too round numbers.... 100ms is in > no way > > magic and also not "correct" it is however a decent description of > reaction times > > in a number of perceptul tasks that can be mis-interpreted as showing > things like > > the brain runs at 10Hz or similar... > > > > > > > > > > So, for any two machines separated by less than 1/10 of a light-secon= d > in > > distance, the total queueing delay has to stabilize in about 1/10 of a > second. > > (I'm using a light-second in a fiber medium, not free-space, as the > speed of light > > in fiber is a lot slower than the speed of light on microwaves, as Wall > Street has > > recently started recoginizing and investing in). > > > > > > I don't see how RTT-fairness can be achieved by some set of bits in > the IP > > header. You can't shorten RTT below about 2/10 of a second in that > desired system > > state. You can only "lengthen" RTT by delaying packets in source or > endpoint > > buffers, because it's unreasonable to manage all the routers. > > > > > > And the endpoints that share a path can't talk to each other and reac= h > a > > decision in on the order of 2/10 of a second. > > > > > > So at the very highest level, what is RTT-fairness's objective functi= on > > optimizing, and how can it work? > > > > > > Can it be done without any change to routers? > > > > Well the goal here seems to undo the RTT-dependence of throughput so a > router can > > equalize per flow throughput and thereby (from its own vantage point) > enforce RTT > > independence, within the amount of memory available. And that already > works today > > for all identifiable flows, but apparently at a computational cost that > larger > > routers do not want to pay. But you knew all that > > > > > > > > > > > > > > > > > > > On Tuesday, April 12, 2022 3:07pm, "Michael Welzl" > > > said: > > > > > > > > > > > > On Apr 12, 2022, at 8:52 PM, Sebastian Moeller > > wrote: > > > Question: is QUIC actually using the spin bit as an essential part of > the > > protocol? > > > The spec says it=E2=80=99s optional: > > https://www.rfc-editor.org/rfc/rfc9000.html#name-latency-spin-bit > > > Otherwise endpoints might just game this if faking their RTT at a > router > > yields an advantage... > > > This was certainly discussed in the QUIC WG. Probably perceived as an > unclear > > incentive, but I didn=E2=80=99t really follow this. > > > Cheers, > > > Michael > > > > > > This is why pping's use of tcp timestamps is elegant, little incentiv= e > for > > the endpoints to fudge.... > > > > > > Regards > > > Sebastian > > > > > > > > > On 12 April 2022 18:00:15 CEST, Michael Welzl > > wrote: > > > Hi, > > > Who or what are you objecting against? At least nothing that I > described > > does what you suggest. > > > BTW, just as a side point, for QUIC, routers can know the RTT today - > using > > the spin bit, which was designed for that specific purpose. > > > Cheers, > > > Michael > > > > > > > > > On Apr 12, 2022, at 5:51 PM, David P. Reed > > wrote: > > > I strongly object to congestion control *in the network* attempting t= o > > measure RTT (which is an end-to-end comparative metric). Unless the > current RTT is > > passed in each packet a router cannot enforce fairness. Period. > > > > > > Today, by packet drops and fair marking, information is passed to the > sending > > nodes (eventually) about congestion. But the router can't know RTT toda= y. > > > > > > The result of *requiring* RTT fairness would be to put the random > bottleneck > > router (chosen because it is the slowest forwarder on a contended path) > become the > > endpoint controller. > > > > > > That's the opposite of an "end-to-end resource sharing protocol". > > > > > > Now, I'm not saying it is impossible - what I'm saying it is asking a= ll > > endpoints to register with an "Internet-wide" RTT real-time tracking an= d > control > > service. > > > > > > This would be the technical equivalent of an ITU central control poin= t. > > > > > > So, either someone will invent something I cannot imagine (a > distributed, > > rapid-convergence algortithm that rellects to *every potential user* of > a shared > > router along the current path the RTT's of ALL other users (and > potential users). > > > > > > IMHO, the wish for RTT fairness is like saying that the entire solar > system's > > gravitational pull should be equalized so that all planets and asteroid= s > have fair > > access to 1G gravity. > > > > > > > > > On Friday, April 8, 2022 2:03pm, "Michael Welzl" > > said: > > > > > > Hi, > > > FWIW, we have done some analysis of fairness and convergence of DCTCP > in: > > > Peyman Teymoori, David Hayes, Michael Welzl, Stein Gjessing: > "Estimating an > > Additive Path Cost with Explicit Congestion Notification", IEEE > Transactions on > > Control of Network Systems, 8(2), pp. 859-871, June 2021. DOI > > 10.1109/TCNS.2021.3053179 > > > Technical report (longer version): > > > > > > https://folk.universitetetioslo.no/michawe/research/publications/NUM-ECN_= report_2019.pdf > > > and there=E2=80=99s also some in this paper, which first introduced o= ur LGC > > mechanism: > > > https://ieeexplore.ieee.org/document/7796757 > > > See the technical report on page 9, section D: a simple trick can > improve > > DCTCP=E2=80=99s fairness (if that=E2=80=99s really the mechanism to sta= y with=E2=80=A6 > > I=E2=80=99m getting quite happy with the results we get with our LGC sc= heme :-) > > ) > > > > > > Cheers, > > > Michael > > > > > > On Apr 8, 2022, at 6:33 PM, Dave Taht wrote: > > > I have managed to drop most of my state regarding the state of variou= s > > > dctcp-like solutions. At one level it's good to have not been keeping > > > up, washing my brain clean, as it were. For some reason or another I > > > went back to the original paper last week, and have been pounding > > > through this one again: > > > > > > Analysis of DCTCP: Stability, Convergence, and Fairness > > > > > > "Instead, we propose subtracting =CE=B1/2 from the window size for ea= ch > > marked ACK, > > > resulting in the following simple window update equation: > > > > > > One result of which I was most proud recently was of demonstrating > > > perfect rtt fairness in a range of 20ms to 260ms with fq_codel > > > https://forum.mikrotik.com/viewtopic.php?t=3D179307 )- and I'm pretty > > > interested in 2-260ms, but haven't got around to it. > > > > > > Now, one early result from the sce vs l4s testing I recall was severe > > > latecomer convergence problems - something like 40s to come into flow > > > balance - but I can't remember what presentation, paper, or rtt that > > > was from. ? > > > > > > Another one has been various claims towards some level of rtt > > > unfairness being ok, but not the actual ratio, nor (going up to the > > > paper's proposal above) whether that method had been tried. > > > > > > My opinion has long been that any form of marking should look more > > > closely at the observed RTT than any fixed rate reduction method, and > > > compensate the paced rate to suit. But that's presently just reduced > > > to an opinion, not having kept up with progress on prague, dctcp-sce, > > > or bbrv2. As one example of ignorance, are 2 packets still paced back > > > to back? DRR++ + early marking seems to lead to one packet being > > > consistently unmarked and the other marked. > > > > > > -- > > > I tried to build a better future, a few times: > > > https://wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org > > > > > > Dave T=C3=A4ht CEO, TekLibre, LLC > > > _______________________________________________ > > > Ecn-sane mailing list > > > Ecn-sane@lists.bufferbloat.net > > > https://lists.bufferbloat.net/listinfo/ecn-sane > > > > > > -- > > > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > > > > > > > _______________________________________________ > Ecn-sane mailing list > Ecn-sane@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/ecn-sane > --=20 Please send any postal/overnight deliveries to: Vint Cerf 1435 Woodhurst Blvd McLean, VA 22102 703-448-0965 until further notice --000000000000ca821505dd08aa2d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
David's last point reminds me of a time-sharing system= I once worked on. We adjusted the scheduling so tasks that needed lower la= tency got priority and we deliberately increased latency for tasks that use= rs assumed would take a while :-)))

v


On Tue, Apr 19, 2022 at 4:40 PM David P. Reed <dpreed@deepplum.com> wrote:

Sebastian - al= l your thoughts here seem reasonable.

=C2=A0=

I woul= d point out only two things:

=C2=A0=

1) 100= ms. is a magic number for human perception. It's basically the order o= f magnitude of humans' ability to respond to unpredictable events outsi= de the human. That's why it is magic. Now humans can actually perceive = intervals much, much shorter (depending on how we pay attention), but usual= ly it is by comparing two events' time ordering. We can even synchroniz= e to external, predictable events with finer resolution (as in Jazz improv = or just good chamber music playing).=C2=A0 A century of careful scientific = research supports this, niot just one experiment. Which is why one should t= ake it seriously as a useful target. (the fact that one can achieve it acro= ss the planet with digital signalling networks makes it a desirable goal fo= r anything interactive between a human and any entity, be it computer or hu= man). If one can do better, of course, that's great. I like that from m= y home computer I can get lots of places in under 8 msec (15 msec RTT).

=C2=A0=

2) giv= en that a particular heavily utilized link might be shared for paths where = the light-speed-in-fiber round trip for active flows varies by an order of = magnitude, why does one try to make fair RTT (as opposed to all other possi= ble metrics on each flow) among flows. It doesn't make any sense to me = why. Going back to human interaction times, it makes sense to me that you m= ight want to be unfair so that most flows get faster than 200 ms. RTT, for = example, penalizing those who are really close to each other anyway.

If the= RTT is already low because congestion has been controlled, you can't m= ake it lower. Basically, the ideal queue state is < 1 packet in the bott= leneck outbound queues, no matter what the RTT through that queue is.

=C2=A0=

=C2=A0=

=C2=A0=

On Thu= rsday, April 14, 2022 5:25pm, "Sebastian Moeller" <moeller0@gmx.de> said:
=

> J= ust indulge me here for a few crazy ideas ;)
>
> > On Apr 1= 4, 2022, at 18:54, David P. Reed <dpreed@deepplum.com> wrote:
> >
> = > Am I to assume, then, that routers need not pay any attention to RTT t= o
> achieve RTT-fairness?
>
> Part of RTT-bias seems cau= sed by the simple fact that tight control loops work
> better than sl= oppy ones ;)
>
> There seem to be three ways to try to remedy = that to some degree:
> 1) the daft one:
> define a reference RT= T (larger than typically encountered) and have all TCPs
> respond as = if encountering that delay -> until the path RTT exceeds that
> re= ference TCP things should be reasonably fair
>
> 2) the flows = communicate with the bottleneck honestly:
> if flows would communicat= e their RTT to the bottleneck the bottleneck could
> partition its re= sources such that signaling (mark/drop) and puffer size is
> bespoke = per-flow. In theory that can work, but relies on either the RTT
> inf= ormation being non-gameably linked to the protocol's operation* or ever= ybody
> being fully veridical and honest
> *) think a protocol = that will only work if the best estimate of the RTT is
> communicated= between the two sides continuously
>
> 3) the router being ve= rbose:
> If routers communicate the fill-state of their queue (global= or per-flow does not
> matter all that much) flows in theory can do = a better job at not putting way too
> much data in flight remedying t= he cost of drops/marks that affects high RTT flows
> more than the sh= orter ones. (The router has little incentive to lie here, if it
> wan= ted to punish a flow it would be easier to simply drop its packets and be d= one
> with).
>
>
> IMHO 3, while theoretically th= e least effective of the three is the only one that
> has a reasonabl= e chance of being employed... or rather is already deployed in the
> = form of ECN (with mild effects).
>
> > How does a server or= client (at the endpoint) adjust RTT so that it is fair?
>
> S= ee 1) above, but who in their right mind would actually implement something= like
> that (TCP Prague did that, but IMHO never in earnest but just= to "address" the
> L4S bullet point RTT-bias reduction).>
> > Now RTT, technically, is just the sum of the instantan= eous queue lengths in
> bytes along the path and the reverse path, pl= us a fixed wire-level delay. And
> routers along any path do not have= correlated queue sizes.
> >
> > It seems to me that RTT = adjustment requires collective real-time cooperation
> among all-or-m= ost future users of that path. The path is partially shared by many
>= servers and many users, none of whom directly speak to each other.
>= >
> > And routers have very limited memory compared to their t= hroughput-RTdelay
> product. So calculating the RTT using spin bits a= nd UIDs for packets seems a bit
> much to expect all routers to do.>
> If posed like this, I guess the better question is, what ca= n/should routers be
> expected to do here: either equitably share the= ir queues or share queue
> inequitably such that throughput is equita= ble. From a pure router point of the
> view the first seems "fai= rest", but as fq_codel and cake show, within reason
> equitable = capacity sharing is possible (so not perfectly and not for every
> po= ssible RTT spread).
>
> >
> > So, what process mea= sures the cross-interactions among all the users of all
> the paths, = and what control-loop (presumably stable and TCP-compatible) actually
&g= t; converges to RTT fairness IRL.
>
> Theoretically nothing, i= n reality on a home link FQ+competent AQM goes a long way
> in that d= irection.
>
>
> >
> > Today, the basis of c= ongestion control in the Internet is that each router is
> a controll= er of all endpoint flows that share a link, and each router is free to
&= gt; do whatever it takes to reduce its queue length to near zero as an aver= age on all
> timescales larger than about 1/10 of a second (a magic n= umber that is directly
> derived from measured human brain time resol= ution).
>
> The typical applies, be suspicious of too round nu= mbers.... 100ms is in no way
> magic and also not "correct"= it is however a decent description of reaction times
> in a number o= f perceptul tasks that can be mis-interpreted as showing things like
>= ; the brain runs at 10Hz or similar...
>
>
> >
&g= t; > So, for any two machines separated by less than 1/10 of a light-sec= ond in
> distance, the total queueing delay has to stabilize in about= 1/10 of a second.
> (I'm using a light-second in a fiber medium,= not free-space, as the speed of light
> in fiber is a lot slower tha= n the speed of light on microwaves, as Wall Street has
> recently sta= rted recoginizing and investing in).
> >
> > I don't = see how RTT-fairness can be achieved by some set of bits in the IP
> = header. You can't shorten RTT below about 2/10 of a second in that desi= red system
> state. You can only "lengthen" RTT by delaying= packets in source or endpoint
> buffers, because it's unreasonab= le to manage all the routers.
> >
> > And the endpoints t= hat share a path can't talk to each other and reach a
> decision = in on the order of 2/10 of a second.
> >
> > So at the ve= ry highest level, what is RTT-fairness's objective function
> opt= imizing, and how can it work?
> >
> > Can it be done with= out any change to routers?
>
> Well the goal here seems to und= o the RTT-dependence of throughput so a router can
> equalize per flo= w throughput and thereby (from its own vantage point) enforce RTT
> i= ndependence, within the amount of memory available. And that already works = today
> for all identifiable flows, but apparently at a computational= cost that larger
> routers do not want to pay. But you knew all that=
>
>
> >
> >
> >
> >
&= gt; > On Tuesday, April 12, 2022 3:07pm, "Michael Welzl" <<= a href=3D"mailto:michawe@ifi.uio.no" target=3D"_blank">michawe@ifi.uio.no>
> said:
> >
> >
> >
> > On= Apr 12, 2022, at 8:52 PM, Sebastian Moeller <moeller0@gmx.de>
> wrote:
> &= gt; Question: is QUIC actually using the spin bit as an essential part of t= he
> protocol?
> > The spec says it=E2=80=99s optional:
= > https://www.rfc-editor.org/rfc/rfc9000.html#name-= latency-spin-bit
> > Otherwise endpoints might just game this = if faking their RTT at a router
> yields an advantage...
> >= This was certainly discussed in the QUIC WG. Probably perceived as an uncl= ear
> incentive, but I didn=E2=80=99t really follow this.
> >= ; Cheers,
> > Michael
> >
> > This is why pping&= #39;s use of tcp timestamps is elegant, little incentive for
> the en= dpoints to fudge....
> >
> > Regards
> > Sebasti= an
> >
> >
> > On 12 April 2022 18:00:15 CEST, M= ichael Welzl <mi= chawe@ifi.uio.no>
> wrote:
> > Hi,
> > Who o= r what are you objecting against? At least nothing that I described
>= does what you suggest.
> > BTW, just as a side point, for QUIC, r= outers can know the RTT today - using
> the spin bit, which was desig= ned for that specific purpose.
> > Cheers,
> > Michael> >
> >
> > On Apr 12, 2022, at 5:51 PM, David P. = Reed <dpreed@de= epplum.com>
> wrote:
> > I strongly object to congest= ion control *in the network* attempting to
> measure RTT (which is an= end-to-end comparative metric). Unless the current RTT is
> passed i= n each packet a router cannot enforce fairness. Period.
> >
>= ; > Today, by packet drops and fair marking, information is passed to th= e sending
> nodes (eventually) about congestion. But the router can&#= 39;t know RTT today.
> >
> > The result of *requiring* RT= T fairness would be to put the random bottleneck
> router (chosen bec= ause it is the slowest forwarder on a contended path) become the
> en= dpoint controller.
> >
> > That's the opposite of an = "end-to-end resource sharing protocol".
> >
> >= Now, I'm not saying it is impossible - what I'm saying it is askin= g all
> endpoints to register with an "Internet-wide" RTT r= eal-time tracking and control
> service.
> >
> > Th= is would be the technical equivalent of an ITU central control point.
&g= t; >
> > So, either someone will invent something I cannot imag= ine (a distributed,
> rapid-convergence algortithm that rellects to *= every potential user* of a shared
> router along the current path the= RTT's of ALL other users (and potential users).
> >
> &= gt; IMHO, the wish for RTT fairness is like saying that the entire solar sy= stem's
> gravitational pull should be equalized so that all plane= ts and asteroids have fair
> access to 1G gravity.
> >
&g= t; >
> > On Friday, April 8, 2022 2:03pm, "Michael Welzl&q= uot; <michawe@if= i.uio.no>
> said:
> >
> > Hi,
> > F= WIW, we have done some analysis of fairness and convergence of DCTCP in:> > Peyman Teymoori, David Hayes, Michael Welzl, Stein Gjessing: &qu= ot;Estimating an
> Additive Path Cost with Explicit Congestion Notifi= cation", IEEE Transactions on
> Control of Network Systems, 8(2)= , pp. 859-871, June 2021. DOI
> 10.1109/TCNS.2021.3053179
> >= ; Technical report (longer version):
> >
> https://folk.universitetetioslo.no/michawe/resea= rch/publications/NUM-ECN_report_2019.pdf
> > and there=E2=80= =99s also some in this paper, which first introduced our LGC
> mechan= ism:
> > https://ieeexplore.ieee.org/document/7796757
> = > See the technical report on page 9, section D: a simple trick can impr= ove
> DCTCP=E2=80=99s fairness (if that=E2=80=99s really the mechanis= m to stay with=E2=80=A6
> I=E2=80=99m getting quite happy with the re= sults we get with our LGC scheme :-)
> )
> >
> > C= heers,
> > Michael
> >
> > On Apr 8, 2022, at 6:= 33 PM, Dave Taht <dave.taht@gmail.com> wrote:
> > I have managed to drop m= ost of my state regarding the state of various
> > dctcp-like solu= tions. At one level it's good to have not been keeping
> > up,= washing my brain clean, as it were. For some reason or another I
> &= gt; went back to the original paper last week, and have been pounding
&g= t; > through this one again:
> >
> > Analysis of DCTCP= : Stability, Convergence, and Fairness
> >
> > "Inst= ead, we propose subtracting =CE=B1/2 from the window size for each
> = marked ACK,
> > resulting in the following simple window update eq= uation:
> >
> > One result of which I was most proud rece= ntly was of demonstrating
> > perfect rtt fairness in a range of 2= 0ms to 260ms with fq_codel
> > https://forum.mikrotik.com/v= iewtopic.php?t=3D179307 )- and I'm pretty
> > interested i= n 2-260ms, but haven't got around to it.
> >
> > Now,= one early result from the sce vs l4s testing I recall was severe
> &= gt; latecomer convergence problems - something like 40s to come into flow> > balance - but I can't remember what presentation, paper, or= rtt that
> > was from. ?
> >
> > Another one ha= s been various claims towards some level of rtt
> > unfairness bei= ng ok, but not the actual ratio, nor (going up to the
> > paper= 9;s proposal above) whether that method had been tried.
> >
>= ; > My opinion has long been that any form of marking should look more> > closely at the observed RTT than any fixed rate reduction metho= d, and
> > compensate the paced rate to suit. But that's prese= ntly just reduced
> > to an opinion, not having kept up with progr= ess on prague, dctcp-sce,
> > or bbrv2. As one example of ignoranc= e, are 2 packets still paced back
> > to back? DRR++ + early marki= ng seems to lead to one packet being
> > consistently unmarked and= the other marked.
> >
> > --
> > I tried to bui= ld a better future, a few times:
> > https://= wayforward.archive.org/?site=3Dhttps%3A%2F%2Fwww.icei.org
> ><= br>> > Dave T=C3=A4ht CEO, TekLibre, LLC
> > _______________= ________________________________
> > Ecn-sane mailing list
>= > E= cn-sane@lists.bufferbloat.net
> > https://lists.bufferbloat= .net/listinfo/ecn-sane
> >
> > --
> > Sent f= rom my Android device with K-9 Mail. Please excuse my brevity.
> >=
>
>

_______________________________________________
Ecn-sane mailing list
Ecn-san= e@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/ecn-sane


--
Please send any postal/ove= rnight deliveries to:
Vint Cerf
1435 Woodhurst Blvd=C2= =A0
McLean, VA 22102
703-448-0965

<= div>until further notice



=
--000000000000ca821505dd08aa2d-- --000000000000d14b5305dd08aa69 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIIPlwYJKoZIhvcNAQcCoIIPiDCCD4QCAQExDzANBglghkgBZQMEAgEFADALBgkqhkiG9w0BBwGg ggzxMIIEtjCCA56gAwIBAgIQeAMYYHb81ngUVR0WyMTzqzANBgkqhkiG9w0BAQsFADBMMSAwHgYD VQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMzETMBEGA1UEChMKR2xvYmFsU2lnbjETMBEGA1UE AxMKR2xvYmFsU2lnbjAeFw0yMDA3MjgwMDAwMDBaFw0yOTAzMTgwMDAwMDBaMFQxCzAJBgNVBAYT AkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMSowKAYDVQQDEyFHbG9iYWxTaWduIEF0bGFz IFIzIFNNSU1FIENBIDIwMjAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCvLe9xPU9W dpiHLAvX7kFnaFZPuJLey7LYaMO8P/xSngB9IN73mVc7YiLov12Fekdtn5kL8PjmDBEvTYmWsuQS 6VBo3vdlqqXZ0M9eMkjcKqijrmDRleudEoPDzTumwQ18VB/3I+vbN039HIaRQ5x+NHGiPHVfk6Rx c6KAbYceyeqqfuJEcq23vhTdium/Bf5hHqYUhuJwnBQ+dAUcFndUKMJrth6lHeoifkbw2bv81zxJ I9cvIy516+oUekqiSFGfzAqByv41OrgLV4fLGCDH3yRh1tj7EtV3l2TngqtrDLUs5R+sWIItPa/4 AJXB1Q3nGNl2tNjVpcSn0uJ7aFPbAgMBAAGjggGKMIIBhjAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0l BBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFHzM CmjXouseLHIb0c1dlW+N+/JjMB8GA1UdIwQYMBaAFI/wS3+oLkUkrk1Q+mOai97i3Ru8MHsGCCsG AQUFBwEBBG8wbTAuBggrBgEFBQcwAYYiaHR0cDovL29jc3AyLmdsb2JhbHNpZ24uY29tL3Jvb3Ry MzA7BggrBgEFBQcwAoYvaHR0cDovL3NlY3VyZS5nbG9iYWxzaWduLmNvbS9jYWNlcnQvcm9vdC1y My5jcnQwNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDovL2NybC5nbG9iYWxzaWduLmNvbS9yb290LXIz LmNybDBMBgNVHSAERTBDMEEGCSsGAQQBoDIBKDA0MDIGCCsGAQUFBwIBFiZodHRwczovL3d3dy5n bG9iYWxzaWduLmNvbS9yZXBvc2l0b3J5LzANBgkqhkiG9w0BAQsFAAOCAQEANyYcO+9JZYyqQt41 TMwvFWAw3vLoLOQIfIn48/yea/ekOcParTb0mbhsvVSZ6sGn+txYAZb33wIb1f4wK4xQ7+RUYBfI TuTPL7olF9hDpojC2F6Eu8nuEf1XD9qNI8zFd4kfjg4rb+AME0L81WaCL/WhP2kDCnRU4jm6TryB CHhZqtxkIvXGPGHjwJJazJBnX5NayIce4fGuUEJ7HkuCthVZ3Rws0UyHSAXesT/0tXATND4mNr1X El6adiSQy619ybVERnRi5aDe1PTwE+qNiotEEaeujz1a/+yYaaTY+k+qJcVxi7tbyQ0hi0UB3myM A/z2HmGEwO8hx7hDjKmKbDCCA18wggJHoAMCAQICCwQAAAAAASFYUwiiMA0GCSqGSIb3DQEBCwUA MEwxIDAeBgNVBAsTF0dsb2JhbFNpZ24gUm9vdCBDQSAtIFIzMRMwEQYDVQQKEwpHbG9iYWxTaWdu MRMwEQYDVQQDEwpHbG9iYWxTaWduMB4XDTA5MDMxODEwMDAwMFoXDTI5MDMxODEwMDAwMFowTDEg MB4GA1UECxMXR2xvYmFsU2lnbiBSb290IENBIC0gUjMxEzARBgNVBAoTCkdsb2JhbFNpZ24xEzAR BgNVBAMTCkdsb2JhbFNpZ24wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDMJXaQeQZ4 Ihb1wIO2hMoonv0FdhHFrYhy/EYCQ8eyip0EXyTLLkvhYIJG4VKrDIFHcGzdZNHr9SyjD4I9DCuu l9e2FIYQebs7E4B3jAjhSdJqYi8fXvqWaN+JJ5U4nwbXPsnLJlkNc96wyOkmDoMVxu9bi9IEYMpJ pij2aTv2y8gokeWdimFXN6x0FNx04Druci8unPvQu7/1PQDhBjPogiuuU6Y6FnOM3UEOIDrAtKeh 6bJPkC4yYOlXy7kEkmho5TgmYHWyn3f/kRTvriBJ/K1AFUjRAjFhGV64l++td7dkmnq/X8ET75ti +w1s4FRpFqkD2m7pg5NxdsZphYIXAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8E BTADAQH/MB0GA1UdDgQWBBSP8Et/qC5FJK5NUPpjmove4t0bvDANBgkqhkiG9w0BAQsFAAOCAQEA S0DbwFCq/sgM7/eWVEVJu5YACUGssxOGhigHM8pr5nS5ugAtrqQK0/Xx8Q+Kv3NnSoPHRHt44K9u bG8DKY4zOUXDjuS5V2yq/BKW7FPGLeQkbLmUY/vcU2hnVj6DuM81IcPJaP7O2sJTqsyQiunwXUaM ld16WCgaLx3ezQA3QY/tRG3XUyiXfvNnBB4V14qWtNPeTCekTBtzc3b0F5nCH3oO4y0IrQocLP88 q1UOD5F+NuvDV0m+4S4tfGCLw0FREyOdzvcya5QBqJnnLDMfOjsl0oZAzjsshnjJYS8Uuu7bVW/f hO4FCU29KNhyztNiUGUe65KXgzHZs7XKR1g/XzCCBNAwggO4oAMCAQICEAHCGaual6pdiigN0q7j W1cwDQYJKoZIhvcNAQELBQAwVDELMAkGA1UEBhMCQkUxGTAXBgNVBAoTEEdsb2JhbFNpZ24gbnYt c2ExKjAoBgNVBAMTIUdsb2JhbFNpZ24gQXRsYXMgUjMgU01JTUUgQ0EgMjAyMDAeFw0yMjAyMjgx OTM4MDhaFw0yMjA4MjcxOTM4MDhaMCAxHjAcBgkqhkiG9w0BCQEWD3ZpbnRAZ29vZ2xlLmNvbTCC ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANOie/CUuHUBgv2/BDRI61p1Mw72H8oAyE05 TidbQ2YGlUDW2xMV5LbhBlp+ax3FkB3VLkPN47QtmTu1Oz84UYV0CghZt4vO8cQslhVc8q9Zrv2G R1wAmPXNJa7OB19wexS3GVYqe2IGtMrc696S7G3Di4HakkudGJv7iAfR+Il7gZh/2Dbbo3Vc7kAG BNggINI0rfbLLL3188Bb6uTHEOhZIH2zUCkeLS/Kzyc5MG2tM2mujLCFo9mDQeAQTJTFOsjl3f+b 6ngwI8TrJnCmjsXfFEgNHZ0QVgPhORPHyAmbvHVujaPE+o9sqpAbNajnuBBFzpX7F331qcGwYlDd OlkCAwEAAaOCAdAwggHMMBoGA1UdEQQTMBGBD3ZpbnRAZ29vZ2xlLmNvbTAOBgNVHQ8BAf8EBAMC BaAwHQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsGAQUFBwMCMB0GA1UdDgQWBBQFPlCzf+Bd+BTsRvjF J6GYn78BmTBMBgNVHSAERTBDMEEGCSsGAQQBoDIBKDA0MDIGCCsGAQUFBwIBFiZodHRwczovL3d3 dy5nbG9iYWxzaWduLmNvbS9yZXBvc2l0b3J5LzAMBgNVHRMBAf8EAjAAMIGaBggrBgEFBQcBAQSB jTCBijA+BggrBgEFBQcwAYYyaHR0cDovL29jc3AuZ2xvYmFsc2lnbi5jb20vY2EvZ3NhdGxhc3Iz c21pbWVjYTIwMjAwSAYIKwYBBQUHMAKGPGh0dHA6Ly9zZWN1cmUuZ2xvYmFsc2lnbi5jb20vY2Fj ZXJ0L2dzYXRsYXNyM3NtaW1lY2EyMDIwLmNydDAfBgNVHSMEGDAWgBR8zApo16LrHixyG9HNXZVv jfvyYzBGBgNVHR8EPzA9MDugOaA3hjVodHRwOi8vY3JsLmdsb2JhbHNpZ24uY29tL2NhL2dzYXRs YXNyM3NtaW1lY2EyMDIwLmNybDANBgkqhkiG9w0BAQsFAAOCAQEAS0oT55lMUM92ZtvZSqigFMQH D5w4NciCEFZqkxVyqPcRj+OiE9XzcWVLwbWCOIMu7z9jbUOBEAWaBqZjtJwtA5bUBnJK8kpBfYvK qwJ/qQV9HczcEMcJCm+g77/wOhG0yxD0j1wOLgK/HVRWbODrPXLXWZ57jwqwL/YO8TWjeMkmpIZI oe+HIWv+ayqC6vDEI1SOMJUCJKUp1oKkz6PUsBodi6mH3BDedE/Ke8JRIlJebk/pZqi9m0UpE6rv 58AQHYIE7fXHpbM5yoEubAivyI5EYu4lCH+3R75vqu/si/AWPS/Ctfelauu5qkTeajKXUCQOCbji /5Hh6RhVSbYOyzGCAmowggJmAgEBMGgwVDELMAkGA1UEBhMCQkUxGTAXBgNVBAoTEEdsb2JhbFNp Z24gbnYtc2ExKjAoBgNVBAMTIUdsb2JhbFNpZ24gQXRsYXMgUjMgU01JTUUgQ0EgMjAyMAIQAcIZ q5qXql2KKA3SruNbVzANBglghkgBZQMEAgEFAKCB1DAvBgkqhkiG9w0BCQQxIgQgTiffGfyw169r HlL1sorfXpVAWO8Z9cv+6/pvL0ihqLMwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG 9w0BCQUxDxcNMjIwNDE5MjEzNjE4WjBpBgkqhkiG9w0BCQ8xXDBaMAsGCWCGSAFlAwQBKjALBglg hkgBZQMEARYwCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMAsGCSqGSIb3DQEBCjALBgkqhkiG9w0B AQcwCwYJYIZIAWUDBAIBMA0GCSqGSIb3DQEBAQUABIIBAAL+08qu4/0YQy+tk412pYzfxT2kwI2E pX+zJRBQLCneJETWW3glBjSdQBTF0uNSFrYCiTY21WxsRw0KKB39yGUgqCo0xRD6EdPmHBTgj+AM k0n4vD18UdVQR0sQCgHaN0RH9GVUiHwb3qG7upuJ1e05el4IOLP4y1nLtOd71zy9j5Jnon+BXoGc uvrx7zCwKN41AcFY+Epbcf7uCry+6rl0IDZvoUkvPMsRcMKAjT5nqGLLiDyliWYjPdtBukKBXlLX RS/QP0W12rq8RMUJHcoZq+J1XUCnHxiJCPsLYTj8h/Mdi+WyYm5JLce2Yv5+246QzKZQNNgP3l4z hFSlo6w= --000000000000d14b5305dd08aa69--