From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 29CFB3B29E for ; Mon, 27 Sep 2021 11:14:39 -0400 (EDT) Received: by mail-il1-x12a.google.com with SMTP id x2so19678396ilm.2 for ; Mon, 27 Sep 2021 08:14:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=3I+iU1I/ek4lSMvpHb8bCUC9qiuDprpvi9m1hSUVcZM=; b=fPgODnnt8SLWYiXAih9H1sGogSGtL1zBVmiLBA69p+jEJS0hgQWAkkNA21mdjzdroV A6bhZ9wgnowOcTfsBq5hOvpt20aJ9GzBhJW7iNL3vY4Ix/OcbGhLxy/WVmyzdSyWZarE 0ertihOIu+OJf10em2qsVv8BWFvvY77B/vQNL8x7DxaA3jZdHwXWpRI4ibw1+S+ILxQR KFto8VkABzhPywmP0JX7Vw1U4EzVIQgawPvTelY1LnRmnESseIFYHwVathoo0ik8VcCi dFA+L30NTr/Zv9ATMmMqTk3oQ51DWKiQUUn0k9xHuFQt7WhOdfvTB3fTHB0NHHA5UCvq uRYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=3I+iU1I/ek4lSMvpHb8bCUC9qiuDprpvi9m1hSUVcZM=; b=7jvTHLtgXIjQJmnnaIHqswWGY1xnfycbneQoxT3VGE5pcW+Wm1c0BVsRXPAC08Pi/n 6QtF3IAwJ+qw5+TZrBKEivnvoAhlKiJJze/RpW8FbIwlvykNAA2a7eYOPwYh3/eUqm5i 0AeHs6Gl2LCUE6up/08ZS0hiblI8g3D4ftSOE9lhAjVsQqAFZM32RZ+98ABkavv93u4e Gw1j3Fj8Y7Nq31W6kZEQLabPmzI3Gz/3j76Rx8s1Jh+T8DU/76GPJ4E5yumnEIc2fxo6 XguA4GoxtqZlWWwFzE0w5BTemIcZTe495j/gq4YC3vrGNH85GRWn99QIyaSMzYWxa4iE wGJA== X-Gm-Message-State: AOAM532Kr3326T4HeZyXTeuDnaRxQZ7FmKfaCVi77YZ0/nx/yZX6PFw/ Ec1fq8Nf+yHkRG5s94IQfebbpvUTlvU+OHsmaRk= X-Google-Smtp-Source: ABdhPJzeoxLvFJ35iKhAwbpc3UNPnnbxd1UXPzRXNT65T14C5mgu8yv6UbT9MV8v21nBagmM80roHd6Flktl0lrQzIg= X-Received: by 2002:a05:6e02:1d83:: with SMTP id h3mr437191ila.274.1632755678317; Mon, 27 Sep 2021 08:14:38 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Mon, 27 Sep 2021 08:14:26 -0700 Message-ID: To: Bob Briscoe Cc: "Mohit P. Tahiliani" , Asad Sajjad Ahmed , ECN-Sane Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Ecn-sane] paper idea: praising smaller packets X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Sep 2021 15:14:39 -0000 I note that I have long clamped the mss at the router, on slower links to values below 600, I've never really got around to analyzing this, but in the presence of fixed length packet fifos elsewhere, a smaller ratio of acks to packets, and in search of better multiplexing for the types of traffic I care about most, it's made sense since I started doing it in the 90s. I care not one whit towards extra cpu usage elsewhere derived from these slow links. :) In striving for a better personal formulation as to how I view the internet, a statement I might start with today would be "the internet is a communications network" , as my priorities have always been voip, videoconferencing, gaming, request/response, and at the very bottom, capacity seeking bulk traffic. And, yes, when the patch went by to linux to count CEs better, just now, I got informed of the progress within bbrv2, which I'm pleased with and will attempt to test sometime in the coming months. A note towards flow queuing... vs fair queuing there is essentially no queueing if you manage to pace single packets below the service time it takes to deliver a round of all the other flows. If you deliver two packets back to back, above the quantum, you get that near 0 + the time it takes to serve a round, and I've often thought there migth be a way to take advantage of that property in non tcp transports, notably videoconferencing. Thank you for pointing me at these other works below. On Mon, Sep 27, 2021 at 7:50 AM Bob Briscoe wrote= : > > Dave, > > On 26/09/2021 21:08, Dave Taht wrote: > > ... an exploration of smaller mss sizes in response to persistent conge= stion > > > > This is in response to two declarative statements in here that I've > > long disagreed with, > > involving NOT shrinking the mss, and not trying to do pacing... > > I would still avoid shrinking the MSS, 'cos you don't know if the > congestion constraint is the CPU, in which case you'll make congestion > worse. But we'll have to differ on that if you disagree. > > I don't think that paper said don't do pacing. In fact, it says "...pace > the segments at less than one per round trip..." > > Whatever, that paper was the problem statement, with just some ideas on > how we were going to solve it. > after that, Asad (added to the distro) did his whole Masters thesis on > this - I suggest you look at his thesis and code (pointers below). > > Also soon after he'd finished, changes to BBRv2 were introduced to > reduce queuing delay with large numbers of flows. You might want to take > a look at that too: > https://datatracker.ietf.org/meeting/106/materials/slides-106-iccrg-updat= e-on-bbrv2#page=3D10 > > > > > https://www.bobbriscoe.net/projects/latency/sub-mss-w.pdf > > > > OTherwise, for a change, I largely agree with bob. > > > > "No amount of AQM twiddling can fix this. The solution has to fix TCP." > > > > "nearly all TCP implementations cannot operate at less than two packets= per RTT" > > Back to Asad's Master's thesis, we found that just pacing out the > packets wasn't enough. There's a very brief summary of the 4 things we > found we had to do in 4 bullets in this section of our write-up for netde= v: > https://bobbriscoe.net/projects/latency/tcp-prague-netdev0x13.pdf#subsubs= ection.3.1.6 > And I've highlighted a couple of unexpected things that cropped up below. > > Asad's full thesis: > Ahmed, A., "Extending TCP for Low Round Trip Delay", > Masters Thesis, Uni Oslo , August 2019, > . > Asad's thesis presentation: > https://bobbriscoe.net/presents/1909submss/present_asadsa.pdf > > Code: > https://bitbucket.org/asadsa/kernel420/src/submss/ > Despite significant changes to basic TCP design principles, the diffs > were not that great. > > A number of tricky problems came up. > > * For instance, simple pacing when <1 ACK per RTT wasn't that simple. > Whenever there were bursts from cross-traffic, the consequent burst in > your own flow kept repeating in subsequent rounds. We realized this was > because you never have a real ACK clock (you always set the next send > time based on previous send times). So we set up the the next send time > but then re-adjusted it if/when the next ACK did actually arrive. > > * The additive increase of one segment was the other main problem. When > you have such a small window, multiplicative decrease scales fine, but > an additive increase of 1 segment is a huge jump in comparison, when > cwnd is a fraction of a segment. "Logarithmically scaled additive > increase" was our solution to that (basically, every time you set > ssthresh, alter the additive increase constant using a formula that > scales logarithmically with ssthresh, so it's still roughly 1 for the > current Internet scale). > > What became of Asad's work? > Altho the code finally worked pretty well {1}, we decided not to pursue > it further 'cos a minimum cwnd actually gives a trickle of throughput > protection against unresponsive flows (with the downside that it > increases queuing delay). That's not to say this isn't worth working on > further, but there was more to do to make it bullet proof, and we were > in two minds how important it was, so it worked its way down our > priority list. > > {Note 1: From memory, there was an outstanding problem with one flow > remaining dominant if you had step-ECN marking, which we worked out was > due to the logarithmically scaled additive increase, but we didn't work > on it further to fix it.} > > > > Bob > > > -- > ________________________________________________________________ > Bob Briscoe http://bobbriscoe.net/ > --=20 Fixing Starlink's Latencies: https://www.youtube.com/watch?v=3Dc9gLo6Xrwgw Dave T=C3=A4ht CEO, TekLibre, LLC