From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D2A2D3CB37 for ; Wed, 7 Feb 2024 07:20:55 -0500 (EST) Received: by mail-pj1-x1034.google.com with SMTP id 98e67ed59e1d1-295c8b795e2so386201a91.0 for ; Wed, 07 Feb 2024 04:20:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707308455; x=1707913255; darn=lists.bufferbloat.net; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=MW3mQvoNAIiaB+NF1H7wN6KhqXR+qUuE98hWJIntemo=; b=g+LHtwCWkYWtgfnyjb6RSQFzI8PeURYCu4/OkLEp/LW1olbuCfPcDvRB4489TVjslC PVuybpPVmNhuf2doNFFNi5i2Uhg2gR+B30SfQT+aH+l0hQpm+OLAdjM3mU/r7QshyeH/ bxnFvUfPrJoxdeqZ1V6PDfOLvX6TjGI9J4vDTWntn/yoGHPaemUTIAWij+S2KnjMp0E5 KX9g2IRHjLLoJbav0ombTfPEY5TVgNs5TNmu9G3C9Qsnh3ZPvWKuXot/ofPdGgdHkwUc mF1YEekW7iOdYVbmfLs/8IqAlCKXit/AqQ1PaDnjI0w2kyWRuiOM3MNR/3FfEiy9wSK4 qAXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707308455; x=1707913255; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MW3mQvoNAIiaB+NF1H7wN6KhqXR+qUuE98hWJIntemo=; b=HsLTuh9pWx+PnN5Gv2N9eFagOo+vVkVfeC1FlM4AsKlbnLahHXH0Y9CbhyGK1CTEX5 mcdVj7+Em4dBOa/vKFaXfK7mGj0eSsgimFu4ihav6mkjsK8zRTeoJy6MocdSsZ66Ay7R 4D9FJj7YIaphd4eObic2FFlI5hjuoPmIyD4zhl565p4LC850lquTvqFzhsT+f/mVCfDb 2TH49nXMsP09tZTfuH7rEb1fNBZYunXRQKKAEFPfrkrFxwwhpCHsUTMWTDobEkYDFgVN nOayQjIzagGSl8WRAjgcNzvu10IR5Azw/N4gsC9QtlZ8foEXLqxFvakfQXKRHKlNukqc jr3g== X-Gm-Message-State: AOJu0YxIrIxn+fpFwxK24t2Vw/OSr65Wj6KDyqHA0H6d/a+lOuJDPI7x cZCLJL06hE1v95ZZl9qrLQhhScHqPv+S0jp/DNPXHQnQCLGLkFBCJ9seUT5AUtkaS7ZzELC41uu 3SusbUuC/Y1Okbb2+Sek8yVroQa8= X-Google-Smtp-Source: AGHT+IFCRBcM7xWtFY+ASji36VEM4u38Gghbgddyb8etSqZQgsml1u+CSvYoUoEleTLLtsShNJbRUw3nxXytt1sBiPM= X-Received: by 2002:a17:90a:3182:b0:296:cca0:ee34 with SMTP id j2-20020a17090a318200b00296cca0ee34mr2429239pjb.31.1707308454458; Wed, 07 Feb 2024 04:20:54 -0800 (PST) MIME-Version: 1.0 References: <1BAA689A-D44E-4122-9AD8-25F6D024377E@ifi.uio.no> In-Reply-To: From: Dave Taht Date: Wed, 7 Feb 2024 07:20:42 -0500 Message-ID: To: Michael Welzl Cc: Linux Kernel Network Developers , bloat@lists.bufferbloat.net Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] Trying to *really* understand Linux pacing X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Feb 2024 12:20:56 -0000 On Wed, Feb 7, 2024 at 7:08=E2=80=AFAM Michael Welzl w= rote: > > Whoa=E2=80=A6 and now I=E2=80=99m even more afraid :-) > > My sincere apologies to anyone whose code I may have mis-represented! I = just tried to get it=E2=80=A6 sorry if there are silly mistakes in there! I think his invention of packet pacing alone, of Eric Dumazet's many inventions, deserves him a place in the Internet Hall of Fame. It make short buffered switches saner, allows interactive traffic to fit in between other traffic (essentially host based FQ), and as the Linux default, has helped power the expansion of ever more bandwidth to ever more people without overloading the edges far more, than his fq_codel did. I also love, that by making it just work, he thoroughly disproved a formerly influential paper on it, circa 2004 or so. And I do not understand the implementation either, and have been trying to come up with ways of improving slow start by varying it. I hope more enlightenment spreads, and your attempt to document it improves. https://docs.google.com/document/d/1-uXnPDcVBKmg5krkG5wYBgaA2yLSFK_kZa7xGDW= c7XU/edit#heading=3Dh.7624xn94jzf6 > > > > On 7 Feb 2024, at 13:05, Dave Taht wrote: > > > > Dear Michael: > > > > Thank you for digging deeply into packet pacing, TSQ, etc. I think > > there are some interesting new possibilities in probing (especially > > during slow start) that could make the core idea even more effective > > than it is. I also tend to think that attempting it in various cloudy > > environments and virtualization schemes, and with certain drivers, the > > side effects are not as well understood as I would like. For example, > > AWS's nitro lacks BQL as does virtio-net. > > > > I think the netdev community, now cc'd, would be interested in your > > document and explorations so far, below. I hope for more > > enlightenment. > > > > On Wed, Feb 7, 2024 at 6:57=E2=80=AFAM Michael Welzl via Bloat > > wrote: > >> > >> Dear de-bloaters of buffers, > >> Esteemed experts of low delay and pacing! > >> > >> I have no longer been satisfied with high-level descriptions of how pa= cing works in Linux, and how it interacts with TSQ (I=E2=80=99ve seen some,= in various papers, over the years) - but I wanted to REALLY understand it.= So, I have dug through the code. > >> > >> I documented this experience here: > >> https://docs.google.com/document/d/1-uXnPDcVBKmg5krkG5wYBgaA2yLSFK_kZa= 7xGDWc7XU/edit?usp=3Dsharing > >> but it has some holes and may have mistakes. > >> > >> Actually, my main problem is that I don=E2=80=99t really know what goe= s on when I configure a larger IW=E2=80=A6 things seem to get quite =E2=80= =9Coff=E2=80=9D there. Why? Anyone up for solving that riddle? ;-) > >> (see the tests I documented towards the end of the document) > >> > >> Generally, if someone who has their hands on files such as tcp_output.= c all the time could take a look, and perhaps =E2=80=9Cfill=E2=80=9D my hol= es, or improve anything that might be wrong, that would be fantastic! I t= hink that anyone should be allowed to comment and make suggestions in this = doc. > >> > >> MANY thanks to whoever finds the time to take a look ! > >> > >> Cheers, > >> Michael > >> > >> _______________________________________________ > >> Bloat mailing list > >> Bloat@lists.bufferbloat.net > >> https://lists.bufferbloat.net/listinfo/bloat > > > > > > > > -- > > 40 years of net history, a couple songs: > > https://www.youtube.com/watch?v=3DD9RGX6QFm5E > > Dave T=C3=A4ht CSO, LibreQos > --=20 40 years of net history, a couple songs: https://www.youtube.com/watch?v=3DD9RGX6QFm5E Dave T=C3=A4ht CSO, LibreQos