From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com [IPv6:2607:f8b0:4864:20::734]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 7C26A3CB35 for ; Sun, 29 Mar 2020 19:52:59 -0400 (EDT) Received: by mail-qk1-x734.google.com with SMTP id v7so17313930qkc.0 for ; Sun, 29 Mar 2020 16:52:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=UBgqjR1b3itIK605F+21TXwBysZU0hs8gJ9bTTeVMRg=; b=l2D5lB48pJ21WurdTOeGIMjaVHENn0tjaCy1N2CBqOpHKyvHcMpoF50a4q2TjC66yl KIlh9cGRH1fx6gvmqT8G9i/ky7lRiip2krUDhZOe8n966wKdCDZYVcfB+a6rmNY3JICV FdZdJOZWh4ahb89LgGf8Wdr0JNhtwtw0sipaJc0LNo1R46DlxVixmrZfi1HRDhiGaO01 enmkaB14BZ/Cv/HdZCmJZ7fY6rlJfWv28v3u+Cs2WlBipjciktQZBnItsYvZdNZ+c0w7 TpVmsOqXqhW5/dulXmmB6akstEdm48jeGVE739Jjeu9YAtwlfL6Am//A0VM1dLKv9bh9 3slQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=UBgqjR1b3itIK605F+21TXwBysZU0hs8gJ9bTTeVMRg=; b=YfS/BlSwQKhNh/f61xa+y0OgJ05nVon8Vow5qhtXmzwUAiGPeHU1f+dmRikM8MahpD 6MZqNa0BiUvWWJ5YF/gPCfOOSlANN+hDCo1fV51cINW0VmAJl0ZRT8jT5eFTwIr7Be1j cbYUDSpr1cK6LfJ9TaudxOGrpZUB5svYWkl1vD2LfhfrOnmcT9WdXeNta6Ut0vZYe5qR PvVfwh4vCvAljcxBbAJU+Q3n3Re5eZ7DGeodxr8glmc6onU1hB4KTU67cBMv+FKJqUrc ebdp9ygt1rFBe+w/y9UtH+dPZpEsCHuTViwiOcH9MvIVKphejETkTtGwYts3h/rVZfHM ILnw== X-Gm-Message-State: ANhLgQ0DzS1srd5FHXTGcO+/28jl0NEzqtJQfKNJZ3ES3AfvrQYnc/eO TY13ZAOnEnpdH/8yhmLyYrYfEtyzTb5ZujTAodI= X-Google-Smtp-Source: ADFU+vtp5oxOj7zN9ZhdSQNupRfYjTRhb9gkgFV7TdnhXc2OpYm3nrppv9dW+9fRCTQmj5RPAL6rU2I+mnCb2xt5StM= X-Received: by 2002:a37:cd0:: with SMTP id 199mr9788350qkm.189.1585525978954; Sun, 29 Mar 2020 16:52:58 -0700 (PDT) MIME-Version: 1.0 References: <875zesret5.fsf@toke.dk> <87r1xgpuhm.fsf@toke.dk> In-Reply-To: From: Aaron Wood Date: Sun, 29 Mar 2020 16:52:47 -0700 Message-ID: To: Dave Taht Cc: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= , bloat Content-Type: multipart/alternative; boundary="000000000000c2034d05a207084e" Subject: Re: [Bloat] Still seeing bloat with a DOCSIS 3.1 modem X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Mar 2020 23:52:59 -0000 --000000000000c2034d05a207084e Content-Type: text/plain; charset="UTF-8" On Sun, Mar 29, 2020 at 12:58 PM Dave Taht wrote: > I just finished doing my first openwrt build in a couple years. (with > AQL) Trying to summon up the moxie to try it. Found my soldiering iron > and usb to serial interfaces.... > That's kept me from rolling my own... I have the interfaces, but not the energy to deal with the troubleshooting. I think I still have an old WNDR3700 in a box somewhere that I could prep as a backup, but I'd rather not go through the hassle. > On Wed, Mar 25, 2020 at 8:58 AM Aaron Wood wrote: > > > > One other thought I've had with this, is that the apu2 is multi-core, > and the i210 is multi-queue. > > > > Cake/htb aren't, iirc, setup to run on multiple cores (as the rate > limiters then don't talk to each other). But with the correct tuple > hashing in the i210, I _should_ be able to split things and do two cores at > 500Mbps each (with lots of compute left over). > > A good test might be sch_mq + cake bandwidth whatever for each hw > queue. irqbalancing also may or may not help. > Bandwidth = 1Gbps or 500Mbps? (I was thinking 500Mbps for that test setup). > > Obviously, that puts a limit on single-connection rates, but as the > number of connections climb, they should more or less even out (I remember > Dave Taht showing the oddities that happen with say 4 streams and 2 cores, > where it's common to end up with 3 streams on the same core). But assuming > that the hashing function results in even sharing of streams, it should be > fairly balanced (after plotting some binomial distributions with higher "n" > values). Still not perfect, especially since streams aren't likely to all > be elephants. > > One reason why we are seeing "tcp rack" pushed so hard is due to cable > modems having multiple channels, and thus ooo packets are probable > when you try to push a stream across those channels. > I don't know anything about the channels and how they're bonded. separate packets on each, or symbols that are spread across all the channels that are used to construct a packet in less time..? I'd expect to see more out of order packets than I do, if they were using them all separately. But then none of the tests really do single-stream gigabit. > Me, I'm reasonably confident we've hit the age of "peak bandwidth" for > most things at up/dl rates above 40Mbit. > Very little of what I do gets to high (>50Mbps) rates. But those that do, I'm glad it's there. > And in the real world at home, a couple hash collissions and unequal > distribution really don't matter for real traffic. > --000000000000c2034d05a207084e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sun, Mar 29, 2020 at 12:58 PM Dave Tah= t <dave.taht@gmail.com> wr= ote:
I just finished d= oing my first openwrt build in a couple years. (with
AQL) Trying to summon up the moxie to try it. Found my soldiering iron
and usb to serial interfaces....

That&#= 39;s kept me from rolling my own...=C2=A0 I have the interfaces, but not th= e energy to deal with the troubleshooting.=C2=A0 I think I still have=C2=A0= an old WNDR3700 in a box somewhere that I could prep as a backup, but I'= ;d rather not go through the hassle.
=C2=A0
On Wed, Mar 25, 2020 at 8:58 AM Aaron Wood <woody77@gmail.com> wrote:
>
> One other thought I've had with this, is that the apu2 is multi-co= re, and the i210 is multi-queue.
>
> Cake/htb aren't, iirc, setup to run on multiple cores (as the rate= limiters then don't talk to each other).=C2=A0 But with the correct tu= ple hashing in the i210, I _should_ be able to split things and do two core= s at 500Mbps each (with lots of compute left over).

A good test might be sch_mq + cake bandwidth whatever for each hw
queue. irqbalancing also may or may not help.

Bandwidth =3D 1Gbps or 500Mbps? =C2=A0(I was thinking 500Mbps for th= at test setup).
=C2=A0
> Obviously, that puts a limit on single-connection rates, but as the nu= mber of connections climb, they should more or less even out (I remember Da= ve Taht showing the oddities that happen with say 4 streams and 2 cores, wh= ere it's common to end up with 3 streams on the same core).=C2=A0 But a= ssuming that the hashing function results in even sharing of streams, it sh= ould be fairly balanced (after plotting some binomial distributions with hi= gher "n" values).=C2=A0 Still not perfect, especially since strea= ms aren't likely to all be elephants.

One reason why we are seeing "tcp rack" pushed so hard is due to = cable
modems having multiple channels, and thus ooo packets are probable
when you try to push a stream across those channels.
<= br>
I don't know anything about the channels and how they'= ;re bonded. =C2=A0separate packets on each, or symbols that are spread acro= ss all the channels that are used to construct a packet in less time..?=C2= =A0 I'd expect to see more out of order packets than I do, if they were= using them all separately.=C2=A0 But then none of the tests really do sing= le-stream gigabit.
=C2=A0
Me, I'm reasonably confident we've hit the age of "peak bandwi= dth" for
most things at up/dl rates above 40Mbit.

Very little of what I do gets to high (>50Mbps) rates.=C2=A0 But those= that do, I'm glad it's there.
=C2=A0
And in the real world at home, a couple hash collissions and unequal
distribution really don't matter for real traffic.
--000000000000c2034d05a207084e--