From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qv1-xf2e.google.com (mail-qv1-xf2e.google.com [IPv6:2607:f8b0:4864:20::f2e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E06643B29D for ; Mon, 7 Jun 2021 13:28:11 -0400 (EDT) Received: by mail-qv1-xf2e.google.com with SMTP id t6so3413207qvp.5 for ; Mon, 07 Jun 2021 10:28:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:mime-version:subject:message-id:references:to:date; bh=iapifNgFyrL+9RyTpFU/AKfoznaQs0rzmHmtEW6h2BY=; b=MphaRNVPe0wvrpnozcpm74Gomg2/OmPZeZT+dGb7fWM75w9MSJNxSlbORRlQLI9+22 Kjdm4rZF/aqFPxtLtMPIUc1blXra4DG2ObHRGK2erYIQqQpPCVniBvCOhbWmU6TgtZhc m43TN3bwI7nw7GXJc5h7urwdXGjudggDjvM9p5JmFd6i2XSjKCfNKmbFan3UQi2Zb3j8 VrjlZRdFTNasaJKae4iAb/xDrf0Te2iQUJ+UFjg4Sc+cdNG1PZjQReJ27jEUpaHVcuW5 Wngb6EuAsyAhvyv9WMNQCIAHeMvoZ+YqDzWM7OblGHabJ+MkKdGZcH3Q83cX48AnNAok jXew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:references :to:date; bh=iapifNgFyrL+9RyTpFU/AKfoznaQs0rzmHmtEW6h2BY=; b=n19nE87JpJXUHQCEZ06CHn2k4QHECbn3RwZnin9tpPsnFZJhE8WeHz37QqGD/A+Vjk 4tbCjJuqK6iNDr0djS/WXn5EyQxle8+5T3Y9pQchEYqvkziEYqwS2xs/fTBWndqplHna MkdRmhTsJSM0ifOdb0zV7fCJdCzuyP86utvcgtSl30fHTM2cCkd26jFkOkMWN7ZpiL4Z 8CRGzgxsvSBI2aG3yltfJDRcRhkBqVymSsn6UrqMA+2IJu2P4uWFeatO/TQDDp5TFCRq 3J+LrFbmua3aBoY50Nc1ywNLtUbgMTEnX3rfeDi3XELjUlxjFecprPvKrfH0fMCOLuAt I4Ew== X-Gm-Message-State: AOAM532d/+cufB0LTXviOnG0YcLJwuKO3M+m6L/pXpCa+8ySeGEd5Y8R +pa/hUxhTmacpgNvwgFTqu5td3oDT4TtUg== X-Google-Smtp-Source: ABdhPJziKK+Jk7k2RCWwOzUFQuCErR1xFzn/xc7qpMHvZeFNTfygnDQ0u0BZssWXEdBza7qC4QDDCA== X-Received: by 2002:a0c:fb4b:: with SMTP id b11mr19317426qvq.51.1623086890954; Mon, 07 Jun 2021 10:28:10 -0700 (PDT) Received: from richs-mbp-pro.lan ([198.55.239.24]) by smtp.gmail.com with ESMTPSA id d85sm4742926qkg.84.2021.06.07.10.28.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Jun 2021 10:28:10 -0700 (PDT) From: Rich Brown Content-Type: multipart/alternative; boundary="Apple-Mail=_67D58041-8DC0-4A55-9E81-A94E7115A555" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Message-Id: References: <20210607133853.045a96d5@babalu> To: bloat Date: Mon, 7 Jun 2021 13:28:10 -0400 X-Mailer: Apple Mail (2.3608.120.23.2.7) Subject: [Bloat] Fwd: Traffic shaping at 10~300mbps at a 10Gbps link X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jun 2021 17:28:12 -0000 --Apple-Mail=_67D58041-8DC0-4A55-9E81-A94E7115A555 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Saw this on the lartc mailing list... For my own information, does = anyone have thoughts, esp. for this quote: "... when the speed comes to about 4.5Gbps download (upload is about = 500mbps), chaos kicks in. CPU load goes sky high (all 24x2.4Ghz physical = cores above 90% - 48x2.4Ghz if count that virtualization is on)..." Thanks. Rich > Begin forwarded message: >=20 > From: "Ethy H. Brito" > Subject: Traffic shaping at 10~300mbps at a 10Gbps link > Date: June 7, 2021 at 12:38:53 PM EDT > To: lartc >=20 >=20 > Hi >=20 > I am having a hard time trying to shape 3000 users at ceil speeds from = 10 to 300mbps in a 7/7Gbps link using HTB+SFQ+TC(filter by IP hashkey = mask) for a few days now tweaking HTB and SFQ parameters with no luck so = far. >=20 > Everything seems right, up 4Gbps overall download speed with shaping = on. > I have no significant packets delay, no dropped packets and no high = CPU average loads (not more than 20% - htop info) >=20 > But when the speed comes to about 4.5Gbps download (upload is about = 500mbps), chaos kicks in. > CPU load goes sky high (all 24x2.4Ghz physical cores above 90% - = 48x2.4Ghz if count that virtualization is on) and as a consequence = packets are dropped (as reported by tc -s class sh ...), RTT goes above = 200ms and a lots of ungry users. This goes from about 7PM to 11 PM every = day. >=20 > If I turn shaping off, everything return to normality immediately and = peaks of not more than 5Gbps (1 second average) are observed and a CPU = load of about 5%. So I infer the uplink is not crowded. >=20 > I use one root HTB qdisc and one root (1:) HTB class. > Then about 20~30 same level (1:xx) inner classes to (sort of) separate = the users by region=20 > And under these inner classes, goes the almost 3000 leaves (1:xxxx).=20= > I have one class with about 900 users and this quantity decreases by = the other inner classes having some of them with just one user. >=20 > Is the way I'm using HTB+SFQ+TC suitable for this job? >=20 > Since the script that creates the shaping environment is too long I do = not post it here. >=20 > What can I inform you guys to help me solve this? > Fragments of code, stats, some measurements? What? >=20 > Thanks. >=20 > Regards >=20 > Ethy --Apple-Mail=_67D58041-8DC0-4A55-9E81-A94E7115A555 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
Saw this on the lartc mailing = list... For my own information, does anyone have thoughts, esp. for this = quote:

"... when the = speed comes to about 4.5Gbps download (upload is about 500mbps), chaos = kicks in. CPU load goes sky high (all 24x2.4Ghz physical cores above 90% = - 48x2.4Ghz if count that virtualization is on)..."

Thanks.

Rich


Begin forwarded message:

From: = "Ethy H. Brito" <ethy.brito@inexo.com.br>
Subject: = Traffic shaping = at 10~300mbps at a 10Gbps link
Date: = June 7, 2021 at 12:38:53 PM = EDT
To: = lartc <lartc@vger.kernel.org>


Hi

I am having a hard time trying to shape 3000 = users at ceil speeds from 10 to 300mbps in a 7/7Gbps link using = HTB+SFQ+TC(filter by IP hashkey mask) for a few days now tweaking HTB = and SFQ parameters with no luck so far.

Everything seems right, up 4Gbps overall download speed with = shaping on.
I have no significant packets delay, no = dropped packets and no high CPU average loads (not more than 20% - htop = info)

But when the speed comes to about = 4.5Gbps download (upload is about 500mbps), chaos kicks in.
CPU load goes sky high (all 24x2.4Ghz physical cores above = 90% - 48x2.4Ghz if count that virtualization is on) and as a consequence = packets are dropped (as reported by tc -s class sh ...), RTT goes above = 200ms and a lots of ungry users. This goes from about 7PM to 11 PM every = day.

If I turn shaping off, everything = return to normality immediately and peaks of not more than 5Gbps (1 = second average) are observed and a CPU load of about 5%. So I infer the = uplink is not crowded.

I use one root HTB = qdisc and one root (1:) HTB class.
Then about 20~30 same = level (1:xx) inner classes to (sort of) separate the users by region
And under these inner classes, goes the almost 3000 leaves = (1:xxxx).
I have one class with about 900 users and this = quantity decreases by the other inner classes having some of them with = just one user.

Is the way I'm using = HTB+SFQ+TC suitable for this job?

Since the = script that creates the shaping environment is too long I do not post it = here.

What can I inform you guys to help me = solve this?
Fragments of code, stats, some measurements? = What?

Thanks.

Regards

Ethy

= --Apple-Mail=_67D58041-8DC0-4A55-9E81-A94E7115A555--