From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id BAC573B29E for ; Thu, 3 Sep 2020 09:32:19 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1599139935; bh=cJbxfnzOQylysk+o00Eq8RJcDw2OdhxKTt59VBM6X+8=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=NLG8MZ+f4gaKKzuB/c6ejYae5RTih1aTAUkpBsue4o612QMFgQO+C/tOiYvZMuBna YMvcmP2smFx4ZRCyr3A5VlTwvNxIiX7AeOzRZCjjDauCKPkArysKWJRt7byxZZQvy3 MfUYxW56i9FVCIsQ52f6bttXSMu7FSQ+vyZlIgw0= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [192.168.250.102] ([134.76.241.253]) by mail.gmx.com (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1McH5a-1kqCV12yqK-00chLz; Thu, 03 Sep 2020 15:32:15 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.15\)) From: Sebastian Moeller In-Reply-To: Date: Thu, 3 Sep 2020 15:32:13 +0200 Cc: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , bloat@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: References: <87mu2bjbf8.fsf@toke.dk> <87eennj64i.fsf@toke.dk> To: Mikael Abrahamsson X-Mailer: Apple Mail (2.3445.104.15) X-Provags-ID: V03:K1:geBVUtRkLWrbillh/PWaUdKEDTlQsSwRYjFue8V1S5TjK0tkn9S QvTjcTGdXyL/nOMnotKMM3PQ61gWvUeHHr3UjaCpZg0Nz9Xz+vbF7nqLdauEuuQsLr8IDXd Je/LMPXGueAGMZ+kmsDx1/WOWLKP4p22sSaekQCrz/jVsnAFRdOy2iIZkzYRQVpZc7OoMH1 aYh/lI91z8PjeSfwYH2nQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:JVbkyf1yzbA=:2zbpk9MVMwTY7Thi3VLdM4 Mc4X0sBxUIhSzB/sJ8QKSo0dWoT9dHM7n2FK2PJbBQzm485ydzzO3zwgL+nGUvpvkwjPxfN2C BZN9kvY17uo8L93OsgDAkl5kAtYknX2pJfdZPUwBxGBE7f+7vgZRbPICc3zfIS8317XGfTp0F 6+qHckoRPVInGWbJHssIxfmSmY+djPKvEkYwFSNf5hEDRAWv3OJlcEovp2MTPXi2yM7wi8zFx zTyjRwKhAPIIuAM3keYZMufAhEfupfativgsORAtFb+F9rL+xV1ruPuEXb40cg/k4rtZORi/e /yCLv74Y7VCxGVT6IwbyTPIXXCm09DhPGOIP3I4IT+WRbVw3WAs8owNa5rfeGFMb8l6uWccsB JLxqM8gGxKLjbQDi7k0A325Tum2PGKLPj5tbgoj0ofSDqBVywdP7lfPRKrNmB39f80xUOgLTD 9tDfl8BiTjIs/JI6XGjmq/KHI71upDqnQASwNa9vWSqicNIuZMw9333bTNTMg2aLkzdkr/lOB RvDZjzwDHi7wggl+E9GZl0fXqJjmAjJ47aIphIvPhC2GpwRrwemwzzMivjubBIKJtVLDxyMaA 8imJW1JkPCwu6gPiKkR+WMH/WPROrg8Fq94UhhPbpbzAusA2C/7F7mOA+XSzzSMD3vnx1W007 e+9aN1M3kMg5HZ58GKm0oLQdHG82tnDZZLkFYKnJmZNgcvFCb2R5BWOaSFgSKqA7q6EJ/AXTc a6CPzyPGhKzp/Zj6X2+rAiDkdE7GwCB9Gw3241tOYohXi4PqC3FyAjxtZotsnWuALVQOEKHfl XfwhHnwFnFf4d+8fvT9/eguhJ/f0FR4hhC78a8FysmAsHZbLzJ9K92n01ujSMXfpIMzgOZuIg vUKkK/46isQlNrnxewpFK6Jgt1iJcB5nH8HKT9NLZPHArDb0O+HiG5aadBsbiOZtfoilE2y7r p06MOAlsVZO842hXl62Wez04lgvI/3UJLCdEt6BcbWRIUF8GzWqKmnG/HPiMOwkGe5yh7YhiF I1uG/IPBGdRCZWqxYYiST6WPvG2RyRJ31/m6UMn95WuKpQzv8Ov+sk2B5vqj+PGCJHQXADJ2v CAGGAzwwKtQ0Bq/DKpT0DGaigG5tOnE4VxDF/3s1uHkXdiucX16ciH3+Aq0PcT2Rnr2yAqqfM fOCE8YPg/kxKafxreLIh+UIAxYLxF2+U3OaXEsCc9LAa1Zg6veBh15u5aubF/DCLGk93e7rXU gb6U3QDAsiv6yu7mH Subject: Re: [Bloat] CAKE in openwrt high CPU X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Sep 2020 13:32:20 -0000 Hi Mikael, > On Sep 3, 2020, at 15:10, Mikael Abrahamsson via Bloat = wrote: >=20 > On Mon, 31 Aug 2020, Toke H=C3=B8iland-J=C3=B8rgensen wrote: >=20 >> And what about when you're running CAKE in 'unlimited' mode? >=20 > I tried this: >=20 > # tc qdisc add dev eth0 root cake bandwidth 900mbit That still employs the cake shaper, so is not equivalent with = unlimited, I believe. [PEDANT_MODE] 900 Mbps without explicit overhead will result in a typical maximum = TCP/IPv4 goodput of 900 * ((1500-20-20)/(1500+14)) =3D 867.899603699 Mbps but since ethernet overhead is actually 38 bytes instead of 14 this = actually occupies=20 (900 * ((1500-20-20)/(1500+14))) * ((1500+38)/(1500-20-20)) =3D = 914.266842801 on the ethernet link which for small packets will become problematic: (900 * ((150-20-20)/(100+14))) * ((150+38)/(150-20-20)) =3D = 1484.21052632 Mbps gross speed out of the 1000.0 Gigabit ethernet = offers. in fact, packet sizes below 202 will spend all the "credit" you got from = reducing the shaper rate to 900 Mbps in the first place. (900 * ((202-20-20)/(202 +14))) * ((202 +38)/(202-20-20)) =3D 1000 =20 Maybe tell cake that you run on ethernet by adding the "ethernet = keyword" which will both take care of the per-packet overhead of 38 = bytes and the minimum packet size on the link of 88 bytes? Please note that for throughput this does not really matter that much, = but latency-under-load is not going to be pretty when too many small = packets are in flight... [/PEDANT_MODE] >=20 > This seems fine from a performance point of view (not that high sirq%, = around 35%) and does seem to limit my upstream traffic correctly. Not = sure it helps though, at these speeds the bufferbloat problem is not = that obvious and easy to test over the Internet :) Mmmh, how did you measure the sirq percentage? Some top versions = show overall percentage with 100% meaning all CPUs, so 35% in a quadcore = could mean 1 fully maxed out CPU (25%) plus an additional 10% spread = over the other three, or something more benign. Better top (so not = busybox's) or htop versions also can show the load per CPU which is = helpful to pinpoint hotspots... Best Regards Sebastian >=20 > root@OpenWrt:~# tc -s qdisc > qdisc noqueue 0: dev lo root refcnt 2 > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc cake 8034: dev eth0 root refcnt 9 bandwidth 900Mbit diffserv3 = triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw = overhead 0 > Sent 1111772001 bytes 959703 pkt (dropped 134, overlimits 221223 = requeues 179) > backlog 0b 0p requeues 179 > memory used: 2751976b of 15140Kb > capacity estimate: 900Mbit > min/max network layer size: 42 / 1514 > min/max overhead-adjusted size: 42 / 1514 > average network hdr offset: 14 >=20 > Bulk Best Effort Voice > thresh 56250Kbit 900Mbit 225Mbit > target 5.0ms 5.0ms 5.0ms > interval 100.0ms 100.0ms 100.0ms > pk_delay 0us 22us 232us > av_delay 0us 6us 7us > sp_delay 0us 4us 5us > backlog 0b 0b 0b > pkts 0 959747 90 > bytes 0 1111935437 39440 > way_inds 0 22964 0 > way_miss 0 275 2 > way_cols 0 0 0 > drops 0 134 0 > marks 0 0 0 > ack_drop 0 0 0 > sp_flows 0 3 1 > bk_flows 0 1 0 > un_flows 0 0 0 > max_len 0 68130 3714 > quantum 1514 1514 1514 >=20 >=20 > --=20 > Mikael Abrahamsson email: = swmike@swm.pp.se_______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat