From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E635B3B2A3 for ; Fri, 17 Mar 2017 16:11:03 -0400 (EDT) Received: from [192.168.1.222] ([80.135.64.143]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MMGWH-1cqNNz0sof-00805g; Fri, 17 Mar 2017 21:11:02 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) From: Sebastian Moeller In-Reply-To: Date: Fri, 17 Mar 2017 21:11:01 +0100 Cc: "bloat@lists.bufferbloat.net" Content-Transfer-Encoding: quoted-printable Message-Id: References: <57A046A6-C0F2-4209-93E5-CA728F4C64EE@gmx.de> To: xnor X-Mailer: Apple Mail (2.3259) X-Provags-ID: V03:K0:gZjBg5qovEypTqdGf1XbAxhLP1uCEto6VdAsSa3HgF8GF9/QyMM 2frmVnJxNwm+3Wxhf1sJsnfk5ET3YPd4likw83QJpiQb3NLSBkXlGjlfj0oWDkU+MysJecW 39LlcQ3Gxo6cJyaRYbxTT66QfV0CLVXPJg4W9eVhwr85f7cQWAPhdA4O/tsD1lYDDmMHCkE MnlTuzykK34zHi1w3MKyw== X-UI-Out-Filterresults: notjunk:1;V01:K0:MTPALuolCvo=:SHXhfe8tFwbS2D+zt2HU1A oy4DGhXqxHXSalMjqsyjpR1vx7KOEvySfCirXc3i5gul4pTA3j+vCd/MVh31L1fgL/JWJvWHs 6unxsEDjNc9QIQ+m7GovHT8U+t/qfeqMafy4yTCXdrN7xUuUv8HZZhIvU4RM1dEbSsFHvnN+3 kuMq+LjdB2D92cMfUhspLVgmv3vJwlWHCRPHWftvCifhATZAStHvNj65tqVL4Qg+m7G6hBu8j gWhSlqz4dDrJr7bGsCFpw6DyDie25sNxZ+x8YS6YjYaLOB4xtf/GUpP9rm6H41HBEUWU6SXTJ TdaEy+5MXigm7oPhVpFE5asQ9QV1UD09kSERbnbu1DE6wlsGMJEJCNwFJMdr6IsppVipGQRg+ bYdYdD8WwZuRJWAjTO34D2y6vTz42sRSda3Q7o/1d4hAA1dbJ+y4uJm5LDf+vTNEMRrzcRG4b Aosb+ellg5QEIHdGdHDt4gjS56N91JzvwMTUutqSVHsWgpu7yGy4xAdNGJ78HE4TpwoOvrXzl em8pgdOh3rL1VWphuHiDvzGxHzS9LlM18lB+l45IBTR4aNpuV3lZ1wFS947qt8WvN2zIlHRf8 b9aBg4Gp9s30J2LinC6/oBa6VdN29kWipfXslZx7qx/Ma9YFxuJ5QozOxnSbYmWEglm1mK0RO EijJqUNQCUwt7MH1gA4ALaGc52c2n4s3YZGSf6PFY6GR8GlqkQRb2uxVUEjjDCfQzzZodu32g VqBiVIdViFSlXUuSukrk4aIBn5NvE2CgcoO3M9hnAzBaYB/oUz6ZRk84s+aBI3NywzoAyhGj2 SGH06AX Subject: Re: [Bloat] Inaccurate rates with HTB/HFSC+fq_codel on router due to VLAN? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Mar 2017 20:11:04 -0000 Hi xnor, > On Mar 17, 2017, at 20:15, xnor wrote: >=20 > Hey, >=20 > please reply to the list address so that everyone can see it. Sure can do, not a secret (at least not intended as a secret). = Then again, if it had been you would have spoiled it effectively... >=20 >=20 >> Just a guess, but I believe /sys/class/net/eth0.2/statistics/rx_bytes = shows the data before HTB/HFSC had a chance to touch the packets. > Of course. So? HTB/HFSC don't shrink packages, and since I'm only = doing TCP and also use fq_codel it should shape ingress quite nicely to = my configured rate. Erm, so TCP tends to probe the link capacity by increasing its = bandwidth repeatedly, given enough flows some will always be touching = against the limit. If you look at the rate packets emerge from the = shaper instead the rate they rare fed into the shaper you would have = IMHO a stronger argument that the shaper does not shape correctly=E2=80=A6= If after the shaper the rate is as configured but too large before it = could mean that some of your senders do not respond nicely to the = slow-down signal, or the slow down signal is unintelligible to the = senders (like the shaper assumes ECN while the senders do set ECN bits = but do not respond properly to the ECN signaling. >=20 >> it would be interesting to repeat the tests but run tcpdump on the = interface that delivers the traffic to the test machine on the internal = LAN. My prediction is that on that port you will pretty much see the = 18000Kbps you expect. > The LAN is connected through eth0.1 which is part of a bridge = interface br-lan (this bridge is the only other interface with an IP = address besides eth0.2). >=20 > With 160 download connections I've just measured (also included the = average bytes per packet, short bpp): >=20 > eth0.2: 20.3 Mbps download (~1400 bpp) > eth0: 21.6 Mbps download (~800 bpp), 19 Mbps upload (~780 bpp) > eth0.1: 18 Mbps upload (~1490 bpp) > br-lan: 18 Mbps upload (~1490 bpp) >=20 > (all numbers approx. accurate to about +/- 0.1 Mbps) I am confused about the reported directions here, but mostly by = the average packet sizes on eth0, why are these at 100-100*800/1490 =3D = 46.3087248322 % of the internal packet sizes? Does this indicate massive = fragmentation on your wan link? >=20 > This is completely saturating the 20 Mbps link and ruins performance. It seems your router only has one port from the SoC to the = switch and the WAN port also lives on the switch, so all packets will = need to first go from WAN port to CPU via eth0 and then out again via = eth0 to the LAN port on the switch, is that a correct interpretation? In = that case you can also instantiate the internet download shaper as = egress/upload shaper on eth0.1 avoiding the ifb dance...=20 >=20 > I've tested to decrease the rate to make it work in the above = scenario: I had to back off with the rate to about 14 Mbps (!) because = then, as you can guess, measured eth0.2 bandwidth drops to below the 20 = Mbps link speed. Well the true link speed should be a hard ceiling... >=20 > With less connections the measured eth0.2 bandwidth is closer to the = configured 18 Mbps and so works fine.. Again, incoming data is first handled and accounted by = eth0/eth0.2 before the IFB sees it, so I very much expect to see more = than the shaper rate at this point. If the shaper emits the correct rate = into your internal network and also drops packets it seems unavoidable = that more than the configured rate worth of data arrives into the = shaper. But I am not an expert and hence might be wrong. >=20 >=20 >> This seems old, if your router is supported you might want to try = lede (https://lede-project.org/releases/17.01/start#), then you could = also use the cake qdisc which has a few new tricks up its sleeve, nicer = than HTB and HFSC... > I am planning to upgrade, but I highly doubt it'll help and it also = doesn't help me clear up the confusion with what is going on here. Well you wou;d jump from kernel 3.18? to a more recent 4.4 = series kernel which might/should have some bugs fixed. But I understand = the desire to understand the current situation better before moving on. >=20 >=20 > It's definitely shaping something. The question is: what? Packets? Best Regards >=20 > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat