From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 7E4353B2AB for ; Sun, 30 Oct 2016 15:27:29 -0400 (EDT) Received: from [192.168.42.241] ([80.135.107.212]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0MN0jA-1byjua0VAm-006bZR; Sun, 30 Oct 2016 20:27:28 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3251\)) From: Sebastian Moeller In-Reply-To: <1477845791.1404.2.camel@yahoo.com> Date: Sun, 30 Oct 2016 20:27:27 +0100 Cc: cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <0D26D833-A8E0-488D-840C-2B63E853C555@gmx.de> References: <1477845791.1404.2.camel@yahoo.com> To: Georgios Amanakis X-Mailer: Apple Mail (2.3251) X-Provags-ID: V03:K0:JXn+2LSUtFkr7q8Ta05oL2wmYfAYjYmZ0s2JdwtWdMTr+7SKI4O fpCxwoF5IuHV5XW0L1HSjsgRFt+lmppYbYsmlb6LhCRyeiSDTjimwORHZmr5zuuU3v0whZw ztvBu/E3n+dxIAL4ZLBbVzzkc7w1GERrUohQVLP5raU7THc1q93AAVcsaOIVcFEzxuBUwX7 Ve+wpBRsYX0LAWuX3hJZA== X-UI-Out-Filterresults: notjunk:1;V01:K0:HMbq+0n9Q5Y=:firVOawShqkG7aKOm4ane9 4BoTRw0FGUa7fncZ3IB5YzgRnjlvOBnjh50Wd/anAeu2P/yijIcAuejYLb8or4/CfdZF1Y75c wVvOmGipXDX8+0qmAA9xxVjOBCL6O0KV2w7k6fCAQPMpLUfy8BiBKTdZ+1bi0RdljlSShRGj3 tRwxzOyHwLyxMafvS+/OqcrZqcdjaqQWCorxOUVLR8z8FVLuq6qv8wdkGiHYYzZQFUtiOj/OB 8Oek4rsNRMhxzA9csrLhX+tW/pC3KLT0UMgycH+OIk0jmy/2WOEst0oqUieNAhIXL8joZZzH5 TGaqbqNupEHXecEwpO4nZG4wuRKqPIBaaTzjdUSjBpi+eaewrL9eqeASty6uyIIWGoXzmp+B5 8C7Pi3+uounrtnHsQl5ubBb46nKnnipE57kn5uPj8x6tQr+WzCrywNJVXXQtIspkCA8Llbwxf KuR4cCQcdYMDZKoHXcYWJ/0V30flBarwoX13i+HstsN/YSXub9/mlR8pVgl/LM9pEDzgS6I7g 49y/3EKsEN5EL3Sz5xm08Xxk6oBp4oZo1tagbyp+XAHHGqUIvJRCyk3B3wkyGpWctNZuWD729 hfrZwK5ypy6VwcMmbIjoa1nS7073Iiqj3GfE4AdhHNTNOJgdV4jMk2c8itgx/2mkG0NaHwn9a uO8yFfxa21/AXou+54YRxdQUn+906crKfSklN4oU4mUGiqaafbplSU93kauzXk6Thsp9pp89u yEOjffyp6x+wISxmzN9nm4GsFRfzjrngr1AzM8cwoYUgN3L8S6S09MTm+BGXzyHqAVUxI6JTg JyaqHqc Subject: Re: [Cake] WAN ingress rate with concurrent downloads X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Oct 2016 19:27:29 -0000 Hi Georgios, > On Oct 30, 2016, at 17:43, Georgios Amanakis = wrote: >=20 > Dear All, >=20 > I think Sebastian's theory has a weak point.=20 Oh, I do not claim to be an expert on these matters, so would = cherish input from others, but I do think my hypothesis is not to be = dismissed completely (see below). >=20 >> Again I believe you just show bit-torrent misbehaving ... >=20 > I can easily reproduce this behaviour if one LAN-host downloads > multiple files using TCP (e.g. 2 or more kernels simultaneously from = ht > tp://www.kernel.org).=20 >=20 >> For me this looks like bwmon accounts the incoming packets before >> cake sees and drops them ... >=20 > If only one download takes place then the bandwidth bmon reports on > WAN-ingress is the same as the limit set in cake. I don't think that > bmon accounts the incoming packets before cake sees them. That could be true, but also should be testable. According to = http://bwmon.sourceforge.net bwmon reads /proc/net/dev to get its = estimates, so =E2=80=9Ccat /proc/net/dev=E2=80=9D should show you what = is in there: Directly after reboot: root@turris:~# cat /proc/net/dev Inter-| Receive | = Transmit face |bytes packets errs drop fifo frame compressed multicast|bytes = packets errs drop fifo colls carrier compressed gre0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 wlan0: 526875 2287 0 0 0 0 0 0 = 693742 2001 0 0 0 0 0 0 sit0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 gretap0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 lo: 2175 26 0 0 0 0 0 0 = 2175 26 0 0 0 0 0 0 ifb4eth1: 740189 2097 0 0 0 0 0 0 = 740189 2097 0 0 0 0 0 0 eth0: 0 0 0 0 0 0 0 0 = 17543 84 0 0 0 0 0 0 eth1: 752467 2167 0 0 0 0 0 0 = 541001 2390 0 0 0 0 0 0 eth2: 0 0 0 0 0 0 0 0 = 17497 83 0 0 0 0 0 0 ip6tnl0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 wlan1: 0 0 0 0 0 0 0 0 = 17875 76 0 0 0 0 0 0 br-lan: 494863 2288 0 0 0 0 0 0 = 627972 1881 0 0 0 0 0 0 ifb0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 teql0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 ifb1: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 After a dslreports speedtest with 16/16 streams: root@turris:~# cat /proc/net/dev Inter-| Receive | = Transmit face |bytes packets errs drop fifo frame compressed multicast|bytes = packets errs drop fifo colls carrier compressed gre0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 wlan0: 20160809 37095 0 0 0 0 0 0 = 48570975 44819 0 0 0 0 0 0 sit0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 gretap0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 lo: 2453 28 0 0 0 0 0 0 = 2453 28 0 0 0 0 0 0 ifb4eth1: 47764269 44632 0 0 0 0 0 0 = 47764269 44632 0 0 0 0 0 0 eth0: 0 0 0 0 0 0 0 0 = 20435 100 0 0 0 0 0 0 eth1: 48135827 45333 0 0 0 0 0 0 = 19357920 36538 0 0 0 0 0 0 eth2: 0 0 0 0 0 0 0 0 = 20389 99 0 0 0 0 0 0 ip6tnl0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 wlan1: 0 0 0 0 0 0 0 0 = 21047 92 0 0 0 0 0 0 br-lan: 19641491 37097 0 0 0 0 0 0 = 47617521 44254 0 0 0 0 0 0 ifb0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 teql0: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 ifb1: 0 0 0 0 0 0 0 0 = 0 0 0 0 0 0 0 0 root@turris:~#=20 Please notice how eth1 reports initially 2167-2097 =3D 70 packets more = than ifb4eth1 (the shapers ingress qdisc is attached to the IFB as linux = does not allow shapers on ingress without the ifb trick, policers = however can directly act in ingress). After the test we see 45333-44632 = =3D 701 Packets more on eth1 than ifb4eth1. I take this as supporting = evidence for my hypothesis... > Furthermore, > Wireshark reports that all bittorrent traffic in my case are TCP > packets, and reports the same bandwidth usage as bmon on WAN-ingress.=20= Well, if both look at the raw enp0s13 interface I would expect = both of them to report the traffic before packets can be dropped. >=20 > The only way I have found that WAN-ingress limit is obeyed, is using > act_police with the in-kernel bandwidth estimator (i.e. tc filter add > dev enp0s13 parent ffff: protocol all estimator 10msec 80msec u32 = match > u32 0 0 police avrate 3300kbit drop flowid :1). As stated above, I believe I recall that policers can directly = act on ingress and hence affect the number of packets reported for that = interface... > But then there is an > discrepancy between WAN-ingress and LAN-egress (i.e. WAN-ingress is at > ~400kbytes/s and LAN-egress on the host running bittorrent is > ~220kbytes/s). Unfortunately I cannot see the bandwidth my cable-modem > syncs at (I searched the web-interface but did not find it; it is a > Comcast-provided Arris TG1682G). With DOCSIS systems there is no clear and easy way to get hod on = the relevant numbers (which would be the numbers your ISP plugged into = the CMTS shaper and information about potentially allowed bursts). >=20 > Noah, could you try and see what happens if instead of bittorrent a > LAN-host downloads multiple files using TCP? I would also be interested in the outcome of that test, please = keep posting to the list. Best Regards Sebastian >=20 > Thank you all for your feedback, > George > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake