From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DF7753B25E for ; Fri, 28 Oct 2016 03:45:54 -0400 (EDT) Received: from [172.16.11.213] ([134.76.241.253]) by mail.gmx.com (mrgmx102) with ESMTPSA (Nemesis) id 0M4TgW-1cpxkr0E2e-00ykaP; Fri, 28 Oct 2016 09:45:53 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3251\)) From: Sebastian Moeller In-Reply-To: <1477618454.22786.3.camel@yahoo.com> Date: Fri, 28 Oct 2016 09:45:46 +0200 Cc: cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <16BBEB07-B0D4-48BD-B18B-491330038CB4@gmx.de> References: <758C5432-757C-4CFA-BA20-48F6B6D32839@yahoo.com> <0FA74BC3-33E0-4CF8-B134-B688EA0A4DBB@yahoo.com> <1477618454.22786.3.camel@yahoo.com> To: Georgios Amanakis X-Mailer: Apple Mail (2.3251) X-Provags-ID: V03:K0:2Z95lZ8j2C7FdE2L9O/p4WLyBDQB1NGEnjf2Q5epiSTEOCVQR92 6hQrANR/3aRPr9y0itCSNa6UcNFcLOAhmQfxA0quBCXl9O97Jfg/0DLbI0meUEuzhzxsq0n 3wIMrClXU253EoUkHeBp0xEdiYjJvNepvkYLVdtTnjganl0D9HAx0RdZW/OhriO/1pAJfaJ W4f7vywVVAr8DT/tz1YMQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:1kpZaz9AOwI=:lKJiG8mfr+wBtcECR7rNNT x/Hi2AjbV97WAYwFr5+AQ49kvHD0djL1Mb2Lnhf0j6jUOjSHbXErwjk21H5Dj1lOEi4LOZNVz 3pVSQahBLSp/I/MlQw6xdZ5n0qAr8mA5NRa9Q8ty/zAmFGCq2NC+BdUzCPSDT5ARR34g8Smky NBGSYA/iaGFwNs6aLggyOdTv8XKDNr36tD05Mljw8zCtz1jBfXoowWzSshpGMJ9VBIJUtGL4g aqZ7XfDkoNBpJV4ryqryoiqCk8oex5xo2ES3xzxp9k8Jl9CHXn5AL8f8c9M723CRn4HQQTKUl C1Ov9hHdl6c+k2LBjVCS9gPs3Xfk5JYiu7yxQqIWi7uqOdvQ7blB4VkHR9k5WuK27qM3OnI8B HAbycywD6K+WLGnQTl8CYE1OcfmNgLTGjfOHGc82YqJKgQqOF19W27hz40l6CADwa6aNPq9Vi wS2agkArmIBUXbzgKLX9oj7RR0E5ScDyX3yTh3tmbdkLPpPFApyiK00GCaRRQPsdwnGuNfAUO APg+nKnN/3vuS1LiziMygO2f4Q6j7cbA0a2YYlH1b9I9s63kODdYFR5jriexZmJNyq6f+TQFI fFstz/PmKZ84ICXudfpOkyHp5YuHJWRH1u/kc26gKcpjT99zxz8KmSlxf5Urrrh4EfFhRUVCm na8E7nEpO5piHeslR4xC8l/1zEdAD9/Xu1bhtGmuop0ME19IHNG9AMLuwjM5XT//Zm9mAFLbE yCDhiua9Wco8gaU/UTgCloi6QdDUojT+hsvezh1wXxv8xrHJEG4DyrwIY/zEN5PNoWd5XzrlS qOMsczd Subject: Re: [Cake] WAN ingress rate with concurrent downloads X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Oct 2016 07:45:55 -0000 Hi Georgios, > On Oct 28, 2016, at 03:34, Georgios Amanakis = wrote: >=20 > Hi Sebastian, >=20 > the ISP is Comcast in MD, USA, residential connection. It is a cable > connection (i.e. the WAN port on the modem is a coaxial one) and not a > DSL one. Interesting, from the bandwidth I had assumed ADSL ;) > Maximum ingress rate is ~450kbytes/s which is what the modem > syncs at. MMh typically sync-rates are soecified in Kbps or 10^3 Bits per second, = not KiloBytes and certainly not 2^10 Bytes; could you clarify what = exactly your modem reports and what the best numbers you get from = speedtests (like dslreoorts.com) without active AQM/cake/htb, please? > By using "test_WAN_dual-isolate__piece_of_cake.qos", "tc -d > qdisc show" reports (enp0s13 is WAN, enp0s14 is LAN): > -------8<-------- > qdisc noqueue 0: dev lo root refcnt 2=20 > qdisc fq_codel 0: dev enp0s14 root refcnt 2 limit 10240p flows 1024 =09= > quantum 1514 target 5.0ms interval 100.0ms ecn=20 > qdisc cake 8002: dev enp0s13 root refcnt 2 bandwidth 900Kbit = besteffort > dual-srchost nat rtt 100.0ms raw=20 > qdisc ingress ffff: dev enp0s13 parent ffff:fff1 ----------------=20 > qdisc fq_codel 0: dev ifb0 root refcnt 2 limit 10240p flows 1024 =09= > quantum 1514 target 5.0ms interval 100.0ms ecn=20 > qdisc cake 8003: dev ifb4enp0s13 root refcnt 2 bandwidth 3300Kbit =09= > besteffort dual-dsthost nat rtt 100.0ms raw=20 > -------8<-------- >=20 > If one of the hosts on LAN is running bittorrent, bmon on the gateway > reports: > RX bps RXpps TX bps TXpps > enp0s14 23.55KiB 303 = 402.87KiB 322 > enp0s13 457.26KiB 369 25.04KiB = 307 >=20 Okay so: 900Kbps =3D 900 * 1000 / (8 * 1024) =3D 109.86328125 KiB 3300Kbps =3D 3300 * 1000 / (8 * 1024) =3D 402.83203125 KiB So it looks like enp0s14-TX actually shows the specified = internet-download rate, (internet upload is not maxed out) >=20 > Setting the ingress rate at 1900kbps (instead of 3300kbps as above), > "tc -d qdisc show" reports: > -------8<-------- > qdisc noqueue 0: dev lo root refcnt 2=20 > qdisc fq_codel 0: dev enp0s14 root refcnt 2 limit 10240p flows 1024 =09= > quantum 1514 target 5.0ms interval 100.0ms ecn=20 > qdisc cake 8008: dev enp0s13 root refcnt 2 bandwidth 900Kbit = besteffort > dual-srchost nat rtt 100.0ms raw=20 > qdisc ingress ffff: dev enp0s13 parent ffff:fff1 ----------------=20 > qdisc fq_codel 0: dev ifb0 root refcnt 2 limit 10240p flows 1024 =09= > quantum 1514 target 5.0ms interval 100.0ms ecn=20 > qdisc cake 8009: dev ifb4enp0s13 root refcnt 2 bandwidth 1900Kbit =09= > besteffort dual-dsthost nat rtt 100.0ms raw=20 > -------8<-------- >=20 > Using bittorrent, bmon on the gateway reports: > RX bps RXpps TX bps =09 > TXpps > enp0s14 19.25KiB 219 232.51KiB 184 > enp0s13 393.36KiB 304 20.25KiB 220 >=20 Okay so: 900Kbps =3D 900 * 1000 / (8 * 1024) =3D 109.86328125 KiB 3300Kbps =3D 1900 * 1000 / (8 * 1024) =3D 231.93359375 KiB And again enp0s14-TX shows the specified internet-download rate, = (internet upload is not maxed out) For me this looks like bwmon accounts the incoming packets before cake = sees and drops them (the enp0s14-TX shows that cake does only transmit = the specified rate into the LAN, so cake does its job)). Then the fact = that enp0s13 (btw these new interface names have not yet grown on me, = yuck) sees more incoming packets simply means that the bit torrent peers = are an unruly bunch and do not react nicely to cakes drops and marks = (are the packets ECN labeled?). Bit-torrent, when using its =C2=B5TP = protocol tries to use increasing delay as signal to throttle, but cake = removes that latency increase under load, which mjight make the torrent = peers not scale back their TX rates based on a wrong believe that there = is more link capacity available. Not that anybody asks me, but I believe = =C2=B5TP actively is a bad thing and all torrent peers shoud default to = TCP instead of =C2=B5TP?UDP, but I digress. > In both cases the ingress rate limit on the WAN interface (enp0s13) is > not obeyed. I verified these data by using tcpdump and doing an IO- > Graph with wireshark. Well that is true in that cake does not run a lower rate on the = ethernet interface, but what it does happens after packets are passed = from the NIC to the OS. Again I believe you just show bit-torrent = misbehaving, a nice illustration why we can=E2=80=99t have nice things, = erm why non-latency-inducing shapers, like cake and HTB+fq_codel have a = hard time ruling in bit-torrent=E2=80=A6 BTW, can you see from your = packet captures how long a typical torrent-flow actually exists and is = active? So are these say a few dozend long-term flows or a swarm of = dozens of flows that come and go on a seconds time-frame? Best Regards Sebastian >=20 > Looking forward to your feedback, P.S.: I am just an amateur in these matters, so other=E2=80=99s on the = cake list will surely be able to give better advice. So @experts, please = holler if my theory above does not make sense... > George > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake