From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 24B3D3B2A4 for ; Fri, 21 Apr 2017 09:25:03 -0400 (EDT) Received: from [172.17.3.29] ([134.76.241.253]) by mail.gmx.com (mrgmx001 [212.227.17.190]) with ESMTPSA (Nemesis) id 0LoEwL-1cMBKZ1smf-00gFR9; Fri, 21 Apr 2017 15:25:02 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Sebastian Moeller In-Reply-To: Date: Fri, 21 Apr 2017 15:25:01 +0200 Cc: Jonathan Morton , cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <2084C319-1EFE-47BC-BCA1-2534A4E2272D@gmx.de> References: <05C0B0C7-4337-4115-AC6B-DA81392FCB34@gmail.com> <22E633CF-5EE0-4B0F-89A8-B790E730FB6C@gmx.de> To: Dendari Marini X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:mcURlGD3miS5KPTgwt9bBlPEitzd4BeovF48X1Zwc+pXEXbBRwg 0dNAfgsMaUrxdxFrHJQtY/7CNmBBYbieXTH7+SEAFBgQIg/tG9YCI3Vq6A3Mb61+1hML8cz 4FuGbwK/cfpXZVAJmdEPx2QascrHvhxv0D8twwzUrr50Emy6enyaCiYoq0avvcldfRERny8 StqAw7SojTANx8ko52DxA== X-UI-Out-Filterresults: notjunk:1;V01:K0:G9nLgELom9Y=:ofJmRMioTNEO2O1GvRzDzG WEspym2AvgxflIJOyN7x1yAqo3vSxDm240LeFuApzXhNtwNMOVcNg2RN4UTPywiZQvZrSqQ0w IlFS/ULDkHiM0RcIaLJVB/ANT7CPk4pfrD+e5agKs81jH7yzACR/AY6ciGjU5Lr/LqQLMJiwk y7azXYTeqqHIkJu9k3uoigIdExg8JATiBnCnXOIN6GY9KBTqWIR8gA1bCcWYvg5JfvGE40s1F p7ejHDhKGNl0yc1wnoHdoxAmunZa1vdihCAQyjbDRfkIunIs0KBUMNZXqALPyrTUN3/Bb/r0t RmWEmKk1HSeZQmFei1MM4AjpVxLSzs7OZzTxMmPJf7nlOF6VgHFapIA9+V2ObNirgDV53OsnO n0CyWQcqQOWVnKFC+h4+DHMWyZV51MFDMMNiYZuB7xgHgiq4dadmkk5qv+scOEZoq9flCuNgR srzQq4UFcUdSFBFXFuQJY3bZtcJaRodI59jZ2BA0nECqqIJH27r5Kch/N40FuQjo2ldqOEeEQ wkbssEgFZAY+J24m/GniJnE9neL68Q0AB9NbHQ3Tp4+fP4L+i2b0Hc1l2kpg7/HlX4ce4xAd4 ufNXm5uVZU82GfU8QgaBv4fedD42ZpJvxlOBjFsInwpYABY2sdKzFDt9UYgePvlezlKCOzNxE xFvidJJSBNjbDX8V0GGrSnAkRsZ5hKyitPWfVzXA/yiaDRb3k3mqDgvsZVAdpESvJAnQMs6UP E56YXIyjE+TE8wbX0+BxQYNl8r0RrJdxusPXvgoW8FnQ7+aK8H5Gy6HW1JRYsrfQGSFjprucD chy8Hzb Subject: Re: [Cake] Getting Cake to work better with Steam and similar applications X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Apr 2017 13:25:04 -0000 Hi Dendari, > On Apr 21, 2017, at 10:34, Dendari Marini wrote: >=20 > Hello, thanks for all of your replies. >=20 > First of all, my connection encapsulation should be ATM LLC and it can = actually reach up to 17.5/1 Mbps, but that's kinda best case scenario = which is why I wanted to play it safe with just 16/.9 (which I should = reach more consistently). Okay, I read ATM LLC together with the fact that you have a = choice of et0 and pppoe0 that you hace 32 bytes of overhead (well = including the 8 byte pppoe header it would be 40): case 40 disp('Connection: PPPoE, LLC/SNAP RFC-2684'); disp('Protocol (bytes): PPP (2), PPPoE (6), Ethernet = Header (14), ATM LLC (3), ATM SNAP (5), ATM pad (2), ATM AAL5 SAR (8) : = Total 40'); overhead_bytes_around_MTU =3D 32; overhead_bytes_in_MTU =3D 8; So please add =E2=80=9Catm overhead 32" to cake on eth0 or =E2=80=9Catm = overhead 40=E2=80=9D to cake instances on pppoe (these packets do not = have the PPPoE header added yet and hence appear 8 bytes to small). With that information you can expect the following maximum TCP/IPv4 = goodput: 17.5 * (48/53) * ((1500 - 8 - 20 -20)/(1500 + 32)) =3D 15.02 Mbps=20 or=20 (17.5 * (48/53) * ((1500 - 8 - 20 -20)/(1500 + 32))) * 1000^2/1024^2 =3D = 14.32 Mibps or 1.79 MBps, still pretty close to the 1.75MBps you seem = to measure. Please note that if you do not specify the =E2=80=9Catm=E2=80=9D= keyword your shaper will not account for the static 100-100*48/53 =3D = 9.43 % rate increase going from ethernet frames to atm=E2=80=99s AAL5, = so to account for that part alone you would need to set your shaper to = 17.5*0.9 =3D 15.75 Mbps, but that completely ignores the other overhead = (and the interger atm cell per ethernet frame rule that AAL5 insists = upon) > Back to the Steam issue. Unfortunately I can't seem to get really = consistent results, mainly because sometimes it's downloading the game = from just a few connections and other times it's downloading from dozens = and dozens connections. The latter is the one giving me more issues both = in terms of latency/packet loss and in terms of evenly splitting the = bandwidth across the hosts. Question: if you set the shaper=E2=80=99s to 50% of line rate = (8.75/0.5?) do you still see that unfairness? And if you add =E2=80=9Catm = overhead 40=E2=80=9D to cake on pppoe0 and set the shaper to 90% of line = rates (15.75/0.9) how does the Steam affect per-host fairness? Also how = transient are these connections team uses? >=20 > One thing that seems to give better results is changing the interface = where Cake is used from eth0 to pppoe0. When I used fq_codel it seemed = to give better results when using eth0 and so I went ahead and did the = same with Cake. As far as I can tell cake can drill down to the required = IP/TCP/UDP fields independent of whether there are VLAN tags or PPPoE = headers so cake should not care (except for the different overhead = specifications you need to add as stated above). BUT if instantiated on = eth0 cake will see pppoe LCP packets and might decide to drop them, = which can take down the link, so out of caution I would still = instantiate on pppoe in your case. >=20 > Anyway more testing needed, will report if I notice any consistent = result. >=20 > By the way this is the thread I opened on the Ubiquiti forums talking = about this issue (not sure if it can give you some more info): = https://community.ubnt.com/t5/EdgeMAX/Smart-Queue-seemingly-not-working-fo= r-Steam-downloads/td-p/1890405 > Also the thread where I got Cake for the ER-X from: = https://community.ubnt.com/t5/EdgeMAX/Cake-and-FQ-PIE-compiled-for-the-Edg= eRouter-devices/td-p/1679844 >=20 > On 20 April 2017 at 20:36, Sebastian Moeller wrote: >=20 > > On Apr 20, 2017, at 18:05, Dendari Marini = wrote: > > > > Hello, thanks for your reply. > > > > Looks like most of your options are okay, including the correct = =E2=80=9Cdual=E2=80=9D modes and =E2=80=9Cingress=E2=80=9D mode in the = right place. However, I think you need to adjust your bandwidth and = overhead settings, otherwise Cake isn=E2=80=99t reliably in control of = the bottleneck queues. Try these to begin with: > > > > =E2=80=A6 bandwidth 850Kbit conservative dual-srchost nat > > > > =E2=80=A6 bandwidth 15Mbit conservative dual-dsthost nat ingress > > > > That should give you correct operation, and you can fine-tune from = there. > > > > Just did quick test with your settings. First thing I noticed is my = final download bandwidth is about 12Mbps, Steam on PC1 downloads at = 1.4-1.5MB/s while downloading a file on PC2 seems to max out at = ~250KB/s. =46rom my understanding I should see each PC download at = ~700KB/s, or am I mistaken? >=20 > Assuming you measured good put in [M|K]iBytes this adds up to = 1.5+0.25 =3D 1.75 * 1024^2 * 8 =3D 14680064 Bits or (1.4+0.25) * 8 = *1024^2 / 1000^2 =3D 13.84 Mbps which seems a bit high for a 16Mbps ADSL = link. I would ecpext something like 16 * (48/53) * ((1500 - 8 - 20 -20) = / (1500 + 32)) =3D 13.73 Mbps TCP/IPv4 goodput=E2=80=A6 so you seem to = be running close to theoretical maximum of your link (assuming I am not = totally off with the overhead (estimated ADSL overhead on top of MTU: 6 = destination MAC + 6 source MAC + 2 ethertype + 3 ATM LLC + 5 ATM SNAP + = 2 ATM pad + 8 ATM AAL5 SAR 32 bytes). But with your shaper set at 15Mbps = without the atm option you will actually accept up to 15 * (53/48) =3D = 16.5625 Mbps on the wire, which probably is above your link bandwidth. = This fits well with the really low number of drops in your cake stats, = you simply never have cake feel that shaping is needed? >=20 > Best Regards >=20 >=20 >=20 >=20 > > > > On 20 April 2017 at 17:32, Jonathan Morton = wrote: > > > >> On 20 Apr, 2017, at 18:23, Dendari Marini = wrote: > >> > >>> Could you post the output of calling =E2=80=9Ctc -s qdisc=E2=80=9D = here on the list please? That should allow to figure out what you = actually told cake to do ;0 > > > >> qdisc cake 8001: dev eth0 root refcnt 2 bandwidth 900Kbit diffserv3 = dual-srchost nat rtt 100.0ms raw > > > >> qdisc cake 8002: dev ifb4eth0 root refcnt 2 bandwidth 16Mbit = diffserv3 dual-dsthost nat ingress rtt 100.0ms raw > > > > Looks like most of your options are okay, including the correct = =E2=80=9Cdual=E2=80=9D modes and =E2=80=9Cingress=E2=80=9D mode in the = right place. However, I think you need to adjust your bandwidth and = overhead settings, otherwise Cake isn=E2=80=99t reliably in control of = the bottleneck queues. Try these to begin with: > > > > =E2=80=A6 bandwidth 850Kbit conservative dual-srchost nat > > > > =E2=80=A6 bandwidth 15Mbit conservative dual-dsthost nat ingress > > > > That should give you correct operation, and you can fine-tune from = there. > > > > - Jonathan Morton > > > > > > >=20 >=20