From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D18F73B2A3 for ; Sat, 22 Apr 2017 12:47:30 -0400 (EDT) Received: from [192.168.42.241] ([80.135.73.48]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MZU7V-1cj3ld3mqD-00LDz9; Sat, 22 Apr 2017 18:47:29 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Sebastian Moeller In-Reply-To: Date: Sat, 22 Apr 2017 18:47:27 +0200 Cc: Dendari Marini , cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <0BA3EE91-C5BC-4155-9D5D-D15D34490A1A@gmx.de> References: <05C0B0C7-4337-4115-AC6B-DA81392FCB34@gmail.com> <22E633CF-5EE0-4B0F-89A8-B790E730FB6C@gmx.de> To: Jonathan Morton X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:e0dlL62j1qnA+FaygunRzw+48GaxDU7hE38gDXGm4lUQonjvkn0 vW37M6/E+pDlBl5ZkWN/2qbzf7XER0er0UE7c0w3+Ed8tIMA7FM4x+va7XZ1K7F6SXzpOyA 7A0rjV6HI7hph5BobFnb2WGWibXwPM0phxcAts2j0VstxfU8HXQNkDjSVZxkT/GL82CJ0bx o4/0GqkzBVlTCjOs9dU7g== X-UI-Out-Filterresults: notjunk:1;V01:K0:JlTs05apvls=:rkRCsLR/R9C01DWGEWON5S 5f7PNzF6qagK/hls/Ew4aOtWDT5NLSYhMoOKydPXiJNmN7bdgJdaHE6lCvjvYM75YpRw2DjDg raatPQHCwKZq8Cv2kkb8iblS+dyayyMdY7MrvFg7zEw7vm26LIhuicprJeLeIS3gnJAPU5s6K 1obpqpLdB5VHkBZ8mDVSAj2693mz3JDp5XubB+tPmw0HjZyKOxvBadxmDcHDFEHz1POrVo93u 84wdFsxDOAFM7Okgsvi5BYjcdVS3eMDr2Mvf4t2R6fZ5dY8ZOBsIPGFBn4tX7X2uA88Bp4dui n2QAQ45sFTV/Tq18VStMHAutfSSkuu0MlemyR12U8gaRwmZPTD/kclF3mxtzb/3vyoj6WMS1e hkOO9JalzrjAcCwK9Lbt2bJCvFUutom2nMo2xGTPlz0UVlegTKXr38bHhaWNbS0cCxg3IGFYM fzqHDiFOlMuAtz3yJ1nkOwtGQUaTPzNwKYy1gZeEk9nK4Jwrbcdt7oKHIfHuNXs4knq711UMA M4LMFjbFCqSE3tMUv6ZvoSPmSchPgN40K4CWEGAt45KGJLFghZHbkSGNHqXnGWrwHmR7NbvZX jUi3IB48quLWmVXbmIwYG5iFk+hcK/7DqfN/9kmGwrAb1xal4NAlB4oJbi9f10bSpQM4I/HUG QIXRZ0K5/YhfaqTE7YvCYPg2u1MeoTPwshG3Kf8r5jFb+M81ZEUN/WGVKev+dkAu/jUT6uZem ckA/8YzRbznByiS4FqipbBIS1yWy25O+M2gStOJU/4fJTfJyzq9X+dRhFeNNu5ymsvcXlYZBT 4ELZxk/ Subject: Re: [Cake] Getting Cake to work better with Steam and similar applications X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Apr 2017 16:47:31 -0000 > On Apr 22, 2017, at 11:36, Jonathan Morton = wrote: >=20 >>> So please add =E2=80=9Catm overhead 32" to cake on eth0 or =E2=80=9Cat= m overhead 40=E2=80=9D to cake instances on pppoe (these packets do not = have the PPPoE header added yet and hence appear 8 bytes to small). >>=20 >> Thanks for your help, will definitely use them. Just wondering if I = use "pppoe-vcmux/bridged-llcsnap" on eth0 or "pppoe-llcsnap" on pppoe0 = would have the same effect? Or are there some other "under-the-hood" = changes when using them? >=20 > On the pppoe interface, use pppoe-vcmux if your modem is set to use = VC-MUX, or pppoe-llcsnap if it=E2=80=99s set to use LLC-SNAP (they might = be described using slightly different terms, but should still be = recognisable as one or the other). This probably depends on your ISP, = and may further vary regionally within the same ISP. In my experience it is rather bothersome trying to get that = information from one=E2=80=99s ISP, this is why I would recommend to = follow the instructions on = https://github.com/moeller0/ATM_overhead_detector and empirically = measure the actual overhead. In that case one ends up with the numeric = overhead, hence my inclination to use that number directly instead of = looking into a table to translate that back into a symbolic keyword=E2=80=A6= especially since say for an overhead of 32 (and 36) there are two = different encapsulation schees that add up to that number: case 32 disp('Connection: Bridged, LLC/SNAP RFC-1483/2684'); disp('Protocol (bytes): Ethernet Header (14), ATM LLC = (3), ATM SNAP (5), ATM pad (2), ATM AAL5 SAR (8) : Total 32'); disp('Connection: PPPoE, VC/Mux RFC-2684'); disp('Protocol (bytes): PPP (2), PPPoE (6), Ethernet = Header (14), ATM pad (2), ATM AAL5 SAR (8) : Total 32=E2=80=99); good luck divining which of those is in use if all you know is the = numeric overhead... >=20 > I really prefer to use the self-explanatory keywords (which is why I = added them in the first place) instead of opaque magic numbers. This is = a point on which Sebastian has long disagreed with me. True, but I am not going to re-hash that here again ;) >=20 >>> Question: if you set the shaper=E2=80=99s to 50% of line rate = (8.75/0.5?) do you still see that unfairness? And if you add =E2=80=9Catm = overhead 40=E2=80=9D to cake on pppoe0 and set the shaper to 90% of line = rates (15.75/0.9) how does the Steam affect per-host fairness? Also how = transient are these connections team uses? >>=20 >> Actually did more testing about this and it seems that as far I have = set the bandwidth to ~15Mbps (so ~15% less of my max speed) and use the = "nat" parameter, the per-host fairness works even without the = "dual-host" and "overhead" parameters. I definitely find this very = interesting, is this behaviour caused by the way Steam downloads games? >=20 > By default, Cake uses triple-isolate mode, which uses information = about both source and destination hosts to perform per-host isolation; = this usually works well regardless of which side of the connection has = the LAN hosts. The =E2=80=9Cdual=E2=80=9D modes let you specify that = fact explicitly, making it a little more robust and predictable. >=20 > Without overhead compensation, Cake will actually use more of the = physical link than it thinks it does - by default it only accounts for = raw IP or Ethernet packets, depending on the type of interface it=E2=80=99= s attached to. With full-size packets as in a bulk download, the = difference is relatively small, so the 15% margin is just about = sufficient to make things work. But with small packets mixed in, the = difference grows, such that Cake might no longer control the bottleneck = with some traffic mixes. All true, to elaborate a bit on the ATM specific issue, due to = AAL5=E2=80=99s insistence that each ethernet frame is packaged into an = integer number of ATM cells (where the unused octets are simply padded = out) the worst case is something like 100%, if a hypothetical packet = would only require 49 Bytes, it will still require two ATM cells of 53 = bytes... >=20 > The =E2=80=9Cconservative=E2=80=9D keyword I recommended earlier = (which is exactly equivalent to Sebastian=E2=80=99s recommendation of = =E2=80=9Catm overhead 48=E2=80=9D) reverses that situation; Cake will = then always end up using *less* of the physical link than it accounts = for, which is safe for troubleshooting with. The keyword is there = specifically so that we do=E2=80=99t have to figure out the precise = overhead profile before tackling more substantive issues. Due to the boundary observation above, one other option is to = start with the shaper set to 50% of link rate, that should have = sufficient wiggle room for all realistic overheads=E2=80=A6 (but = honestly on a known ATM link I would always run the = ATM_overhead_detector to get the precise number). >=20 > At any rate, it has nothing to do with Steam specifically. >=20 >>> As far as I can tell cake can drill down to the required IP/TCP/UDP = fields independent of whether there are VLAN tags or PPPoE headers so = cake should not care (except for the different overhead specifications = you need to add as stated above). BUT if instantiated on eth0 cake will = see pppoe LCP packets and might decide to drop them, which can take down = the link, so out of caution I would still instantiate on pppoe in your = case. >>=20 >> Yeah, with further testing it seems the interface wasn't the culprit = but I'll still do all my testing on pppoe0 just to be safe. >>=20 >> Anyway I was wondering if there's some kind of manual for Cake and = the various parameters, I'm looking to set it up best way possible but = there are some parameters which I'm not sure what they do (one of them = being "ingress=E2=80=9D). >=20 > With the correct version of iproute2 installed, just issue =E2=80=9Cman = tc-cake=E2=80=9D. That=E2=80=99s the official documentation. >=20 > Currently it doesn=E2=80=99t have the ingress keyword yet. That=E2=80=99= ll be fixed soon. >=20 >> Also while reading on the bufferbloat.net Cake page I noticed a = possible "fix" for BitTorrent (by setting it as "background", = https://www.bufferbloat.net/projects/codel/wiki/Cake/#diffserv-support), = I'm wondering if this can be done with Steam too? >=20 > It=E2=80=99s possible, if you can figure out which traffic is Steam in = the first place, and write filters to match on it. This is complicated = by the fact that Valve runs a sophisticated CDN to handle their rather = impressive bandwidth load. >=20 > - Jonathan Morton >=20