From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D20793B2A4 for ; Mon, 24 Apr 2017 13:32:24 -0400 (EDT) Received: from [172.17.3.29] ([134.76.241.253]) by mail.gmx.com (mrgmx002 [212.227.17.190]) with ESMTPSA (Nemesis) id 0M2Ldk-1cB0lG3fc4-00s8pf; Mon, 24 Apr 2017 19:32:14 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Sebastian Moeller In-Reply-To: Date: Mon, 24 Apr 2017 19:32:11 +0200 Cc: David Lang , cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: References: <05C0B0C7-4337-4115-AC6B-DA81392FCB34@gmail.com> <22E633CF-5EE0-4B0F-89A8-B790E730FB6C@gmx.de> <0BA3EE91-C5BC-4155-9D5D-D15D34490A1A@gmx.de> <00DDAA0B-7D99-489B-BA2D-1F20289409B3@gmx.de> <2FFBF256-2932-4FC7-AD1F-0D7CEE111809@gmx.de> <8D8540DA-7AA3-4366-9488-DC1C266C598E@gmx.de> <709E9DC1-9FAA-4849-8981-3AFA0DC83132@gmx.de> To: Dendari Marini X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:auS6NAL0Fku4k8jcZdxw/NyVczW8yW556ipl9o0TzXOJMOUzalj orQYHGeE3s5zg5/sm+yCjjrIfj8f/+gpQc5M3N2RCkM9Ngt6hCl88a1BPpODyeHwDu2j21A sOgGpOW/L5h2NRTJZtwBqvY/vEZB0ZKOYY7b7xVhN8S0IXGtJomtm9P1OCHkxj8QjSnbpEZ n1sg6IDU1nZMwHEqcfbYA== X-UI-Out-Filterresults: notjunk:1;V01:K0:NfB5N+nNOPM=:jxj2Q+6RjSJB/9v2lN1ZsF xYG+aMusktz0odB5q9aLkKendEYTPZG6Jb+uUCFjjfXgtcuv/YZULYKwNHlIKlq89F5vXi3kk +LlEanUewu/s9T5EDvN80RCWDoJwk2IQc6lRr9B56uzVkEJp30bDfvT7UOHs3WM3mqvKId3CM /sq8RDy6MoExLIrNE3UW7+5bpfZ6uKXqrBB4i+D710ZKzS+Z6qfg8mlPNcV5K0Unj99Bi1ZLN gYaBYyflbbKF1Xlq+P+BQhXgqbXGbeKeegZOmlqWBChdPy9iI+wiGIjhvwATgXEmKFAl1divB mNRSPfNHFz1Z6IQNHSs0jt9k/uTV/x9jrQIPdsK//kNvPB5yHDISQSSF4kFib1/b2PgTODGtL SCIwyM5Fy0JzIuyN7vuoGctaQHZZNfNo35A6OjmXO8x/Idjno7GsmgysHVashQxa0UpkDuJ64 2TGqwA6YqYNwamyuE4MBP0bCLe0WBkuG7uSqZOpbPl5QrlvYKmpEYm43UBUF6ikWz1cJKi8yf zPEhz+vld/68PC0aLCR3+GCZqaxr6qlXCat9JjCheWKVI4Iv7A6dNqsC1PueraDLBbhNxIUf1 BxCSoRRVeKBRr3xb5ey8Hk7/a0ogyY4YtGhpljaQJpbeedCQ5h2jV6vIvNUIsUQKBmzEgPjFu F2UjKx3IqrZG89SIRq0/p0dTZHxWRktLYqw4qzMmHeoJaJ9AeFuuU3RiO/Tihjbjn2Lh9tTDb 0hbgvoX3FzeBnADYV0VMAwFFLXGzcbwA83R5AlhKz1jZNiz4jsLiUTdo46hNa4bRcsQcacTsa fdLLS+ZZICKq+C4OZkDEU/mMaU4DvDWC4Aeon7Z+aH9xGnSSKjlSuqeDPjTxrWCpPvMmCFYDo nO9Es50BEGZ5e4G0v0FQ== Subject: Re: [Cake] Getting Cake to work better with Steam and similar applications X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Apr 2017 17:32:25 -0000 Okay, so the fix was simply to reduce the hrping reported sizes by 20, as = hrping reports the full IP size of its probe, while gnu/bsd ping only = reports ICMP size so for the hrping case the sizes where 20 bytes to = large. So in essence 40 is the correct number for the ADSL-leg of your = link. Your ISP might have an upstream shaper that might even have a = larger overhead*, but I really doubt that. Interestingly enough even the = truncated 8.8.8.8 data gets processed reasonably sanely (in the end it = only affects a single datapoint the last one so the detector is = amazingly robust enough). Sorry for having you act as beta tester, and thanks for helping debug = this ;) Anyway I committed that change and after a new pull reported = values should also be correct for the windows case=E2=80=A6 (in my = defense I only had PTM vdsl2 data available when adding the processing = for hrping data and obviously did not look to closely at the output as = that showed as expected no ATM carrier, so the precise numerical value = was not meaningful anyway). I guess that shows that the hrping script = probably has not seen any real world usage. Best Regards Sebastian *) my code only ever should see the overhead on the ATM link assuming = its probe traffic does not fully saturate the potential upstream = shaper=E2=80=A6 > On Apr 24, 2017, at 15:49, Dendari Marini wrote: >=20 > Hi Sebastian, >=20 > Did few quick tests using your recommendations and the results is = still "overhead 20". Plot: http://imgur.com/a/5c3iv Text: = https://drive.google.com/open?id=3D0B7vEuplJWEIkRVBhX2xGUmNUMVE (NOTE: I = used PINGSPERSIZE =3D 100 instead of 1000, as I didn't have much time) >=20 > On 24 April 2017 at 14:35, Sebastian Moeller wrote: > Hi Dendari, >=20 > thanks, >=20 >=20 > > On Apr 24, 2017, at 14:08, Dendari Marini = wrote: > > > > Hello, > > > > Could you share the two output plots somewhere, so I can have a look = at those? (Also I might want tto ask for the text file that actually was = generated by the ping collector script, just so I can run and = confirm/de-bug things my self). > > > > Sure thing. The plot images: http://imgur.com/a/qDtA0 >=20 > Okay, these _look_ reasonable on first sight... >=20 >=20 > > And the output text file: = https://drive.google.com/open?id=3D0B7vEuplJWEIkc1ozbUZRSGstajQ >=20 > =E2=80=A6but this actually shows that 8.8.8.8 truncated the size of = the ICMP responses (the runs of =E2=80=9Cbytes=3D92=E2=80=9D in the log = file also it only returned sizes from 44 to 92 so not exactly what we = asked for). I would not really trust these results, even though the RTTs = should be dominated by your slow egress link. Please redo these against = a different ICMP target server (netperf-east.bufferbloat.net might do, = depending on your location, that on is on the east coast of the U.S.). I = have to admit, that the hrping/windows overhead extraction did not have = much exercise so far and might be not be robust enough. I will have a = look at your log file, but this really requires a different target = server. In the unix script there is the following: >=20 >=20 > echo "To run measurements supply the TARGET IP address as first = agument to ${0} this script." > echo "Use traceroute 8.8.8.8 to get a list of increasingly distant = hosts, pick the first host out of your network (ideally the DSLAM)." > echo "Test whether the selected host responds to ping: 'ping -s16 = -c 1 target.IP.address.quad' : this needs to actually return non zero = RTTs." > echo "If the hosts does not reply to the pings take the next host = from the traceroute (moving closer to 8.8.8.8), repeat until you find a = replying host." > echo "Once the main script is started have a quick look at the = logfile, to see whether the RTTs stay close to the initial test RTT." > echo "If the RTTs have increased a lot, the PINGPERIOD might be = too short, and the host might have put us on a slow path; either = increase PINGPERIOD or try the next host..." > echo "" > echo "Here is the traceroute (might take a while):" > echo "" > traceroute 8.8.8.8 > echo "" > echo "Alternatively you might try to use googles infrastructure by = running: ${0} gstatic.com " > echo "Please note that gstatic.com might not return arbitrarily = sized ICMP probes, so check the log file care-fully.=E2=80=9D >=20 > Which unfortunately does not appear in the windows script yet, but = this still seems great advise, please note that at the time this was = written, 8.8.8.8 still replied with =E2=80=9Ccorrectly=E2=80=9D=E2=80=99 = sized ICMP echo responses just like gstatic.com di initially, now both = seem to truncate their responses. This is somewhat to be expected, but = maybe somewhere along the path there is another server that will still = respond properly... >=20 >=20 > Best Regards > Sebastian >=20 >=20 >=20 > > > > On 24 April 2017 at 13:34, Sebastian Moeller = wrote: > > Hello, > > > > > > > On Apr 24, 2017, at 10:41, Dendari Marini = wrote: > > > > > > Hello, > > > > > > Probably correct, but you do not have to resort to believing, you = can actually try to measure that ;) In case I have been too subtle = before, have a look at https://github.com/moeller0/ATM_overhead_detector = and follow the instructions there... > > > > > > I just used your script and it estimated an overhead of 20 bytes, = so should I use "overhead 20 atm" or am I missing something? In the last = few days I've been using "pppoe-llcsnap" ("overhead 40 atm") without any = evident issue, should I change it? > > > > Hmm, 20 seems rather interesting and something I never saw = before. Could you share the two output plots somewhere, so I can have a = look at those? (Also I might want tto ask for the text file that = actually was generated by the ping collector script, just so I can run = and confirm/de-bug things my self). I am not saying 20 is impossible, = just that it is improbable enough to require more scrutiny. > > > > > > Best Regards > > Sebastian > > > > > > > > > > FWIW here's a quick example on ingress ppp that I tested using = connmark > > > the connmarks (1 or 2 or unmarked) being set by iptables rules on = outbound > > > connections/traffic classes. > > > > > > Unfortunately I'm really not sure how to apply those settings to = my case, it's something I've never done so some hand-holding is probably = needed, sorry. At the moment I've limited the Steam bandwidth using the = built-in Basic Queue and DPI features from the ER-X. They're easy to set = up but aren't really ideal, would rather prefer Cake would take care = about it more dynamically. > > > > > > Anyway about the Steam IP addresses I've noticed, in the almost = three weeks of testing, they're almost always the same IP blocks (most = of which can be found on the Steam Support website, = https://support.steampowered.com/kb_article.php?ref=3D8571-GLVN-8711). I = believe it would be a good starting point for limiting Steam, what do = you think? > > > > > > On 24 April 2017 at 09:55, Sebastian Moeller = wrote: > > > Hi David, > > > > > > > On Apr 23, 2017, at 14:32, David Lang wrote: > > > > > > > > On Sun, 23 Apr 2017, Sebastian Moeller wrote: > > > > > > > >>> About the per-host fairness download issue: while it's kinda = resolved I still feel like it's mainly related to Steam, as normally = downloading files from PC1 and PC2 halved the speed as expected even at = full bandwidth (so no overhead, no -15%). > > > >> > > > >> This might be true, but for cake to meaningfully resolve = bufferbloat you absolutely _must_ take care to account for encapsulation = and overhead one way or another. > > > > > > > > well, one way to account for this overhead is to set the allowed = bandwidth low enough. Being precise on this overhead lets you get closer = to the actual line rate, but if you have enough bandwidth, it may not = really matter (i.e. if you have a 100Mb connection and only get 70Mb out = of it, you probably won't notice unless you go looking) > > > > > > Violent agreement. But note that with AAL5=E2=80=99s rule = to always use an integer number of ATM cells per user packet the = required bandwidth sacrifice to statically cover the worst case gets = ludicrous (theoretical worst case: requiring 2 53 byte ATM cells for on = 49 Byte data packet: 100 * 49 / (53 * 2) =3D 46.2% and this is on top of = any potential unaccounted overhead inside the 49 Byte packet). Luckily = the ATM padding issue is not as severe for bigger packets=E2=80=A6 but = still to statically fully solve modem/dslam bufferbloat the required = bandwidth sacrifice seems excessive=E2=80=A6 But again you are right, = there might be users who do not mind to go to this length. For this = reason I occasionally recommend to start the bandwidth at 50% to = certainly rule out overhead/encapsulation accounting issues (mind you = take 50% as starting point from which to ramp up=E2=80=A6) > > > > > > Best Regards > > > Sebastian. > > > > > > > > > > > > > > David Lang > > > > > > > > > > >=20 >=20