From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D709B3B2A4 for ; Mon, 24 Apr 2017 09:49:53 -0400 (EDT) Received: by mail-wm0-x22d.google.com with SMTP id r190so68444521wme.1 for ; Mon, 24 Apr 2017 06:49:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=uIu4QfkHGXohz5nPNtOgOlTEJUq6BKqB7o8d57RsK1E=; b=XRiRNdYWuK3kaLVmQzeLgmmbADYJ9G50MWZV4cFcRWkHGD3EhTALzl/bRpFB4oRoqh D1rwqgXJVFlSo1KXBp2xHR85FYQknrgP6iIdLjQfG8+5fKAFpZRFSoTDiBs3iNC0yvGy yDYV25uoP1mYzb2yslXskW12fErkhKXgL3dD+j6hcAJTYTZaSNxp70M9xlurbDFLghfh UlCiQ0J30qqWzWMd3BM+Jg/YqirE9ItME63eZK3D3PiWcxsrR1YbBLPA0KPyxX5242/2 fe2GgVsXsONQHTTWohXeznM+zr7bwFDEmgZN1I+GCCsBIjj48M7GYr5/rUGVZI4kuPWT rqhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=uIu4QfkHGXohz5nPNtOgOlTEJUq6BKqB7o8d57RsK1E=; b=eJ7A7qHkmNgZA9Y7mjnR3pUbhVHoXo2Mk7ShXiLDChsx2uR2mlrihl2eJ0ZtorfiOW b3YIFiCyolGX1qvPYgWbNw0byjXnTEqdjvzlavHSpk8OA7kq/8LBUNisr0nOXNCp4UV/ TqUXExa9GlpVo59GzWiTL8iayVs9E4apB722tnEADLLd/1RLbMdkAcbNN+0xQuqx88dZ L61rSKOljs+H+ga51j+m2xYeqElireETitdH56ipBzubvZ3teA5XZrzx5shjbOG7bwEn gNb8XrC3nYmfzO9bf17aGJ7OME9+Z2Ao9DCNF/ULomr5crtWg03/wd3H58Qhjfb5wmPG f0dg== X-Gm-Message-State: AN3rC/5BWaX8PlBzw459Th5yFUVYPSKS4SKoG0t7wry4VybZMzXOwslq lKvanLP51ruooHNWe1eB0QXe/5S7WD3XE6M= X-Received: by 10.80.174.230 with SMTP id f35mr1172635edd.157.1493041792714; Mon, 24 Apr 2017 06:49:52 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.169.59 with HTTP; Mon, 24 Apr 2017 06:49:51 -0700 (PDT) In-Reply-To: <709E9DC1-9FAA-4849-8981-3AFA0DC83132@gmx.de> References: <05C0B0C7-4337-4115-AC6B-DA81392FCB34@gmail.com> <22E633CF-5EE0-4B0F-89A8-B790E730FB6C@gmx.de> <0BA3EE91-C5BC-4155-9D5D-D15D34490A1A@gmx.de> <00DDAA0B-7D99-489B-BA2D-1F20289409B3@gmx.de> <2FFBF256-2932-4FC7-AD1F-0D7CEE111809@gmx.de> <8D8540DA-7AA3-4366-9488-DC1C266C598E@gmx.de> <709E9DC1-9FAA-4849-8981-3AFA0DC83132@gmx.de> From: Dendari Marini Date: Mon, 24 Apr 2017 15:49:51 +0200 Message-ID: To: Sebastian Moeller Cc: David Lang , cake@lists.bufferbloat.net Content-Type: multipart/alternative; boundary=f403045c2154b08e2f054de9e193 Subject: Re: [Cake] Getting Cake to work better with Steam and similar applications X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Apr 2017 13:49:54 -0000 --f403045c2154b08e2f054de9e193 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Sebastian, Did few quick tests using your recommendations and the results is still "overhead 20". Plot: http://imgur.com/a/5c3iv Text: https://drive.google.com/open?id=3D0B7vEuplJWEIkRVBhX2xGUmNUMVE (NOTE: I used PINGSPERSIZE =3D 100 instead of 1000, as I didn't have much time) On 24 April 2017 at 14:35, Sebastian Moeller wrote: > Hi Dendari, > > thanks, > > > > On Apr 24, 2017, at 14:08, Dendari Marini wrote: > > > > Hello, > > > > Could you share the two output plots somewhere, so I can have a look at > those? (Also I might want tto ask for the text file that actually was > generated by the ping collector script, just so I can run and > confirm/de-bug things my self). > > > > Sure thing. The plot images: http://imgur.com/a/qDtA0 > > Okay, these _look_ reasonable on first sight... > > > > And the output text file: https://drive.google.com/open?id=3D > 0B7vEuplJWEIkc1ozbUZRSGstajQ > > =E2=80=A6but this actually shows that 8.8.8.8 truncated the size of the I= CMP > responses (the runs of =E2=80=9Cbytes=3D92=E2=80=9D in the log file also = it only returned > sizes from 44 to 92 so not exactly what we asked for). I would not really > trust these results, even though the RTTs should be dominated by your slo= w > egress link. Please redo these against a different ICMP target server ( > netperf-east.bufferbloat.net might do, depending on your location, that > on is on the east coast of the U.S.). I have to admit, that the > hrping/windows overhead extraction did not have much exercise so far and > might be not be robust enough. I will have a look at your log file, but > this really requires a different target server. In the unix script there = is > the following: > > > echo "To run measurements supply the TARGET IP address as first > agument to ${0} this script." > echo "Use traceroute 8.8.8.8 to get a list of increasingly distant > hosts, pick the first host out of your network (ideally the DSLAM)." > echo "Test whether the selected host responds to ping: 'ping -s16 -c = 1 > target.IP.address.quad' : this needs to actually return non zero RTTs." > echo "If the hosts does not reply to the pings take the next host fro= m > the traceroute (moving closer to 8.8.8.8), repeat until you find a replyi= ng > host." > echo "Once the main script is started have a quick look at the > logfile, to see whether the RTTs stay close to the initial test RTT." > echo "If the RTTs have increased a lot, the PINGPERIOD might be too > short, and the host might have put us on a slow path; either increase > PINGPERIOD or try the next host..." > echo "" > echo "Here is the traceroute (might take a while):" > echo "" > traceroute 8.8.8.8 > echo "" > echo "Alternatively you might try to use googles infrastructure by > running: ${0} gstatic.com " > echo "Please note that gstatic.com might not return arbitrarily sized > ICMP probes, so check the log file care-fully.=E2=80=9D > > Which unfortunately does not appear in the windows script yet, but this > still seems great advise, please note that at the time this was written, > 8.8.8.8 still replied with =E2=80=9Ccorrectly=E2=80=9D=E2=80=99 sized ICM= P echo responses just like > gstatic.com di initially, now both seem to truncate their responses. This > is somewhat to be expected, but maybe somewhere along the path there is > another server that will still respond properly... > > > Best Regards > Sebastian > > > > > > > On 24 April 2017 at 13:34, Sebastian Moeller wrote: > > Hello, > > > > > > > On Apr 24, 2017, at 10:41, Dendari Marini wrote= : > > > > > > Hello, > > > > > > Probably correct, but you do not have to resort to believing, you can > actually try to measure that ;) In case I have been too subtle before, ha= ve > a look at https://github.com/moeller0/ATM_overhead_detector and follow > the instructions there... > > > > > > I just used your script and it estimated an overhead of 20 bytes, so > should I use "overhead 20 atm" or am I missing something? In the last few > days I've been using "pppoe-llcsnap" ("overhead 40 atm") without any > evident issue, should I change it? > > > > Hmm, 20 seems rather interesting and something I never saw > before. Could you share the two output plots somewhere, so I can have a > look at those? (Also I might want tto ask for the text file that actually > was generated by the ping collector script, just so I can run and > confirm/de-bug things my self). I am not saying 20 is impossible, just th= at > it is improbable enough to require more scrutiny. > > > > > > Best Regards > > Sebastian > > > > > > > > > > FWIW here's a quick example on ingress ppp that I tested using connma= rk > > > the connmarks (1 or 2 or unmarked) being set by iptables rules on > outbound > > > connections/traffic classes. > > > > > > Unfortunately I'm really not sure how to apply those settings to my > case, it's something I've never done so some hand-holding is probably > needed, sorry. At the moment I've limited the Steam bandwidth using the > built-in Basic Queue and DPI features from the ER-X. They're easy to set = up > but aren't really ideal, would rather prefer Cake would take care about i= t > more dynamically. > > > > > > Anyway about the Steam IP addresses I've noticed, in the almost three > weeks of testing, they're almost always the same IP blocks (most of which > can be found on the Steam Support website, https://support.steampowered. > com/kb_article.php?ref=3D8571-GLVN-8711). I believe it would be a good > starting point for limiting Steam, what do you think? > > > > > > On 24 April 2017 at 09:55, Sebastian Moeller wrote: > > > Hi David, > > > > > > > On Apr 23, 2017, at 14:32, David Lang wrote: > > > > > > > > On Sun, 23 Apr 2017, Sebastian Moeller wrote: > > > > > > > >>> About the per-host fairness download issue: while it's kinda > resolved I still feel like it's mainly related to Steam, as normally > downloading files from PC1 and PC2 halved the speed as expected even at > full bandwidth (so no overhead, no -15%). > > > >> > > > >> This might be true, but for cake to meaningfully resolve > bufferbloat you absolutely _must_ take care to account for encapsulation > and overhead one way or another. > > > > > > > > well, one way to account for this overhead is to set the allowed > bandwidth low enough. Being precise on this overhead lets you get closer = to > the actual line rate, but if you have enough bandwidth, it may not really > matter (i.e. if you have a 100Mb connection and only get 70Mb out of it, > you probably won't notice unless you go looking) > > > > > > Violent agreement. But note that with AAL5=E2=80=99s rule to = always > use an integer number of ATM cells per user packet the required bandwidth > sacrifice to statically cover the worst case gets ludicrous (theoretical > worst case: requiring 2 53 byte ATM cells for on 49 Byte data packet: 100= * > 49 / (53 * 2) =3D 46.2% and this is on top of any potential unaccounted > overhead inside the 49 Byte packet). Luckily the ATM padding issue is not > as severe for bigger packets=E2=80=A6 but still to statically fully solve > modem/dslam bufferbloat the required bandwidth sacrifice seems excessive= =E2=80=A6 > But again you are right, there might be users who do not mind to go to th= is > length. For this reason I occasionally recommend to start the bandwidth a= t > 50% to certainly rule out overhead/encapsulation accounting issues (mind > you take 50% as starting point from which to ramp up=E2=80=A6) > > > > > > Best Regards > > > Sebastian. > > > > > > > > > > > > > > David Lang > > > > > > > > > > > > --f403045c2154b08e2f054de9e193 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Sebastian,


On 24 April 2017 at 14:35, Sebastian Moeller = <moeller0@gmx.de> wrote:
Hi Dendari,

thanks,


> On Apr 24, 2017, at 14:08, Dendari Marini <dendari92@gmail.com> wrote:
>
> Hello,
>
> Could you share the two output plots somewhere= , so I can have a look at those? (Also I might want tto ask for the text fi= le that actually was generated by the ping collector script, just so I can = run and confirm/de-bug things my self).
>
> Sure thing. The plot images: http://imgur.com/a/qDtA0

=C2=A0 =C2=A0 =C2=A0 =C2=A0 Okay, these _look_ reasonable on first s= ight...


> And the output text file: https= ://drive.google.com/open?id=3D0B7vEuplJWEIkc1ozbUZRSGstajQ
=E2=80=A6but this actually shows that 8.8.8.8 truncated the size of = the ICMP responses (the runs of =E2=80=9Cbytes=3D92=E2=80=9D in the log fil= e also it only returned sizes from 44 to 92 so not exactly what we asked fo= r). I would not really trust these results, even though the RTTs should be = dominated by your slow egress link. Please redo these against a different I= CMP target server (netperf-east.bufferbloat.net might do, dep= ending on your location, that on is on the east coast of the U.S.). I have = to admit, that the hrping/windows overhead extraction did not have much exe= rcise so far and might be not be robust enough. I will have a look at your = log file, but this really requires a different target server. In the unix s= cript there is the following:


=C2=A0 =C2=A0 echo "To run measurements supply the TARGET IP address a= s first agument to ${0} this script."
=C2=A0 =C2=A0 echo "Use traceroute 8.8.8.8 to get a list of increasing= ly distant hosts, pick the first host out of your network (ideally the DSLA= M)."
=C2=A0 =C2=A0 echo "Test whether the selected host responds to ping: &= #39;ping -s16 -c 1 target.IP.address.quad' : this needs to actually ret= urn non zero RTTs."
=C2=A0 =C2=A0 echo "If the hosts does not reply to the pings take the = next host from the traceroute (moving closer to 8.8.8.8), repeat until you = find a replying host."
=C2=A0 =C2=A0 echo "Once the main script is started have a quick look = at the logfile, to see whether the RTTs stay close to the initial test RTT.= "
=C2=A0 =C2=A0 echo "If the RTTs have increased a lot, the PINGPERIOD m= ight be too short, and the host might have put us on a slow path; either in= crease PINGPERIOD or try the next host..."
=C2=A0 =C2=A0 echo ""
=C2=A0 =C2=A0 echo "Here is the traceroute (might take a while):"=
=C2=A0 =C2=A0 echo ""
=C2=A0 =C2=A0 traceroute 8.8.8.8
=C2=A0 =C2=A0 echo ""
=C2=A0 =C2=A0 echo "Alternatively you might try to use googles infrast= ructure by running: ${0} gstatic.com "
=C2=A0 =C2=A0 echo "Please note that gstatic.com might not return arbitra= rily sized ICMP probes, so check the log file care-fully.=E2=80=9D

Which unfortunately does not appear in the windows script yet, but this sti= ll seems great advise, please note that at the time this was written, 8.8.8= .8 still replied with =E2=80=9Ccorrectly=E2=80=9D=E2=80=99 sized ICMP echo = responses just like gstatic.com di initially, now both seem to truncate their = responses. This is somewhat to be expected, but maybe somewhere along the p= ath there is another server that will still respond properly...


Best Regards
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = Sebastian



>
> On 24 April 2017 at 13:34, Sebastian Moeller <moeller0@gmx.de> wrote:
> Hello,
>
>
> > On Apr 24, 2017, at 10:41, Dendari Marini <dendari92@gmail.com> wrote:
> >
> > Hello,
> >
> > Probably correct, but you do not have to resort to believing, you= can actually try to measure that ;) In case I have been too subtle before,= have a look at https://github.com/moeller0/AT= M_overhead_detector and follow the instructions there...
> >
> > I just used your script and it estimated an overhead of 20 bytes,= so should I use "overhead 20 atm" or am I missing something? In = the last few days I've been using "pppoe-llcsnap" ("over= head 40 atm") without any evident issue, should I change it?
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Hmm, 20 seems rather interesting and = something I never saw before. Could you share the two output plots somewher= e, so I can have a look at those? (Also I might want tto ask for the text f= ile that actually was generated by the ping collector script, just so I can= run and confirm/de-bug things my self). I am not saying 20 is impossible, = just that it is improbable enough to require more scrutiny.
>
>
> Best Regards
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Sebastian
>
>
> >
> > FWIW here's a quick example on ingress ppp that I tested usin= g connmark
> > the connmarks (1 or 2 or unmarked) being set by iptables rules on= outbound
> > connections/traffic classes.
> >
> > Unfortunately I'm really not sure how to apply those settings= to my case, it's something I've never done so some hand-holding is= probably needed, sorry. At the moment I've limited the Steam bandwidth= using the built-in Basic Queue and DPI features from the ER-X. They're= easy to set up but aren't really ideal, would rather prefer Cake would= take care about it more dynamically.
> >
> > Anyway about the Steam IP addresses I've noticed, in the almo= st three weeks of testing, they're almost always the same IP blocks (mo= st of which can be found on the Steam Support website, https://support.steampowered.com/kb_article.php= ?ref=3D8571-GLVN-8711). I believe it would be a good starting poin= t for limiting Steam, what do you think?
> >
> > On 24 April 2017 at 09:55, Sebastian Moeller <moeller0@gmx.de> wrote:
> > Hi David,
> >
> > > On Apr 23, 2017, at 14:32, David Lang <david@lang.hm> wrote:
> > >
> > > On Sun, 23 Apr 2017, Sebastian Moeller wrote:
> > >
> > >>> About the per-host fairness download issue: while it= 's kinda resolved I still feel like it's mainly related to Steam, a= s normally downloading files from PC1 and PC2 halved the speed as expected = even at full bandwidth (so no overhead, no -15%).
> > >>
> > >>=C2=A0 =C2=A0 =C2=A0 This might be true, but for cake to = meaningfully resolve bufferbloat you absolutely _must_ take care to account= for encapsulation and overhead one way or another.
> > >
> > > well, one way to account for this overhead is to set the all= owed bandwidth low enough. Being precise on this overhead lets you get clos= er to the actual line rate, but if you have enough bandwidth, it may not re= ally matter (i.e. if you have a 100Mb connection and only get 70Mb out of i= t, you probably won't notice unless you go looking)
> >
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Violent agreement. But note that= with AAL5=E2=80=99s rule to always use an integer number of ATM cells per = user packet the required bandwidth sacrifice to statically cover the worst = case gets ludicrous (theoretical worst case: requiring 2 53 byte ATM cells = for on 49 Byte data packet: 100 * 49 / (53 * 2) =3D 46.2% and this is on to= p of any potential unaccounted overhead inside the 49 Byte packet). Luckily= the ATM padding issue is not as severe for bigger packets=E2=80=A6 but sti= ll to statically fully solve modem/dslam bufferbloat the required bandwidth= sacrifice seems excessive=E2=80=A6 But again you are right, there might be= users who do not mind to go to this length. For this reason I occasionally= recommend to start the bandwidth at 50% to certainly rule out overhead/enc= apsulation accounting issues (mind you take 50% as starting point from whic= h to ramp up=E2=80=A6)
> >
> > Best Regards
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Sebastian.
> >
> >
> > >
> > > David Lang
> >
> >
>
>


--f403045c2154b08e2f054de9e193--