From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x230.google.com (mail-qt0-x230.google.com [IPv6:2607:f8b0:400d:c0d::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 37CE63B2A3 for ; Sat, 22 Apr 2017 08:50:27 -0400 (EDT) Received: by mail-qt0-x230.google.com with SMTP id y33so86723755qta.2 for ; Sat, 22 Apr 2017 05:50:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:cc; bh=xNKarZfTodyZGmHx1HPwPJQuN333G9UMDYUI4iKQgfA=; b=bN9f8RwXw4fgpgoZgKR8rIsr5S2MW7pZxriMa1EFz/tjRSJjg6pwYZUPCsSc5EzPvM c/nTMTflfN/V7Vf8muQwidN5E/iCTN5QM8pkqzfD8oWZ0XsztpFm027AAMBrDa07/32B FFWKXi/+tHGy3q1qeOCsELIVxQyA5eOu1n/PXd+R0RD7VBXARdPxf6ZDQC0Tl7uKA7V3 sHBZdPPnpXX+C4dOeWKVwQRkfhub/Jda0+wYj7Z7u7ggfPm31y0ivqyW6LN+edL4dvNl nQQG90UcADwKQhKa48e/UrCsRLF6+S7BPRUHNzXuwI4wl6iaQKi0IsLXbRrcG7uLDgVs lY6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:cc; bh=xNKarZfTodyZGmHx1HPwPJQuN333G9UMDYUI4iKQgfA=; b=aDpXe4hvMSI1OZw+SmMeMBgQaPq1Dc5ZfCoVeERavkMVPetugUb0AS4KIHG1ou9GAJ ypYS9uS4mdUuOqrlSs3A4Si557rHvPKm8lvnq7kVuyOVF8yJ7QeN9jmxVwxZjhoAqoNC yqxU+A4rx5iHcAxTk3MtjIdS20rdiKYxOruT+k4TXF6xQamwAuNKq4UDl6jt0eEGunrR xV8OBIC2WFofm4FuxPUCmYpnN297eZfpg092me8F8Nufeww0QjOIbgwUbUNG2i8HPFZy SAuqsetus4ADSPPC96luNT25kqvceBNPM7YNR9/3EsRUTWnNijotBNZQdILv4wTkS14Z mfKA== X-Gm-Message-State: AN3rC/5gnmtuGu7KM2FPlMLO7ptyP3IpaIq0spntj1OWO6HQ+5HAvYbn PmMoqVfffyb9kA7gvR/Ii21zrwz9qg== X-Received: by 10.200.35.27 with SMTP id a27mt18452977qta.2.1492865426522; Sat, 22 Apr 2017 05:50:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.81.9 with HTTP; Sat, 22 Apr 2017 05:50:25 -0700 (PDT) Received: by 10.140.81.9 with HTTP; Sat, 22 Apr 2017 05:50:25 -0700 (PDT) In-Reply-To: References: <05C0B0C7-4337-4115-AC6B-DA81392FCB34@gmail.com> <22E633CF-5EE0-4B0F-89A8-B790E730FB6C@gmx.de> From: xnoreq Date: Sat, 22 Apr 2017 14:50:25 +0200 Message-ID: Cc: cake@lists.bufferbloat.net, Dendari Marini Content-Type: multipart/alternative; boundary=001a113f479a727c8e054dc0d1d0 Subject: Re: [Cake] Getting Cake to work better with Steam and similar applications X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Apr 2017 12:50:27 -0000 --001a113f479a727c8e054dc0d1d0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable FYI, when I looked at Steam traffic a few years ago it was very bursty, meaning that there is nothing transmitted for a short period and then there's a burst that uses up all link bandwidth for a short while. Internet was unusable without limiting Steam downloads quite a lot. Now I don't have that problem anymore. Cake's ingress mode works almost as well as I have expected (still need to set bw about 1Mbps lower on a 20Mbps link - but that's ok). Maybe the Steam CDN now also uses a saner network scheduler (like fq with pacing). I'd guess so anyway. On 22 Apr 2017 11:36, "Jonathan Morton" wrote: > >> So please add =E2=80=9Catm overhead 32" to cake on eth0 or =E2=80=9Cat= m overhead 40=E2=80=9D to > cake instances on pppoe (these packets do not have the PPPoE header added > yet and hence appear 8 bytes to small). > > > > Thanks for your help, will definitely use them. Just wondering if I use > "pppoe-vcmux/bridged-llcsnap" on eth0 or "pppoe-llcsnap" on pppoe0 would > have the same effect? Or are there some other "under-the-hood" changes wh= en > using them? > > On the pppoe interface, use pppoe-vcmux if your modem is set to use > VC-MUX, or pppoe-llcsnap if it=E2=80=99s set to use LLC-SNAP (they might = be > described using slightly different terms, but should still be recognisabl= e > as one or the other). This probably depends on your ISP, and may further > vary regionally within the same ISP. > > I really prefer to use the self-explanatory keywords (which is why I adde= d > them in the first place) instead of opaque magic numbers. This is a poin= t > on which Sebastian has long disagreed with me. > > >> Question: if you set the shaper=E2=80=99s to 50% of line rate (8.75/0.= 5?) do > you still see that unfairness? And if you add =E2=80=9Catm overhead 40=E2= =80=9D to cake on > pppoe0 and set the shaper to 90% of line rates (15.75/0.9) how does the > Steam affect per-host fairness? Also how transient are these connections > team uses? > > > > Actually did more testing about this and it seems that as far I have se= t > the bandwidth to ~15Mbps (so ~15% less of my max speed) and use the "nat" > parameter, the per-host fairness works even without the "dual-host" and > "overhead" parameters. I definitely find this very interesting, is this > behaviour caused by the way Steam downloads games? > > By default, Cake uses triple-isolate mode, which uses information about > both source and destination hosts to perform per-host isolation; this > usually works well regardless of which side of the connection has the LAN > hosts. The =E2=80=9Cdual=E2=80=9D modes let you specify that fact explic= itly, making it a > little more robust and predictable. > > Without overhead compensation, Cake will actually use more of the physica= l > link than it thinks it does - by default it only accounts for raw IP or > Ethernet packets, depending on the type of interface it=E2=80=99s attache= d to. > With full-size packets as in a bulk download, the difference is relativel= y > small, so the 15% margin is just about sufficient to make things work. B= ut > with small packets mixed in, the difference grows, such that Cake might n= o > longer control the bottleneck with some traffic mixes. > > The =E2=80=9Cconservative=E2=80=9D keyword I recommended earlier (which i= s exactly > equivalent to Sebastian=E2=80=99s recommendation of =E2=80=9Catm overhead= 48=E2=80=9D) reverses > that situation; Cake will then always end up using *less* of the physical > link than it accounts for, which is safe for troubleshooting with. The > keyword is there specifically so that we do=E2=80=99t have to figure out = the > precise overhead profile before tackling more substantive issues. > > At any rate, it has nothing to do with Steam specifically. > > >> As far as I can tell cake can drill down to the required IP/TCP/UDP > fields independent of whether there are VLAN tags or PPPoE headers so cak= e > should not care (except for the different overhead specifications you nee= d > to add as stated above). BUT if instantiated on eth0 cake will see pppoe > LCP packets and might decide to drop them, which can take down the link, = so > out of caution I would still instantiate on pppoe in your case. > > > > Yeah, with further testing it seems the interface wasn't the culprit bu= t > I'll still do all my testing on pppoe0 just to be safe. > > > > Anyway I was wondering if there's some kind of manual for Cake and the > various parameters, I'm looking to set it up best way possible but there > are some parameters which I'm not sure what they do (one of them being > "ingress=E2=80=9D). > > With the correct version of iproute2 installed, just issue =E2=80=9Cman t= c-cake=E2=80=9D. > That=E2=80=99s the official documentation. > > Currently it doesn=E2=80=99t have the ingress keyword yet. That=E2=80=99= ll be fixed soon. > > > Also while reading on the bufferbloat.net Cake page I noticed a > possible "fix" for BitTorrent (by setting it as "background", > https://www.bufferbloat.net/projects/codel/wiki/Cake/#diffserv-support), > I'm wondering if this can be done with Steam too? > > It=E2=80=99s possible, if you can figure out which traffic is Steam in th= e first > place, and write filters to match on it. This is complicated by the fact > that Valve runs a sophisticated CDN to handle their rather impressive > bandwidth load. > > - Jonathan Morton > > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake > --001a113f479a727c8e054dc0d1d0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
FYI, when I looked at Steam traffic a few years ago it wa= s very bursty, meaning that there is nothing transmitted for a short period= and then there's a burst that uses up all link bandwidth for a short w= hile.
Internet was unusable without limiting Steam downloa= ds quite a lot.

Now I don&= #39;t have that problem anymore. Cake's ingress mode works almost as we= ll as I have expected (still need to set bw about 1Mbps lower on a 20Mbps l= ink - but that's ok).
Maybe the Steam CDN now al= so uses a saner network scheduler (like fq with pacing). I'd guess so a= nyway.


On 22 Apr 2017 1= 1:36, "Jonathan Morton" <chromatix99@gmail.com> wrote:
>> So please add =E2=80=9Catm overhead 32" = to cake on eth0 or =E2=80=9Catm overhead 40=E2=80=9D to cake instances on p= ppoe (these packets do not have the PPPoE header added yet and hence appear= 8 bytes to small).
>
> Thanks for your help, will definitely use them. Just wondering if I us= e "pppoe-vcmux/bridged-llcsnap" on eth0 or "pppoe-llcsnap&qu= ot; on pppoe0 would have the same effect? Or are there some other "und= er-the-hood" changes when using them?

On the pppoe interface, use pppoe-vcmux if your modem is set to use VC-MUX,= or pppoe-llcsnap if it=E2=80=99s set to use LLC-SNAP (they might be descri= bed using slightly different terms, but should still be recognisable as one= or the other).=C2=A0 This probably depends on your ISP, and may further va= ry regionally within the same ISP.

I really prefer to use the self-explanatory keywords (which is why I added = them in the first place) instead of opaque magic numbers.=C2=A0 This is a p= oint on which Sebastian has long disagreed with me.

>> Question: if you set the shaper=E2=80=99s to 50% of line rate (8.7= 5/0.5?) do you still see that unfairness? And if you add =E2=80=9Catm overh= ead 40=E2=80=9D to cake on pppoe0 and set the shaper to 90% of line rates (= 15.75/0.9) how does the Steam affect per-host fairness? Also how transient = are these connections team uses?
>
> Actually did more testing about this and it seems that as far I have s= et the bandwidth to ~15Mbps (so ~15% less of my max speed) and use the &quo= t;nat" parameter, the per-host fairness works even without the "d= ual-host" and "overhead" parameters. I definitely find this = very interesting, is this behaviour caused by the way Steam downloads games= ?

By default, Cake uses triple-isolate mode, which uses information about bot= h source and destination hosts to perform per-host isolation; this usually = works well regardless of which side of the connection has the LAN hosts.=C2= =A0 The =E2=80=9Cdual=E2=80=9D modes let you specify that fact explicitly, = making it a little more robust and predictable.

Without overhead compensation, Cake will actually use more of the physical = link than it thinks it does - by default it only accounts for raw IP or Eth= ernet packets, depending on the type of interface it=E2=80=99s attached to.= =C2=A0 With full-size packets as in a bulk download, the difference is rela= tively small, so the 15% margin is just about sufficient to make things wor= k.=C2=A0 But with small packets mixed in, the difference grows, such that C= ake might no longer control the bottleneck with some traffic mixes.

The =E2=80=9Cconservative=E2=80=9D keyword I recommended earlier (which is = exactly equivalent to Sebastian=E2=80=99s recommendation of =E2=80=9Catm ov= erhead 48=E2=80=9D) reverses that situation; Cake will then always end up u= sing *less* of the physical link than it accounts for, which is safe for tr= oubleshooting with.=C2=A0 The keyword is there specifically so that we do= =E2=80=99t have to figure out the precise overhead profile before tackling = more substantive issues.

At any rate, it has nothing to do with Steam specifically.

>> As far as I can tell cake can drill down to the required IP/TCP/UD= P fields independent of whether there are VLAN tags or PPPoE headers so cak= e should not care (except for the different overhead specifications you nee= d to add as stated above). BUT if instantiated on eth0 cake will see pppoe = LCP packets and might decide to drop them, which can take down the link, so= out of caution I would still instantiate on pppoe in your case.
>
> Yeah, with further testing it seems the interface wasn't the culpr= it but I'll still do all my testing on pppoe0 just to be safe.
>
> Anyway I was wondering if there's some kind of manual for Cake and= the various parameters, I'm looking to set it up best way possible but= there are some parameters which I'm not sure what they do (one of them= being "ingress=E2=80=9D).

With the correct version of iproute2 installed, just issue =E2=80=9Cman tc-= cake=E2=80=9D.=C2=A0 That=E2=80=99s the official documentation.

Currently it doesn=E2=80=99t have the ingress keyword yet.=C2=A0 That=E2=80= =99ll be fixed soon.

> Also while reading on the bufferbloat.net Cake page I noticed a possi= ble "fix" for BitTorrent (by setting it as "background"= , https://www.bufferbloat.net/= projects/codel/wiki/Cake/#diffserv-support), I'm wonderin= g if this can be done with Steam too?

It=E2=80=99s possible, if you can figure out which traffic is Steam in the = first place, and write filters to match on it.=C2=A0 This is complicated by= the fact that Valve runs a sophisticated CDN to handle their rather impres= sive bandwidth load.

=C2=A0- Jonathan Morton

_______________________________________________
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake
--001a113f479a727c8e054dc0d1d0--