[Cake] Getting Cake to work better with Steam and similar applications

[Jonathan Morton]

>> So please add “atm overhead 32" to cake on eth0 or “atm overhead 40” to cake instances on pppoe (these packets do not have the PPPoE header added yet and hence appear 8 bytes to small).
> 
> Thanks for your help, will definitely use them. Just wondering if I use "pppoe-vcmux/bridged-llcsnap" on eth0 or "pppoe-llcsnap" on pppoe0 would have the same effect? Or are there some other "under-the-hood" changes when using them?

On the pppoe interface, use pppoe-vcmux if your modem is set to use VC-MUX, or pppoe-llcsnap if it’s set to use LLC-SNAP (they might be described using slightly different terms, but should still be recognisable as one or the other).  This probably depends on your ISP, and may further vary regionally within the same ISP.

I really prefer to use the self-explanatory keywords (which is why I added them in the first place) instead of opaque magic numbers.  This is a point on which Sebastian has long disagreed with me.

>> Question: if you set the shaper’s to 50% of line rate (8.75/0.5?) do you still see that unfairness? And if you add “atm overhead 40” to cake on pppoe0 and set the shaper to 90% of line rates (15.75/0.9) how does the Steam affect per-host fairness? Also how transient are these connections team uses?
> 
> Actually did more testing about this and it seems that as far I have set the bandwidth to ~15Mbps (so ~15% less of my max speed) and use the "nat" parameter, the per-host fairness works even without the "dual-host" and "overhead" parameters. I definitely find this very interesting, is this behaviour caused by the way Steam downloads games?

By default, Cake uses triple-isolate mode, which uses information about both source and destination hosts to perform per-host isolation; this usually works well regardless of which side of the connection has the LAN hosts.  The “dual” modes let you specify that fact explicitly, making it a little more robust and predictable.

Without overhead compensation, Cake will actually use more of the physical link than it thinks it does - by default it only accounts for raw IP or Ethernet packets, depending on the type of interface it’s attached to.  With full-size packets as in a bulk download, the difference is relatively small, so the 15% margin is just about sufficient to make things work.  But with small packets mixed in, the difference grows, such that Cake might no longer control the bottleneck with some traffic mixes.

The “conservative” keyword I recommended earlier (which is exactly equivalent to Sebastian’s recommendation of “atm overhead 48”) reverses that situation; Cake will then always end up using *less* of the physical link than it accounts for, which is safe for troubleshooting with.  The keyword is there specifically so that we do’t have to figure out the precise overhead profile before tackling more substantive issues.

At any rate, it has nothing to do with Steam specifically.

>> As far as I can tell cake can drill down to the required IP/TCP/UDP fields independent of whether there are VLAN tags or PPPoE headers so cake should not care (except for the different overhead specifications you need to add as stated above). BUT if instantiated on eth0 cake will see pppoe LCP packets and might decide to drop them, which can take down the link, so out of caution I would still instantiate on pppoe in your case.
> 
> Yeah, with further testing it seems the interface wasn't the culprit but I'll still do all my testing on pppoe0 just to be safe.
> 
> Anyway I was wondering if there's some kind of manual for Cake and the various parameters, I'm looking to set it up best way possible but there are some parameters which I'm not sure what they do (one of them being "ingress”).

With the correct version of iproute2 installed, just issue “man tc-cake”.  That’s the official documentation.

Currently it doesn’t have the ingress keyword yet.  That’ll be fixed soon.

> Also while reading on the bufferbloat.net Cake page I noticed a possible "fix" for BitTorrent (by setting it as "background", https://www.bufferbloat.net/projects/codel/wiki/Cake/#diffserv-support), I'm wondering if this can be done with Steam too?

It’s possible, if you can figure out which traffic is Steam in the first place, and write filters to match on it.  This is complicated by the fact that Valve runs a sophisticated CDN to handle their rather impressive bandwidth load.

 - Jonathan Morton