[Cake] Getting Cake to work better with Steam and similar applications

[Sebastian Moeller]

> On Apr 22, 2017, at 11:36, Jonathan Morton <chromatix99 at gmail.com> wrote:
> 
>>> So please add “atm overhead 32" to cake on eth0 or “atm overhead 40” to cake instances on pppoe (these packets do not have the PPPoE header added yet and hence appear 8 bytes to small).
>> 
>> Thanks for your help, will definitely use them. Just wondering if I use "pppoe-vcmux/bridged-llcsnap" on eth0 or "pppoe-llcsnap" on pppoe0 would have the same effect? Or are there some other "under-the-hood" changes when using them?
> 
> On the pppoe interface, use pppoe-vcmux if your modem is set to use VC-MUX, or pppoe-llcsnap if it’s set to use LLC-SNAP (they might be described using slightly different terms, but should still be recognisable as one or the other).  This probably depends on your ISP, and may further vary regionally within the same ISP.

	In my experience it is rather bothersome trying to get that information from one’s ISP, this is why I would recommend to follow the instructions on https://github.com/moeller0/ATM_overhead_detector and empirically measure the actual overhead. In that case one ends up with the numeric overhead, hence my inclination to use that number directly instead of looking into a table to translate that back into a symbolic keyword… especially since say for an overhead of 32 (and 36) there are two different encapsulation schees that add up to that number:

case 32
		disp('Connection: Bridged, LLC/SNAP RFC-1483/2684');
		disp('Protocol (bytes): Ethernet Header (14), ATM LLC (3), ATM SNAP (5), ATM pad (2), ATM AAL5 SAR (8) : Total 32');

		disp('Connection: PPPoE, VC/Mux RFC-2684');
		disp('Protocol (bytes): PPP (2), PPPoE (6), Ethernet Header (14), ATM pad (2), ATM AAL5 SAR (8) : Total 32’);

good luck divining which of those is in use if all you know is the numeric overhead...



> 
> I really prefer to use the self-explanatory keywords (which is why I added them in the first place) instead of opaque magic numbers.  This is a point on which Sebastian has long disagreed with me.

	True, but I am not going to re-hash that here again ;)

> 
>>> Question: if you set the shaper’s to 50% of line rate (8.75/0.5?) do you still see that unfairness? And if you add “atm overhead 40” to cake on pppoe0 and set the shaper to 90% of line rates (15.75/0.9) how does the Steam affect per-host fairness? Also how transient are these connections team uses?
>> 
>> Actually did more testing about this and it seems that as far I have set the bandwidth to ~15Mbps (so ~15% less of my max speed) and use the "nat" parameter, the per-host fairness works even without the "dual-host" and "overhead" parameters. I definitely find this very interesting, is this behaviour caused by the way Steam downloads games?
> 
> By default, Cake uses triple-isolate mode, which uses information about both source and destination hosts to perform per-host isolation; this usually works well regardless of which side of the connection has the LAN hosts.  The “dual” modes let you specify that fact explicitly, making it a little more robust and predictable.
> 
> Without overhead compensation, Cake will actually use more of the physical link than it thinks it does - by default it only accounts for raw IP or Ethernet packets, depending on the type of interface it’s attached to.  With full-size packets as in a bulk download, the difference is relatively small, so the 15% margin is just about sufficient to make things work.  But with small packets mixed in, the difference grows, such that Cake might no longer control the bottleneck with some traffic mixes.

	All true, to elaborate a bit on the ATM specific issue, due to AAL5’s insistence that each ethernet frame is packaged into an integer number of ATM cells (where the unused octets are simply padded out) the worst case is something like 100%, if a hypothetical packet would only require 49 Bytes, it will still require two ATM cells of 53 bytes...

> 
> The “conservative” keyword I recommended earlier (which is exactly equivalent to Sebastian’s recommendation of “atm overhead 48”) reverses that situation; Cake will then always end up using *less* of the physical link than it accounts for, which is safe for troubleshooting with.  The keyword is there specifically so that we do’t have to figure out the precise overhead profile before tackling more substantive issues.

	Due to the boundary observation above, one other option is to start with the shaper set to 50% of link rate, that should have sufficient wiggle room for all realistic overheads… (but honestly on a known ATM link I would always run the ATM_overhead_detector to get the precise number).

> 
> At any rate, it has nothing to do with Steam specifically.
> 
>>> As far as I can tell cake can drill down to the required IP/TCP/UDP fields independent of whether there are VLAN tags or PPPoE headers so cake should not care (except for the different overhead specifications you need to add as stated above). BUT if instantiated on eth0 cake will see pppoe LCP packets and might decide to drop them, which can take down the link, so out of caution I would still instantiate on pppoe in your case.
>> 
>> Yeah, with further testing it seems the interface wasn't the culprit but I'll still do all my testing on pppoe0 just to be safe.
>> 
>> Anyway I was wondering if there's some kind of manual for Cake and the various parameters, I'm looking to set it up best way possible but there are some parameters which I'm not sure what they do (one of them being "ingress”).
> 
> With the correct version of iproute2 installed, just issue “man tc-cake”.  That’s the official documentation.
> 
> Currently it doesn’t have the ingress keyword yet.  That’ll be fixed soon.
> 
>> Also while reading on the bufferbloat.net Cake page I noticed a possible "fix" for BitTorrent (by setting it as "background", https://www.bufferbloat.net/projects/codel/wiki/Cake/#diffserv-support), I'm wondering if this can be done with Steam too?
> 
> It’s possible, if you can figure out which traffic is Steam in the first place, and write filters to match on it.  This is complicated by the fact that Valve runs a sophisticated CDN to handle their rather impressive bandwidth load.
> 
> - Jonathan Morton
>