[Cake] Frontier FIOS Framing

[Sebastian Moeller]

> On Sep 23, 2019, at 06:26, Jonathan Morton <chromatix99 at gmail.com> wrote:
> 
>> I have searched every nook and cranny of the bloated internet looking for any information I can find on whether Frontier/Verizon FIOS (assuming the only difference between the service offered by both Frontier and Verizon is in name only) requires any special framing parameters passed on to sch_cake's overhead settings. Most mentions of cake/fq/scm/etc and FIOS are ether very dated and inconclusive or I find messages and forum posts asking questions a lot like this one.
> 
> I don't know precisely what framing FIOS uses.  However, most provisioning shapers used by cable/fibre ISPs operate on Ethernet frames, so if you use the "ethernet" keyword you should match what the shaper is doing.  The proof of the pudding is in the eating, of course.

	Ethernet will assume 38 bytes  of overhead, including the "silent" inter-frame-gap overhead (which essentially is a gap in transmission sized so that one could transmit 12 octets instead), for real active ethernet over fiber that should be correct (maybe it needs another 4 bytes if a vlan tag is used).
	But according to https://www.lightreading.com/gigabit/fttx/verizon-preps-next-major-broadband-upgrade/d/d-id/722062 verizon uses GPON, and all we know about GPON is that is uses a 5 byte GEM header which replaces parts of the ethernet overhead (IFG, Preamble, SFD, -> 12+7+1 = 20Bytes) for an estimated per packet overhead of 38-20+5 = 23 Bytes. But I do not know how much additional "hidden" overhead GPON adds to each packet. So "ethernet's" 38 bytes should be a decent safe bet (it is always better to over-estimate per-packet-overhead, as otherwise small packets will fool the shaper).


> 
>> Currently this is what I have and am also curious if I should be using the "nat" keyword for both ingress and egress? I'm not entirely sure - see below:
> 
> If your box is doing NAT *and* you are using a flow-mode that depends on accurate internal host information, then you should have the "nat" keyboard on in both directions.  Otherwise it's more efficient to switch it off, though leaving it on does no harm otherwise.
> 
> The default flow-mode is "triple-isolate", which does use internal host information.  So do the "dual-srchost" and "dual-dsthost" modes, which are more precise but need you to specify which direction the traffic is flowing.  The "besteffort" and "flows" modes do not, but you should only use those if you're deliberately experimenting with something.
> 
>> In absence of framing compensation I figured I should just go extreme by reserving more bandwidth than the qdisc needs because I also read somewhere I think that mentioned that if you don't compensate and are incorrect everything stops working as opposed to if you over compensate you might lose out on bandwidth but you'll still win in the latency department.
> 
> That's approximately correct, close enough for actual practice.  It's also why we included the "conservative" keyword, which applies the maximum amount of framing compensation that is ever likely to be seen in the wild - much more than you'd expect to see on a cable/fibre link, but only slightly more than on most ADSL lines.

	I am not 100% sure the "likely to be seen in the wild" assumption still holds, ds-lite will easily add another 40 bytes of overhead for IPv4 packets (as will other IPv4 in IPv6 encapsulations)...

> 
> The overhead compensation matters more with small packets than with the larger ones used for bulk transfers; for the latter, reserving a little more bandwidth will appear to make everything work.  

	+1, for any given packet-size under-estimation of either gross-shaper-rate or per-packet-overhead can be compensated by over-estimating the other, and since most speedtests employ large packets that can lead to under-estimation of the overhead which will cause bufferbloat if there are enough small packets in flight (for which effective transmit duration estimation will be too small with too little overhead leading to observable bufferbloat).


> For fibre I would try "ethernet" and reserve about 1% bandwidth each way, then if possible test to see whether there is any bloat.

	First the iteration, perform a speedtest and plug the resulting goodput numbers into cake as gross-shaper-rates, and then you can slowly increase these limits until you see latency under load increase more than you are willing to accept. I would first keep the ingress/download rate at goodput number and optimize egress/upload and then do the same for the ingress rate. Ideally use a bidirectionally saturating speedtest (like flent's RRUL or RRUL_CS8), as that will typically be more sensitive and show a higher magnitude of latency under load increase for the same settings than doing the up-/downloading tests sequentially. 

Best Regards
	Sebastian

> 
> - Jonathan Morton
> _______________________________________________
> Cake mailing list
> Cake at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake