[Cake] Ethernet frame overheads

[Sebastian Moeller]

Hi list,


On Sep 30, 2015, at 12:53 , Kevin Darbyshire-Bryant <kevin at darbyshire-bryant.me.uk> wrote:

> Greetings fellow cake eaters :-)
> 
> Sebastian raised a little flag in the brain cell yesterday in his
> questioning/discussion of cake stats & kernel accounted for overheads. 
> As he stated the kernel considers the overhead of an ethernet packet to
> be 14 bytes but forgets about the 4 byte frame check sequence and
> pre/post ambles which in the end make a packet on the wire worth 1538
> bytes  (must use octets, must use octets)
> 
> Jonathan pointed out that cake's overhead parameter/s are used for
> timing purposes, which to me makes it all the more important that
> overheads are got right.
> 
> As much as I'm loath to suggest yet another option to cake, should it
> not by default assume ethernet 'on-the-wire' framing overheads
> (preamble& start of frame=8, FCS=4, interpacket gap=12) totalling 24
> octets, not forgetting VLAN tag/s at 4 octets each and the already
> accounted for 14 octets of MAC source,dest & frame type/length for
> timing purposes?  Ref: https://en.wikipedia.org/wiki/Ethernet_frame

	At least we could come up with a nice symbolic option “ether-layer1” or “ether-phy” or some-such that adds the missing preamble and inter-frame gap (IFG), we already have "ether-fcs”, maybe, since this is just tc that needs modification we can go wild and do “ether-all” to include IFG, preamble, MAC-header, and FCS… for people wanting to shape an ethernet link...

> 
> For ethernet links this is going to be important.  For other links
> 'transporting' ethernet at slower rates (I'm thinking VDSL2 modems & the
> like) I suspect their overheads and pure slowness of link swamp the
> timing discrepancy.

	A lot of the link types used by ISPs actually contain an ethernet MAC header as part of the transferred data (I believe of the common ones only IPoA and PPPoA avoided that basically useless 14 bytes…), so those 14 bytes truly are relevant on-the-wire overhead for those links, while the layer1 stuff does not seem to be transferred and FCS being potentially/likely used in VDSL2. The real ethernet overhead commes into play the moment one tries to shape a VDSL2 link to 100Mbps on a modem connected vie 100Mbps ethernet… (best solution, don’t do that then use a 1Gbps link to the modem ;) )





Too many details:

# this is just from the standards, I have never seen a VDSL1 link in real life:
VDSL1 (see G.993.1 Annex H): 2 Byte PPP + 6 Byte PPPoE + 4 Byte VLAN + 1 Byte Opening Flag Sequence (OFS), 1 Byte Address Field (AF), 1 Byte Control Field (CF), 2 Byte TC-CRC (PTM-FCS), 1 Byte Closing Flag Sequence (CFS) (CFS might be unused for back-2-back PTM frames?(*)) = 18 byte, or 17 byte (if CFS is unused, it seems the CFS is essentially free as it will be omitted if there is back2back data to send). This all seems to be HDLC encoded, which statistically costs 1/128 if I understood Jonathan correctly, but with tailored nasty packet payloads might be much larger if I understand the HDLC typed used correctly.

# my ISP uses PPPoE and a VLAN that is terminated by the MODEM
PPPOE-VLAN-VDSL2 (IEEE 802.3-2012 61.3 relevant for VDSL2): 2 Byte PPP + 6 Byte PPPoE + 4 Byte VLAN + 1 Byte Start of Frame (S), 1 Byte End of Frame (Ck), 2 Byte TC-CRC (PTM-FCS), = 16 Byte

FAST-ETHERNET: 7 Byte Preamble + 1 Byte start of frame delimiter (SFD) + 12 Byte inter frame gap (IFG) = 20 Byte

(see G.992.3 Annex N N.1 note 4 neither Preamble und nor SFD are transferred over VDSL, the IFG does not even consist out of bytes that could be transferred it is simply a temporal break taking the same time as a transfer of 12 bytes would take)

COMMON for pppoe-VDSL2 and fast-ethernel: 4 Byte Frame Check Sequence (FCS) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype) = 18 byte

Now let’s see what happens if we try to feed a VLAN and PPPoE terminating modem-router via fast-ethernet:
max Paket size 1492 Byte (due to 8 Byte PPP/PPPoE Overhead between Modem and BRAS)

FE: Payload 1492: (no VLANs no PPPoE encapsulation)
Layer 1+: (1492 (Payload = pMTU) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype) + 4 (FCS) + 7 (preamble) + 1 (start of frame delimiter) + 12 (interframe gap)) = 1492+6+6+2+4+7+1+12 = 1530
payload-factor: (1492/1530) = 0.975163398693
payload rate: 100 * payload-factor = 100 * (1492/1530) = 97.5163398693 Mbps

VDLS2: 109.341 Mbps “link rate”

theoretisches VDSL100 (ohne G.INP) mit exakt 100Mbps "link rate" 
Layer1+: (1492 (Nutzlast = pMTU) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype) + 4 FCS + 2 (PPP), + 6 (PPPoE) + 4 VLAN + 5 (PTM Framing) = 1492+6+6+2+4+2+6+4+5 = 1527 
payload-factor: (1492/1527) = 0.977079240341
encapsulation-factor (for PTM)(***): 64/65 = 0.984615384615

payload rate: 109.341 * payload-factor * encapsulation-factor  = 109.341 * (1492/1527) * (64/65) = 105.191208584 Mbps
Loss of VDSL2-rate caused by  FE-Bottleneck:
(100* 105.191208584/97.5163398693)-100 = 7.9%

Starting at around 100Mbps fast-ethernet turns into a bottleneck for VDSL2-links, so in that situation a downlink shaper needs to account for that by shaping to a lower speed; an upload shaper however could be replaced by BQL, assuming BQL works well enough… The example calculation is not the worst case as the overhead on the VDSL2 link could be lower and on the FE link it could be larger, but at least the values match what a large european ISP uses in its home market...

Best Regards
	Sebastian




> 
> 'ethernet' & 'ethernetotw' flags?
> 
> What don't I understand properly here?
> 
> Kevin
> 
> _______________________________________________
> Cake mailing list
> Cake at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake