From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 599983B2A4 for ; Tue, 5 Sep 2017 04:01:31 -0400 (EDT) Received: from [192.168.250.101] ([134.76.241.253]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0M9OMc-1dcWYR05s6-00ClZC; Tue, 05 Sep 2017 10:01:29 +0200 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Sebastian Moeller In-Reply-To: Date: Tue, 5 Sep 2017 10:01:27 +0200 Cc: cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Dennis Fedtke X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:PmLvGenMSWBmeIJ3y86XTQo++0nxKBVpejR2JZrf+CYadbzRAA3 YhVbroOYnwrTK3sFrcp35O7nM8mkQRda6KDHSKKpzuYL49NsOBR0FXg4q5gy5fRD/aBVcAu 2F4jun0Xz7HvwFxiQmkeMfgNGb5u83PRD+XavOXUbfMUggGrOCwFoef/zH9E9jVjAmL4NfV cRSwLFQLY98UYwqv+edzQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:TnCj3eFcQ6g=:C/QpT0tVAwMThI0lfbXFkI riLRzTzpx1wG8KQHwiTO6mahsqZq3ym8+X6h4322vAd7J63BpbXDP1OWVdrEvljLsBySlJk4+ Zu2WgXXar6Fz2iSyEtetHlqidNz8qZr9ATvMTY3YUcFoF7a6aCdgn+fh0bF6Ja0VUkidx2YG3 D+jsbwP/cueRxU/nxDOJvzXFNbU91xp3RT8h0suHnuQG9T4rKkWWZh8H/FC/SMv2SF0rcOPHe DZyt67eI4a2JbUGgn6sLSiBUCdFNk6/tqQuS8+HYwzj01eCg3mEHs+BL43/ze7wnsYWdeOG05 MNLN1f8xeV/dJiOqcOYeRwqMCBJ/RE3xaJ17v938ii4QXfNuDz+dYyiiN1/qbihBfpByrNmma DTsrCPuEDeaIT5RjqRt0MUMw1WbqNZCOREKjGEOg9cm4cdz9qj9HnKWQP/0DxkTo5i3zILeLo fA7pfQuYrsateUN1fY4BY5X8S3w4KccAfG3ryknGMVpHjqZh3q2rAnYHtRHMCE8eP1E9GNeQU XfE2O+KtDdsNkQWkk0pXLUP0aVnzQa8lBgRC5GKG815+fjTVVrrZRTYerlBtX9Zy7mes05spj fdAAtHhqBjh4mOjWRTl8lDkkcje+5HXRaZlOJ3qmW1iL10A7rk/jO5XA8rtPbYVzqnZX1icJ7 Ui7BQHEms5XwxOT9ZEs2QhlfIFMVsBtV470nve+7JpImm3i7Hxw81Iog2WqNZmeEEczz/sIDv WAxLc01zkhrAUUF6QTbQrcxrw6sw7EsYFG/xq5glfhaqfvnxyU+GX7EIkGpDUPVTPNvF0X2Py PMDiI362P/1b0cO2TSOdCQ/WZTwAowkX+tplt9puud7MNJKzN0= Subject: Re: [Cake] overhead and mpu X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2017 08:01:31 -0000 Hi Dennis, > On Sep 5, 2017, at 08:00, Dennis Fedtke = wrote: >=20 > Hi Ryan, >=20 > Thanks for you answers. > Lets assume again ethernet over docsis connection at 50 Mbit/s. As the other's already mentioned, on DOCSIS it is customary to = lie to the customer regarding the provisioned bandwidth; fortunately the = "lie" is actually in favor of the customer, so nobody needs to feel bad = about this ;) > So to get 50 Mbit/s ethernet perfomance, the docsis link speed needs = to be set at a higher link speed to compensate for the 18 header = ethernet overhead, or? > or > Assuming: > The docsis link syncs at exactly 50Mbit/s. When cake is set to exactly = 50Mbit/s, it is actually set at a higher speed then the link actually = is. On DOCSIS networks according to Greg White of cablelabs: For downstream packets, the DOCSIS MAC header consists of the standard = 6-byte MAC Header, plus 6 bytes for the DS EHDR and 5 bytes for the = Downstream Privacy EH Element. So, 17+14+4=3D35 bytes of per-packet = overhead for a downstream DOCSIS MAC Frame. For upstream packets, there will typically be only the 6-byte MAC = Header, plus the 4 byte Upstream Privacy EH version 2 Element, so = 10+14+4=3D28 bytes of per-packet overhead for an Upstream DOCSIS MAC = Frame. The 14+4 denotes the 14 byte ethernet header (6 byte dest mac, 6 byte = src mac, 2 byte ether type) plus the 4 byte ethernet frame check = sequence FCS. Interestingly, the above DICSIS observations seem irrelevant for end = customers as the shaper used in DOCSIS systems that limits a users = maximal bandwidth does completely ignore DOCSIS overhead and only = includes ethernet frames including their frame check sequence (FCS 4 = Byte). Tocite the relevant section from the Docsis standard = (http://www.cablelabs.com/specification/docsis-3-0-mac-and-upper-layer-pro= tocols-interface-specification/): "C.2.2.7.2 Maximum Sustained Traffic Rate 632 This parameter is the rate = parameter R of a token-bucket-based rate limit for packets. R is = expressed in bits per second, and MUST take into account all MAC frame = data PDU of the Service Flow from the byte following the MAC header HCS = to the end of the CRC, including every PDU in the case of a Concatenated = MAC Frame. This parameter is applied after Payload Header Suppression; = it does not include the bytes suppressed for PHS. The number of bytes = forwarded (in bytes) is limited during any time interval T by Max(T), as = described in the expression: Max(T) =3D T * (R / 8) + B, (1) where the = parameter B (in bytes) is the Maximum Traffic Burst Configuration = Setting (refer to Annex C.2.2.7.3). NOTE: This parameter does not limit = the instantaneous rate of the Service Flow. The specific algorithm for = enforcing this parameter is not mandated here. Any implementation which = satisfies the above equation is conformant. In particular, the = granularity of enforcement and the minimum implemented value of this = parameter are vendor specific. The CMTS SHOULD support a granularity of = at most 100 kbps. The CM SHOULD support a granularity of at most 100 = kbps. NOTE: If this parameter is omitted or set to zero, then there is = no explicitly-enforced traffic rate maximum. This field specifies only a = bound, not a guarantee that this rate is available." So in essence DOCSIS users need to only account for 18 Bytes of ethernet = overhead in both ingress and egress directions under non-congested = conditions. >=20 > For mpu why 64? > I assume minimum 46bytes ethernet payload + 18bytes header? Well your DOCSIS modem will simply grab the ethernet packets = including the FCS and those are simply guranteed to be >=3D 64 bytes. = The linux kernel however might see smaller packets if the payload is = smaller (but note for TCP/IPv4 with its 40 bytes of header the = difference is quite small anyway). Interestingly if you use a single = VLAN tag the ethernet MPU still stays at 64 bytes, only the payload = section (including padding) shrinks to 42 bytes... >=20 > But why does cake use hard_header_len for the header size which is 14 = bytes? I believe this is used internally so cake can deduce the size of = the automatically added overhead (the linux kernel will add 14 bytes on = ethernet interfaces automatically, which while certainly justifiable are = not the ideal value for an ethernet shaper); with this cake allows the = user to specify absolute overhead (so if you specify overhead 18, cake = will give you 14 - 14 + 18 =3D 18, while tc's stab method will give you = 14 + 18 =3D 32). So cake is doing the right thing here (the only thing = it could do even better is to report the size of the automatically = corrected in-kernel hard_header_len, but then the audience for that = information is probably to small to justify this feature). >=20 > I don't know how packet sheduler system works, but maybe it is not = needed to include the ethernet header. All you really need to give cake is a precise estimate of the = bandwidth and the effective on-the-wire-size of the data packets; for = DOCSIS this means overhead of 18 bytes* >=20 > cake does work on ip pakets or? so this is layer3 i think. > the hole thing with ethernet headers is happening in layer2. > This would change the minimum packet size to 46 or? =46rom sch_cake.c: static inline u32 cake_overhead(struct cake_sched_data *q, u32 in) { u32 out =3D in + q->rate_overhead; =20 if (q->rate_mpu && out < q->rate_mpu) { out =3D q->rate_mpu; } =20 if (q->rate_flags & CAKE_FLAG_ATM) { out +=3D 47; out /=3D 48; out *=3D 53; } else if(q->rate_flags & CAKE_FLAG_PTM) { // the following adds one byte per 64 bytes or part = thereof // this is conservative and easier to calculate than the = precise value out +=3D (out / 64) + !!(out % 64); } =20 return out; =20 } As you can see cake first adds the overhead and then makes sure the = packet size is >=3D MPU... This will do the right thing for DOCSIS, even = though it is conceivable that there are encapsulations out that can not = be fully described with this (think full ethernet frames with additional = overhead, where size+full_overhead might be >=3D mpu but the correct = method would calculate size+ethernet_overhead >=3D mpu and then add = outer_layer_overhead on top of this, but this is largely academic until = something like this gets rolled-out in large numbers...) Best Regards Sebastian P.S.: unlike Ryan and Jonathan I would always use "overhead 18 mpu 64" = instead of "docsis" because I prefer explicitness over terseness, but = that is a matter of taste and "tc -s qdisc" will helpfully expand docsis = into "overhead 18 mpu 64" (which is great). >=20 > Thanks. >=20 > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake *) The 18 bytes DOCSIS overhead are actually a lie, the true per-packet = overhead on docsis is far greater. Without congestion all this = additional overhead is simply "payed" for the user out of the shared = gross bandwidth and everybody is happy. But under congestion this, IMHO, = can lead to massive unfairness in that a user's shate of the gross = bandwidth will correlate (bandwidth being equal) with the inversre of = the packet size; put differently small packets use relatively more of = the gross bandwidth than large packets, and I have not found out how = docss gurantees fairness to all the CPE in a segment, but I digress.=20