[Cake] overhead and mpu

[Sebastian Moeller]

Hi Dennis,

> On Sep 7, 2017, at 09:58, Dennis Fedtke <dennisfedtke at gmail.com> wrote:
> 
> Hi Sebastian,
> 
> here is the outout:
> 
> config queue
>         option debug_logging '0'
>         option interface 'eth1.100'
>         option qdisc 'cake'
>         option enabled '1'
>         option qdisc_advanced '1'
>         option qdisc_really_really_advanced '1'
>         option linklayer 'none'
>         option egress_ecn 'NOECN'
>         option upload '19000'
>         option verbosity '5'
>         option ingress_ecn 'ECN'
>         option script 'layer_cake.qos'
>         option download '150000'
>         option squash_ingress '1'
>         option squash_dscp '1'
>         option iqdisc_opts 'docsis nat'
>         option eqdisc_opts 'docsis nat'

Ah, there goes my typo or forgotten keyword hypothesis...
	I bet the reason is that on the IFB the vlan tag might not be visible, while on egress it is; unexpectedly cake will not report the overhead if the specified overhead equals the hard_header_len (@Jonathan, what is the rationale for the behavior? could you live with reporting the hard_header_len unconditionally somewhere in the "tc -s qdisc" output?). Since with a vlan tag the kernel accounts for 18 bytes (a vlan tag takes 4 bytes and 14+4=18) cake simply omits this during reporting. Now why the kernel deos not seem to see/account for the VLAN tag on ingress I do not know. But this certainly shows that cake does the right thing by automatically adjusting for hard_header_len... this certainly is more complicated than it should be...


> 
> But also see my last email please.
> 
> it seems that both devices use different default overhead (14 vs 18)

	I agree, but I have no time to dive into the kernel sources to figure out why...

Best Regards
	Sebastian

> 
> Thanks.
> 
> 
> Am 07.09.2017 um 09:47 schrieb Sebastian Moeller:
>> Hi Dennis,
>> 
>>> On Sep 7, 2017, at 06:11, Dennis Fedtke <dennisfedtke at gmail.com> wrote:
>>> 
>>> Hi everyone!
>>> 
>>> thanks for your answers again.
>>> 
>>> So what about from should i expect from using the docsis keyword?
>>> (This layer cake script + nat docsis set in advanced config)
>>> 
>>> tc -s qdisc show dev eth1.100:
>>> qdisc cake 8064: root refcnt 2 bandwidth 19Mbit diffserv3 triple-isolate nat rtt 100.0ms raw mpu 64
>>>  Sent 246233450 bytes 1316430 pkt (dropped 65, overlimits 126706 requeues 0)
>>>  backlog 0b 0p requeues 0
>>>  memory used: 2372672b of 4Mb
>>>  capacity estimate: 19Mbit
>>>                  Bulk   Best Effort      Voice
>>>   thresh      1187Kbit      19Mbit    4750Kbit
>>>   target        15.3ms       5.0ms       5.0ms
>>>   interval     110.3ms     100.0ms      10.0ms
>>>   pk_delay         0us       298us        10us
>>>   av_delay         0us        13us         2us
>>>   sp_delay         0us         1us         1us
>>>   pkts               0     1315692         803
>>>   bytes              0   246260830       59314
>>>   way_inds           0         733           0
>>>   way_miss           0        5012          29
>>>   way_cols           0           0           0
>>>   drops              0          65           0
>>>   marks              0           0           0
>>>   sp_flows           0           1           0
>>>   bk_flows           0           0           0
>>>   un_flows           0           0           0
>>>   max_len            0        1514         405
>>> 
>>> tc -s qdisc show dev ifb4eth1.100
>>> qdisc cake 8065: root refcnt 2 bandwidth 150Mbit diffserv3 triple-isolate nat wash rtt 100.0ms noatm overhead 18 via-ethernet mpu 64
>>>  Sent 869520280 bytes 2153104 pkt (dropped 30, overlimits 270811 requeues 0)
>>>  backlog 0b 0p requeues 0
>>>  memory used: 624320b of 7500000b
>>>  capacity estimate: 150Mbit
>>>                  Bulk   Best Effort      Voice
>>>   thresh      9375Kbit     150Mbit   37500Kbit
>>>   target         5.0ms       5.0ms       5.0ms
>>>   interval     100.0ms     100.0ms      10.0ms
>>>   pk_delay       121us       448us         1us
>>>   av_delay        40us       158us         0us
>>>   sp_delay         2us         2us         0us
>>>   pkts           58805     1362429      731900
>>>   bytes       77433974   748216627    43914060
>>>   way_inds           0      118167           0
>>>   way_miss           5        5911           3
>>>   way_cols           0           0           0
>>>   drops              1          29           0
>>>   marks              0           0           0
>>>   sp_flows           0           0           0
>>>   bk_flows           0           0           1
>>>   un_flows           0           0           0
>>>   max_len         1486        1514          90
>>> 
>>> Why is there no overhead showing for egress?
>> Could you post post the output of "cat /etc/config/sqm" (assuming you use LEDE/OpenWrt); i have a hunch that you might have forgotten to put the docsis keyword into both advanced configuration fields (or there is a typo n one of those words).
>> 
>> 
>>> Why is there overhead18 + via-ethernet showing for ingress?
>> 	This tells you that cake is using ethernet_payload + 18 bytes overhead, and that it automatically effectively subtracted the 14 bytes overhead the kernel added before applying its 18 bytes (so that you do not need to worry about how much the kernel added).
>> 
>> BTW, cake reports the maximum of the pkt_len field of all packets as max_len and on ethernet interfaces the kernel automatically adds 14 bytes (already to skb->len), this is why you see max_len 1514 instead of 1500. If you use tc's stab method for overhead accounting things will look different, as stab will add the full header size to pkt_len, so using tc stab you would probably see 1532 for an ecplicit overhead of 18 (as tc stab does not take the already accounted for header into account so if you request 18 on an ethernet interface you will effectively get 18+14 = 32.) It really is  a great feature of cake to undo the kernel's automagic length adjustment so that one can request overhead on top of the ethernet payload/the IP packets.
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> Thanks again =)
>>> 
>