[Cake] overheads or rate calculation changed?

[Sebastian Moeller]

Hi Andy,


> On Dec 23, 2017, at 00:38, Andy Furniss <adf.lists at gmail.com> wrote:
> 
> Jonathan Morton wrote:
>>> On 21 Dec, 2017, at 2:54 am, Andy Furniss <adf.lists at gmail.com> wrote:
>>> 
>>>    refactor cake_advance_shaper and ack_filter
>>> 
>>>    cake_advance_shaper now returns a modified len argument to
>>>    reflect cake_overhead.
>>>    skb_ack_filter variable replaced with ack
>> Fixed.  At one point cake_advance_shaper() was still getting a packet length with overhead correction already applied, and was then applying it a second time.
> 
> Thanks, seems good now, well, as it was before anyway :-)
> 
> Still a bit confusing for users having to look at max_len in stats to judge what overhead to use if they know what over ip they want.
> 
> Maybe some hint in the man page about turn off gro and look at max_len would help when using raw.
> 
> IIRC this was discussed before, I think conclusion was it's the kernel/ppp being different/wrong sometimes in what it reports to cake.
> 
> My specific case is pppoe where I need IP + 34. I can see 1500 as max_len, which = IP length so raw overhead 34 is correct.
> 
> On a pure eth to get the same I need raw overhead 20 as max_len is 1514.
> 
> This is the (confusing) output from pppoe
> 
> ./tc qdisc add dev ppp0 handle 1:0 root cake bandwidth 19690kbit raw overhead 34 diffserv4 dual-srchost nat rtt 200ms
> 
> qdisc cake 1: dev ppp0 root refcnt 2 bandwidth 19690Kbit diffserv4 dual-srchost nat rtt 200.0ms noatm overhead 56 via-ethernet total_overhead 56 hard_header_len 22

	This is still broken, the overhead the should just be 34, and hard_header_len should be 0, even though for pppoe 22 is not completely wrong (14 + 8). I currently see the same with PPPoE+VLAN I get 26. So either that is a change in current cake or the kernel actually started to report a more precise hard_header_len...


> Sent 776346756 bytes 883288 pkt (dropped 284882, overlimits 1429302 requeues 0)
> backlog 0b 0p requeues 0
> memory used: 4196096b of 4Mb
> capacity estimate: 19690Kbit
>                   Bulk  Best Effort        Video        Voice
>  thresh       1230Kbit    19690Kbit     9845Kbit     4922Kbit
>  target         14.8ms       10.0ms       10.0ms       10.0ms
>  interval      204.8ms      200.0ms      200.0ms      200.0ms
>  pk_delay        186us        1.6ms          0us        112us
>  av_delay         11us         94us          0us          4us
>  sp_delay          1us          4us          0us          4us
>  pkts             1160      1166876            0          134
>  bytes          457795   1202631653            0        10176
>  way_inds            0         3813            0            0
>  way_miss         1126        10503            0            3
>  way_cols            0            0            0            0
>  drops               0       284882            0            0
>  marks               0            0            0            0
>  ack_drop            0            0            0            0
>  sp_flows            0            1            0            0
>  bk_flows            0            1            0            0
>  un_flows            0            0            0            0
>  max_len           746         1500            0           76
> 
> 
> 
> On eth raw overhead 34 gives different results, only seeing max_len gives the clue to take 14 to get IP + 34.
> 
> tc qdisc add dev enp6s0 handle 1:0 root cake bandwidth 19690kbit raw overhead 34
> 
> qdisc cake 1: dev enp6s0 root refcnt 2 bandwidth 19690Kbit diffserv3 triple-isolate rtt 100.0ms noatm overhead 48 via-ethernet total_overhead 48 hard_header_len 14

	This looks like expected, the in-kernel overhead is added to the specified overhead, just like in the past. I really really wished cake would report the packet size before and _after_ its overhead addition making sanity checking much easier. BTW tc's stab might be useful as an external check... I also wonder whether the kernel's behaviour in regards to ppp interfaces changed between kernel 4.4 and newer ones, I see the same weirdness:
root at router:~# uname -a
Linux router 4.9.67 #0 Sun Dec 17 11:53:32 2017 mips GNU/Linux
excerpt from tc -s:
qdisc cake 800f: dev pppoe-wan root refcnt 2 bandwidth 9545Kbit diffserv3 dual-srchost nat rtt 100.0ms noatm overhead 34 via-ethernet total_overhead 34 hard_header_len 26 mpu 64 

I note that hard_header_len 26 sort of matches the situation on the ground: 
Mine:
14 (SRCMAC, DSTMAC, ethertype) + 8 (PPP, PPPOE), + 4 (VLAN) = 26
Yours: 
14 (SRCMAC, DSTMAC, ethertype) + 8 (PPP, PPPOE) = 22

I guess I should try with different kernels and recent sch_cake...




Best Regards
	Sebastian


> Sent 24102094 bytes 15934 pkt (dropped 1, overlimits 31841 requeues 0)
> backlog 0b 0p requeues 0
> memory used: 27Kb of 4Mb
> capacity estimate: 19690Kbit
>                   Bulk  Best Effort        Voice
>  thresh       1230Kbit    19690Kbit     4922Kbit
>  target         14.8ms        5.0ms        5.0ms
>  interval      109.8ms      100.0ms      100.0ms
>  pk_delay          0us        1.7ms          0us
>  av_delay          0us        1.7ms          0us
>  sp_delay          0us        229us          0us
>  pkts                0        15935            0
>  bytes               0     24103608            0
>  way_inds            0            0            0
>  way_miss            0            4            0
>  way_cols            0            0            0
>  drops               0            1            0
>  marks               0            0            0
>  ack_drop            0            0            0
>  sp_flows            0            0            0
>  bk_flows            0            1            0
>  un_flows            0            0            0
>  max_len             0         1514            0
> _______________________________________________
> Cake mailing list
> Cake at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake