[Cake] A CAKE success story

[Kevin Darbyshire-Bryant]

Hi All,

Never afraid of sharing my mistakes if nothing else to help prevent others from falling in to the same trap I offer the following lesson/s :-)

As you know I’ve been working on a convoluted scheme to store & subsequently restore DSCP values to/from firewall marks.  Linux 5.3 features ‘act_ctinfo’ for the restore side of things, the store being an out of tree ‘hack’.  I’ve a cobbled together sqm-script that combines the ingredients in a suitable manner. act_ctinfo is a tc filter and perhaps surprisingly I used an instance of this DSCP restoration on both the ingress path AND the egress.  The egress path is there as an ‘optimisation’ e.g. we go through iptables once, decide on a DSCP value, store it and then that value is applied to all subsequent packets for the connection on both ingress and egress paths.  The point is to exercise CAKE’s diffserv capabilities a bit more often.

I use CAKE’s diffserv4 profile, mainly so that I have somewhere to de-prioritise to rather than trying to prioritise everything up, in other words I try to put the unimportant stuff in bulk and let everything else stay in best effort (or video or voice if it’s easy to do). An example of de-prioritising is MS Onedrive.  But I encountered a strange issue, namely that on my 80mbit down/20mbit up with onedrive uploads going I could only get about half bandwidth in the download direction from onedrive.  Traffic in other tins was fine, but a onedrive up & down in ‘bulk’ would be less performant.

Fiddling around proved it was something to do with the tc filter invocation, especially on the egress side - even if I got ‘act_ctinfo’ to do nothing performance still dropped off.  Another symptom was cake’s stats only showed a single ‘bulk’ flow, even though there were at least 5 flows.

In a demonstration of what happens when you cargo cult existing code, this was my ‘egress’ tc filter invocation:

$TC filter add dev $IFACE protocol all prio 10 u32 match u32 0 0 flowid 1:1 action \
        ctinfo dscp 0xfc000000 0x02000000

And this is what it should have been:

$TC filter add dev $IFACE protocol all prio 10 u32 match u32 0 0 action \
        ctinfo dscp 0xfc000000 0x02000000

I haven’t delved into the tc man page yet, but I suspect ‘flowid 1:1’ overrides all the flow ids, so cake was doing the right thing and basically ignoring flow isolation - yikes!

There’s a similar incantation used on the ingress side of things in a few of the sqm-scripts - I suspect they’re harmless in this case because it’s part of the IFB redirection and effectively gets ignored.  I removed the ‘flowid’ on my ingress path as well :-)


This nasty has been lurking in my setup for a while now and I always felt there was something occasionally odd going on, it’s now fixed!


Cheers,

Kevin D-B

gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20190921/8d5de72f/attachment.sig>