[Cake] Using firewall connmarks as tin selectors

[Ryan Mounce]

On Sun, 3 Mar 2019 at 22:22, Kevin Darbyshire-Bryant
<kevin at darbyshire-bryant.me.uk> wrote:
>
> Be afraid, be very afraid.
>
> I’ve woken up with two ideas in my head, one is bad, the other is very bad.  The bad one is already implemented and lurking in the mine branch of my cake git tree:
>
> The bad idea:
>
> An extension of the ‘fwmark’ tin allocation idea is to get cake to automagically update the conntrack mark based on the DSCP tin allocation chosen on egress.  That way, well behaved applications using DSCP (e.g. dropbear) get their return path packets similarly classified on ingress.  Badly behaved applications can have iptables rules put in place to ‘manually’ add fwmarks as is already done.
>
>
> The very bad idea:
>
> And it’s bad ‘cos it’s sort of incompatible with the existing fwmark implementation as described above.  So an awful lot of our shenanigans above is due to DSCP not traversing the internet particularly well.  The solution above abstracts DSCP into ’tins’ which we put into fwmarks.  Another approach would be to put the DSCP *into* the fwmark.  CAKE could (optionally) copy the FWMARK contained DSCP into the diffserv field onto the actual packets.  Voila DSCP traversal across ’tinternet with tin/bandwidth allocation in our local domain preserved.

Ooh now you've got my attention - equal parts horrifying and
brilliant. Just making sure I have my head around this, I think it
would be perfect for my use case.

My ISP requires traffic to be washed on egress, and they wash it on
ingress. I set the DSCP in some applications and mangle some others
with iptables on the router. Either way, DSCP is present by the time
packets reach cake on egress, which dutifully uses DSCP to sort into
tins before washing. This works great, albeit with each packet for
certain flows being inefficiently mangled.

So the first change is that cake will copy the DSCP bits of a packet
into the fwmark on egress - if the fwmark hasn't already been set by
cake. This allows the impact of my iptables mangle rules to be
minimised as only packets without a cake-populated fwmark need their
DSCP to be mangled. Cake then uses the fwmark to sort packets into
tins, and in my case the DSCP is washed on egress.

Then on ingress, cake uses the same fwmark to sort packets into tins.
And cake can copy the DSCP bits back from the fwmark to packets on
ingress. Magic!

The overwriting of the DSCP bits from fwmark could be called
'staining', as opposed to DSCP 'bleaching'. But this doesn't sound
very appealing when we're baking delicious CAKE - we're in the kitchen
not the laundry! So how about dyeing, or colouring?

-Ryan