Re: [Cake] customizing Cake's isolation with ipsets, tc-flow and eBPF

[Pete Heist <pete@heistp.net>]

On Sun, 2021-06-06 at 21:59 +0200, Toke Høiland-Jørgensen wrote:
> Pete Heist <pete@heistp.net> writes:
> 
> > I've always wanted a way to customize Cake's host and flow isolation
> > in
> > a way that would be usable e.g. for small ISPs, and this is what I
> > came
> > up with:
> > 
> > https://github.com/heistp/cake-custom-isolation
> > 
> > ipsets are used to set the skb priority or mark, then tc-flow or a
> > simple eBPF classifier is used in a child filter of cake to get the
> > major and minor class IDs set, which override the host and flow
> > hashes.
> 
> Very cool! Awesome to see the customisation options being used for
> something neat like this! :)
> 
> > To show it in action, the cakeiso.sh script sets up a netns
> > environment
> > and runs competition between two "subscribers" and three flows, two
> > TCP
> > flows and one unresponsive UDP flow. Several configurations are run
> > to
> > show what is and isn't possible.
> > 
> > If anyone knows of a simpler way than eBPF to get both the major and
> > minor class ID set from ipsets, I'd like to hear it, but the included
> > classifiers are at least very simple one-liners...
> 
> Well, you could go the other way? Instead of ipset, just do the
> classification in eBPF and use a BPF map to store the IP addresses.
> There's even an LPM map type, so you can use arbitrary prefix lengths
> for each class (or not, and just use a hashmap)...

True that, I started something like that at some point:

https://github.com/heistp/tc-users/

but I think I got a little overzealous with it. I'm not sure if/when
I'll get back to that, but the ipset solution seems to be "good enough"
for what I (and my ISP) needs. I'm glad you slipped the tc filter
overrides in before Cake went out the door. :)

This doesn't do away with the possible need for a full-blown ISP qdisc
one day, with configurable subscriber tiers, handling of higher loads,
etc, but at least it's something for the little guys.

Pete

> -Toke