Re: [Cake] customizing Cake's isolation with ipsets, tc-flow and eBPF

["Toke Høiland-Jørgensen" <toke@toke.dk>]

Pete Heist <pete@heistp.net> writes:

> On Sun, 2021-06-06 at 21:59 +0200, Toke Høiland-Jørgensen wrote:
>> Pete Heist <pete@heistp.net> writes:
>> 
>> > I've always wanted a way to customize Cake's host and flow isolation
>> > in
>> > a way that would be usable e.g. for small ISPs, and this is what I
>> > came
>> > up with:
>> > 
>> > https://github.com/heistp/cake-custom-isolation
>> > 
>> > ipsets are used to set the skb priority or mark, then tc-flow or a
>> > simple eBPF classifier is used in a child filter of cake to get the
>> > major and minor class IDs set, which override the host and flow
>> > hashes.
>> 
>> Very cool! Awesome to see the customisation options being used for
>> something neat like this! :)
>> 
>> > To show it in action, the cakeiso.sh script sets up a netns
>> > environment
>> > and runs competition between two "subscribers" and three flows, two
>> > TCP
>> > flows and one unresponsive UDP flow. Several configurations are run
>> > to
>> > show what is and isn't possible.
>> > 
>> > If anyone knows of a simpler way than eBPF to get both the major and
>> > minor class ID set from ipsets, I'd like to hear it, but the included
>> > classifiers are at least very simple one-liners...
>> 
>> Well, you could go the other way? Instead of ipset, just do the
>> classification in eBPF and use a BPF map to store the IP addresses.
>> There's even an LPM map type, so you can use arbitrary prefix lengths
>> for each class (or not, and just use a hashmap)...
>
> True that, I started something like that at some point:
>
> https://github.com/heistp/tc-users/
>
> but I think I got a little overzealous with it. I'm not sure if/when
> I'll get back to that, but the ipset solution seems to be "good enough"
> for what I (and my ISP) needs. I'm glad you slipped the tc filter
> overrides in before Cake went out the door. :)

Yes! I would love to take credit for the idea, but this came from
the netdev review. I agree though, great that it ended up in there!

> This doesn't do away with the possible need for a full-blown ISP qdisc
> one day, with configurable subscriber tiers, handling of higher loads,
> etc, but at least it's something for the little guys.

Yeah; I have something in the pipeline that will hopefully end up being
useable for that, but may take a little while to get there :)

-Toke