[Cake] flow isolation for ISPs

[Jonathan Morton]

> On 7 Apr, 2017, at 11:13, Pete Heist <peteheist at gmail.com> wrote:
> 
>> On Apr 6, 2017, at 11:26 AM, Pete Heist <peteheist at gmail.com> wrote:
>> 
>>> On Apr 6, 2017, at 11:11 AM, Jonathan Morton <chromatix99 at gmail.com> wrote:
>>> 
>>> On 6 Apr, 2017, at 11:27, Pete Heist <peteheist at gmail.com> wrote:
>>>> 
>>>> There is a table of member ID to a list of MAC addresses for the member, so if there could somehow be fairness based on that table and by MAC address, that could solve it, but I don’t see how it could be implemented.
>>> 
>>> One option would be to use HTB with FLOWER filters to sort out the subscribers into classes, and use Cake or fq_codel as a child qdisc per class.  Remember that Cake can be used in “unlimited” mode to rely on an external shaping source.
> 
> One more thought, would it be possible for Cake to optionally include the packet’s mark in the hash?
> 
> I know it’s additional functionality, and another keyword, but it could get you out of the business of the myriad of ways people might want to do flow isolation, and you’d still have a catch-all answer for such cases.
> 
> There could be a keyword ‘hash-mark’, let’s say, which first includes the mark in the hash, then does on to deal with any other flow isolation keywords as usual. So for example if I have ‘hash-mark’ and ‘dual-srchost’, the hash is first on the mark, then by source host, then by flow. I could set the mark to be the member number with iptables.

That isn’t really how hashing works; there is no “first, second, third” structure, just an accumulation of entropy which is all mashed together.  In order to run the triple-isolation algorithm at all, I have to take separate hashes of the relevant host addresses, alongside the general 5-tuple hash.

However, it would be possible to use the “mark” directly as one of the host identifiers which triple-isolate operates on to provide that layer of fairness.  That’s probably what you meant.

Since this wouldn’t unduly complicate the configuration interface, it could be a feasible way of adding this functionality for modest installations, up to a strict maximum of 1024 subscribers (and a recommended maximum somewhat below that).

> It looks like the mark could be obtained from the ‘mark' field of the sk_buff struct, but I don’t know the validity of the field in various cases. For example, I don’t think I can set the mark on ingress before it reaches a qdisc on an IFB device.

It has been suggested, in the context of using the “mark” for Diffserv purposes, that Linux’ conntrack facility could preserve the mark between directions of flow.  Cake can already query conntrack for NAT awareness.

 - Jonathan Morton