[Cake] flow isolation for ISPs

[Toke Høiland-Jørgensen]

Pete Heist <peteheist at gmail.com> writes:

>  On Apr 6, 2017, at 11:33 AM, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
>
>  Once upon a time I implemented something like this; it was basically a
>  PHP script that would generate an HTB bucket (with sfq as leaf qdisc;
>  this was pre-fq_codel) per subscriber ID and use tc filter to map the
>  list of IPs registered to that customer into the right bucket. The HTB
>  shaper was used to enforce the bandwidth each customer was paying for.
>
>  Did it work? Yup, mostly. Was it ugly? Oh boy, yes!
>
> Oh my, ok, so it is possible. It can take a while to apply many qdiscs
> and filters on lower end devices, so I picture some delay while
> modifying the list or restarting the routers, but I’m just exploring
> options now, so it is one.

What I did was 10 years ago on what was fairly high-end x86 hardware at
the time; scaling to 10s (or maybe low 100s) of users. So depending on
the scale, it's probably doable on somewhat cheaper hardware now. But
yeah, if you need to modify things often, you may have problems.

>  The fq_codel qdisc does have support for arbitrary tc filters to replace
>  the default hashing, BTW. If you don't need the cake shaper, that might
>  be a solution?
>
> I see, I found mention of it in Chapter 6 of a draft RFC that it looks
> like you wrote, actually
> (https://tools.ietf.org/html/draft-ietf-aqm-fq-codel-06#section-6). :)
> To try it out, am I heading the right direction by looking at tc
> filter’s skbedit action, or is that just for MQ devices?
> (http://man7.org/linux/man-pages/man8/tc-skbedit.8.html)
>
> I also saw this mention of “We are not aware of any deployments
> utilising the custom classification feature"
> https://tools.ietf.org/html/draft-ietf-aqm-fq-codel-02#section-5.1.1,
> so not sure how often this has been tried. :)

Yeah, haven't actually heard of anyone using the feature in production.
It's basically this section from the 'classful qdiscs' section of 'man
tc':

       When a packet enters a classful qdisc it can be classified to one of the classes within. Three criteria are available, although not all qdiscs will use all three:

       tc filters
              If tc filters are attached to a class, they are consulted first for relevant instructions. Filters can match on all fields of a packet header, as well as on the firewall mark applied by ipchains or iptables.

So you can basically use the full capabilities of tc-filter in place of
the built-in hashing of fq_codel. The tc-u32 man page has some examples,
which is probably a good starting point.

If you do try this out and feel like writing up a small
example/tutorial, I'm happy to add a link (or the whole thing) somewhere
on bufferbloat.net :)

-Toke