[Cake] flow isolation for ISPs

[Pete Heist]

> On Apr 7, 2017, at 1:13 PM, Sebastian Moeller <moeller0 at gmx.de> wrote:
> 
> Hi Peter,
> 
>> On Apr 7, 2017, at 11:37, Pete Heist <peteheist at gmail.com <mailto:peteheist at gmail.com>> wrote:
>> 
>> Ok, I’m still getting familiar with how triple-isolate is implemented. For example, I was surprised in my test setup that no fairness is enforced when four client IPs connect to a single server IP, but I understand from this discussion (https://github.com/dtaht/sch_cake/issues/46) that that is actually what is expected. We would probably use dual-srchost and dual-dsthost in the backhaul, which seems to work very well, and in the backhaul we have the information to specify that in both directions. (Also, there is no NAT to deal with at this level.)

I didn’t write that very well before, so just to clarify, there’s nothing more we need to specify for dual-srchost and dual-dsthost to work, just that we control both directions of the flow so can use dual-srchost on upstream egress and dual-dsthost on downstream egress.

>> Just to see if I understand the marking proposal, here's the behavior I would expect: if there are two TCP flows (on egress) with mark 1 and one with mark 2, that together saturate the link, the measured rate of the two flows with mark 1 will add up to the rate of the single flow with mark 2. Is that right? And would you still add a keyword to specify that the mark should be used at all?
>> 
>> I’m not sure where the 1024 limit comes from, but it would probably be fine in our case as of now, with 800 members. Even in the future, I don’t think occasional collisions would be a big problem, and I think there are things we could do to minimize them.
> 
> Seeing your 800 members I remember a discussion over at the lede forum, https://forum.lede-project.org/t/lede-as-a-dedicated-qos-bufferbloat-appliance/1861/27?u=moeller0 <https://forum.lede-project.org/t/lede-as-a-dedicated-qos-bufferbloat-appliance/1861/27?u=moeller0> where orangetek, used cake on a wired backhaul for approximately 600 end users. He reported for number of concurrent flows: “As far as i can tell, around 25k-30k during busy hours.” 
> He also increased the number of CAKE_BINs in the code to 64k. So depending on your user’s 1024 might be a bit tight, given that you still ideally want flows to not share bins if possible (sure cake is great in avoiding sharing unless impossible, but with enough flows you might want/need to simply hard code your cake instances for higher limits).

I see, so the 1024 limit probably comes from the CAKE_QUEUES define. :)

So far, we’re not looking to use Cake on the main Internet router. I’m just not sure yet if it would be appropriate for a gigabit uplink (that also doesn’t reach saturation, as far as can be discerned from mrtg plots). We’re taking things step by step, and looking at some of the backhaul routers first, where there can sometimes be congestion.

As for an average peak number of concurrent flows on the Internet router I could find out from the admin. Total throughput for the gigabit Internet uplink is on a public page: https://www.lbcfree.net/mrtg/10.101.254.194_24.html, but not flow counts.

Thanks for the tip...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20170407/770c8c7f/attachment.html>