[Cake] Large number of Flows

[Cake] Large number of Flows

[Mike]

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]

Will cake support a large number of flows like over a thousand per linux box without any modifications.  I did see that there was a qdisc issue for fq_codel on a large scale.  We would be using linux kernel 4.19 which has cake already in it.  Any help or issues that might be encountered in scaling would be appreciated.



Thanks
Mike Thompson


[-- Attachment #2: Type: text/html, Size: 952 bytes --]

Re: [Cake] Large number of Flows

[Dave Taht]

fq_codel, Cake etc, supports an infinite number of flows.

It has a limited number of "queues" that can get mapped to flows, but
it's usually ok if a collision happens.

The 1024 queue tradeoff is based on the observation that usually a max
of a few hundred active flows exist, and furthermore,
excessive fair queueing tends to defeat the purpose of the aqm of
keeping overall flow lengths short. Collisions of two fat flows are
rare.

You can recompile cake with more queues if you like (fq_codel has a
soft limit of 64k queues). We don't have much data on 10GigE+
behaviors. It was kind
of my assumption more queues would help in the 40GigE world, but
that's usually got hardware mq (64 or more), and what I'm seeing there
is 64 default fq_codel instances, 64k
queues essentially, and I think that's WAY too much....


On Mon, Feb 17, 2020 at 6:07 AM Mike <mike@surfglobal.net> wrote:
>
> Will cake support a large number of flows like over a thousand per linux box without any modifications.  I did see that there was a qdisc issue for fq_codel on a large scale.  We would be using linux kernel 4.19 which has cake already in it.  Any help or issues that might be encountered in scaling would be appreciated.
>
>
>
> Thanks
> Mike Thompson
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729

Re: [Cake] Large number of Flows

[Dave Taht]

On Mon, Feb 17, 2020 at 7:52 AM Mike <mike@surfglobal.net> wrote:
>
> So 1024 is the max queues that it supports as is, so if I had 1500 users with their own traffic shaping setup per user it would be unsupported without recompiling the kernel?  Is there a command to see how many is used and available?

I think we are starting from two very different reference points for
what we can accomplish. Cake's primary use case is for the CPE, or
customer edge router shaping egress and ingress to suit the customers
needs, and usually below the ISPs (badly) shaped rate.

Each cake instance only does one bandwidth, with that 1024 queue default.

If you are looking to have a bandwidth per subscriber managed
centrally, It is certainly feasible and desirable to use cake from the
ISP premise. How I do it is I have one virtual interface per
"subscriber", managed by a route table (e.g. ip route 192.168.1.99 via
dev cust1; ip -6 route aa::bb:cc::/48 via dev cust1) to which I add
the cake instance, bandwidth parameter, etc

Others like preseem do things like a transparent bridge in between the
switch and the edge (dsl bras, etc

You can setup an htb or drr + one instance of cake per subscriber if you like.

> Also I saw on the fq_codel page they talk about issues with cores and netem but Cake doesn’t seem to use netem to delay packets etc based on the man page, so is the core issue still a factor?

Depends on your requirements. htb + anything or cake tend to lock the
processing to a single core. It doesn't in the case I describe above,
but I've not tried to push it past 10Gbit.

>
>
> On February 17, 2020 at 9:34:59 AM, Dave Taht (dave.taht@gmail.com) wrote:
>
> fq_codel, Cake etc, supports an infinite number of flows.
>
> It has a limited number of "queues" that can get mapped to flows, but
> it's usually ok if a collision happens.
>
> The 1024 queue tradeoff is based on the observation that usually a max
> of a few hundred active flows exist, and furthermore,
> excessive fair queueing tends to defeat the purpose of the aqm of
> keeping overall flow lengths short. Collisions of two fat flows are
> rare.
>
> You can recompile cake with more queues if you like (fq_codel has a
> soft limit of 64k queues). We don't have much data on 10GigE+
> behaviors. It was kind
> of my assumption more queues would help in the 40GigE world, but
> that's usually got hardware mq (64 or more), and what I'm seeing there
> is 64 default fq_codel instances, 64k
> queues essentially, and I think that's WAY too much....
>
>
> On Mon, Feb 17, 2020 at 6:07 AM Mike <mike@surfglobal.net> wrote:
> >
> > Will cake support a large number of flows like over a thousand per linux box without any modifications. I did see that there was a qdisc issue for fq_codel on a large scale. We would be using linux kernel 4.19 which has cake already in it. Any help or issues that might be encountered in scaling would be appreciated.
> >
> >
> >
> > Thanks
> > Mike Thompson
> >
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
>
>
>
> --
> Make Music, Not War
>
> Dave Täht
> CTO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-831-435-0729



-- 
Make Music, Not War

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729