[Cake] [CAKE] Rate is much lower than expected - CPU load is higher than expected

Jose Blanquicet blanquicet at gmail.com
Tue Jun 23 09:05:28 EDT 2020


Hi Toke,

Thanks for your reply.

On Mon, Jun 22, 2020 at 5:47 PM Toke Høiland-Jørgensen <toke at redhat.com> wrote:
> > We have an embedded system with limited CPU resources that acts as a
> > gateway to provide Internet access from LTE to a private USB-NCM
> > network (And also to a Wi-Fi private network but we will work on it
> > later). Our problem is that the bandwidth on LTE and USB link is
> > higher than what the system is able to handle thus it reaches 100% of
> > CPU load when we perform a simple speed test from a device on the
> > private network.
>
> What speeds were you getting without shaping?

Between 35 and 40Mbps.

> > Therefore, we want to limit the bandwidth to avoid system getting
> > saturated in such use-case. To do so, we thought to use the CAKE on
> > the USB interface. For instance, we tried:
> >
> >     tc qdisc replace root dev eth0 cake bandwidth 20mbit ethernet
> > internet flowblind nonat besteffort nowash
> >
> > It worked correctly and the maximum rate was limited but there are two
> > things that are worrying us:
> >
> > 1) The maximum rate reached after applying CAKE was in between 12Mbps
> > and 15Mbps which is quite lower than the 20Mbps we are configuring, we
> > were expecting around 18-19. Why? Is there something in the parameters
> > we are doing wrong? Please take into account that our goal is to limit
> > the rate but adding as little CPU load as possible.
>
> Hmm, are you actually running out of CPU? I.e., is the CPU pegged at
> 100% when you hit this limit? What kind of platform are you running on?
> And what kernel and CAKE versions are you using?

I checked the CPU with top and there is still free CPU to be used. We
also tried with lower values like 10 and it is again far away from the
configured limit.

We have just a percentage of an ARM Cortex A7 (1.2GHz) because the
rest is reserved for modem. We are now trying to optimize all the
applications in the system but LTE<->WIFI/USB data transfer is indeed
the
use-case that puts our system in crisis.

The kernel version is 3.18 and for we are using the latest commit on
master branch (9d79e2b) for CAKE. In case, we could change CAKE but
not the kernel version, at most some specific patches.

> > 2) The CPU load added by CAKE was not negligible for our system. In
> > fact, we compared the CPU load when limitation was done by CAKE and by
> > the device on the private network, e.g. curl tool with parameter
> > "--limit-rate". As a result, we found that the CPU load when using
> > CAKE was 30%. Is there any way to make it lighter with a different
> > configuration?
>
> No, you've already turned off most of the features that might incur
> overhead, so I don't think there's anything more you can do
> configuration-wise to improve CPU load. Shaping does tend to use up a
> lot of CPU, so it's not too surprising you run into issues here.

Could you please help us to identify which one is still active? We
thought we had already turned off all the features not needed to apply
a limitation with a single queue (Besteffor mode).

> We did recently get a pull request whose author states that he was
> seeing a 1/3 improvement in performance from it. See:
> https://github.com/dtaht/sch_cake/pull/136
>
> You could try this; if your ingress network device driver has the same
> issue with skbs being allocated in smaller bits, you may see a similar
> increase with this patch. For a quick test you could also just try
> commenting out the call to cake_handle_diffserv() entirely since you're
> running in besteffort mode anyway :)

Interesting. We will try this, we commented out the call to
cake_handle_diffserv() as you said and just to be sure, we also
applied the 2nd commit of the PR. I will be back soon with news.

Thanks,
Jose Blanquicet


More information about the Cake mailing list