[Cake] [CAKE] Rate is much lower than expected - CPU load is higher than expected

[Toke Høiland-Jørgensen]

Jose Blanquicet <blanquicet at gmail.com> writes:

> Hi everyone,
>
> We have an embedded system with limited CPU resources that acts as a
> gateway to provide Internet access from LTE to a private USB-NCM
> network (And also to a Wi-Fi private network but we will work on it
> later). Our problem is that the bandwidth on LTE and USB link is
> higher than what the system is able to handle thus it reaches 100% of
> CPU load when we perform a simple speed test from a device on the
> private network.

What speeds were you getting without shaping?

> Therefore, we want to limit the bandwidth to avoid system getting
> saturated in such use-case. To do so, we thought to use the CAKE on
> the USB interface. For instance, we tried:
>
>     tc qdisc replace root dev eth0 cake bandwidth 20mbit ethernet
> internet flowblind nonat besteffort nowash
>
> It worked correctly and the maximum rate was limited but there are two
> things that are worrying us:
>
> 1) The maximum rate reached after applying CAKE was in between 12Mbps
> and 15Mbps which is quite lower than the 20Mbps we are configuring, we
> were expecting around 18-19. Why? Is there something in the parameters
> we are doing wrong? Please take into account that our goal is to limit
> the rate but adding as little CPU load as possible.

Hmm, are you actually running out of CPU? I.e., is the CPU pegged at
100% when you hit this limit? What kind of platform are you running on?
And what kernel and CAKE versions are you using?

> 2) The CPU load added by CAKE was not negligible for our system. In
> fact, we compared the CPU load when limitation was done by CAKE and by
> the device on the private network, e.g. curl tool with parameter
> "--limit-rate". As a result, we found that the CPU load when using
> CAKE was 30%. Is there any way to make it lighter with a different
> configuration?

No, you've already turned off most of the features that might incur
overhead, so I don't think there's anything more you can do
configuration-wise to improve CPU load. Shaping does tend to use up a
lot of CPU, so it's not too surprising you run into issues here.

We did recently get a pull request whose author states that he was
seeing a 1/3 improvement in performance from it. See:
https://github.com/dtaht/sch_cake/pull/136

You could try this; if your ingress network device driver has the same
issue with skbs being allocated in smaller bits, you may see a similar
increase with this patch. For a quick test you could also just try
commenting out the call to cake_handle_diffserv() entirely since you're
running in besteffort mode anyway :)

-Toke