From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-x242.google.com (mail-lj1-x242.google.com [IPv6:2a00:1450:4864:20::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id C4F2D3B29E for ; Tue, 23 Jun 2020 09:06:06 -0400 (EDT) Received: by mail-lj1-x242.google.com with SMTP id 9so23365031ljc.8 for ; Tue, 23 Jun 2020 06:06:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=m8yT2X4pV1gn58niCbx6qY+8n8nsCdXlX7bW1nFLajY=; b=Q9fXIUAXFOSXWVPTWnDaC22uLpgD/eDYNRzJ3C8a46otwjwYqD97ce6GHoXGieY7VA DDj+WFyPvjf7S1Dq16Q2FoxhluwgrqQmCxVl6iAqosnTXFu4YAaMSI8xK6315+AtmVdz msskEqb/e827KcPFES3O8D+Eq7m7uIjn/Lt83peO4YWHiSoDYFdAxgrartpsLZarzyW9 /mLb4fJM8OoZSYKZVdrMd4w8bL7m1fApE34Z75bAN73SZzCees2U7Pi/gd6W+j+x1vkf 1zHzFknpTau2zW/q6PvKT9SLbRpUx6xC3LeJGnrOBoQhQrmlLVN2GIbrIg0j5hy/trK+ hTZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=m8yT2X4pV1gn58niCbx6qY+8n8nsCdXlX7bW1nFLajY=; b=TDeH1lkH29pSvznc22fAbFVrOitBJHdS9ugDdgErvG5KUhyHQxbkHw3eFl5MObdq7o ezqAjrJ4vrIHVzj+ZLvWVkzkAnqNNUAxh76VXPabSMvIOtRPe5ec6PovU/IrEYPYl6Pq KzSLZeG5/eqjHRpKcjRnn5DB1DpoySqRIpSYBSiVPD8Y/Pb2jidHTi1VDL7u0lckIs1i o+cGtTfwqBtF5v9fxIpWUgixhyE9GY0580fNvlNg4sPg40BJI01S4sT8b3RMj6nhXWZI gIOwInBANVc8zzyJuDNr9KK8+60EF5VjUMQuXN+pPs6QUYKOKnrRsMZN4R/noBJElyIj ZVrA== X-Gm-Message-State: AOAM531zBTLM7ltYKC20VGnh3FxJXj9qljKWr9xux/C50MvvXXQ/BPKt Nkfg4DoIDQKI7g4eRDA3AWC1WDpcRdYx5qPGAdA= X-Google-Smtp-Source: ABdhPJyUhMg4zBnFNP1sGQ+Augdt99xL/U1KZcr5nIoc/74SxjP3NB/tEizZRt9PnFVYy4T62iO5dslM8jXzuKbKJ8Y= X-Received: by 2002:a2e:864b:: with SMTP id i11mr1318031ljj.64.1592917565549; Tue, 23 Jun 2020 06:06:05 -0700 (PDT) MIME-Version: 1.0 References: <877dvzt84w.fsf@toke.dk> In-Reply-To: <877dvzt84w.fsf@toke.dk> From: Jose Blanquicet Date: Tue, 23 Jun 2020 15:05:28 +0200 Message-ID: To: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= Cc: cake@lists.bufferbloat.net, marco maniezzo Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] [CAKE] Rate is much lower than expected - CPU load is higher than expected X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2020 13:06:07 -0000 Hi Toke, Thanks for your reply. On Mon, Jun 22, 2020 at 5:47 PM Toke H=C3=B8iland-J=C3=B8rgensen wrote: > > We have an embedded system with limited CPU resources that acts as a > > gateway to provide Internet access from LTE to a private USB-NCM > > network (And also to a Wi-Fi private network but we will work on it > > later). Our problem is that the bandwidth on LTE and USB link is > > higher than what the system is able to handle thus it reaches 100% of > > CPU load when we perform a simple speed test from a device on the > > private network. > > What speeds were you getting without shaping? Between 35 and 40Mbps. > > Therefore, we want to limit the bandwidth to avoid system getting > > saturated in such use-case. To do so, we thought to use the CAKE on > > the USB interface. For instance, we tried: > > > > tc qdisc replace root dev eth0 cake bandwidth 20mbit ethernet > > internet flowblind nonat besteffort nowash > > > > It worked correctly and the maximum rate was limited but there are two > > things that are worrying us: > > > > 1) The maximum rate reached after applying CAKE was in between 12Mbps > > and 15Mbps which is quite lower than the 20Mbps we are configuring, we > > were expecting around 18-19. Why? Is there something in the parameters > > we are doing wrong? Please take into account that our goal is to limit > > the rate but adding as little CPU load as possible. > > Hmm, are you actually running out of CPU? I.e., is the CPU pegged at > 100% when you hit this limit? What kind of platform are you running on? > And what kernel and CAKE versions are you using? I checked the CPU with top and there is still free CPU to be used. We also tried with lower values like 10 and it is again far away from the configured limit. We have just a percentage of an ARM Cortex A7 (1.2GHz) because the rest is reserved for modem. We are now trying to optimize all the applications in the system but LTE<->WIFI/USB data transfer is indeed the use-case that puts our system in crisis. The kernel version is 3.18 and for we are using the latest commit on master branch (9d79e2b) for CAKE. In case, we could change CAKE but not the kernel version, at most some specific patches. > > 2) The CPU load added by CAKE was not negligible for our system. In > > fact, we compared the CPU load when limitation was done by CAKE and by > > the device on the private network, e.g. curl tool with parameter > > "--limit-rate". As a result, we found that the CPU load when using > > CAKE was 30%. Is there any way to make it lighter with a different > > configuration? > > No, you've already turned off most of the features that might incur > overhead, so I don't think there's anything more you can do > configuration-wise to improve CPU load. Shaping does tend to use up a > lot of CPU, so it's not too surprising you run into issues here. Could you please help us to identify which one is still active? We thought we had already turned off all the features not needed to apply a limitation with a single queue (Besteffor mode). > We did recently get a pull request whose author states that he was > seeing a 1/3 improvement in performance from it. See: > https://github.com/dtaht/sch_cake/pull/136 > > You could try this; if your ingress network device driver has the same > issue with skbs being allocated in smaller bits, you may see a similar > increase with this patch. For a quick test you could also just try > commenting out the call to cake_handle_diffserv() entirely since you're > running in besteffort mode anyway :) Interesting. We will try this, we commented out the call to cake_handle_diffserv() as you said and just to be sure, we also applied the 2nd commit of the PR. I will be back soon with news. Thanks, Jose Blanquicet