From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 9540F3B2A4 for ; Mon, 22 Jun 2020 11:47:41 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592840861; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=D05HhaihY03qByB9u3rBYCf70uDkum4ra/9qNADxG14=; b=alB8z0Xgq/s7hGE7HQYBlOiI/mTlBZcHexpWH/9vZ7ATVN8wxHdV4Ko5n6nx5V2MLObOPp pMbXVLHHjyYdmfE4o5quDXAcO68KhleKYgM1GYo9nb6AE3aRq0QFIDE7OMA6BXNWsqbjuH wt6h7nbEqxKRkPyBBuX/sEnGIbDPx2E= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-513-qJqem_7rO0esZyjKog-iwQ-1; Mon, 22 Jun 2020 11:47:29 -0400 X-MC-Unique: qJqem_7rO0esZyjKog-iwQ-1 Received: by mail-wm1-f71.google.com with SMTP id h25so167396wmb.0 for ; Mon, 22 Jun 2020 08:47:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=D05HhaihY03qByB9u3rBYCf70uDkum4ra/9qNADxG14=; b=LgwTM1CTA59s5qMHPztp8s4Y5ruhdIno7EoK0W4ASHJL4wvGG9Dbu9KHFTr7XVEMr/ 09M+QPZDysurB4+xwE6/afCJSWA+5lJ7blhcgeTGbuufX3tOT+JfbwwH+hDLc+TdBlk7 aungA7ysa+XVCjo+Lp/fbFUTQheWmKh84HbT8AZQTwySu4OdgnWXMiR/Fa48xwYnna+8 OooiNwY6ULGDFaPgPrewoES3xONjHgIQqJMEURasfqnNMrDiwCal8kBP+wB8iu5A2IF3 vYthoQTbGpwZKcqqYG4N0OeYO46DVAwoqW/2yCJoLiB1OB0iwsMojhJOSKUrIlNUEu3C 5nWQ== X-Gm-Message-State: AOAM5318NfsQT7ogDFcpE18erB7JdRQjXIjA3mQABmJxXIFAyHyJTSQx 4tz3nMe5DyR322xBbw1wosOQiTV4qF3gwW4uQswcnPdsfJuhQyxKeNuNU2fGp6514zfa/5I+e2e TDPWcCPMSliLnwB6HeZiv0Q== X-Received: by 2002:adf:f445:: with SMTP id f5mr11250483wrp.339.1592840848802; Mon, 22 Jun 2020 08:47:28 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxHazsT0tmaYxZeSAbPfRBVAXXKqSbqsgqBadWehtsY47X7lxW1/qjseVBi1RIyKWrHWBlU1w== X-Received: by 2002:adf:f445:: with SMTP id f5mr11250470wrp.339.1592840848588; Mon, 22 Jun 2020 08:47:28 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id o10sm18404057wrj.37.2020.06.22.08.47.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jun 2020 08:47:27 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 21029181502; Mon, 22 Jun 2020 17:47:27 +0200 (CEST) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= To: Jose Blanquicet , cake@lists.bufferbloat.net Cc: marco maniezzo In-Reply-To: References: X-Clacks-Overhead: GNU Terry Pratchett Date: Mon, 22 Jun 2020 17:47:27 +0200 Message-ID: <877dvzt84w.fsf@toke.dk> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Subject: Re: [Cake] [CAKE] Rate is much lower than expected - CPU load is higher than expected X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2020 15:47:41 -0000 Jose Blanquicet writes: > Hi everyone, > > We have an embedded system with limited CPU resources that acts as a > gateway to provide Internet access from LTE to a private USB-NCM > network (And also to a Wi-Fi private network but we will work on it > later). Our problem is that the bandwidth on LTE and USB link is > higher than what the system is able to handle thus it reaches 100% of > CPU load when we perform a simple speed test from a device on the > private network. What speeds were you getting without shaping? > Therefore, we want to limit the bandwidth to avoid system getting > saturated in such use-case. To do so, we thought to use the CAKE on > the USB interface. For instance, we tried: > > tc qdisc replace root dev eth0 cake bandwidth 20mbit ethernet > internet flowblind nonat besteffort nowash > > It worked correctly and the maximum rate was limited but there are two > things that are worrying us: > > 1) The maximum rate reached after applying CAKE was in between 12Mbps > and 15Mbps which is quite lower than the 20Mbps we are configuring, we > were expecting around 18-19. Why? Is there something in the parameters > we are doing wrong? Please take into account that our goal is to limit > the rate but adding as little CPU load as possible. Hmm, are you actually running out of CPU? I.e., is the CPU pegged at 100% when you hit this limit? What kind of platform are you running on? And what kernel and CAKE versions are you using? > 2) The CPU load added by CAKE was not negligible for our system. In > fact, we compared the CPU load when limitation was done by CAKE and by > the device on the private network, e.g. curl tool with parameter > "--limit-rate". As a result, we found that the CPU load when using > CAKE was 30%. Is there any way to make it lighter with a different > configuration? No, you've already turned off most of the features that might incur overhead, so I don't think there's anything more you can do configuration-wise to improve CPU load. Shaping does tend to use up a lot of CPU, so it's not too surprising you run into issues here. We did recently get a pull request whose author states that he was seeing a 1/3 improvement in performance from it. See: https://github.com/dtaht/sch_cake/pull/136 You could try this; if your ingress network device driver has the same issue with skbs being allocated in smaller bits, you may see a similar increase with this patch. For a quick test you could also just try commenting out the call to cake_handle_diffserv() entirely since you're running in besteffort mode anyway :) -Toke