From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x244.google.com (mail-qt0-x244.google.com [IPv6:2607:f8b0:400d:c0d::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 29C473BA8E for ; Tue, 11 Sep 2018 14:43:05 -0400 (EDT) Received: by mail-qt0-x244.google.com with SMTP id z8-v6so29337983qto.9 for ; Tue, 11 Sep 2018 11:43:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=WrdPK5yHebDITTb0wVhHSJOTafT98EIRfGyEQUEoNVE=; b=BAo9CPmZYXPdYsGQLUmQBnIfPbqkB9r5lDooU74ymVLh4lN3YjIqwhIMx14hcBd8h2 mTTLjEXy3nnfLO0HK+oTLdTC9P85fggRnVcV0T6/FVTvXYDozWebfKS71K/SMGMGbuLP YCYFHz4oBSvhh74LnYjmsnQyvxFKXrHHfy09kABxNKe2L5xpLsz7VZcYDjlum+L5nwcW LjclY0wYjn9pLnI1DHQcieEVMAxiZWQwNPOXu2FA+dfVoo8CMDz0xxz/1zp2Ki8UMOqT utiUn4UYEhvLBn4mDtYAC9JN+77VGvApgAqmM83ugf4P6WaQix3JL2yVDF/JxDx9YmKn 5/jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=WrdPK5yHebDITTb0wVhHSJOTafT98EIRfGyEQUEoNVE=; b=DvVLQFBkxquHzGTi6D7W0euIL4KyemEwM4srwxZ0vjahiodzhkp1PWuqa2rc4OWhkh tRfRtRfY3qovPdpv3crAz0KKe1El7I+c1fLZUVu7d/u+nh+KLmAErdqz/vpygx+9CLrx Bhf+Zz1eyF3XKnfICU8LZatpm+EIcBwG6YtHO85COMsvqQVGajLwRHTQUcqmKmbF8UQq yKG2EFMfGPC8t60qoKiPYhRqecoXThmELurGegZb6BrQdRO+DKmbkajtyqyP7ZM4712C s/2lZc0m+UICVrBN+BL03Sa6cwz2BF5Nq+0Ny8pXhPWYbo/Oy47DVli43kJB774bacQu oVDQ== X-Gm-Message-State: APzg51CFFeUVnjotyQz+AmOh6mvwQnj2rJD3nnw1VCk4tMvcWTf3lvKI QmGRV6kjv6mVn9nEaiC71P0rvgxZupLiaejM1H0= X-Google-Smtp-Source: ANB0Vdbfc7cSJn4riQExLs1xb7sIev7wtWlvsCc27dJSjUfAg1vzMuZnnsy0A76BGjc0gtfYidLXRZ9khIenrcyntOI= X-Received: by 2002:ac8:2ea1:: with SMTP id h30-v6mr20668627qta.135.1536691384591; Tue, 11 Sep 2018 11:43:04 -0700 (PDT) MIME-Version: 1.0 References: <87zhwxzh8o.fsf@toke.dk> <139B295B-7371-43DE-B472-DE629C9B8432@heistp.net> <87efe65wol.fsf@toke.dk> <6C556301-015B-4903-AE5A-F22D3517FFCC@heistp.net> <79F47FB1-6B00-4753-930B-950FB8CD3850@gmx.de> <49EBEB93-D8A1-4704-BF5C-9490FF77AED4@heistp.net> In-Reply-To: From: Dave Taht Date: Tue, 11 Sep 2018 11:42:52 -0700 Message-ID: To: Pete Heist Cc: Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] Cake vs fq_codel and c/burst on an ER-X bridge X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2018 18:43:05 -0000 Next time I have sufficient spare brain cells I would like to try shaping across cores. I don't know how to share a bit of data between qdiscs as yet, so my plan to prove it out, is to use a static variable for time_next_packet (has to be 32 bits also), and atomically update it between the ksoftirq threads on each hardware queue. https://github.com/dtaht/fq_codel_fast/ Don't know if we can round effectively to ns at 32 bits That would do less work per core with larger intervals between reschedules with sufficient multiplexing. In addition to that problem, shaping inbound across cores is currently serialized to one core (I think) by the sch_ingress qdisc. There was some new work *way* earlier in the path https://lwn.net/Articles/763056/ recently - and to try and find some why to bypass the ingress and mirred portions of the path, and just pull directly from that would help. On Tue, Sep 11, 2018 at 11:29 AM Dave Taht wrote: > > that was me shaping at 900. > On Tue, Sep 11, 2018 at 11:27 AM Pete Heist wrote: > > > > > > > On Sep 11, 2018, at 10:20 AM, Dave Taht wrote: > > > > > > What I "fixed" was on the apu2 with the burst/cburst change, I went > > > from completely bottlenecked on one softirq to having 3 eat cpu, and > > > from 400mbps to 900mbps. Now, that's a quad core and the e1000 (?) > > > driver. The edgerouter X is a dual core, and you did see a small > > > improvement in throughput, but I'd hoped for more. > > > > > > That=E2=80=99s rather dramatic. When you do your tests, do you set the = shaper to a high number and see how high actual throughput goes, or do you = do it some other way? Just in case, I tried setting the ER-X to 350Mbit. ht= b+fq_codel topped out at 149Mbit, and with c/burst 96000 186 Mbit, so in bo= th types of tests it was about a 25% increase, not small, but unfortunately= not 2x! > > > > Pete > > > > > -- > > Dave T=C3=A4ht > CEO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-669-226-2619 --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619