From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id C956A3BA8E for ; Tue, 11 Sep 2018 14:45:05 -0400 (EDT) Received: by mail-wm0-x231.google.com with SMTP id j192-v6so2073443wmj.1 for ; Tue, 11 Sep 2018 11:45:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=X8yFHAeOstTMxaiMqZkWddTJAl93LsADq7KrCzydUlE=; b=XBPp2DqyHP9WdpbbIHmry0NIv4Frmv3gzLr9YMigzhTGSINp+eXyMiUuoWeYODKUUZ HVL+iAYNG7ioLbO40zMHKVToMENf1P3iuFBWjkxaBK6JShUYct5rjm8TQT16CSGvXA5V zFLiNwb+nsuGCqcqC+N0LhmOXLLNsJycmGhJdEpPHXDt9T/7UUROY36P1qlMWYFRGHgf PPJQ0TD+AZigM3rMQdL7fDcUGslby3cTaoEu6M4Nct3ZvvLX+z/4fW0mLpkbx1LZRGSq odg26rSj6WSRpMM2CUSyKRWcMToRD7O32Dsq2USEgP/BZnJKnVjZ/EJtpPELnyCWSCkM cwUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=X8yFHAeOstTMxaiMqZkWddTJAl93LsADq7KrCzydUlE=; b=Spz7OUjyFnjex9PmR4AAx/80om5BxuECJu3htkF82ig69U5YQZy6SsaxFfbqjoEMhQ fmMuKK6Zr90DSfIrN9eXkpxbz7EJ2RlZi27UUt8TUDfW7TYJp5MJWOXGyOlCHycph6X6 w22z+IOS5vZ0oikTcUk8/I327YtjqYr/MB8F8I1PzhHyBmd9r645LlGutP76lBlrELCT d9pyUGphM8qQEJarEcT2Jv2uay5rAFst12bwHICWIii2uO0OeN1R+S4WZYm/lORrF+RT C2hzvXlelhfdcI8O11y9lL3B5Ptp6lou8fYBGcv+vGLvv3ZMHfOcxH4GRt1wd680Av/C R0Ow== X-Gm-Message-State: APzg51DgocMJvpro3/jtEoe4lJ6T5+oj1vOLHMN5ka7erO+bpZKnfatz BgzrMCE+wMtG9AQG8YyEHnTtmRkRH/Q= X-Google-Smtp-Source: ANB0VdbydoCczd6t5OWiCeA82DXuN/D2bjZ4u6ura8ScgTniDcERT22wuWUhrE6G83LseWFi/6kG9A== X-Received: by 2002:a7b:c10c:: with SMTP id w12-v6mr2283389wmi.132.1536691504860; Tue, 11 Sep 2018 11:45:04 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id v46-v6sm20870868wrc.63.2018.09.11.11.45.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 11:45:04 -0700 (PDT) From: Pete Heist Message-Id: <5F35DA9A-C881-4D7F-9F9E-E3CFC49F8A25@heistp.net> Content-Type: multipart/alternative; boundary="Apple-Mail=_80B3DF32-A74E-4689-96BF-E52B9A31D3E8" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Tue, 11 Sep 2018 20:45:02 +0200 In-Reply-To: <5C7BA623-F83E-4B93-9997-1099DEC2392C@gmx.de> Cc: Cake List To: Sebastian Moeller References: <87zhwxzh8o.fsf@toke.dk> <139B295B-7371-43DE-B472-DE629C9B8432@heistp.net> <87efe65wol.fsf@toke.dk> <6C556301-015B-4903-AE5A-F22D3517FFCC@heistp.net> <79F47FB1-6B00-4753-930B-950FB8CD3850@gmx.de> <8E0ECFFC-37A0-4DC3-91C9-27793B1C18E5@heistp.net> <5C7BA623-F83E-4B93-9997-1099DEC2392C@gmx.de> X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] Cake vs fq_codel and c/burst on an ER-X bridge X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2018 18:45:06 -0000 --Apple-Mail=_80B3DF32-A74E-4689-96BF-E52B9A31D3E8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Sep 11, 2018, at 8:28 PM, Sebastian Moeller = wrote: >>=20 >> Yeah, good point, I left nat there because I had one port configured = for routing and the other for the bridge and was sometimes swapping = between the two. I realize now I actually sent the numbers for routing, = not bridging. Bridging without =E2=80=98nat=E2=80=99 looks a bit higher = (155 Mbit for cake instead of 135 Mbit). I would re-do all these tests = for completeness but I=E2=80=99m out of time now. >=20 > Ouch, a ten percent bandwidth cost for the nat feature certainly = answeers the question whether nat should be the default=E2=80=A6 That probably has a lot to do with routing vs bridging though also. If I = turn QoS off, the ER-X does about 250Mbit when routing and 280Mbit with = the soft bridge, so that=E2=80=99s probably most of that difference. = I=E2=80=99m not seeing a throughput difference above random noise = between =E2=80=98nat=E2=80=99 and =E2=80=98nonat=E2=80=99. When I = benchmarked it before I saw an ~1.5% CPU difference, not nothing. >>> The last time we discussed the bust issue, I could not manage to = see any difference with or without a specified burst, but I strongly = believe I simply did not properly test. Btw, this is unidirectional = shaping or with bidirectional saturation? >>=20 >> Unidirectional. I definitely see a difference, but I wonder what = criteria we (and I) used for =E2=80=9Cout of CPU=E2=80=99 in the past. >=20 > So totally unscientifically me yardstick was as long as = throughput increases more or less linearly with configured shaper = bandwidth things are fine, and then at the candidate bandwidths I ran = "top -d 1" and monitored both idle% ad sirq% with idle falling below 5% = being a strong indicator of bottlenecking on cpu cycles. Dlakelan over = at github (https://github.com/dlakelan/routerperf = ) is working on a small side = project that aims for tighter multi-core aware logging of cpu usage on a = router, but that has not left the early prototype stage. Ok, my frustration with the testing has also been variable results from = run to run. My inner self is saying, yes, do some testing, but don=E2=80=99= t spend too much time on it when it has this stochastic side to it.= --Apple-Mail=_80B3DF32-A74E-4689-96BF-E52B9A31D3E8 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On = Sep 11, 2018, at 8:28 PM, Sebastian Moeller <moeller0@gmx.de> = wrote:

Yeah, good point, I = left nat there because I had one port configured for routing and the = other for the bridge and was sometimes swapping between the two. I = realize now I actually sent the numbers for routing, not bridging. = Bridging without =E2=80=98nat=E2=80=99 looks a bit higher (155 Mbit for = cake instead of 135 Mbit). I would re-do all these tests for = completeness but I=E2=80=99m out of time now.

= Ouch, a ten = percent bandwidth cost for the nat feature certainly answeers the = question whether nat should be the = default=E2=80=A6

That probably has a lot to do with routing vs = bridging though also. If I turn QoS off, the ER-X does about 250Mbit = when routing and 280Mbit with the soft bridge, so that=E2=80=99s = probably most of that difference. I=E2=80=99m not seeing a throughput = difference above random noise between =E2=80=98nat=E2=80=99 and = =E2=80=98nonat=E2=80=99. When I benchmarked it before I saw an ~1.5% CPU = difference, not nothing.

The last = time we discussed the bust issue, I could not manage to see any = difference with or without a specified burst, but I strongly believe I = simply did not properly test. Btw, this is unidirectional shaping or = with bidirectional saturation?

Unidirectional. I definitely see a difference, but I wonder = what criteria we (and I) used for =E2=80=9Cout of CPU=E2=80=99 in the = past.

= So totally = unscientifically me yardstick was as long as throughput increases more = or less linearly with configured shaper bandwidth things are fine, and = then at the candidate bandwidths I ran "top -d 1" and monitored both = idle% ad sirq% with idle falling below 5% being a strong indicator of = bottlenecking on cpu cycles. Dlakelan over at github (https://github.com/dlakelan/routerperf) is working on a small side = project that aims for tighter multi-core aware logging of cpu usage on a = router, but that has not left the early prototype stage.

Ok, my = frustration with the testing has also been variable results from run to = run. My inner self is saying, yes, do some testing, but don=E2=80=99t = spend too much time on it when it has this stochastic side to = it.
= --Apple-Mail=_80B3DF32-A74E-4689-96BF-E52B9A31D3E8--