From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-x229.google.com (mail-wr0-x229.google.com [IPv6:2a00:1450:400c:c0c::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 0621E3B29E for ; Sat, 16 Jun 2018 15:20:25 -0400 (EDT) Received: by mail-wr0-x229.google.com with SMTP id d8-v6so12922994wro.4 for ; Sat, 16 Jun 2018 12:20:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=htr4olwWaehXen9JugMaJu7XJ/KCjT61a8Zsrpd8reY=; b=dC/gZaxfXEdpzJ8vpDJ/U+/WTbPaM5tW86hjqIjiIABXnH9QOTIwrlFc0Pu8QUYLab s9Yo/nxdLEODbfX1Jifrv8ehSffr6DWm65pEkSeY9VHT2kUmrH7VSe2rX2gT/3TgQOPL 4dvmCm8DtKguOoevHBuKHnivtX3cVP+tUQlyT1rhS3MwF8rIv+rj4Pofw7SGeJz2urUt 1dyp9Z0MiGC32oToL2oMl5xx4K4UeiJDplxL8mAW07c30dZfUEDoFUN2k2+g7ibqDBqN o+AMDM1aLSX7vtQXI/vJJsIhtesjL4A/L9yHiXo+2VZtWvg0Y0L3g51aVKLKM2sjnu8E lbQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=htr4olwWaehXen9JugMaJu7XJ/KCjT61a8Zsrpd8reY=; b=hNtNWB6lY8tuz5YGDuiuXiTbEEeok9829td5Gc422LqM4Ctr6AeAQ/VAqwFr+/oZgc wBy14X6NaDB3RixkVJ0VOnuy8ysH1rklfOJgSFXN+HVyqpULwhjr5P6PDZts4lCQinkw mUFeliPR8LBK4rqiynOO0tYp4uqZJlbpvUPC21cYpv6wgC0P+FQUDUjZ+7RgNuvwyOl1 MU+p1x3UXnZX24hQETKTO1xzzIe3focDV61u5wmxMjrvQZfrkOohWpsUCaTLuv5p3eyV 2O1me4kQs5A2eaABZDTEdsWySeD5Dtg/roAYasG4KE1oB0a0xShvr84YoDl98cLj2CgK xUrQ== X-Gm-Message-State: APt69E1XNxdcPEcccTQ0Qfjsmo9aQjbl1yXsnV+KwZYFLegr/hMvz8Z+ MNvmnixnIUuj3JjRy7K6j2NDDVcX0PI= X-Google-Smtp-Source: ADUXVKI3g8ZmDcyLGEcvHofjS/+AvBc3Zmm6WGlYbx7CLaRUhYJV3MrH+r22Uz6IzZobjbqwWhKfXQ== X-Received: by 2002:adf:bb10:: with SMTP id r16-v6mr5289026wrg.244.1529176823957; Sat, 16 Jun 2018 12:20:23 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id u108-v6sm22215903wrc.40.2018.06.16.12.20.23 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 16 Jun 2018 12:20:23 -0700 (PDT) Content-Type: multipart/alternative; boundary="Apple-Mail=_682D37AC-7BD1-438B-B478-98C27CE376BC" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Pete Heist In-Reply-To: Date: Sat, 16 Jun 2018 21:20:22 +0200 Cc: cake@lists.bufferbloat.net Message-Id: <7A2F7A22-9A94-49EE-835E-180CFBD9951D@heistp.net> References: <87wov5hzvq.fsf@toke.dk> <20180611125425.792d4dcc@xeon-e3> <6500246E-2382-48FD-8FC7-A15452D42042@heistp.net> To: Michel Blais X-Mailer: Apple Mail (2.3124) Subject: Re: [Cake] Bandwidith rate by host instead of global while using [dual-]srchost and [dual-]dsthost X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jun 2018 19:20:25 -0000 --Apple-Mail=_682D37AC-7BD1-438B-B478-98C27CE376BC Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jun 16, 2018, at 5:33 PM, Michel Blais = wrote: >=20 > I doubt any WISP (Ubnt main market) using Edgerouter for routing would = activate codel since it would hurt badly forwarding performance. Anyway, = just like for CMTS, WISP bottleneck is the wireless radio, not the = router. Hi Michel- badly hurt forwarding performance in what way? The bottleneck is not always the wireless radio, particularly when the = link is not over-provisioned. I=E2=80=99m working with a WISP that uses = Ubiquiti gear and APUs in their backhaul that do soft rate limiting (on = their stable links) with HTB and SFQ. In my lab tests using NSM5s and = APUs, both Cake and fq_codel outperform SFQ in both inter-flow and = (especially) intra-flow latency under load, so that=E2=80=99s why = we=E2=80=99re working on switching to one or the other. HTB is needed = though for customized fairness, as you noted in your original question. Out-of-the-box performance of NSM5s with airMAX enabled leaves something = left to be desired as far as latency under load goes, so one should = either over-provision or do some kind of shaping. Compare: airMAX enabled, default queueing: https://www.drhleny.cz/nsm5-airmax/default_rrulbe/ = airMAX disabled, cake queueing (enabling airMAX here hurts inter-flow = latency): https://www.drhleny.cz/nsm5/eg_csrt_rrulbe_eg_cake_40mbit/ = By the way, if you note the isochronous spikes in the output, that = appears to be due to a bug in the NSM5=E2=80=99s Ethernet or internal = switch driver that has been fixed in LEDE/OpenWRT. If you or anyone else = happens to have an NSM5, I=E2=80=99d appreciate your testing and adding = to the thread, where I didn=E2=80=99t yet manage to convince Ubiquiti = that it=E2=80=99s real and an issue: = https://community.ubnt.com/t5/airMAX-Installation/NanoStation-M5-ping-spik= es-about-once-per-second-even-just-to/td-p/2358704/page/2 = I=E2=80=99m actively writing an article on this topic and will post when = finished... --Apple-Mail=_682D37AC-7BD1-438B-B478-98C27CE376BC Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On Jun 16, 2018, at 5:33 PM, Michel Blais <michel@targointernet.com> wrote:

I doubt = any WISP (Ubnt main market) using Edgerouter for routing would activate = codel since it would hurt badly forwarding performance. Anyway, just = like for CMTS, WISP bottleneck is the wireless radio, not the = router.

Hi Michel- badly hurt forwarding = performance in what way?

The bottleneck is not always the wireless radio, particularly = when the link is not over-provisioned. I=E2=80=99m working with a WISP = that uses Ubiquiti gear and APUs in their backhaul that do soft rate = limiting (on their stable links) with HTB and SFQ. In my lab tests using = NSM5s and APUs, both Cake and fq_codel outperform SFQ in both inter-flow = and (especially) intra-flow latency under load, so that=E2=80=99s why = we=E2=80=99re working on switching to one or the other. HTB is needed = though for customized fairness, as you noted in your original = question.

Out-of-the-box performance of NSM5s with airMAX enabled = leaves something left to be desired as far as latency under load goes, = so one should either over-provision or do some kind of shaping. = Compare:

airMAX = enabled, default queueing:
https://www.drhleny.cz/nsm5-airmax/default_rrulbe/

airMAX disabled, cake = queueing (enabling airMAX here hurts inter-flow latency):
https://www.drhleny.cz/nsm5/eg_csrt_rrulbe_eg_cake_40mbit/<= /div>

By the way, if = you note the isochronous spikes in the output, that appears to be due to = a bug in the NSM5=E2=80=99s Ethernet or internal switch driver that has = been fixed in LEDE/OpenWRT. If you or anyone else happens to have an = NSM5, I=E2=80=99d appreciate your testing and adding to the thread, = where I didn=E2=80=99t yet manage to convince Ubiquiti that it=E2=80=99s = real and an issue:

https://community.ubnt.com/t5/airMAX-Installation/NanoStation-M= 5-ping-spikes-about-once-per-second-even-just-to/td-p/2358704/page/2

I=E2=80=99m = actively writing an article on this topic and will post when = finished...

= --Apple-Mail=_682D37AC-7BD1-438B-B478-98C27CE376BC--