From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 984963BA8E for ; Sat, 28 Jul 2018 16:00:21 -0400 (EDT) Received: by mail-wm0-x231.google.com with SMTP id y9-v6so9274483wma.5 for ; Sat, 28 Jul 2018 13:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=Iitc0B0fIvUEMaskN4YKTWIO5n2ksMOJ/pV+PAC/lU0=; b=TStLtmk5UNIgY5+b0YsM10DhDWmPcAisMXxNdNBiQ2NvT7JYzJGdBzYfKvlAexmTkS rM8bQCy3nDqbala522vskuFaOi7HcwnrvftukDqQR167jps1CK5hev0xQx2lAQUIhyB5 3mQuGmsV4iMiu+cIWfBBWiiEc86/V1WQvMmiFd+vSE7KjZMfhZanhGcY3jHPfYG5AVrj SzCET2xa+jOlfTtwC9Or/yus3i5IX63IzIFRKMCC6dc6UAAoZXgrnYzaJjWHlDS7oGnh 8VeLnmshFs3bCXzztYJxNCPLlvcybeaUxaxcqcXPqhR3zgWIkgTbDU07XM4khyNCFfg2 2Nfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=Iitc0B0fIvUEMaskN4YKTWIO5n2ksMOJ/pV+PAC/lU0=; b=PSv19CPVnj4ePoMOddKNXvOtm6tiL2oiEbRLWFrY/XgJ2bWi5abE/U0olT4P3wKdpy pBYmiBl1dYRWJwWXjqkUEUD9CtNeP6LjQK6HZRgJ2QeCBC+z0PBRynuTq/FRhd13C5co HPxYL3jNRX+s/vlr97ZF1AVD8yO/ZVoEMuVDTMe5E9MEsargZwn5lfjR3zBEFfAtQMS2 GCZWMcFJSv8GOzRPYBBH3EPSPA3/CID96fOmLS+VSNN2mj0bpsJpjsU0eKN65tPgk+dh zkATQ0vCfdJ2xetqMq2FC5jwIPzXdazxsJHJD5A4/OryD4zGvWJqn309Zy8KHrboqhuA ad9Q== X-Gm-Message-State: AOUpUlGF7suC/PEVH3+2D+dvRnbillPuYe+oug55+B/4x7VMhQ5EfuCG BxXlQapWFRI5qD3ujS+KykskAg== X-Google-Smtp-Source: AAOMgpc4yuhiXfEqtmExIFB43hiaLb0j5APRUnf2vGHR2Cn36+7nyNu94/5Qd2tHmesWzHBQWc02Fg== X-Received: by 2002:a1c:864c:: with SMTP id i73-v6mr9489987wmd.40.1532808020646; Sat, 28 Jul 2018 13:00:20 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id x62-v6sm6790046wmg.1.2018.07.28.13.00.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 28 Jul 2018 13:00:20 -0700 (PDT) From: Pete Heist Message-Id: <27E52C41-7488-4A75-8CB4-810604DFB32C@heistp.net> Content-Type: multipart/alternative; boundary="Apple-Mail=_EBCA0487-3038-4C9F-9DEA-191C51A60887" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Sat, 28 Jul 2018 22:00:18 +0200 In-Reply-To: Cc: Cake List To: Dave Taht References: <1357421162.31089.1531812291583@webmail.strato.de> <1c323544b3076c0ab31b887d6113f25f572e41ae.camel@coverfire.com> <87woth28rw.fsf@toke.dk> <87tvol1z6h.fsf@toke.dk> <8980FA6C-508B-43E1-8C23-6EBC4A10499A@heistp.net> <842E73E8-368B-4F53-9A6C-31C10420536E@heistp.net> <8BDEED8F-3DDC-4CE2-9E56-8B0FD9CD6C04@heistp.net> X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] isp economics X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jul 2018 20:00:21 -0000 --Apple-Mail=_EBCA0487-3038-4C9F-9DEA-191C51A60887 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jul 28, 2018, at 9:03 PM, Dave Taht wrote: >=20 > under load on the NanoStation 5 AC Loco=E2=80=99s I got for the = camp=E2=80=99s backhaul. Is it really that good? This is in contrast to = the 50+ms I see with rrul_be on the NanoStation M5 (without controlling = the queue).=20 >=20 > ubnt both cases? doubt it's the same bandwidth both cases. should be = proportional to the latency you are seeing. running 2x2 incurs a latency = penalty also. Higher bandwidth on the newer AC Loco vs the M5 but not a huge = difference (120Mbit one-way vs 90Mbit one-way). Both 20 MHz channels. = The AC Loco defaults to 80 MHz channels, which is excessive for this = application. =20 > This test is straight AP to AP though, with probably 1 flow up and 1 = down plus ping, so I want to get 2-4 more of these and do rrul_be = through the Ethernet ports, to get more flows and UDP, and see how it = looks then. > =20 > Run more flows. SFQ is per packet fq. They have right-sized buffers = when the link is running at close to the configured rate, not when it's = stuggling. I think that=E2=80=99s the primary reason (the way ubnt does their = test), unfortunately don=E2=80=99t have two free to test at the moment. > I also seem to remember they reduced the txop to ~2ms. turned off = 802.11e. I've recommended this for years now in the general case.=20 Re 802.11e, what=E2=80=99s interesting is when you run =E2=80=99athstats=E2= =80=99 on the NSM5 (older), there=E2=80=99s a breakdown of BK, BE, VI = and VO packet stats. On the AC Loco (newer), there=E2=80=99s not. That = does imply, but doesn=E2=80=99t prove, that they don=E2=80=99t use these = queues, at least for point-to-point connections. I think they have to = have 802.11e to be compliant, but I don=E2=80=99t know if they're = mapping everything to one queue or not. It looks like airMAX is now _not_ used at all for point-to-point = connections, whereas it used to be on their older gear. This is good, as = my testing showed airMAX only adds a bit of inter-flow latency for = point-to-point. > Their 100mbit ethernet devices also do flow control and are more often = the bottleneck than not, so the wifi runs empty more often.=20 The AC Loco has Gbit Ethernet thankfully. Looks to me like all of their = AC gear does now. Sorry for diverging too far from the ISP topic, and on the Cake list=E2=80= =A6 --Apple-Mail=_EBCA0487-3038-4C9F-9DEA-191C51A60887 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On = Jul 28, 2018, at 9:03 PM, Dave Taht <dave.taht@gmail.com>= wrote:

under load on the NanoStation 5 AC Loco=E2=80=99= s I got for the camp=E2=80=99s backhaul. Is it really that good? This is = in contrast to the 50+ms I see with rrul_be on the NanoStation M5 = (without controlling the queue). 

ubnt both cases? doubt = it's the same bandwidth both cases. should be proportional to the = latency you are seeing. running 2x2 incurs a latency penalty = also.

Higher bandwidth on the newer AC Loco vs the M5 = but not a huge difference (120Mbit one-way vs 90Mbit one-way). Both 20 = MHz channels. The AC Loco defaults to 80 MHz channels, which is = excessive for this application.
 
This test is straight AP to AP though, with probably 1 flow = up and 1 down plus ping, so I want to get 2-4 more of these and do = rrul_be through the Ethernet ports, to get more flows and UDP, and see = how it looks then.
 
Run more flows. SFQ is per packet = fq. They have right-sized buffers when the link is running at close to = the configured rate, not when it's = stuggling.

I = think that=E2=80=99s the primary reason (the way ubnt does their test), = unfortunately don=E2=80=99t have two free to test at the = moment.

I also seem = to remember they reduced the txop to ~2ms. turned off 802.11e. I've = recommended this for years now in the general = case. 

Re 802.11e, what=E2=80=99s interesting is when you = run =E2=80=99athstats=E2=80=99 on the NSM5 (older), there=E2=80=99s a = breakdown of BK, BE, VI and VO packet stats. On the AC Loco (newer), = there=E2=80=99s not. That does imply, but doesn=E2=80=99t prove, that = they don=E2=80=99t use these queues, at least for point-to-point = connections. I think they have to have 802.11e to be compliant, but I = don=E2=80=99t know if they're mapping everything to one queue or = not.

It looks like airMAX is now = _not_ used at all for point-to-point connections, whereas it used to be = on their older gear. This is good, as my testing showed airMAX only adds = a bit of inter-flow latency for point-to-point.

Their 100mbit ethernet devices = also do flow control and are more often the bottleneck than not, so the = wifi runs empty more often. 

The AC Loco has Gbit Ethernet thankfully. Looks to = me like all of their AC gear does now.

Sorry for diverging too far from the ISP topic, and on the = Cake list=E2=80=A6

= --Apple-Mail=_EBCA0487-3038-4C9F-9DEA-191C51A60887--