From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x244.google.com (mail-wm0-x244.google.com [IPv6:2a00:1450:400c:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 68B513BA8E for ; Fri, 7 Sep 2018 06:00:52 -0400 (EDT) Received: by mail-wm0-x244.google.com with SMTP id b19-v6so14168999wme.3 for ; Fri, 07 Sep 2018 03:00:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=tx49fYeuf6f9IWJNBYoacBZTmVq7vEpkCadPvwLl2yY=; b=e72MZKZCjtAAV563B3y0i6unZyJKf0l/jJN66Yk0HJt9tVEF33KK752Bd43cT7V4tf 8NURdn0ri9nIPUO7RmOWhQRsNV5205BGLyyXsrcgADu29XTHdPBNoZJMVHqO/qfVYtXn kZEU60VRRd2hqHG9pGOP8G1SFPAI2RZ3HrRiHnRDGQ6LHtJkKZ6GcNSfESmMrFZjnZ3u ZKuCg+6cIBkP5KOTf8mEa/gAZtHDCiyDhKx50Xbfvpw/HmY9neVg4u/AHOX5k//PfQOK mlsD5SjES8dRTL/8z7R80+NsWBo9t1bbo6Blm+B3lTXri/A/Ds96xaRD2S8Spu3Z4elA nICA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=tx49fYeuf6f9IWJNBYoacBZTmVq7vEpkCadPvwLl2yY=; b=QrGS1QwBVXqiEntoNvjfzVn3DgSIXU5QeonojLHTc5YaAXYdNG896QZfxn73kfmwU4 gsxYfCDFMTO/D9n3MsYZk4QFgP4sO1mcybIKORYux+D8+2eB/wx1J8d89pxFdQHNsELL VmD98zRtBsBn7nhOBWq9kVlEYw+uowRd1gc45CR6ngMS5BQuETuCl05xHHtmMf2OCJIn VEIj3EG0xpNMdUG8RWPtKPVp6ylVcchtBlfb0T8+mIW+3O7Kv+M96PK73Y6BgcFUWVrT WShBQe7omLCSXEqiOCeX65mDhIS2BUtEXlAoSUxpJvJJ1b+W6wP2hJYFMPzkad0EHBIO zv8A== X-Gm-Message-State: APzg51AYaiQMr/cD/hBtC2XdN5aQCBQULrpHIs9L0PRFDjyFCtvkTIf2 B6zuqbrJvNnQJ42LLJRXGbgZaw== X-Google-Smtp-Source: ANB0VdZwx/nqP7G9w+oaEMle+vX+INWuaKNJK0tIxEc/wfyMrI5vACfVKLlbqX74GY6px3zB4q+bqQ== X-Received: by 2002:a1c:b404:: with SMTP id d4-v6mr4844610wmf.28.1536314451451; Fri, 07 Sep 2018 03:00:51 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id c9-v6sm7330994wrt.77.2018.09.07.03.00.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Sep 2018 03:00:51 -0700 (PDT) From: Pete Heist Message-Id: <3BA06BE5-FBD2-4FC8-86BB-786679A66147@heistp.net> Content-Type: multipart/alternative; boundary="Apple-Mail=_0D33A20D-FB94-4B0E-A7CE-F09EB30F0D13" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Fri, 7 Sep 2018 12:00:49 +0200 In-Reply-To: <1CFBCFB1-AC8B-4172-A4EB-1CE65DE62EF6@gmail.com> Cc: Cake List To: Jonathan Morton References: <2CB335A2-E18F-430F-9331-9AD9B3238D84@heistp.net> <1CFBCFB1-AC8B-4172-A4EB-1CE65DE62EF6@gmail.com> X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] FreeNet backhaul X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2018 10:00:52 -0000 --Apple-Mail=_0D33A20D-FB94-4B0E-A7CE-F09EB30F0D13 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Sep 7, 2018, at 1:03 AM, Jonathan Morton = wrote: >=20 >> On 7 Sep, 2018, at 1:37 am, Pete Heist wrote: >>=20 >> This router is an old ALIX with kernel 2.6.26, but on the other hand = it does have hfsc + esfq (a variant of sfq with host fairness) deployed, = so if it=E2=80=99s actually controlling the queue, one might suspect = that sfq it could control inter-flow latency at least somewhat. >=20 > ESFQ has two important faults: it doesn't explicitly control the = length of individual queues (only tail-drops when a global limit is = reached), and it suffers from hash collisions at the full "birthday = problem" rate. So some of your measurement traffic is likely colliding = with real traffic and suffering accordingly. Ah, ok, that is important. > That still makes ESFQ far better than a dumb FIFO. I=E2=80=99ve heard tales of the way things were. As a contrast, the router I=E2=80=99m on: = https://www.heistp.net/downloads/vysina_ping.pdf = The big difference = here is this router=E2=80=99s uplink is licensed spectrum full-duplex = 100Mbit, whereas Jerab from earlier is 5GHz WiFi (2x NSM5). The shift = around June was an upgrade from ALIX to APU. I haven=E2=80=99t seen evidence yet of backhaul links running at = saturation for long periods. When I watch throughputs in real-time I do = see pulses though that probably don't show up in the long-term MRTG = throughput graphs. I wonder what queue lengths look like at millisecond = resolution during these events.= --Apple-Mail=_0D33A20D-FB94-4B0E-A7CE-F09EB30F0D13 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On = Sep 7, 2018, at 1:03 AM, Jonathan Morton <chromatix99@gmail.com> wrote:

On 7 Sep, 2018, at 1:37 = am, Pete Heist <pete@heistp.net> wrote:

This router is an old ALIX with kernel 2.6.26, but on the = other hand it does have hfsc + esfq (a variant of sfq with host = fairness) deployed, so if it=E2=80=99s actually controlling the queue, = one might suspect that sfq it could control inter-flow latency at least = somewhat.

ESFQ has two = important faults: it doesn't explicitly control the length of individual = queues (only tail-drops when a global limit is reached), and it suffers = from hash collisions at the full "birthday problem" rate.  So some = of your measurement traffic is likely colliding with real traffic and = suffering accordingly.

Ah, ok, that is important.

That still makes ESFQ far better than a dumb FIFO.

I=E2=80=99ve heard tales of the way things were.

As a contrast, the = router I=E2=80=99m on: https://www.heistp.net/downloads/vysina_ping.pdf The = big difference here is this router=E2=80=99s uplink is licensed spectrum = full-duplex 100Mbit, whereas Jerab from earlier is 5GHz WiFi (2x NSM5). = The shift around June was an upgrade from ALIX to APU.

I haven=E2=80=99t seen = evidence yet of backhaul links running at saturation for long periods. = When I watch throughputs in real-time I do see pulses though that = probably don't show up in the long-term MRTG throughput graphs. I wonder = what queue lengths look like at millisecond resolution during these = events.
= --Apple-Mail=_0D33A20D-FB94-4B0E-A7CE-F09EB30F0D13--