From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id CCBAF3B29E for ; Sun, 10 Feb 2019 02:34:27 -0500 (EST) Received: by mail-wm1-x336.google.com with SMTP id b11so11232273wmj.1 for ; Sat, 09 Feb 2019 23:34:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:content-transfer-encoding:mime-version:subject:message-id:date :to; bh=rdd+6imXAQBrjXUeGe5m6zkTdZvZry9NofHR0vK3si0=; b=YHkUbM9eYWthydZXqjxovUiCrPFnuFNiJd29/EJu4/6o3+i2gFuDid2lPAyN97O0gj zFyrtigIcIeqn4CUqpJqcTGPkMw0qMtHfKecjMkUvyUxkHm6/N3btsnmRmO3tndCLmYP 7vGMbCOvwez3yzWNIeUD9Za81BnEth4h7TYzVr/3jXVGiVugrrABxifKSVoZhzVDdvqd 51dEOsHqRacKWIJ7fxzJmQ9msxKBYxtLoTTubnHftygYZrO6StGQMZ4Zq/nH45sFPN73 8RlCPLMPZsEGxOPufknkJKzr4k7ejXtAIKOnfgI/0urSEdv9Dv6wrraxaLBvyJZzlxCH NxJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-id:date:to; bh=rdd+6imXAQBrjXUeGe5m6zkTdZvZry9NofHR0vK3si0=; b=Fc8VZ56DiIlIEYrXpgPH/H/QznkxyB3OTgaZBB/yrrLka8fxvqMZKSyEmVre3LinaR u5qj9q93dIxLW4Jf9Ib1Vf/za3rloOfCBfLGIqChfs/6zyGkcRDWhMUNGOVDnVKdv3d7 Jc+AOQQt9SyJepBbSRYK0KCSteI/9aTyqutIyUN9exKiUq7vfbsaAO4p4XPGgEW+ZSzd +7BpVbKvxyRfKwPwgYocTTQbASizomiAYoXxwZUO19IK0qOrGHJEIjY92MJ3ZaKcQY8W AJobc10xgK+knhdWNWrcLAXMwNpOtFrLZ1LqVkukILkYQOBo5hpdvsbyrlmiCS/XUvN0 8+rg== X-Gm-Message-State: AHQUAuZk02qL70vQxnzxVXmxGGHyKhj73TTioIDOUi5iZuSZ9OMjuo5h pCMEiSKBFNvG+3DnerunVMBB95e57Ks= X-Google-Smtp-Source: AHgI3Ibb/OuWnMrMI50TBc1lcmYMdfppLL6fongdrKywBq1O87/S9jPm/suzNFBTJvrn0EVse/UxNA== X-Received: by 2002:a5d:4546:: with SMTP id p6mr22478700wrr.35.1549784066559; Sat, 09 Feb 2019 23:34:26 -0800 (PST) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id v6sm5293274wro.57.2019.02.09.23.34.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 09 Feb 2019 23:34:25 -0800 (PST) From: Pete Heist Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Message-Id: Date: Sun, 10 Feb 2019 08:34:24 +0100 To: Cake List X-Mailer: Apple Mail (2.3445.9.1) Subject: [Cake] FreeNet update X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Feb 2019 07:34:28 -0000 We=E2=80=99ve got hfsc+fq_codel on one of our licensed spectrum = full-duplex 100-Mbit backhaul links. I hope to switch to Cake, but I=E2=80= =99m not yet sure the 3.16.7 kernel we=E2=80=99re using is stable enough = to handle it. I=E2=80=99d like to report how much better things are now even with = fq_codel, but we keep finding (and fixing) problems elsewhere in the = network that prevent us from actually filling the queue. The short list: 1) Switched TDD framing modes (Ubiquiti=E2=80=99s "flexible new" for = now) as fixed framing was increasing RTT and killing download throughput = (done) 2) Increased stability of CPE links (setting max tx rate, etc) to = prevent loss / latency spikes that affect upstream ACKs and thus = download throughput (done) 3) rx-vlan-offload must be broken in the r8169 driver in 3.16, was = causing throughput problems for VLAN routing (fixed) 4) The current unsolved killer is a 0.03% downstream packet loss problem = on the backhaul link itself, which per the Mathis equation does us no = good I can fill the queue on upload and show clear results with an rrul_up = test, but not download yet, because of problem #4. One thing that had to be handled right in the sqm setup script was = VLANs, which I imagine most ISPs will have in the backhaul. The original = qos script, using sfq, was adding qdiscs to both the VLAN devices = (eth0.3300, etc) and the main devices on egress and ingress. This caused = multiple problems: - Routed packets would pass through three queues in each direction = unnecessarily, and ingress shaping wasn=E2=80=99t needed in the backhaul = anyway - One-armed routers were effectively made half-duplex, because a qdisc = was being added to eth0 without any filtering by VLAN, putting Internet = facing and customer facing traffic on the same queue Now I use tc filters to assign VLANs to queues and apply all qdiscs on = eth0 under one hfsc/htb link sharing hierarchy. So, one router almost done and 60 more to go, as we work to bloat up our = queues so we can debloat them. :) Sigh, I hope to get back to development soon. Enough for now, and thanks = again to the list for the help=E2=80=A6