From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com [IPv6:2a00:1450:4864:20::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E7E973B29E for ; Wed, 23 Oct 2019 05:54:55 -0400 (EDT) Received: by mail-lj1-x231.google.com with SMTP id d1so20333144ljl.13 for ; Wed, 23 Oct 2019 02:54:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=V31nqLwgCe9vmQjeAUjywDaCQupRmUQKJ5V54RqncoQ=; b=gYlp25KVtY+nu+xvVGJpsUBuX6bgoIpHX6ow1UXr111LPKHZSpTLyrCr0V7SR1WmCy UFkPWnENQj95QKfqAA0NEZY9WcFKMTXqJORAXn7QVo6FGHIZJlH9tpO6O33kYjNjvAFR 58X9oc/O3ku/IYsyywJTnSLBCZUtIDSKwvvFGfH3CZHAx0JoCco9F108dak1xTmQJQRH KLtTl2RHDFnCDAu5PlU5rZsjGB+WtnaHfxr34x4+N17wJkdlo0nQnQAJPr2viGivrSt0 3jkJmvoWlhlmhtRuzJBgSMyLHR1s2pzXZJCiWxCnWmZekKWMENZkz89gIHgwsusWDbhq cm/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=V31nqLwgCe9vmQjeAUjywDaCQupRmUQKJ5V54RqncoQ=; b=NmL0fGfq1xAWxPTa28HpWFM7MHfuJmrZz9IrrEGFShRIR1mrLdXBGS2RyzjZOiXiaZ q75yGBBJlYwJj+20qXW29km+6yiBYGD+/Rk0o/FFcpdlVXTqzgOXYOb49Ajgiq8IFr7u zlhNclUxuHjrWdYzf+HjcPG42jJY6kz1QyAYSb8lSr3/VaGG33gXrCCkOBW8j/LoN1Rx /OL1M78uVDX3YCqOTtR0r1lzUCtWKae0HtadGa5Gq8XFujg5dhFJo7DNbyQfDJMtjMCu Sfd2Jzm4p1II8y9FL7PZ7O2nE4YO+ifE+YrZ/k8suZEyikGj9dhGvgoeQVgePTm08eWl dAdw== X-Gm-Message-State: APjAAAXYPLsZTU74vl4ImPtaxt/H/SSVe2trE9IVEuZmkVpmJmv/Wyju tLaJ7QG5MQX460mC0cB7NnU= X-Google-Smtp-Source: APXvYqzE1ILYO67ff7rTc+EeCfogUKlggtlRZpw2rBzEmIRkyfJK2vmyKO6/rflxG6HSDFzOQWABJQ== X-Received: by 2002:a2e:7312:: with SMTP id o18mr17073430ljc.216.1571824494752; Wed, 23 Oct 2019 02:54:54 -0700 (PDT) Received: from jonathartonsmbp.lan (83-245-240-171-nat-p.elisa-mobile.fi. [83.245.240.171]) by smtp.gmail.com with ESMTPSA id d27sm4000603lfb.3.2019.10.23.02.54.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Oct 2019 02:54:54 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Jonathan Morton In-Reply-To: <1571815685531.95922@telenor.com> Date: Wed, 23 Oct 2019 12:54:52 +0300 Cc: grobier@icow-systems.com, bloat@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <76711963-5D5C-413E-B58E-94DD02C41766@gmail.com> References: <775be8cd393d47aabdac5097e9ae054c@icow-systems.com> <51AC6A1E-FF96-4B9C-B322-4EA986AE4BAA@gmail.com> <1571815685531.95922@telenor.com> To: erik.taraldsen@telenor.com X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Bloat] Bufferbloat on 4G Connexion X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2019 09:54:56 -0000 > On 23 Oct, 2019, at 10:28 am, = wrote: >=20 > If you could influence the 4G vendors to de-bloat their equipment, = would you recommend BQL, L4S or codel/cake? BQL is a good start, but needs to go with something else to actually fix = the problem. All it does is move (most of) the queue out of hardware = and into software. The software side then needs to capitalise on having = control of the bottleneck queue, by implementing FQ, AQM, or both. L4S is a complete waste of time. It requires basically replacing the = entire Internet, including endpoints, to work properly. If the = endpoints are not replaced, then the middleboxes behave like a = conventional AQM (see below). Codel would be a HUGE help. It's a well-tested example of an AQM that's = specifically designed to work with TCP, and should work equally well = with other protocols designed to be TCP friendly. There are other AQMs, = such as PIE, BLUE, and even the old standby WRED, that would also be = major improvements over the status quo, though they don't perform as = well on common TCP traffic as Codel. The basic requirement here is to = bring the maximum *standing* queue down from hundreds or thousands of = milliseconds to tens or singles; Codel achieves 5ms with its default = Internet-tuned settings. I use a slight variant of Codel in Cake, which I call COBALT. The main = algorithm is just a refactoring of Codel, but I've also addressed two = blind spots in Codel's design. One is the behaviour upon reactivating = very shortly after the standing queue was initially brought under = control; COBALT keeps better track of previous state in that case. The = other is addressing heavy load from non-TCP-friendly traffic, which = requires a probabilistic drop function rather than a scheduled drop or = mark. To take care of the latter, I've overlaid a version of the BLUE = algorithm. I believe Codel and COBALT should be reasonably straightforward to = implement in hardware. I could help to explain the algorithmic details = and rationale to hardware engineers if need be, and help them determine = the right design tradeoff. FQ's main benefit is allowing "sparse" flows of traffic, which tend to = be more sensitive to latency, to bypass queues of "bulk" traffic which = tends to be more throughput sensitive. This amounts to reducing = "inter-flow induced latency", where AQM focuses on reducing "intra-flow = induced latency". The main difficulty is allegedly in identifying flows = and maintaining the per-flow state and individual queues; these should = not be insurmountable obstacles, but I will admit the complexity of = implementing this in hardware is higher than for a plain AQM. I would note, however, that 4G already incurs some of this complexity = through the need to individually queue, aggregate, schedule, and = transmit a stream of traffic to each mobile station. It should be = relatively easy to add an AQM instance per station, thereby ensuring = that congestion signals incurred by one subscriber saturating their own = capacity don't spill over into other subscribers' traffic. Combining AQM with FQ is the gold standard, minimising both inter-flow = and intra-flow induced latency. That's what fq_codel and Cake do. If = that was deployed in the right places in a 4G network, you could = consider the problem solved. For an estimate of implementation = complexity, take an FQ implementation and add an AQM for each flow's = queue. As a middle ground I could suggest one of my latest projects, CNQ (Cheap = Nasty Queuing). This aims to provide some of the benefits of FQ+AQM, = but with only slightly higher implementation complexity than a plain = AQM, at the cost of inferior flow-isolating performance than true FQ. = It should therefore be more suitable for high-volume nodes than an = FQ-based algorithm. A software implementation for demo purposes is = presently in the works, and we should have some initial performance data = fairly soon. I would seriously consider deploying CNQ on a per-station = basis in 4G. - Jonathan Morton