From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-x241.google.com (mail-lj1-x241.google.com [IPv6:2a00:1450:4864:20::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id EC6663B29E for ; Sun, 8 Sep 2019 13:27:06 -0400 (EDT) Received: by mail-lj1-x241.google.com with SMTP id a22so10492917ljd.0 for ; Sun, 08 Sep 2019 10:27:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=V4FJpXaqXSNZ1dWk6HKSSJHRHFLNJa3J4JWK7ZGV9gA=; b=EwEg81h7OaVrqCUFP4stKqqDcnRVG+r6JlUlg9sXkYHaq/QF9vwkAx4lNeY5xl86lB wRss0oJfjn31oxoEZS3aszzGWDF+l4VsGD4mh1NpE1APiBRtygUUDPE9/RpBq7GS16rc aTK9FquK7TSkvgxtIpHCV+Wht8J3smX9gwmwd61OWs7cyXsPyqXwTCZiSEIKm8+9jpXy zcLi/Bc6hXdTwucEJeHC61jxh1TS1Ex3JSTmRNsn+BvvGETbsMt1cggcwEvzGEWSSpmt C/GX7IvtacxWWBCqp5k9+v6XN18huMFnepVtUgACXymSl89NleurSwDy7q3nbyErjrRW pP0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=V4FJpXaqXSNZ1dWk6HKSSJHRHFLNJa3J4JWK7ZGV9gA=; b=Oh6uj5OJy/TgZHLo40DbXWZ6i76sCwHshkZTPRy2+mFlTcz9wZ7nq5ahOirwudhI3C giCyFZTXmL2yfDylPFaAdU895cZ3W6ZjDBA/iSCvtChnyfxP3FG6s7zWfjg85iamPDRy hcLA+OyshCJESqQ241rdsuKD/Cn9iILK//K95On3Z1/xW23LELSQNn5TWDrIktvqryMP fWCnS1ok2TREsD8CENeTtARI1FTrXbZQbuY19mDMcLIKoKOVgNemrhBImdwsnugVgMQ/ Gba7UGIJ7K1B+m2H2QVHmObsdyoSkYfIPFpmyIwaJxKVa75rtfQJlmXh7d0fu2UGE8sB pThg== X-Gm-Message-State: APjAAAXJprwm+SrqUH0VMzFD833mkqa8h94fJ3AbKxq9eH8c51jGlJ+O 1Tx86Vnz0CS1wr9kdJjOY8I= X-Google-Smtp-Source: APXvYqwQah9J8EhcAD7vrPWqsyT9hLZbMNEOKVTGntfcj26BqFGffapTIfeduV/BZT1Z2hREjJGMeQ== X-Received: by 2002:a2e:9b43:: with SMTP id o3mr1929240ljj.214.1567963625796; Sun, 08 Sep 2019 10:27:05 -0700 (PDT) Received: from jonathartonsmbp.lan (83-245-237-193-nat-p.elisa-mobile.fi. [83.245.237.193]) by smtp.gmail.com with ESMTPSA id t6sm2086779lji.3.2019.09.08.10.27.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 08 Sep 2019 10:27:05 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Jonathan Morton In-Reply-To: Date: Sun, 8 Sep 2019 20:27:03 +0300 Cc: cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: References: <2825CE14-2109-4580-A086-9701F4D3ADF0@gmail.com> <18b1c174-b88d-4664-9aa8-9c42925fc14c@www.fastmail.com> <9a90111b-2389-4dc6-8409-18c40f895540@www.fastmail.com> <43F02160-E691-4393-A0C0-8AB4AD962700@gmail.com> To: Justin Kilpatrick X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] Fighting bloat in the face of uncertinty X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Sep 2019 17:27:07 -0000 >> You could also set it back to 'internet' and progressively reduce the=20= >> bandwidth parameter, making the Cake shaper into the actual = bottleneck.=20 >> This is the correct fix for the problem, and you should notice an=20 >> instant improvement as soon as the bandwidth parameter is correct. >=20 > Hand tuning this one link is not a problem. I'm searching for a set of = settings that will provide generally good performance across a wide = range of devices, links, and situations.=20 >=20 > =46rom what you've indicated so far there's nothing as effective as a = correct bandwidth estimation if we consider the antenna (link) a black = box. Expecting the user to input expected throughput for every link and = then managing that information is essentially a non-starter.=20 >=20 > Radio tuning provides some improvement, but until ubiquiti starts = shipping with Codel on non-router devices I don't think there's a good = solution here.=20 >=20 > Any way to have the receiving device detect bloat and insert an ECN? That's what the qdisc itself is supposed to do. > I don't think the time spent in the intermediate device is detectable = at the kernel level but we keep track of latency for routing decisions = and could detect bloat with some accuracy, the problem is how to = respond. As long as you can detect which link the bloat is on (and in which = direction), you can respond by reducing the bandwidth parameter on that = half-link by a small amount. Since you have a cooperating network, = maintaining a time standard on each node sufficient to observe one-way = delays seems feasible, as is establishing a normal baseline latency for = each link. The characteristics of the bandwidth parameter being too high are easy = to observe. Not only will the one-way delay go up, but the received = throughput in the same direction at the same time will be lower than = configured. You might use the latter as a hint as to how far you need = to reduce the shaped bandwidth. Deciding when and by how much to *increase* bandwidth, which is = presumably desirable when link conditions improve, is a more difficult = problem when the link hardware doesn't cooperate by informing you of its = status. (This is something you could reasonably ask Ubiquiti to = address.) I would assume that link characteristics will change slowly, and run an = occasional explicit bandwidth probe to see if spare bandwidth is = available. If that probe comes through without exhibiting bloat, *and* = the link is otherwise loaded to capacity, then increase the shaper by an = amount within the probe's capacity of measurement - and schedule a = repeat. A suitable probe might be 100x 1500b packets paced out over a second, = bypassing the shaper. This will occupy just over 1Mbps of bandwidth, = and can be expected to induce 10ms of delay if injected into a saturated = 100Mbps link. Observe the delay experienced by each packet *and* the = quantity of other traffic that appears between them. Only if both are = favourable can you safely open the shaper, by 1Mbps. Since wireless links can be expected to change their capacity over time, = due to eg. weather and tree growth, this seems to be more generally = useful than a static guess. You could deploy a new link with a = conservative "guess" of say 10Mbps, and just probe from there. - Jonathan Morton=