From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 6FDFE3B29E for ; Sun, 8 Sep 2019 10:29:31 -0400 (EDT) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 7B7534BE; Sun, 8 Sep 2019 10:29:30 -0400 (EDT) Received: from imap2 ([10.202.2.52]) by compute4.internal (MEProxy); Sun, 08 Sep 2019 10:29:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=althea.net; h= mime-version:message-id:in-reply-to:references:date:from:to:cc :subject:content-type; s=fm1; bh=sVlpX+7s43XEemuAUZIoFC63smGQ9GD b+3FCWqh/7kU=; b=ZAdFVu7037+wurm95XqHEMoPhIHLKmfXT0IOyAXSYEDzH0S 2ZXBINbQuA84BD7TsLCfEoBsvCr70NG3o8tHNGX5DT8mOspx/wtnHRVR7Hwt9K8P gtZd5yFt1yTJ0pfzkodMgQKgbVp0PX0/l4bx9yIHb72G+BjhNp6Ys2P21P+IGh1U 1D9Rf7gBmOfQs4degTFux93eyzbtK4wDuu41GonFgnQgErntULOzFfv7KgTt2ypr /BGUVBeXQu8OJMD5qNhGvv5L82Lk80d1O9wrR/JKrPeooictTlouSumGsmrbmAgd oj+bdnpmLcb7evGowGDepPT3vm3p9fJz3zfwbDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=sVlpX+ 7s43XEemuAUZIoFC63smGQ9GDb+3FCWqh/7kU=; b=y4+Meo6qJ7R3U4MQdroVNX gJPMU4hJMt9NjNoNv5q5cluQ5yp08LB+FYYGDClFucgFpMIfFSmXp5TIgthHuf58 lIb41L0OeRUwzSE6WP5O6gAXRRMIrXjLqRAuDjXemFQH1SV5BJHqqiG5x/i5Kcca Fwc08NUVDQ8aX0Kx42dhV5ID9+6rkVxz5Kq7Z+UFUiZojEb37APJuNWvmSK3zVvq C7NkA1b3nMnC4RNXQgkCNgm5mY/xtmEJTJHwZkBsYpqsInTTbOnGuR6Dfb95j5Wp 4WWc62fpor8DNFPDdzGv2Sd0j+BFvnfXKqa91+B5OewRbSwnjgWGcjX+MKAzORJg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrudekgedgjeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpedflfhushhtihhnucfmihhlphgrthhrihgtkhdfuceojhhu shhtihhnsegrlhhthhgvrgdrnhgvtheqnecurfgrrhgrmhepmhgrihhlfhhrohhmpehjuh hsthhinhesrghlthhhvggrrdhnvghtnecuvehluhhsthgvrhfuihiivgeptd X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id CA546E00A3; Sun, 8 Sep 2019 10:29:29 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.1.7-188-g385deb1-fmstable-20190905v2 Mime-Version: 1.0 Message-Id: In-Reply-To: <43F02160-E691-4393-A0C0-8AB4AD962700@gmail.com> References: <2825CE14-2109-4580-A086-9701F4D3ADF0@gmail.com> <18b1c174-b88d-4664-9aa8-9c42925fc14c@www.fastmail.com> <9a90111b-2389-4dc6-8409-18c40f895540@www.fastmail.com> <43F02160-E691-4393-A0C0-8AB4AD962700@gmail.com> Date: Sun, 08 Sep 2019 10:29:09 -0400 From: "Justin Kilpatrick" To: "Jonathan Morton" Cc: cake@lists.bufferbloat.net Content-Type: text/plain Subject: Re: [Cake] Fighting bloat in the face of uncertinty X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Sep 2019 14:29:31 -0000 > The second-order effect I mentioned is related to the 'target' > parameter. Checking the code, I am reminded that while Cake itself can > have 'target' set from userspace, there actually isn't a parameter to > the tc module which allows setting it independently of 'rtt'. But > there *is* a table in q_cake.c (in tc) which you can temporarily extend > with the following entries for experimentation: You are correct. My sampling was flawed and the 'metro' profile is not actually making any difference. the main contributor to bloat reduction was a bandwidth parameter left over from too much use of `tc qdisc change` rather than add/del. > You could also set it back to 'internet' and progressively reduce the > bandwidth parameter, making the Cake shaper into the actual bottleneck. > This is the correct fix for the problem, and you should notice an > instant improvement as soon as the bandwidth parameter is correct. Hand tuning this one link is not a problem. I'm searching for a set of settings that will provide generally good performance across a wide range of devices, links, and situations. >From what you've indicated so far there's nothing as effective as a correct bandwidth estimation if we consider the antenna (link) a black box. Expecting the user to input expected throughput for every link and then managing that information is essentially a non-starter. Radio tuning provides some improvement, but until ubiquiti starts shipping with Codel on non-router devices I don't think there's a good solution here. Any way to have the receiving device detect bloat and insert an ECN? I don't think the time spent in the intermediate device is detectable at the kernel level but we keep track of latency for routing decisions and could detect bloat with some accuracy, the problem is how to respond. -- Justin Kilpatrick justin@althea.net On Sat, Sep 7, 2019, at 8:59 PM, Jonathan Morton wrote: > > On 8 Sep, 2019, at 3:03 am, Justin Kilpatrick wrote: > > > > So you believe that setting the target RTT closer to the path latency was not the main contributor to reducing bloat? Is there a configuration I could use to demonstrate that one way or the other? > > The second-order effect I mentioned is related to the 'target' > parameter. Checking the code, I am reminded that while Cake itself can > have 'target' set from userspace, there actually isn't a parameter to > the tc module which allows setting it independently of 'rtt'. But > there *is* a table in q_cake.c (in tc) which you can temporarily extend > with the following entries for experimentation: > > static struct cake_preset presets[] = { > {"datacentre", 5, 100}, > {"lan", 50, 1000}, > {"metro", 500, 10000}, > {"regional", 1500, 30000}, > {"internet", 5000, 100000}, > {"oceanic", 15000, 300000}, > {"satellite", 50000, 1000000}, > {"interplanetary", 50000000, 1000000000}, > + > + {"metro-loose", 5000, 10000}, > + {"internet-tight", 500, 100000}, > }; > > If the effect is genuinely due to marking rate, then 'metro-loose' > should behave like 'metro' and 'internet-tight' should behave like > 'internet', to a first-order approximation. If, on the other hand, > it's due to the second-order interaction with CPU scheduling latency, > the reverse may be true. The latter is not something you should be > counting on, as it will insert random AQM marking even when the link is > not actually saturated. > > You could also set it back to 'internet' and progressively reduce the > bandwidth parameter, making the Cake shaper into the actual bottleneck. > This is the correct fix for the problem, and you should notice an > instant improvement as soon as the bandwidth parameter is correct. > > - Jonathan Morton