From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-x235.google.com (mail-qk0-x235.google.com [IPv6:2607:f8b0:400d:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 0EB833B29E for ; Tue, 24 Jul 2018 17:58:37 -0400 (EDT) Received: by mail-qk0-x235.google.com with SMTP id t79-v6so3669148qke.4 for ; Tue, 24 Jul 2018 14:58:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=csss8UQcC9N05VF4OsdbcEReUlbm5qspBU88btSfi5U=; b=BTxxvryGZC5XIxBHPllztn+UttkqbENm540a/lPdP/wEfOM6OhyUNn+n5jbap1onqL tFT6h5sz2yy0C0+R3272D+FRFslzRnrfcf5L0lTynoCA3C8JGPm0P9yYlXgKMq5dUImq rglWVOnrIbaLQ5GwCmJV93zG3qVz6ZKD1+29iQkV+IYqCAnFf4u3XY7v0NtdEvtvO3/5 2gD4r9LWEJ1Sd9AX223fxAHxhGQCI8fE9JGlQdgW5YRR0eov+kwjmlM35d84Nhtd+pxV 3ZxkjDwR1MOaj6lOp+cNFV65IPuSSGyrKoDELE+PvXvbmlYU0XAI50dIG3oG45BTMKP8 FJpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=csss8UQcC9N05VF4OsdbcEReUlbm5qspBU88btSfi5U=; b=eln4nsowtWsIT47iE+TAbGFMzygg65hZgi1Y9L0tShmAjLXwNXs1KhDKQzuEJoE9Zo Lg2vYEPSFj2i7X4W11R/x66zU3pT9ehj2dNRiGeYu8SkSHLBLNl3HAPUzCWZduvKNoH+ TBCtR2xtytZt8uE28IYT0pQcTp5dzEwUgKMqvKPzl8pBjUzTJ0YA+gG/XvxTlxeGNvaf GCbMvcZPhKhfOEZQEnE93yR8y3REcLAHw7piIN8UG1cbAEbvlwqjQNIjXQvZZCRdvwGt f/VBWubexvdY3Rdi8upD/s1n56wbE+QGhBBYHsjCy/qH7yREeNcH+L4blsHick6+6dvM uZhQ== X-Gm-Message-State: AOUpUlEM3AMri2ep8DDBIZ8D4OIs/bncpsCIAM5EG/BNClt8GNoFowRy LohxKtLW80i6iqLvXPHMNhEV9Gla5f74r+O0wwQ= X-Google-Smtp-Source: AAOMgpcqBrh6gFzdRs/L4KUSvlOAW1xy2tRn+nvEXZy7EkbUXpUfzzRmYWsY/P/0lDWMXw8qp8pghU67Zd1wxW98LFw= X-Received: by 2002:a37:2121:: with SMTP id h33-v6mr17101916qkh.319.1532469516400; Tue, 24 Jul 2018 14:58:36 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Tue, 24 Jul 2018 14:58:21 -0700 Message-ID: To: Benjamin Cronce Cc: bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] No backpressure "shaper"+AQM X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jul 2018 21:58:37 -0000 On Tue, Jul 24, 2018 at 2:39 PM Benjamin Cronce wrote: > > > > On Tue, Jul 24, 2018 at 3:57 PM Dave Taht wrote: >> >> On Tue, Jul 24, 2018 at 1:11 PM Benjamin Cronce wrot= e: >> > >> > Maybe the Bobbie idea already would do this, but I did not see it expl= icitly mentioned on its wiki. >> >> you are basically correct below. bobbie's core idea was a kinder, >> gentler, deficit mode policer, with something fq_codel-like aimed at >> reducing accumulated bytes to below the set rate. >> >> this is way different from conventional policers >> >> > The below is all about shaping ingress, not egress. >> > >> > My issue is that my ISP already does a good job with their AQM but not= hing is perfect and their implementation of rate limiting has a kind of bur= st at the start. According to speedtests, my 250Mb connection starts off ar= ound 500Mb/s and slowly drops over the next few seconds until a near perfec= t 250Mb/s steady state. >> >> ideally their htb shaper would use fq_codel as the underlying qdisc. >> Or at least reduce their burst size to something saner. I hope it's >> not a policer. >> >> You can usually "see" a policer in action. Long strings of packets are >> dropped in a row. > > > I feel as if this new configuration is not quite a policer as it feels mu= ch less abrupt as the old configuration. It used to have massive loss spike= s that wrecked havoc on other flows and make the fat TCP flows have a kind = of rebound. Their newer setup seems to be gentler. While there is an increa= sed rate of loss as it attempt to "slowly" settle at the provisioned rate, = it's not like the cliff it used to be, it actually has a slope. well, ask 'em. >> >> >> > The burst at the beginning adds a certain amount of destabilization to= the TCP flows since the window quickly grows to 500Mb and then has to back= down by dropping. If I add my own traffic shaping and AQM, I can reduce the= reported TCP retransmissions from ~3% to ~0.1%. >> >> sure. >> >> >> >> >> > >> > The problem that I'm getting is by adding my own shaping, a measurable= amount of the benefit of their AQM is lost. While I am limited to Codel, H= FSC+Codel, or FairQ+Codel for now, I am actually doing a worse job of anti-= bufferbloat than my ISP is. Fewer latency spices according to DSLReports. >> >> ? measured how. > > Just looking visual at the DSLReport graphs, I more normally see maybe a = few 40ms-150ms ping spikes, while my own attempts to shape can get me sever= al 300ms spikes. I would really need a lot more samples and actually run th= e numbers on them, but just causally looking at them, I get the sense that = mine is worse. too gentle we are perhaps. out of cpu you may be. >> >> >> > >> > This also does not touch on that the act of adding back-pressure in it= s nature increases latency. I cannot say if it's a fundamental requirement = in order to better my current situation, but I am curious if there's a bett= er way. A thought that came to me is that like Bobbie, do a light touch as = the packets have already made their way and you don't want to aggressively = drop packets, but at the same time, I want the packets that already made th= e journey to mostly unhindered enter my network. >> > >> > That's when I thought of a backpressure-less AQM. >> >> I like the restating of what policers do.... > > I think I need to look at the definition of a policer. I always through t= hem as a strict cut-off. I'm not talking about mass dropping packets beyond= a rate, just doing something like Codel where a packet here and there get = dropped at an increasing rate until the observed rate normalizes. bobs below the set rate long enough to drain the queue upstream. >> >> >> >Instead of having backpressure and measuring sojourn time as a function= of how long it takes packets to get scheduled, predict an estimated sojour= n time based on the observed rate of flow, but allow packets to immediately= vacate the queue. The AQM would either mark ECN or drop the packet, but ne= ver delay the packet. >> >> aqms don't delay packets. shapers do. > > My described "AQM" is not a shaper in that it does not schedule packets(p= ossibly FIFO and at line rate), but does understand bandwidth. It neither d= elays packets nor has a strict cut-off. It essentially would allow packets = to flow through at line rate, but if the "average" rate gets too high, it m= ay decide to drop/mark the next packet. It might be described as bufferless= shaping where the goal is to minimize packet-loss. Shaping purely by a gen= tle rate of increasing loss. sure. > > Of course this whole thought may be total rubbish, but I figured I'd thro= w it out there. not rubbish, could be better than policing. >> >> >> > In summary, my ISP seems to have better latency with their AQM, but du= e to their shaper, loss during the burst is much higher than desirable. >> > >> > Maybe this will be mostly moot once I get fq_codel going on pfSense, b= ut I do find it an interesting issue. >> >> I thought it's been in there for a while? > > Technically, but not practically. It should be easily available via the U= I with 2.4.4 which is slowly nearing release. >> >> >> > _______________________________________________ >> > Bloat mailing list >> > Bloat@lists.bufferbloat.net >> > https://lists.bufferbloat.net/listinfo/bloat >> >> >> >> -- >> >> Dave T=C3=A4ht >> CEO, TekLibre, LLC >> http://www.teklibre.com >> Tel: 1-669-226-2619 --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619