[Bloat] No backpressure "shaper"+AQM

[Dave Taht]

On Tue, Jul 24, 2018 at 2:39 PM Benjamin Cronce <bcronce at gmail.com> wrote:
>
>
>
> On Tue, Jul 24, 2018 at 3:57 PM Dave Taht <dave.taht at gmail.com> wrote:
>>
>> On Tue, Jul 24, 2018 at 1:11 PM Benjamin Cronce <bcronce at gmail.com> wrote:
>> >
>> > Maybe the Bobbie idea already would do this, but I did not see it explicitly mentioned on its wiki.
>>
>> you are basically correct below. bobbie's core idea was a kinder,
>> gentler, deficit mode policer, with something fq_codel-like aimed at
>> reducing accumulated bytes to below the set rate.
>>
>> this is way different from conventional policers
>>
>> > The below is all about shaping ingress, not egress.
>> >
>> > My issue is that my ISP already does a good job with their AQM but nothing is perfect and their implementation of rate limiting has a kind of burst at the start. According to speedtests, my 250Mb connection starts off around 500Mb/s and slowly drops over the next few seconds until a near perfect 250Mb/s steady state.
>>
>> ideally their htb shaper would use fq_codel as the underlying qdisc.
>> Or at least reduce their burst size to something saner. I hope it's
>> not a policer.
>>
>> You can usually "see" a policer in action. Long strings of packets are
>> dropped in a row.
>
>
> I feel as if this new configuration is not quite a policer as it feels much less abrupt as the old configuration. It used to have massive loss spikes that wrecked havoc on other flows and make the fat TCP flows have a kind of rebound. Their newer setup seems to be gentler. While there is an increased rate of loss as it attempt to "slowly" settle at the provisioned rate, it's not like the cliff it used to be, it actually has a slope.

well, ask 'em.

>>
>>
>> > The burst at the beginning adds a certain amount of destabilization to the TCP flows since the window quickly grows to 500Mb and then has to backdown by dropping. If I add my own traffic shaping and AQM, I can reduce the reported TCP retransmissions from ~3% to ~0.1%.
>>
>> sure.
>>
>>
>>
>>
>> >
>> > The problem that I'm getting is by adding my own shaping, a measurable amount of the benefit of their AQM is lost. While I am limited to Codel, HFSC+Codel, or FairQ+Codel for now, I am actually doing a worse job of anti-bufferbloat than my ISP is. Fewer latency spices according to DSLReports.
>>
>> ? measured how.
>
> Just looking visual at the DSLReport graphs, I more normally see maybe a few 40ms-150ms ping spikes, while my own attempts to shape can get me several 300ms spikes. I would really need a lot more samples and actually run the numbers on them, but just causally looking at them, I get the sense that mine is worse.

too gentle we are perhaps. out of cpu you may be.

>>
>>
>> >
>> > This also does not touch on that the act of adding back-pressure in its nature increases latency. I cannot say if it's a fundamental requirement in order to better my current situation, but I am curious if there's a better way. A thought that came to me is that like Bobbie, do a light touch as the packets have already made their way and you don't want to aggressively drop packets, but at the same time, I want the packets that already made the journey to mostly unhindered enter my network.
>> >
>> > That's when I thought of a backpressure-less AQM.
>>
>> I like the restating of what policers do....
>
> I think I need to look at the definition of a policer. I always through them as a strict cut-off. I'm not talking about mass dropping packets beyond a rate, just doing something like Codel where a packet here and there get dropped at an increasing rate until the observed rate normalizes.

bobs below the set rate long enough to drain the queue upstream.

>>
>>
>> >Instead of having backpressure and measuring sojourn time as a function of how long it takes packets to get scheduled, predict an estimated sojourn time based on the observed rate of flow, but allow packets to immediately vacate the queue. The AQM would either mark ECN or drop the packet, but never delay the packet.
>>
>> aqms don't delay packets. shapers do.
>
> My described "AQM" is not a shaper in that it does not schedule packets(possibly FIFO and at line rate), but does understand bandwidth. It neither delays packets nor has a strict cut-off. It essentially would allow packets to flow through at line rate, but if the "average" rate gets too high, it may decide to drop/mark the next packet. It might be described as bufferless shaping where the goal is to minimize packet-loss. Shaping purely by a gentle rate of increasing loss.

sure.

>
> Of course this whole thought may be total rubbish, but I figured I'd throw it out there.

not rubbish, could be better than policing.

>>
>>
>> > In summary, my ISP seems to have better latency with their AQM, but due to their shaper, loss during the burst is much higher than desirable.
>> >
>> > Maybe this will be mostly moot once I get fq_codel going on pfSense, but I do find it an interesting issue.
>>
>> I thought it's been in there for a while?
>
> Technically, but not practically. It should be easily available via the UI with 2.4.4 which is slowly nearing release.
>>
>>
>> > _______________________________________________
>> > Bloat mailing list
>> > Bloat at lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/bloat
>>
>>
>>
>> --
>>
>> Dave Täht
>> CEO, TekLibre, LLC
>> http://www.teklibre.com
>> Tel: 1-669-226-2619



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619