[Ecn-sane] [Bloat] when does the CoDel part of fq_codel help in the real world?

[Dave Taht]

On Mon, Nov 26, 2018 at 1:30 PM Jonathan Morton <chromatix99 at gmail.com> wrote:
>
> > On 26 Nov, 2018, at 9:08 pm, Pete Heist <pete at heistp.net> wrote:
> >
> > So I just thought to continue the discussion- when does the CoDel part of fq_codel actually help in the real world?
>
> Fundamentally, without Codel the only limits on the congestion window would be when the sender or receiver hit configured or calculated rwnd and cwnd limits (the rwnd is visible on the wire and usually chosen to be large enough to be a non-factor), or when the queue overflows.  Large windows require buffer memory in both sender and receiver, increasing costs on the sender in particular (who typically has many flows to manage per machine).

You can run devices out of memory more easily with our current ecn
implementations. I am seeing folk cut memory per instance to 256k in,
for example, the gluon project.

we end up dropping (which is better than the device crashing), and in
the fq_codel case, *bulk* head dropping. I see a bifurcation on the
data when we do this, and
I have a one line patch to the fq_codel bulk dropper that appears to
make things better when extremely memory constrained like this, but
haven't got around to fully evaluating it:

https://github.com/dtaht/fq_codel_fast/commit/a524fc2e39dc291199b9b04fb890ea1548f17641

would rather like more to try the memory limits we are seeing in the
field. 32MB (fq_codel default) is waaaay too much. 4MB is way too much
even for gbit, I think. 256k? well, given the choice between crashing
or not...

>
> Queue overflow tends to result in burst loss and head-of-line blocking in the receiver, which is visible to the user as a pause and subsequent jump in the progress of their download, accompanied by a major fluctuation in the estimated time to completion.  The lost packets also consume capacity upstream of the bottleneck which does not contribute to application throughput.  These effects are independent of whether overflow dropping occurs at the head or tail of the bottleneck queue, though recovery occurs more quickly (and fewer packets might be lost) if dropping occurs from the head of the queue.
>
> From a pure throughput-efficiency standpoint, Codel allows using ECN for congestion signalling instead of packet loss, potentially eliminating packet loss and associated lead-of-line blocking entirely.  Even without ECN, the actual cwnd is kept near the minimum necessary to satisfy the BDP of the path, reducing memory requirements and significantly shortening the recovery time of each loss cycle, to the point where the end-user may not notice that delivery is not perfectly smooth, and implementing accurate completion time estimators is considerably simplified.

I wish we had fractional cwnd below 1 and/or that pacing did not rely
on cwnd at all. too many flows at any rate you choose, can end up
marking 100% of packets, still not run you out of memory, and cause
delay.

>
> An important use-case is where two sequential bottlenecks exist on the path, the upstream one being only slightly higher capacity but lacking any queue management at all.  This is presently common in cases where home CPE implements inbound shaping on a generic ISP last-mile link.  In that case, without Codel running on the second bottleneck, traffic would collect in the first bottleneck's queue as well, greatly reducing the beneficial effects of FQ implemented on the second bottleneck.  In this topology, the overall effect is inter-flow as well as intra-flow.
>
> The combination of Codel with FQ is done in such a way that a separate instance of Codel is implemented for each flow.  This means that congestion signals are only sent to flows that require them, and non-saturating flows are unmolested.  This makes the combination synergistic, where each component offers an improvement to the behaviour of the other.

also a good bullet point somewhere!

>
>  - Jonathan Morton
>
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740