From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 27B193B29E for ; Tue, 27 Nov 2018 04:24:18 -0500 (EST) Received: by mail-qk1-x72c.google.com with SMTP id y16so14109972qki.7 for ; Tue, 27 Nov 2018 01:24:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Yw/CaTmwgxZdaqLGpSrc8yN1hwPyFP1SP/IKnURQyFs=; b=k4fBPBsCgKaxzc6GeP49CnPHE/W1apx9U8nsIlBqhuKZuI9pEc7RzAD8848vebuCsC G5pJGkG1fqsE+ojZUQo13fu1PAiZckH0cDqEzIELzRcRha6R9S7zOkDwbN+1ttE6845U dnsgfbhfIvYdyzw1EukgE53IyuPTyqo33lz6u+IFcqEEI8Q0Cwsbo6IDQY06zDvAOuQn c5ppB0QZYMWQmBK2B3BaV6k0JHTOdjr3OjZTysOkYKlbyyfXYwr4KlU1HkoiqMXY9qud eIymH87jGEk+JtJlN8zGUSGz4ilMQsHwZlJZ1CCWPLICCwzk34kL5amLmvVPulGFZBDC 3zog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Yw/CaTmwgxZdaqLGpSrc8yN1hwPyFP1SP/IKnURQyFs=; b=oxmeRJ85EiNsqM2cnS9xDGLs+1Lzw9GJTJ0C+x8jiRgyuhjArlpDrp7nmQEEyEaskx XC82wEMG+jc3fMfz09m6gXOij5gthnaqnn28nxxjoLrG4k22oZHeBtRH/X/GbqOrQWMT wzUuXY784M/SKNp49M3whww5mLWiqarjQHfEf0GAWz6AZfqF+1VzPvF9eoHk3Kzdti4S eBVvGc3mY0ZgIKANghBhgBJBUgHd2q4rwgKsXLFCL+DU2JtO6nS62T+ri6JDSrBwVsL3 25+5i2S2pi7wzinc+IpzXYs1AaGQHlhTyRNBPnyhFAHoIb6KW149WhL5y6nCSN6rPeEi Rrng== X-Gm-Message-State: AA+aEWYI6AQql6BU3M+ZMjh39XgOj9yiCxkzoA1yj+Y8rHDbMTU2PlQF /gFHtxaPZAfNylUrXOeGR/WPgKt/FI+PctAVnQ8= X-Google-Smtp-Source: AFSGD/VhvYdk9SuR2hUSYbx2kvC2EA6lM9knQDS16s4WmcKU/55EfyCGtpC3GCOKMdCCeOmxKZHimvQSmCFtcYAM4Xw= X-Received: by 2002:a37:5a05:: with SMTP id o5mr28422304qkb.126.1543310657515; Tue, 27 Nov 2018 01:24:17 -0800 (PST) MIME-Version: 1.0 References: <65EAC6C1-4688-46B6-A575-A6C7F2C066C5@heistp.net> In-Reply-To: From: Luca Muscariello Date: Tue, 27 Nov 2018 10:24:06 +0100 Message-ID: To: Jonathan Morton Cc: pete@heistp.net, bloat Content-Type: multipart/alternative; boundary="00000000000084f5b6057ba203a3" Subject: Re: [Bloat] when does the CoDel part of fq_codel help in the real world? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Nov 2018 09:24:18 -0000 --00000000000084f5b6057ba203a3 Content-Type: text/plain; charset="UTF-8" I think that this is a very good comment to the discussion at the defense about the comparison between SFQ with longest queue drop and FQ_Codel. A congestion controlled protocol such as TCP or others, including QUIC, LEDBAT and so on need at least the BDP in the transmission queue to get full link efficiency, i.e. the queue never empties out. This gives rule of thumbs to size buffers which is also very practical and thanks to flow isolation becomes very accurate. Which is: 1) find a way to keep the number of backlogged flows at a reasonable value. This largely depends on the minimum fair rate an application may need in the long term. We discussed a little bit of available mechanisms to achieve that in the literature. 2) fix the largest RTT you want to serve at full utilization and size the buffer using BDP * N_backlogged. Or the other way round: check how much memory you can use in the router/line card/device and for a fixed N, compute the largest RTT you can serve at full utilization. 3) there is still some memory to dimension for sparse flows in addition to that, but this is not based on BDP. It is just enough to compute the total utilization of sparse flows and use the same simple model Toke has used to compute the (de)prioritization probability. This procedure would allow to size FQ_codel but also SFQ. It would be interesting to compare the two under this buffer sizing. It would also be interesting to compare another mechanism that we have mentioned during the defense which is AFD + a sparse flow queue. Which is, BTW, already available in Cisco nexus switches for data centres. I think that the the codel part would still provide the ECN feature, that all the others cannot have. However the others, the last one especially can be implemented in silicon with reasonable cost. On Mon 26 Nov 2018 at 22:30, Jonathan Morton wrote: > > On 26 Nov, 2018, at 9:08 pm, Pete Heist wrote: > > > > So I just thought to continue the discussion- when does the CoDel part > of fq_codel actually help in the real world? > > Fundamentally, without Codel the only limits on the congestion window > would be when the sender or receiver hit configured or calculated rwnd and > cwnd limits (the rwnd is visible on the wire and usually chosen to be large > enough to be a non-factor), or when the queue overflows. Large windows > require buffer memory in both sender and receiver, increasing costs on the > sender in particular (who typically has many flows to manage per machine). > > Queue overflow tends to result in burst loss and head-of-line blocking in > the receiver, which is visible to the user as a pause and subsequent jump > in the progress of their download, accompanied by a major fluctuation in > the estimated time to completion. The lost packets also consume capacity > upstream of the bottleneck which does not contribute to application > throughput. These effects are independent of whether overflow dropping > occurs at the head or tail of the bottleneck queue, though recovery occurs > more quickly (and fewer packets might be lost) if dropping occurs from the > head of the queue. > > From a pure throughput-efficiency standpoint, Codel allows using ECN for > congestion signalling instead of packet loss, potentially eliminating > packet loss and associated lead-of-line blocking entirely. Even without > ECN, the actual cwnd is kept near the minimum necessary to satisfy the BDP > of the path, reducing memory requirements and significantly shortening the > recovery time of each loss cycle, to the point where the end-user may not > notice that delivery is not perfectly smooth, and implementing accurate > completion time estimators is considerably simplified. > > An important use-case is where two sequential bottlenecks exist on the > path, the upstream one being only slightly higher capacity but lacking any > queue management at all. This is presently common in cases where home CPE > implements inbound shaping on a generic ISP last-mile link. In that case, > without Codel running on the second bottleneck, traffic would collect in > the first bottleneck's queue as well, greatly reducing the beneficial > effects of FQ implemented on the second bottleneck. In this topology, the > overall effect is inter-flow as well as intra-flow. > > The combination of Codel with FQ is done in such a way that a separate > instance of Codel is implemented for each flow. This means that congestion > signals are only sent to flows that require them, and non-saturating flows > are unmolested. This makes the combination synergistic, where each > component offers an improvement to the behaviour of the other. > > - Jonathan Morton > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --00000000000084f5b6057ba203a3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I think that this is a very good co= mment to the discussion at the defense about the comparison between=C2=A0
SFQ with longest queue drop and FQ_Codel.
=

A congestion controlled proto= col such as TCP or others, including QUIC, LEDBAT and so on
need at least the BDP in the transmission queue to get full link = efficiency, i.e. the queue never empties out.
This g= ives rule of thumbs to size buffers which is also very practical and thanks= to flow isolation becomes very accurate.

=
Which is:=C2=A0

1) find a way to keep the number of backlogged flows at a reasona= ble value.=C2=A0
This largely depends on the minimum= fair rate an application may need in the long term.
We discussed a little bit of available mechanisms to achieve that in the l= iterature.

2) fix the la= rgest RTT you want to serve at full utilization and size the buffer using B= DP * N_backlogged.=C2=A0=C2=A0
Or the other way roun= d: check how much memory you can use=C2=A0
in the ro= uter/line card/device and for a fixed N, compute the largest RTT you can se= rve at full utilization.=C2=A0

3) there is still some memory to dimension for sparse flows in addit= ion to that, but this is not based on BDP.=C2=A0
It = is just enough to compute the total utilization of sparse flows and use the= same simple model Toke has used=C2=A0
to compute th= e (de)prioritization probability.

This procedure would allow to size FQ_codel but also SFQ.
It would be interesting to compare the two under this buffer sizing.=C2= =A0
It would also be interesting to compare another mechanism tha= t we have mentioned during the defense
which is AFD=C2=A0+ a spar= se flow queue. Which is, BTW, already available in Cisco nexus switches for= data centres.

I think that the the codel part wou= ld still provide the ECN feature, that all the others cannot have.
However the others, the last one especially can be implemented in silicon= with reasonable cost.





On Mon 26 Nov 2018 at 22:30, Jonathan Morton <chromatix99@gmail.com> wr= ote:
> On 26 Nov, 2018, at 9:08 = pm, Pete Heist <pet= e@heistp.net> wrote:
>
> So I just thought to continue the discussion- when does the CoDel part= of fq_codel actually help in the real world?

Fundamentally, without Codel the only limits on the congestion window would= be when the sender or receiver hit configured or calculated rwnd and cwnd = limits (the rwnd is visible on the wire and usually chosen to be large enou= gh to be a non-factor), or when the queue overflows.=C2=A0 Large windows re= quire buffer memory in both sender and receiver, increasing costs on the se= nder in particular (who typically has many flows to manage per machine).
Queue overflow tends to result in burst loss and head-of-line blocking in t= he receiver, which is visible to the user as a pause and subsequent jump in= the progress of their download, accompanied by a major fluctuation in the = estimated time to completion.=C2=A0 The lost packets also consume capacity = upstream of the bottleneck which does not contribute to application through= put.=C2=A0 These effects are independent of whether overflow dropping occur= s at the head or tail of the bottleneck queue, though recovery occurs more = quickly (and fewer packets might be lost) if dropping occurs from the head = of the queue.

>From a pure throughput-efficiency standpoint, Codel allows using ECN for co= ngestion signalling instead of packet loss, potentially eliminating packet = loss and associated lead-of-line blocking entirely.=C2=A0 Even without ECN,= the actual cwnd is kept near the minimum necessary to satisfy the BDP of t= he path, reducing memory requirements and significantly shortening the reco= very time of each loss cycle, to the point where the end-user may not notic= e that delivery is not perfectly smooth, and implementing accurate completi= on time estimators is considerably simplified.

An important use-case is where two sequential bottlenecks exist on the path= , the upstream one being only slightly higher capacity but lacking any queu= e management at all.=C2=A0 This is presently common in cases where home CPE= implements inbound shaping on a generic ISP last-mile link.=C2=A0 In that = case, without Codel running on the second bottleneck, traffic would collect= in the first bottleneck's queue as well, greatly reducing the benefici= al effects of FQ implemented on the second bottleneck.=C2=A0 In this topolo= gy, the overall effect is inter-flow as well as intra-flow.

The combination of Codel with FQ is done in such a way that a separate inst= ance of Codel is implemented for each flow.=C2=A0 This means that congestio= n signals are only sent to flows that require them, and non-saturating flow= s are unmolested.=C2=A0 This makes the combination synergistic, where each = component offers an improvement to the behaviour of the other.

=C2=A0- Jonathan Morton

_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat
--00000000000084f5b6057ba203a3--