[Codel] slow start improvement

[Codel] slow start improvement

[Dave Taht]

I am very happy to be seeing various advances in slow start techniques.

https://www.researchgate.net/profile/Li-Lingang-2/publication/372708933_Small_Chunks_can_Talk_Fast_Bandwidth_Estimation_without_Filling_up_the_Bottleneck_Link/links/64d1a210806a9e4e5cf75162/Small-Chunks-can-Talk-Fast-Bandwidth-Estimation-without-Filling-up-the-Bottleneck-Link.pdf


-- 
40 years of net history, a couple songs:
https://www.youtube.com/watch?v=D9RGX6QFm5E
Dave Täht CSO, LibreQos

Re: [Codel] [Bloat] slow start improvement

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 2916 bytes --]

What I am missing in this and similar papres are tests what happens if the proposed scheme is actually used quantitatively over the internet...
The inherent idea seems to be if one would know the available capacity one could 'jump' the cwnd immediately to that window... (ignoring the fact the rwnd typically takes a while to increase accordingly*). My gut feeling tells me this will make dynamics at bottleneck queues even more volatile, not sure whether that will result in an overall better outcome.
te 
Sidenote: this is again a packet pair method with a side helping of "delay" increase measurements (inside the driver stack, so conceptually related to BQL/AQL) so the challenges are all the same.


*) Finally, the rwnd selection module is used to determine whether the value of receiver window (rwnd) embedded in the ACK packet should be ignored, according to the judgement whether it reveals the exhaustion of the receiver’s buffer, thus to remove the restriction of rwnd on slow start acceleration.
Erm, I think this paper should have been rejected on this argument alone... this is exactly the mind set (I know better then my communication partner) that results in a non- or sub-optimally working internet... I wish that those that do not appreciate slow-start would leave their fingers off it.
Not saying that slow-start is perfect, but if you ignore the components that make slow-start effective your replacement likely will not cut it. The fact that slow-strat gradually ramps up the cwin (and pretty aggressively) is one of its features and not a bug, as the alternative of jumping directly to the appropriate capacity for each flow requires an oracle... so a "perfect" solution is clearly out of reach and all we are talking about is different shades of "good enough" (and to repeat myself, whether a solution is good enough does not solely depend on whether the solution if implemented at a single end-node delivers "better" numbers for that end-node but also on its effect on the rest of the network).**

**) I occasionally wish for a tit-for-tat scheduler that is generous at first but will "retaliate" if a flow abuses that generosity...





On 28 December 2023 04:50:59 CET, Dave Taht via Bloat <bloat@lists.bufferbloat.net> wrote:
I am very happy to be seeing various advances in slow start techniques.

https://www.researchgate.net/profile/Li-Lingang-2/publication/372708933_Small_Chunks_can_Talk_Fast_Bandwidth_Estimation_without_Filling_up_the_Bottleneck_Link/links/64d1a210806a9e4e5cf75162/Small-Chunks-can-Talk-Fast-Bandwidth-Estimation-without-Filling-up-the-Bottleneck-Link.pdf <https://www.researchgate.net/profile/Li-Lingang-2/publication/372708933_Small_Chunks_can_Talk_Fast_Bandwidth_Estimation_without_Filling_up_the_Bottleneck_Link/links/64d1a210806a9e4e5cf75162/Small-Chunks-can-Talk-Fast-Bandwidth-Estimation-without-Filling-up-the-Bottleneck-Link.pdf>



[-- Attachment #2: Type: text/html, Size: 4009 bytes --]

Re: [Codel] [Bloat] slow start improvement

[Dave Taht]

In general my hope for the bufferbloat email list is to close the loop
between industry, open source, and academia. Academic authors (now
cc´d) have a tendency to not publish sources (?), and as the wait from
test to publication is so long, move onto other things, even if it is
a promising technique that could use further development and eyeballs.
Me, I wanted to know what wifi they tested for this, and do strongly
feel that slow start in the field is presently much larger than widely
recognised in academia coming from various cdns.

On Thu, Dec 28, 2023 at 5:17 AM Sebastian Moeller <moeller0@gmx.de> wrote:
>
> What I am missing in this and similar papres are tests what happens if the proposed scheme is actually used quantitatively over the internet...
> The inherent idea seems to be if one would know the available capacity one could 'jump' the cwnd immediately to that window... (ignoring the fact the rwnd typically takes a while to increase accordingly*). My gut feeling tells me this will make dynamics at bottleneck queues even more volatile, not sure whether that will result in an overall better outcome.
> te
> Sidenote: this is again a packet pair method with a side helping of "delay" increase measurements (inside the driver stack, so conceptually related to BQL/AQL) so the challenges are all the same.
>
>
> *) Finally, the rwnd selection module is used to determine whether the value of receiver window (rwnd) embedded in the ACK packet should be ignored, according to the judgement whether it reveals the exhaustion of the receiver’s buffer, thus to remove the restriction of rwnd on slow start acceleration.
> Erm, I think this paper should have been rejected on this argument alone... this is exactly the mind set (I know better then my communication partner) that results in a non- or sub-optimally working internet... I wish that those that do not appreciate slow-start would leave their fingers off it.
> Not saying that slow-start is perfect, but if you ignore the components that make slow-start effective your replacement likely will not cut it. The fact that slow-strat gradually ramps up the cwin (and pretty aggressively) is one of its features and not a bug, as the alternative of jumping directly to the appropriate capacity for each flow requires an oracle... so a "perfect" solution is clearly out of reach and all we are talking about is different shades of "good enough" (and to repeat myself, whether a solution is good enough does not solely depend on whether the solution if implemented at a single end-node delivers "better" numbers for that end-node but also on its effect on the rest of the network).**
>
> **) I occasionally wish for a tit-for-tat scheduler that is generous at first but will "retaliate" if a flow abuses that generosity...
>
>
>
>
>
> On 28 December 2023 04:50:59 CET, Dave Taht via Bloat <bloat@lists.bufferbloat.net> wrote:
>>
>> I am very happy to be seeing various advances in slow start techniques.
>>
>> https://www.researchgate.net/profile/Li-Lingang-2/publication/372708933_Small_Chunks_can_Talk_Fast_Bandwidth_Estimation_without_Filling_up_the_Bottleneck_Link/links/64d1a210806a9e4e5cf75162/Small-Chunks-can-Talk-Fast-Bandwidth-Estimation-without-Filling-up-the-Bottleneck-Link.pdf
>>
>>


-- 
40 years of net history, a couple songs:
https://www.youtube.com/watch?v=D9RGX6QFm5E
Dave Täht CSO, LibreQos

Re: [Codel] [Bloat] slow start improvement

[Dave Taht]

Thx for getting back to us.

A specific question regarding your AP, is what wifi drivers were on
it? Many (but not enough) have the fq_codel fq+AQM algorithm on it,
notably all the openwrt based mt76, mt79, ath9k, ath10k, I think
ath11k, and iwl, as described in this paper:
https://www.cs.kau.se/tohojo/airtime-fairness/

A feature of this is some ECN support, which if triggered is a sure
fire way to know you have overshot and can back down to 5ms or so
delay.

On Sun, Dec 31, 2023 at 4:33 AM 李林刚 <linganglee@126.com> wrote:
>
> Dear Täht:
>
>
>
> Thank you very much for your interest in our work and the valuable advices you gave!
>
>
>
> As described in Part IV.A of the paper, the WiFi in our experiment works in the 2.4GHz band (802.11b/g/n mixed mode with 40/20 MHz bandwidth). In fact, in order to verify that our work works even in lower throughput networks, we also set the WiFi AP to 802.11g only and 802.11b only modes respectively, and the experimental results show that the smaller the available bandwidth of the network, the weaker the acceleration effect of FBE is (due to space constraints we didn't include these data in the paper).
>
>
>
> Regarding your comment that FBE will make dynamics at bottleneck queues even more volatile, in our test results the bandwidth estimated by FBE does not deviate a lot from the real bandwidth and does not seriously disturb the bottleneck queue. In addition, existing congestion control algorithms (e.g. cubic and BBR) usually cause large queues when exiting the slow start phase, while FBE doesn't have a large impact on the queue compared to them.
>
>
>
> As you said, the rwnd selection module is not a good design, but we think that rwnd may could be revisited in the right context for existing receivers that usually have a large receive buffer if we want to speed up the startup process. Therefore, we used it as an advanced design in this paper, but of course this requires more detailed analysis and more careful selection, which is one of the improvements we can consider in the future.
>
>
>
> It is a very good suggestion to test if FBE also works in more network environments, but due to the lack of various test environments, we are mainly testing via WiFi and speedtest public servers right now. If the experimental conditions are available in the future, we will further evaluate FBE as well.
>
>
>
> We couldn't agree more with you that slow-start is very important. It is not that we feel slow-start is not good and we have to modify it, but we think that in the current network scenarios where the available bandwidth is getting larger and larger, slow-start may have room for improvements. Therefore, we come up with our design, and hope that our explorations may be beneficial to the future optimization of the slow start.
>
>
>
> Thank you again for your valuable comments, which have given us a deeper understanding of network optimization and for our future researches. Wishing you a happy new year!
>
>
>
> Sincerely,
>
>
>
> Li
>
>
>
>
>
> At 2023-12-28 21:37:54, "Dave Taht" <dave.taht@gmail.com> wrote:
> >In general my hope for the bufferbloat email list is to close the loop
> >between industry, open source, and academia. Academic authors (now
> >cc´d) have a tendency to not publish sources (?), and as the wait from
> >test to publication is so long, move onto other things, even if it is
> >a promising technique that could use further development and eyeballs.
> >Me, I wanted to know what wifi they tested for this, and do strongly
> >feel that slow start in the field is presently much larger than widely
> >recognised in academia coming from various cdns.
> >
> >On Thu, Dec 28, 2023 at 5:17 AM Sebastian Moeller <moeller0@gmx.de> wrote:
> >>
> >> What I am missing in this and similar papres are tests what happens if the proposed scheme is actually used quantitatively over the internet...
> >> The inherent idea seems to be if one would know the available capacity one could 'jump' the cwnd immediately to that window... (ignoring the fact the rwnd typically takes a while to increase accordingly*). My gut feeling tells me this will make dynamics at bottleneck queues even more volatile, not sure whether that will result in an overall better outcome.
> >> te
> >> Sidenote: this is again a packet pair method with a side helping of "delay" increase measurements (inside the driver stack, so conceptually related to BQL/AQL) so the challenges are all the same.
> >>
> >>
> >> *) Finally, the rwnd selection module is used to determine whether the value of receiver window (rwnd) embedded in the ACK packet should be ignored, according to the judgement whether it reveals the exhaustion of the receiver’s buffer, thus to remove the restriction of rwnd on slow start acceleration.
> >> Erm, I think this paper should have been rejected on this argument alone... this is exactly the mind set (I know better then my communication partner) that results in a non- or sub-optimally working internet... I wish that those that do not appreciate slow-start would leave their fingers off it.
> >> Not saying that slow-start is perfect, but if you ignore the components that make slow-start effective your replacement likely will not cut it. The fact that slow-strat gradually ramps up the cwin (and pretty aggressively) is one of its features and not a bug, as the alternative of jumping directly to the appropriate capacity for each flow requires an oracle... so a "perfect" solution is clearly out of reach and all we are talking about is different shades of "good enough" (and to repeat myself, whether a solution is good enough does not solely depend on whether the solution if implemented at a single end-node delivers "better" numbers for that end-node but also on its effect on the rest of the network).**
> >>
> >> **) I occasionally wish for a tit-for-tat scheduler that is generous at first but will "retaliate" if a flow abuses that generosity...
> >>
> >>
> >>
> >>
> >>
> >> On 28 December 2023 04:50:59 CET, Dave Taht via Bloat <bloat@lists.bufferbloat.net> wrote:
> >>>
> >>> I am very happy to be seeing various advances in slow start techniques.
> >>>
> >>> https://www.researchgate.net/profile/Li-Lingang-2/publication/372708933_Small_Chunks_can_Talk_Fast_Bandwidth_Estimation_without_Filling_up_the_Bottleneck_Link/links/64d1a210806a9e4e5cf75162/Small-Chunks-can-Talk-Fast-Bandwidth-Estimation-without-Filling-up-the-Bottleneck-Link.pdf
> >>>
> >>>
> >
> >
> >--
> >40 years of net history, a couple songs:
> >https://www.youtube.com/watch?v=D9RGX6QFm5E
> >Dave Täht CSO, LibreQos



-- 
40 years of net history, a couple songs:
https://www.youtube.com/watch?v=D9RGX6QFm5E
Dave Täht CSO, LibreQos

Re: [Codel] [Bloat] slow start improvement

[Jonathan Morton]

> On 28 Dec, 2023, at 12:17 pm, Sebastian Moeller via Bloat <bloat@lists.bufferbloat.net> wrote:
> 
> The inherent idea seems to be if one would know the available capacity one could 'jump' the cwnd immediately to that window... (ignoring the fact the rwnd typically takes a while to increase accordingly*). 

Yes, I've just got to the bit about selectively ignoring rwnd - that's a straight violation of TCP.  There may be scope for optimising congestion control in various ways, but rwnd is a fundamental part of the protocol that predates congestion control itself; it implements TCP's original function of "flow control".  Sending data outside the rwnd invites the receiver invoking RST, or even firewall action, which I can guarantee will have a material impact on flow completion time!

Slow-start already increases cwnd to match the BDP in at most 20 RTTs, and that's the extreme condition, starting from an IW of 1 segment and ramping up to the maximum possible window of 2^30 bytes (assuming an MSS of at least 1KB, which is usual).  The more recent standard of having IW=10 already shortens that by 3-4 RTTs.  It's an exponential process, so even quite large changes in available bandwidth don't affect the convergence time very much.  TCP's adaptation to changes in the BDP after slow-start is considerably slower, even with CUBIC.

I also note a lack of appreciation as to how HyStart (and HyStart++) works.  Their delay-sensitive criterion is triggered not when the cwnd exceeds the BDP, but at an earlier point when the packet bursts (issued at double the natural ack-clocked rate) cause a meaningful amount of temporary queue delay.  This queuing is normally drained almost immediately after it occurs, *precisely because* the cwnd has not yet reached the true path BDP.  This allows slow-start to transition to congestion-avoidance smoothly, without a multiplicative-decrease episode.  HyStart++ adds a further phase of exponential growth on a more cautious schedule, but with essentially the same principle in mind.

The irony is that they rely on precisely the same phenomenon of short-term queuing, but observe it in the form of the limited delivery rate of a burst, rather than an increase in delay on the later packets of the burst.

 - Jonathan Morton