[Bloat] slow start: small chunks can talk

[Bloat] slow start: small chunks can talk

[Dave Taht]

Promising approach:

https://ieeexplore.ieee.org/document/10188775
-- 
Podcast: https://www.youtube.com/watch?v=bxmoBr4cBKg
Dave Täht CSO, LibreQos

[Bloat] Another passive bandwidth estimation method

[Michael Menth]

Hi all,

we've recently developed a passive method for finding a link's capacity 
(in a different context). You find the algorithm in III.B.5 in
https://ieeexplore.ieee.org/document/9954450
The approach ist tested in V.B for 1, 10, and 100 Gb/s links on a Linux 
server and provides sufficiently accurate results for bandwidth 
utilizations of 25%. The method is likely to work also for lower 
utilizations, but this was not an issue in this work. The method is 
applicable only by a link's head-end node. It does not work for end 
systems to find the bottleneck bandwidth on some unknown intermediate 
node. However, it can deliver useful information for scheduling 
algorithms in forwarding nodes, which is the use case in this paper, and 
which may be of interest to some readers on this list.

Kind regards

Michael


Am 01.08.2023 um 00:36 schrieb Dave Taht via Bloat:
> Promising approach:
>
> https://ieeexplore.ieee.org/document/10188775

-- 
Prof. Dr. habil. Michael Menth
University of Tuebingen
Faculty of Science
Department of Computer Science
Chair of Communication Networks
Sand 13, 72076 Tuebingen, Germany
phone: (+49)-7071/29-70505
fax: (+49)-7071/29-5220
mailto:menth@uni-tuebingen.de
http://kn.inf.uni-tuebingen.de

Re: [Bloat] Another passive bandwidth estimation method

[Sebastian Moeller]

Hi Michael,

that "teaser" you wrote is certainly interesting. Would you be able to distribute author copies to those of us that do not subscribe to IEEExplore, please?

Regards
	Sebastian


> On Aug 1, 2023, at 09:32, Michael Menth via Bloat <bloat@lists.bufferbloat.net> wrote:
> 
> Hi all,
> 
> we've recently developed a passive method for finding a link's capacity (in a different context). You find the algorithm in III.B.5 in
> https://ieeexplore.ieee.org/document/9954450
> The approach ist tested in V.B for 1, 10, and 100 Gb/s links on a Linux server and provides sufficiently accurate results for bandwidth utilizations of 25%. The method is likely to work also for lower utilizations, but this was not an issue in this work. The method is applicable only by a link's head-end node. It does not work for end systems to find the bottleneck bandwidth on some unknown intermediate node. However, it can deliver useful information for scheduling algorithms in forwarding nodes, which is the use case in this paper, and which may be of interest to some readers on this list.
> 
> Kind regards
> 
> Michael
> 
> 
> Am 01.08.2023 um 00:36 schrieb Dave Taht via Bloat:
>> Promising approach:
>> 
>> https://ieeexplore.ieee.org/document/10188775
> 
> -- 
> Prof. Dr. habil. Michael Menth
> University of Tuebingen
> Faculty of Science
> Department of Computer Science
> Chair of Communication Networks
> Sand 13, 72076 Tuebingen, Germany
> phone: (+49)-7071/29-70505
> fax: (+49)-7071/29-5220
> mailto:menth@uni-tuebingen.de
> http://kn.inf.uni-tuebingen.de
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Another passive bandwidth estimation method

[Michael Menth]

Hi all,

here is a private copy for personal use:

http://atlas.cs.uni-tuebingen.de/menth/papers/Menth21-Sub-6-accepted.pdf

Kind regards

Michael


Am 01.08.2023 um 09:51 schrieb Sebastian Moeller:
> Hi Michael,
>
> that "teaser" you wrote is certainly interesting. Would you be able to distribute author copies to those of us that do not subscribe to IEEExplore, please?
>
> Regards
> 	Sebastian
>
>
>> On Aug 1, 2023, at 09:32, Michael Menth via Bloat <bloat@lists.bufferbloat.net> wrote:
>>
>> Hi all,
>>
>> we've recently developed a passive method for finding a link's capacity (in a different context). You find the algorithm in III.B.5 in
>> https://ieeexplore.ieee.org/document/9954450
>> The approach ist tested in V.B for 1, 10, and 100 Gb/s links on a Linux server and provides sufficiently accurate results for bandwidth utilizations of 25%. The method is likely to work also for lower utilizations, but this was not an issue in this work. The method is applicable only by a link's head-end node. It does not work for end systems to find the bottleneck bandwidth on some unknown intermediate node. However, it can deliver useful information for scheduling algorithms in forwarding nodes, which is the use case in this paper, and which may be of interest to some readers on this list.
>>
>> Kind regards
>>
>> Michael
>>
>>
>> Am 01.08.2023 um 00:36 schrieb Dave Taht via Bloat:
>>> Promising approach:
>>>
>>> https://ieeexplore.ieee.org/document/10188775
>> -- 
>> Prof. Dr. habil. Michael Menth
>> University of Tuebingen
>> Faculty of Science
>> Department of Computer Science
>> Chair of Communication Networks
>> Sand 13, 72076 Tuebingen, Germany
>> phone: (+49)-7071/29-70505
>> fax: (+49)-7071/29-5220
>> mailto:menth@uni-tuebingen.de
>> http://kn.inf.uni-tuebingen.de
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat

-- 
Prof. Dr. habil. Michael Menth
University of Tuebingen
Faculty of Science
Department of Computer Science
Chair of Communication Networks
Sand 13, 72076 Tuebingen, Germany
phone: (+49)-7071/29-70505
fax: (+49)-7071/29-5220
mailto:menth@uni-tuebingen.de
http://kn.inf.uni-tuebingen.de

Re: [Bloat] Another passive bandwidth estimation method

[Dave Taht]

On Tue, Aug 1, 2023 at 1:24 AM Michael Menth <menth@uni-tuebingen.de> wrote:
>
> Hi all,
>
> here is a private copy for personal use:
>
> http://atlas.cs.uni-tuebingen.de/menth/papers/Menth21-Sub-6-accepted.pdf

Very interesting, thank you. So great you have code on github!

I appreciate the mention of cake, and I figure it would scale worse
than fq-pie in the 100Gbit scenario you present, but do not know.
However, it does come with a built in forground (respecting EF, and a
few other diffserv codepoints), and a background service class,
respecting both CS1 and LE, and it would be interesting to know how
that differs from DSCD?

> Kind regards
>
> Michael
>
>
> Am 01.08.2023 um 09:51 schrieb Sebastian Moeller:
> > Hi Michael,
> >
> > that "teaser" you wrote is certainly interesting. Would you be able to distribute author copies to those of us that do not subscribe to IEEExplore, please?
> >
> > Regards
> >       Sebastian
> >
> >
> >> On Aug 1, 2023, at 09:32, Michael Menth via Bloat <bloat@lists.bufferbloat.net> wrote:
> >>
> >> Hi all,
> >>
> >> we've recently developed a passive method for finding a link's capacity (in a different context). You find the algorithm in III.B.5 in
> >> https://ieeexplore.ieee.org/document/9954450
> >> The approach ist tested in V.B for 1, 10, and 100 Gb/s links on a Linux server and provides sufficiently accurate results for bandwidth utilizations of 25%. The method is likely to work also for lower utilizations, but this was not an issue in this work. The method is applicable only by a link's head-end node. It does not work for end systems to find the bottleneck bandwidth on some unknown intermediate node. However, it can deliver useful information for scheduling algorithms in forwarding nodes, which is the use case in this paper, and which may be of interest to some readers on this list.
> >>
> >> Kind regards
> >>
> >> Michael
> >>
> >>
> >> Am 01.08.2023 um 00:36 schrieb Dave Taht via Bloat:
> >>> Promising approach:
> >>>
> >>> https://ieeexplore.ieee.org/document/10188775
> >> --
> >> Prof. Dr. habil. Michael Menth
> >> University of Tuebingen
> >> Faculty of Science
> >> Department of Computer Science
> >> Chair of Communication Networks
> >> Sand 13, 72076 Tuebingen, Germany
> >> phone: (+49)-7071/29-70505
> >> fax: (+49)-7071/29-5220
> >> mailto:menth@uni-tuebingen.de
> >> http://kn.inf.uni-tuebingen.de
> >>
> >> _______________________________________________
> >> Bloat mailing list
> >> Bloat@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/bloat
>
> --
> Prof. Dr. habil. Michael Menth
> University of Tuebingen
> Faculty of Science
> Department of Computer Science
> Chair of Communication Networks
> Sand 13, 72076 Tuebingen, Germany
> phone: (+49)-7071/29-70505
> fax: (+49)-7071/29-5220
> mailto:menth@uni-tuebingen.de
> http://kn.inf.uni-tuebingen.de
>


-- 
Podcast: https://www.youtube.com/watch?v=bxmoBr4cBKg
Dave Täht CSO, LibreQos

Re: [Bloat] Another passive bandwidth estimation method

[Michael Menth]

Hi Dave,

well, I don't see DSCD as a competitor with cake or fq-pie. DSCD is not 
primarily an aqm mechanism but its central objective is prioritizing ABE 
traffic without harming BE traffic. This idea is more than 20 years old, 
but scalable algorithms were not available, DSCD fills this gap. 
However, some implementation details of DSCD may be useful for other aqm 
mechanisms or scheduling mechanisms, e.g., when it comes to rate 
calculation etc.

Kind regards

Michael

Am 01.08.2023 um 10:37 schrieb Dave Taht:
> On Tue, Aug 1, 2023 at 1:24 AM Michael Menth <menth@uni-tuebingen.de> wrote:
>> Hi all,
>>
>> here is a private copy for personal use:
>>
>> http://atlas.cs.uni-tuebingen.de/menth/papers/Menth21-Sub-6-accepted.pdf
> Very interesting, thank you. So great you have code on github!
>
> I appreciate the mention of cake, and I figure it would scale worse
> than fq-pie in the 100Gbit scenario you present, but do not know.
> However, it does come with a built in forground (respecting EF, and a
> few other diffserv codepoints), and a background service class,
> respecting both CS1 and LE, and it would be interesting to know how
> that differs from DSCD?
>
>> Kind regards
>>
>> Michael
>>
>>
>> Am 01.08.2023 um 09:51 schrieb Sebastian Moeller:
>>> Hi Michael,
>>>
>>> that "teaser" you wrote is certainly interesting. Would you be able to distribute author copies to those of us that do not subscribe to IEEExplore, please?
>>>
>>> Regards
>>>        Sebastian
>>>
>>>
>>>> On Aug 1, 2023, at 09:32, Michael Menth via Bloat <bloat@lists.bufferbloat.net> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> we've recently developed a passive method for finding a link's capacity (in a different context). You find the algorithm in III.B.5 in
>>>> https://ieeexplore.ieee.org/document/9954450
>>>> The approach ist tested in V.B for 1, 10, and 100 Gb/s links on a Linux server and provides sufficiently accurate results for bandwidth utilizations of 25%. The method is likely to work also for lower utilizations, but this was not an issue in this work. The method is applicable only by a link's head-end node. It does not work for end systems to find the bottleneck bandwidth on some unknown intermediate node. However, it can deliver useful information for scheduling algorithms in forwarding nodes, which is the use case in this paper, and which may be of interest to some readers on this list.
>>>>
>>>> Kind regards
>>>>
>>>> Michael
>>>>
>>>>
>>>> Am 01.08.2023 um 00:36 schrieb Dave Taht via Bloat:
>>>>> Promising approach:
>>>>>
>>>>> https://ieeexplore.ieee.org/document/10188775
>>>> --
>>>> Prof. Dr. habil. Michael Menth
>>>> University of Tuebingen
>>>> Faculty of Science
>>>> Department of Computer Science
>>>> Chair of Communication Networks
>>>> Sand 13, 72076 Tuebingen, Germany
>>>> phone: (+49)-7071/29-70505
>>>> fax: (+49)-7071/29-5220
>>>> mailto:menth@uni-tuebingen.de
>>>> http://kn.inf.uni-tuebingen.de
>>>>
>>>> _______________________________________________
>>>> Bloat mailing list
>>>> Bloat@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat
>> --
>> Prof. Dr. habil. Michael Menth
>> University of Tuebingen
>> Faculty of Science
>> Department of Computer Science
>> Chair of Communication Networks
>> Sand 13, 72076 Tuebingen, Germany
>> phone: (+49)-7071/29-70505
>> fax: (+49)-7071/29-5220
>> mailto:menth@uni-tuebingen.de
>> http://kn.inf.uni-tuebingen.de
>>
>
-- 
Prof. Dr. habil. Michael Menth
University of Tuebingen
Faculty of Science
Department of Computer Science
Chair of Communication Networks
Sand 13, 72076 Tuebingen, Germany
phone: (+49)-7071/29-70505
fax: (+49)-7071/29-5220
mailto:menth@uni-tuebingen.de
http://kn.inf.uni-tuebingen.de

Re: [Bloat] slow start: small chunks can talk

[Bless, Roland (TM)]

Hi Dave,

On 01.08.23 at 00:36 Dave Taht via Bloat wrote:
> Promising approach:
> 
> https://ieeexplore.ieee.org/document/10188775

It's a pity that neither the authors nor the reviewers were aware of 
earlier related work that you also sent here to the list:

L. Guo and J. Y. B. Lee, "TCP-FLASH - A Fast Reacting TCP for Modern 
Networks," in IEEE Access, vol. 9, pp. 68861-68879, 2021, doi: 
10.1109/ACCESS.2021.3077612.

The fast launch phase and especially the CWnd-Compensated Bandwidth 
Estimator (CCBE) are quite similar and a comparison of both approaches
would have been interesting...

Regards,
  Roland

Re: [Bloat] slow start: small chunks can talk

[Sebastian Moeller]

Hi Roland.

> On Aug 7, 2023, at 10:48, Bless, Roland (TM) via Bloat <bloat@lists.bufferbloat.net> wrote:
> 
> Hi Dave,
> 
> On 01.08.23 at 00:36 Dave Taht via Bloat wrote:
>> Promising approach:
>> https://ieeexplore.ieee.org/document/10188775
> 
> It's a pity that neither the authors nor the reviewers were aware of earlier related work that you also sent here to the list:
> 
> L. Guo and J. Y. B. Lee, "TCP-FLASH - A Fast Reacting TCP for Modern Networks," in IEEE Access, vol. 9, pp. 68861-68879, 2021, doi: 10.1109/ACCESS.2021.3077612.

	Interesting link. This still uses essentially a packet-pair measuring method (send packet back2back, look at time difference in resulting ACK packets to estimate bottleneck bandwidth in the forward direction). These are still not robust or reliable... (parallel path, reordering, or simply congestion on the reverse path, ACK filtering, ACK compression by GRO on the receiver end, ...). Now, well possible that packet-pair data, while not perfect, might still be good enough for the intended purpose... And even for a traditional slow-start having an educated guess when to leave the exponential growth phase could be helpful...

	More over, let's assume any of these (let's short circuit the probing phase) will actually be deployed at scale; how will the network cope with the much more aggressive ramp-up of such flows (essentially initial-window as one batch the switch to estimated capacity once the bottleneck rate is estimated). It is fun to see a single/few of such flows do the right thing, but what about having the majority of flows use such methods? (Which will likely invalidate the bottleneck rate estimate quickly).

	I guess I might be too cautious here, but I personally see the exponential ramp-up in more traditional slow-start already as pretty aggressive and yet more adaptive to changing capacity than jumping to the estimated capacity essentially cold (I have similar hesitation with the careful resume internet draft).

Regards
	Sebastian

P.S.: "To tackle this problem, FLASH is designed to suppress
the AWnd constraint unless it is zero. This opportunistic
transmission technique, first proposed by Liu and Lee [6],
allows FLASH to send data beyond the AWnd limit up to
CWnd amount of packets inflight. "

This is quite an euphemism "opportunistic transmission technique" for simply ignoring parts of the protocol...

P.P.S.: Dave's link is behind a pay-wall, so I have no idea about that paper beyond the abstract... but if they also relay on packet-pair bandwidth estimates and an faster-than-slow-start ramp-up, I would expect similar issues.



> 
> The fast launch phase and especially the CWnd-Compensated Bandwidth Estimator (CCBE) are quite similar and a comparison of both approaches
> would have been interesting...
> 
> Regards,
> Roland
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] slow start: small chunks can talk

[Dave Taht]

In general these papers seem quite close to a couple ideas I have been
working on for a few years, and getting close to publication on. I do
not know what to do about it. Soliciting input here:

https://www.linkedin.com/feed/update/urn:li:activity:7092544418255171585/

On Mon, Aug 7, 2023 at 3:18 AM Sebastian Moeller <moeller0@gmx.de> wrote:
>
> Hi Roland.
>
> > On Aug 7, 2023, at 10:48, Bless, Roland (TM) via Bloat <bloat@lists.bufferbloat.net> wrote:
> >
> > Hi Dave,
> >
> > On 01.08.23 at 00:36 Dave Taht via Bloat wrote:
> >> Promising approach:
> >> https://ieeexplore.ieee.org/document/10188775
> >
> > It's a pity that neither the authors nor the reviewers were aware of earlier related work that you also sent here to the list:
> >
> > L. Guo and J. Y. B. Lee, "TCP-FLASH - A Fast Reacting TCP for Modern Networks," in IEEE Access, vol. 9, pp. 68861-68879, 2021, doi: 10.1109/ACCESS.2021.3077612.
>
>         Interesting link. This still uses essentially a packet-pair measuring method (send packet back2back, look at time difference in resulting ACK packets to estimate bottleneck bandwidth in the forward direction). These are still not robust or reliable... (parallel path, reordering, or simply congestion on the reverse path, ACK filtering, ACK compression by GRO on the receiver end, ...). Now, well possible that packet-pair data, while not perfect, might still be good enough for the intended purpose... And even for a traditional slow-start having an educated guess when to leave the exponential growth phase could be helpful...
>
>         More over, let's assume any of these (let's short circuit the probing phase) will actually be deployed at scale; how will the network cope with the much more aggressive ramp-up of such flows (essentially initial-window as one batch the switch to estimated capacity once the bottleneck rate is estimated). It is fun to see a single/few of such flows do the right thing, but what about having the majority of flows use such methods? (Which will likely invalidate the bottleneck rate estimate quickly).
>
>         I guess I might be too cautious here, but I personally see the exponential ramp-up in more traditional slow-start already as pretty aggressive and yet more adaptive to changing capacity than jumping to the estimated capacity essentially cold (I have similar hesitation with the careful resume internet draft).
>
> Regards
>         Sebastian
>
> P.S.: "To tackle this problem, FLASH is designed to suppress
> the AWnd constraint unless it is zero. This opportunistic
> transmission technique, first proposed by Liu and Lee [6],
> allows FLASH to send data beyond the AWnd limit up to
> CWnd amount of packets inflight. "
>
> This is quite an euphemism "opportunistic transmission technique" for simply ignoring parts of the protocol...
>
> P.P.S.: Dave's link is behind a pay-wall, so I have no idea about that paper beyond the abstract... but if they also relay on packet-pair bandwidth estimates and an faster-than-slow-start ramp-up, I would expect similar issues.
>
>
>
> >
> > The fast launch phase and especially the CWnd-Compensated Bandwidth Estimator (CCBE) are quite similar and a comparison of both approaches
> > would have been interesting...
> >
> > Regards,
> > Roland
> >
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>


-- 
Podcast: https://www.youtube.com/watch?v=bxmoBr4cBKg
Dave Täht CSO, LibreQos