[Cake] bbr vs all the aqms, cake winning...

[Cake] bbr vs all the aqms, cake winning...

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 151 bytes --]

Lots of flent, too:

https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0304609&type=printable

-- 
Dave Täht CSO, LibreQos

[-- Attachment #2: Type: text/html, Size: 512 bytes --]

Re: [Cake] bbr vs all the aqms, cake winning...

[David P. Reed]

Interesting. Worth diving into.

Here's my reaction, though:

1. "Winning" (in this post title): there's no clear win here at all. Why?
   A. Very narrow tests, not applicable to real world situations. That's not a bad approach to a research project - but you can't generalize outside the VERY SPECIFIC test setup (a ping vs. one full-on TCP stream, no competing network activity from other OS's and stacks). If I built a car that won the world speed record on the Bonneville Flats test, is my winning car even relevant to driving in LA freeways or Manhattan streets or even Montana back roads? Not bloody likely. The traffic from modern web browsers in a lecture hall on a typical college campus, including smartphones, has a serious congestion problem to solve. Nobody even knows how to study any of the algorithms suitability here. This is terrible to assert a "win" for.

2. Google is actively pushing QUIC to replace all of TCP, and its congestion management is not even implemented much less tested - and BBR isn't relevant to QUIC at all. Similarly, there's more and more WebRTC traffic and RTP traffic - many SMB's are seeing very high percentages of such traffic at their interconnect point. Why no consideration of that at all? These guys are living in an ancient past - networking as it was 2 decades ago. Again, to do simple, controlled research, that's OK. But dammit, why is the research community focusing in their 20 year rearview mirror?

3. The problem of interoperability and compatibility of technologies in the real Internet still has not been addressed. My buddy Dave Clark said, about when RED was being discussed, that the interaction between different flows should at least start with "TCP compatibility" - by which he meant whatever a flow or (whatever QUIC does, which is lots of UDP pairs at once) does to adapt under congestion, it should NOT damage "ordinary TCP". That was in an IRTF meeting, I don't think Dave ever recorded that observation. But it is a MINIMUM requirement. Now there are others. As "diffserv" keeps being bruited about (and I personally still think the idea is totally broken for a dozen technical reasons), there isn't one "diffserv", but the meaning of diffserv marks operationally (queue handling, routing paths, ...) is NOT interoperable at all - and the last time Dave tried to get a number of major Internet Access Providers to try to define interoperability for any subset of diffserv codepoints, they made 0 progress for a number of reasons, and they tried. Yet, this community of congestion control researchers continue to act like "diffserv" is an answer to some question.

The Internet is evolving rapidly, at the edges using "running code and rarely testing before deployment". That can only work if all the implementers cooperate, share research and address interoperability and compatibility.

I know there are no research funds available, unless you can use GPT or some other "Generative AI" name in the title. Hell, I bet Comcast will even take its own research budget and give it over to some projects with AI in the name and GPUs in the hardware. (Jason Livingood, I feel sorry for you, but I'd recommend turning your attention to optimizing audio chat services research rather than congestion, if you want to save your job).

[I'm actively working on using Machine Learning in real-time systems right now, advising a startup, but I don't think compatibility, interoperability, evolution of the Internet protocols will benefit from any of that - if anything, it will just make the congestion, interoperability, compatibility problems harder to solve and more important than ever.]

So I really care about the research directions here. And while this paper isn't bad, the concerns I have above really need to be recognized, because 5 years from now, the game won't be the game that the title of this post says has been "won".
(we aren't gonna be racing in Bonneville Salt Flats).




On Thursday, September 26, 2024 18:21, "Dave Taht via Cake" <cake@lists.bufferbloat.net> said:

> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
> Lots of flent, too:
> 
> https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0304609&type=printable
> 
> --
> Dave Täht CSO, LibreQos
> 

Re: [Cake] bbr vs all the aqms, cake winning...

[David Lang]

David P. Reed wrote:

> 2. Google is actively pushing QUIC to replace all of TCP, and its congestion 
> management is not even implemented much less tested - and BBR isn't relevant 
> to QUIC at all. Similarly, there's more and more WebRTC traffic and RTP 
> traffic - many SMB's are seeing very high percentages of such traffic at their 
> interconnect point. Why no consideration of that at all? These guys are living 
> in an ancient past - networking as it was 2 decades ago. Again, to do simple, 
> controlled research, that's OK. But dammit, why is the research community 
> focusing in their 20 year rearview mirror?

because they don't have real-world experience to know the difference between 
networking as it is taught (and how it may exist on a simple network with well 
behaved traffic) and the wild-wild-west of the public web that desktop/mobile 
users experience.

> I know there are no research funds available, unless you can use GPT or some 
> other "Generative AI" name in the title. Hell, I bet Comcast will even take 
> its own research budget and give it over to some projects with AI in the name 
> and GPUs in the hardware. (Jason Livingood, I feel sorry for you, but I'd 
> recommend turning your attention to optimizing audio chat services research 
> rather than congestion, if you want to save your job).

could the community try and produce 'traffic simulators' that implement these 
various protocols with a more realistic traffic pattern? something that can be 
turned up or down with a few presets of the mix that we can make available for 
the academics to use for their testing?

David Lang

Re: [Cake] bbr vs all the aqms, cake winning...

[David P. Reed]

On Friday, September 27, 2024 17:43, "David Lang" <david@lang.hm> said:

> 
> could the community try and produce 'traffic simulators' that implement these
> various protocols with a more realistic traffic pattern? something that can be
> turned up or down with a few presets of the mix that we can make available for
> the academics to use for their testing?

It's a pretty good idea.

However, some things I think about...it's important to remember that just recording traffic streams and playing them back filters out almost all of the "control" dynamics.
In the real Internet. It's the control feedback that causes instability and bufferbloat - everything is feedback system among a large set of endpoints - typically hundreds of them at minimum. It's hard for those who think from a perspective of one "server", or from one "client", or between a "pair" to realize that networks aren't like that. Every "independent" but concurrent activity feeds back significant "noise" into the other activities, changing latency in every protocol, etc.

Even the dynamic behavior of a single router out-queue affects everything, especially, as you know, the queues upstream and downstream.

THat's one reason that I find any work that tries to find ways to store packets in router queues ("background service") to be dubious, because that's one of the things that always increases dynamic instability. Always is my belief based on much experience and experimentation - there may even be a theorem there, but it's tricky to find the premises of such theorem.

However, this suggests a "stress simulator" that might help create realistic experiments - instead of simulating traffic, simulate instabilities in routers by injecting packets into queues for mix of flows based on some probabilistic model that acts like "real Internet". These injected packets might, for example, have invalid checksums on purpose to avoid harming any endpoint stacks. So such a tool would be a lot easier to deploy.

RRUL isn't typical, but it at least is a simple setup that pushes certain AQM challenges into a real bottleneck queue, if there is one bottleneck. Simulating, say, a highly optimized "single page web site" programmed in JavaScript with some rich UI packages, even in the single bottleneck case with multiple edge smartphones running other apps (like Zoom and streaming audio) - one can't really tell if that covers any other cases of interest.

> 
> David Lang
> 

Re: [Cake] bbr vs all the aqms, cake winning...

[David Lang]

On Sat, 28 Sep 2024, David P. Reed wrote:

> On Friday, September 27, 2024 17:43, "David Lang" <david@lang.hm> said:
>
>> 
>> could the community try and produce 'traffic simulators' that implement these
>> various protocols with a more realistic traffic pattern? something that can be
>> turned up or down with a few presets of the mix that we can make available for
>> the academics to use for their testing?
>
> It's a pretty good idea.
>
> However, some things I think about...it's important to remember that just recording traffic streams and playing them back filters out almost all of the "control" dynamics.

That's why I was saying 'traffic simulator' rather than 'traffic replay'. A good 
simulator is a much harder problem

The simulator needs both ends to send and receive traffid, dealing with delays, 
failures, and whatever the control system does.

This means that the simulator isn't 'play these packets' (for all the reasons 
you gave) but rather is managing the simulation at layer 7, not at layer 2. It 
would be things like: 'open these connections and send data' The data can be 
junk, but it's timing and size match real traffic. There are a lot of different 
types of traffic, and only ISPs can provide information about the mix of the 
traffic

for types, off the top of my head I know of: one-way streaming such as youtube, 
two way streaming such as zoom, bulk downloads, dependencies (DNS, http library 
calls, cache checks), gaming, bulk uploads, etc

the control software would have knobs to adjust the mix of traffic types, the 
number of endpoints (on each end), the volume of data, and report on failures 
(hard failures where the data doesn't get through, soft failures where there is 
'too much' latency for streaming/gaming', user perceived quality (page load 
times for simulated complex pages), etc

and then throw this sort of traffic between the endpoints and see how it changes 
with different control systems.

> However, this suggests a "stress simulator" that might help create realistic 
> experiments - instead of simulating traffic, simulate instabilities in routers 
> by injecting packets into queues for mix of flows based on some probabilistic 
> model that acts like "real Internet". These injected packets might, for 
> example, have invalid checksums on purpose to avoid harming any endpoint 
> stacks. So such a tool would be a lot easier to deploy.

simulating failures is useful once you have a good case to start with. Based on 
this discussion, I don't believe that they are starting from a good point, so 
injecting failures won't get them to where they can tell if they can handle 
"real world" traffic well.

A problem injector is also easier than the type of simulation master I'm talking 
about.

David Lang

Re: [Cake] [Bloat] bbr vs all the aqms, cake winning...

[Livingood, Jason]

On 9/28/24, 21:35, "Bloat on behalf of David Lang via Bloat" <bloat-bounces@lists.bufferbloat.net <mailto:bloat-bounces@lists.bufferbloat.net> on behalf of bloat@lists.bufferbloat.net <mailto:bloat@lists.bufferbloat.net>> wrote:
> That's why I was saying 'traffic simulator' rather than 'traffic replay'. A good 
simulator is a much harder problem

[JL] Further to David R's point - there is so much variation in client, device, home network, CPE, etc. that simulation just never catches all the corner cases (and so IMO not worth the time investment in most cases). That's why in my experience large operators of networks and applications will of course run automatic QA tests in a lab, but quickly move on to phased deployment - with a gradually expanded deployment that is coupled with alerting for changes in operations patterns/performance/etc. 

Re: [Cake] [Bloat] bbr vs all the aqms, cake winning...

[Juliusz Chroboczek]

> Similarly, there's more and more WebRTC traffic and RTP traffic

I know you know that, but just to make sure the audience doesn't get the
wrong idea: good implementations of WebRTC do perform congestion control.
Unfortunately, the details depend on the implementation, and the details
are not always published.

> I know there are no research funds available, unless you can use [...]
> some other "Generative AI" name in the title.

That's not true.  You can also say "quantum".

-- Juliusz