[Ecn-sane] [tsvwg] Fwd: my backlogged comments on the ECT(1) interim call

Wed Apr 29 07:21:11 EDT 2020

Hi Bob,

more below in-line.

> On Apr 29, 2020, at 12:32, Bob Briscoe <ietf at bobbriscoe.net> wrote:
> 
> Sebastian,
> 
> On 29/04/2020 10:46, Sebastian Moeller wrote:
>> Hi Bob,
>> 
>> 
>>> On Apr 29, 2020, at 11:31, Bob Briscoe <ietf at bobbriscoe.net> wrote:
>>> 
>>> Dave,
>>> 
>>> Please don't tar everything with the same brush. Inline...
>>> 
>>> On 27/04/2020 20:26, Dave Taht wrote:
>>>> just because I read this list more often than tsvwg.
>>>> 
>>>> ---------- Forwarded message ---------
>>>> From: Dave Taht <dave.taht at gmail.com>
>>>> Date: Mon, Apr 27, 2020 at 12:24 PM
>>>> Subject: my backlogged comments on the ECT(1) interim call
>>>> To: tsvwg IETF list <tsvwg at ietf.org>
>>>> Cc: bloat <bloat at lists.bufferbloat.net>
>>>> 
>>>> 
>>>> It looks like the majority of what I say below is not related to the
>>>> fate of the "bit". The push to take the bit was
>>>> strong with this one, and me... can't we deploy more of what we
>>>> already got in places where it matters?
>>>> 
>>>> ...
>>>> 
>>>> so: A) PLEA: From 10 years now, of me working on bufferbloat, working
>>>> on real end-user and wifi traffic and real networks....
>>>> 
>>>> I would like folk here to stop benchmarking two flows that run for a long time
>>>> and in one direction only... and thus exclusively in tcp congestion
>>>> avoidance mode.
>>> [BB] All the results that the L4S team has ever published include short flow mixes either with or without long flows.
>>>     2020: http://folk.uio.no/asadsa/ecn-fbk/results_v2.2/full_heatmap_rrr/
>>>     2019: http://bobbriscoe.net/projects/latency/dctth_journal_draft20190726.pdf#subsection.4.2
>>>     2019: https://www.files.netdevconf.info/f/febbe8c6a05b4ceab641/?dl=1
>>>     2015: http://bobbriscoe.net/projects/latency/dctth_preprint.pdf#subsection.7.2
>>> 
>>> I think this implies you have never actually looked at our data, which would be highly concerning if true.
>> 	[SM] Bob, please take the time to read what Dave is asking for here, it is rather specific, and as far as I can tell has never been tested in all the years.
>> 
>>> Regarding asymmetric links, as you will see in the 2015 and 2019 papers, our original tests were conducted over Al-Lu's broadband testbed with real ADSL lines, real home routers, etc. When we switched to a Linux testbed, we checked we were getting identical results to the testbed that used real broadband kit, but I admit we omitted to emulate the asymmetric upstream. As I said, we can add asymmetric tests back again, and we should.
>> 	[SM] You tested an asymmetric link, with no AQM on the uplink and also no saturating traffic on the uplink, this is not the test Dave has been championing for years a fully saturating load by 4 or more capacity-seeking flows per direction.
>> 	As far as I can tell in all of the testing you did, you never got around to test this  for end-user rather important condition: the whole family/group is using the internet access link to its max. It is exactly that condition where latencies typically go through the roof and people resort to crude behavioral solutions (do not game while someone vide-conferences, and the like).
>> 	The pure fact that you never tested this really IMHO demonstrates that magnitude of testing is no good proxy for quality of testing, especially since Dave repeatedly asked for bi-directionally saturating loads to be tested.
> 
> [BB] Fair enough, it's true there was not saturating traffic in the uplink.

	[SM] Yes, please report the outcome of such a test, as this s not an irrelevant corner-case, but one of the conditions where L4S needs to shine if you want it to succeed. The idea here would be obviously to instantiate one L4S AQM instance per direction, just as the RFC's propose L4S to be rolled-out. 

QUESTON: I take it that you actually have tested topologies with one DualQ instance per direction, but you never attempted to saturate bot directions simultaneously, or did you only ever test topologies with a single AQM instance?

> 
> Until we introduce ACK congestion control (host) or RFC3449-compliant ACK thinning (network), such tests would just prove the need for better ACK congestion control or good ACK thinning, and not properly test the thing we're both trying to test.

	[SM] Pardon me? Please go and try standard SQM with either HTB+fq_codel or just cake and revisit the idea about ACK traffic to be an unsurmountable challenge. And for your entertainment cake actually has a optional ACK filter... 
	What I want to say, I have tested that condition already and the relative low-latency solution I use copes with that rather nasty traffic pattern pretty well, from reading the L4S drafts my understanding that L4S should also perform admirably here. 

> 
> IOW, if you want to test a solution to problem A, there's no point using a scenario that deliberately shows up problem B, for which a solution isn't included in the test environment.

	[SM] So you are saying that L4S is not suitable for bi-directionally saturating traffic? I have long complained about the lack of testing against adversarial traffic patterns in L4S, but here we are not talking adversarial traffic, but really the bread and butter traffic, a solution that claims to deliver low-latency, low-loss and high throughput needs to cope with gracefully.
	IMHO, ACK thinning is a red herring, as the acceptable performance of SQM (without ACK thinning) under these conditions demonstrates. 

> 
> As is made clear in the informative part of the ECN++ draft ( https://tools.ietf.org/html/draft-ietf-tcpm-generalized-ecn-05#section-4.4.1 ), AccECN feedback provides an easy basis to integrate ACK congestion control with data congestion control.

	[SM] All fine, but if L4S really depends on this for normal operation, this needs to be honestly described in the L4S RFC IMHO, especially what the expected consequences are of not fixing ACK congestion.

> However, when you're digging the foundations for others to build on -

	[SM] How about we let history be the judge of that?

> you can't do all the building work as well - you have to rely on other people to take up that mantle (hence the pointers I gave to the other work on sender control of ACK thinning, that are now snipped from this thread).

	[SM] Well, if you promise the world, the onus is on you to deliver it. If, as it seems, you just agreed, that L4S as currently designed and implemented will not gracefully deal with bi-directionally saturating loads... Puzzled, why you believe the releasing L4S into the wild before that open flank is closed? 

> 
> This is something else I meant to say to Dave. When testing distributed systems, you have to test each change one at a time, to discover which changes are good and which are bad.

	[SM] And then you need to test whether what you learned in the isolated/reduced cases also holds for the interactions...

> That's why there is a role for tests with simple traffic mixes, even tho they're not realistic. Yes, it's also necessary to show a system works as a whole in a realistic setting (hence the tests I pointed to with real applications), but for testing purposes, changing multiple things at once gives little insight into potential problems or actual problems.

	[SM] I am puzzled, if you consider bi-directional saturating traffic to be theoretically problematic for L4S, why did you not actually go and confirm that, and then try to figure out remedies INSIDE of the L4S framework (and with that I mean the ietf relevant portions, if you consider TCP Prague as part of L4S, It would be nice to see an RFC draft for that)

Regards
	Sebastian

> 
> Regards
> 
> 
> 
> Bob
> 
> 
> 
>> 
>> Best Regards
>> 	Sebastian
>> 
>> 
>> 
>> 
>> 
>> 
>> 
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/