From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id BCC3A3B29E for ; Tue, 28 Apr 2020 15:04:45 -0400 (EDT) Received: by mail-wr1-x431.google.com with SMTP id x17so25244426wrt.5 for ; Tue, 28 Apr 2020 12:04:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ieee.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=1nuo5xpCAWkkRrmwqkhKtuacGW+yGKC3L9wELpc0Lrs=; b=Kdv7BRzM00OrA/9vQunCWnGn9FsMEou01pCBizAD4QAhRgAhDw/qsocMo4viLAH1bH qYqZJKBT6hF6qSc2yMaYQXjLD40dsNu4yqbQ5yVdvU+E0CPoSNYYOxUC4OsJjzm8dH6t JMs7/Mjx5wg4Tj8ZN6ewi4sNxPMfOM3iZEElA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1nuo5xpCAWkkRrmwqkhKtuacGW+yGKC3L9wELpc0Lrs=; b=N/VWuaq27ANr0UP3+0m0wRRdvQ0u3s+TcJPQy3JhthJk1PeKjDxbz+D6B2QPea9FmN VXaKGhJeQxP+dTVoKYimvrpuUEqkh5R2UAZE6e9oJCdMt/ZucPNBJgWIcOT67+P0HOmU xX3QTHtYwhwqfPSTioe39mrOeUPhH2e2xiVithl8VZ9JpLenn8QEm84EY+D1+Uj4fSIX XJnRn9mJy5XyvsiW0imfRYpzB06vHfjTXgluVa08SlJIKLBx3EgOkeRfkfudORIaCNq4 ZAC/+jkxkEQmQagpkIborFDenUNrxRsctxGVaKiTR9wjFd2zURn9FVHc5Mi28f3gOHkh rYtg== X-Gm-Message-State: AGi0Pua6EVs/yJ5DRFKYUqhGDg8xglDwPnwOaFE4IjFKFZQQylxjesG6 cgAGXIwKURLBxKx9lz9m1tsixGMdmqmi0nLZL3LBTQ== X-Google-Smtp-Source: APiQypIOxFqNlAk1X+GfTFTYNGqh35zgd5JuYSPPC3FUN8ipPlCR4PmMdeXzzbkl3pPlSTMVSFYycWTIZEV9WKbxCZY= X-Received: by 2002:adf:f1c6:: with SMTP id z6mr27791589wro.361.1588100684572; Tue, 28 Apr 2020 12:04:44 -0700 (PDT) MIME-Version: 1.0 References: <744B09AC-F251-4FB8-84B4-6628BB79E1F6@akamai.com> In-Reply-To: <744B09AC-F251-4FB8-84B4-6628BB79E1F6@akamai.com> From: Luca Muscariello Date: Tue, 28 Apr 2020 21:04:33 +0200 Message-ID: To: "Holland, Jake" Cc: Dave Taht , bloat , tsvwg IETF list Content-Type: multipart/alternative; boundary="0000000000002c126105a45e810e" Subject: Re: [Bloat] my backlogged comments on the ECT(1) interim call X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Apr 2020 19:04:46 -0000 --0000000000002c126105a45e810e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Jake, Thanks for the notes. Very useful. The other issue with the meeting was that the virtual mic queue control channel was the WebEx Meeting chat that does not exist in WebEx Teams. So, I had to switch to Meetings and lost some pieces of the discussion. Yes there might be a terminology difference. Elastic traffic is usually used in the sense of bandwidth sharing not just to define variable bit rates. The point is that there are incentives to cheat in L4S. There is a priority queue that my application can enter by providing as input ECT(1). Applications such as on-line meetings will have a relatively low and highly paced rate. This traffic is conformant to dualQ L queue but is unresponsive to congestion notifications. This is especially true for FEC streams which could be used to ameliorate the media quality in presence of losses(e.g. Wi-Fi) or increased jitter. That was one more point on why using ECT(1) as input assumes trust or a black list after being caught. In both cases the ECT(1) as input is DoSable. On Tue, Apr 28, 2020 at 7:12 PM Holland, Jake wrote: > Hi Luca, > > > > To your point about the discussion being difficult to follow: I tried to > capture the intent of everyone who commented while taking notes: > > https://etherpad.ietf.org:9009/p/notes-ietf-interim-2020-tsvwg-03 > > > > I think this was intended to take the place of a need for everyone to > re-send the same points to the list, but of course some of the most cruci= al > points could probably use fleshing out with on-list follow up. > > > > It got a bit rough in places because I was disconnected a few times and > had to cut over to a local text file, and I may have failed to correctly > understand or summarize some of the comments, so there=E2=80=99s chances = I might > have missed something, but I did my best to capture them all. > > > > I encourage people to review comments and check whether they came out mor= e > or less correct, and to offer formatting and cleanup suggestions if there= =E2=80=99s > a good way to make it easier to follow. > > > > I had timestamps at the beginning of each main point of discussion, with > the intent that after the video is published it would be easier to go bac= k > and check precisely what was said. It looks like someone has been making > cleanup edits that removed the first half of those so far, but my local > text file still has most of those and I can go back and re-insert them if > it seems useful. > > > > @Luca: during your comments in particular I think there might have been a > disruption--I had a =E2=80=9Cfirst comment missed, please check video=E2= =80=9D placeholder > and I may have misunderstood the part about video elasticity, but my > interpretation at the time was that Stuart was claiming that video was > elastic in that it would adjust downward to avoid overflowing a loaded > link, and I thought you were claiming that it was not elastic in that it > would not exceed a maximum rate, which I summarized as perhaps a semantic > disagreement, but if you=E2=80=99d like to help clean that up, it might b= e useful. > > > > From this message, it sounds like the key point you were making was that > it also will not go below a certain rate, and perhaps that quality can st= ay > relatively good in spite of high network loss? > > > > Best regards, > > Jake > > > > *From: *Luca Muscariello > *Date: *Tuesday, April 28, 2020 at 1:54 AM > *To: *Dave Taht > *Cc: *tsvwg IETF list , bloat > > *Subject: *Re: [Bloat] my backlogged comments on the ECT(1) interim call > > > > Hi Dave and list members, > > > > It was difficult to follow the discussion at the meeting yesterday. > > Who said what in the first place. > > > > There have been a lot of non-technical comments such as: this solution > > is better than another in my opinion. "better" has often been used > > as when evaluating the taste of an ice cream: White chocolate vs black > chocolate. > > This has taken a significant amount of time at the meeting. I haven't > learned > > much from that kind of discussion and I do not think that helped to make > > much progress. > > > > If people can re-make their points in the list it would help the debate. > > > > Another point that a few raised is that we have to make a decision as fas= t > as possible. > > I dismissed entirely that argument. Trading off latency with resilience o= f > the Internet > > is entirely against the design principle of the Internet architecture > itself. > > Risk analysis is something that we should keep in mind even when > deploying any experiment > > and should be a substantial part of it. > > > > Someone claimed that on-line meeting traffic is elastic. This is not true= , > I tried to > > clarify this. These applications (WebEx/Zoom) are low rate, a typical > maximum upstream > > rate is 2Mbps and is not elastic. These applications have often a > stand-alone app > > that is not using the browser WebRTC stack (the standalone app typically > works better). > > > > A client sends upstream one or two video qualities unless the video camer= a > is switched off. > > In presence of losses, FEC is used but it is still non elastic. > > Someone claimed (at yesterday's meeting) that fairness is not an issue > (who cares, I heard!) > > Well, fairness can constitute a differentiation advantage between two > companies that are > > commercializing on-line meetings products. Unless at the IETF we accept > > "law-of-the-jungle" behaviours from Internet applications developers, we > should be careful > > about making such claims. > > Any opportunity to cheat, that brings a business advantage WILL be used. > > > > /Luca > > > > TL;DR > > To Dave: you asked several times what Cisco does on latency reduction in > > network equipment. I tend to be very shy when replying on these questions > > as this is not vendor neutral. If chairs think this is not appropriate fo= r > > the list, please say it and I'll reply privately only. > > > > What I write below can be found in Cisco products data sheets and is not > > trade secret. There are very good blog posts explaining details. > > Not surprisingly Cisco implements the state of the art on the topic > > and it is totally feasible to do-the-right-thing in software and hardware= . > > > > Cisco implements AFD (one queue + a flow table) accompanied by a priority > queue for > > flows that have a certain profile in rate and size. The concept is well > known and well > > studied in the literature. AFD is safe and can well serve a complex > traffic mix when > > accompanied by a priority queue. This prio-queue should not be confused > with a strict > > priority queue (e.g. EF in diffserv). There are subtleties related to the > DOCSIS > > shared medium which would be too long to describe here. > > > > This is available in Cisco CMTS for the DOCSIS segment. Bottleneck traffi= c > > does not negatively impact non-bottlenecked-traffic such as an on-line > meeting like > > the WebEx call we had yesterday. It is safe from a network neutrality > point-of-view > > and no applications get hurt. > > > > Cisco implements AFD+prio also for some DC switches such as the Nexus 9k. > There > > is a blog post written by Tom Edsal online that explains pretty well how > that works. > > This includes mechanisms such as p-fabric to approximate SRPT (shortest > remaining processing time) > > and minimize flow completion time for many DC workloads. The mix of the t= wo > > brings FCT minimization AND latency minimization. This is silicon and > scales at any speed. > > For those who are not familiar with these concepts, please search the > research work of Balaji > > Prabhakar and Ron Pang at Stanford. > > > > Wi-Fi: Cisco does airtime fairness in Aironet but I think in the Meraki > series too. > > The concept is similar to what described above but there are several > queues, one per STA. > > Packets are enqueued in the access (category) queue at dequeue time from > the air-time > > packet scheduler. > > > > On Mon, Apr 27, 2020 at 9:24 PM Dave Taht wrote: > > It looks like the majority of what I say below is not related to the > fate of the "bit". The push to take the bit was > strong with this one, and me... can't we deploy more of what we > already got in places where it matters? > > ... > > so: A) PLEA: From 10 years now, of me working on bufferbloat, working > on real end-user and wifi traffic and real networks.... > > I would like folk here to stop benchmarking two flows that run for a long > time > and in one direction only... and thus exclusively in tcp congestion > avoidance mode. > > Please. just. stop. Real traffic looks nothing like that. The internet > looks nothing like that. > The netops folk I know just roll their eyes up at benchmarks like this > that prove nothing and tell me to go to ripe meetings instead. > When y'all talk about "not looking foolish for not mandating ecn now", > you've already lost that audience with benchmarks like these. > > Sure, setup a background flow(s) like that, but then hit the result > with a mix of > far more normal traffic? Please? networks are never used unidirectionally > and both directions congesting is frequent. To illustrate that problem... > > I have a really robust benchmark that we have used throughout the > bufferbloat > project that I would like everyone to run in their environments, the flen= t > "rrul" test. Everybody on both sides has big enough testbeds setup that a > few > hours spent on doing that - and please add in asymmetric networks > especially - > and perusing the results ought to be enlightening to everyone as to the > kind > of problems real people have, on real networks. > > Can the L4S and SCE folk run the rrul test some day soon? Please? > > I rather liked this benchmark that tested another traffic mix, > > ( > https://www.cablelabs.com/wp-content/uploads/2014/06/DOCSIS-AQM_May2014.p= df > > ) > > although it had many flaws (like not doing dns lookups), I wish it > could be dusted off and used to compare this > new fangled ecn enabled stuff with the kind of results you can merely get > with packet loss and rtt awareness. It would be so great to be able > to directly compare all these new algorithms against this benchmark. > > Adding in a non ecn'd udp based routing protocol on heavily > oversubscribed 100mbit link is also enlightening. > > I'd rather like to see that benchmark improved for a more modernized > home traffic mix > where it is projected there may be 30 devices on the network on average, > in a few years. > > If there is any one thing y'all can do to reduce my blood pressure and > keep me engaged here whilst you > debate the end of the internet as I understand it, it would be to run > the rrul test as part of all your benchmarks. > > thank you. > > B) Stuart Cheshire regaled us with several anecdotes - one concerning > his problems > with comcast's 1Gbit/35mbit service being unusable, under load, for > videoconferencing. This is true. The overbuffering at the CMTSes > still, has to be seen to be believed, at all rates. At lower rates > it's possible to shape this, with another device (which is what > the entire SQM deployment does in self defense and why cake has a > specific docsis ingress mode), but it is cpu intensive > and requires x86 hardware to do well at rates above 500Mbits, presently. > > So I wish CMTS makers (Arris and Cisco) were in this room. are they? > > (Stuart, if you'd like a box that can make your comcast link pleasurable > under all workloads, whenever you get back to los gatos, I've got a few > lying around. Was so happy to get a few ietfers this past week to apply > what's off the shelf for end users today. :) > > C) I am glad bob said the L4S is finally looking at asymmetric > networks, and starting to tackle ack-filtering and accecn issues > there. > > But... I would have *started there*. Asymmetric access is the predominate > form > of all edge technologies. > > I would love to see flent rrul test results for 1gig/35mbit, 100/10, 200/= 10 > services, in particular. (from SCE also!). "lifeline" service (11/2) > would be good > to have results on. It would be especially good to have baseline > comparison data from the measured, current deployment > of the CMTSes at these rates, to start with, with no queue management in > play, then pie on the uplink, then fq_codel on the uplink, and then > this ecn stuff, and so on. > > D) The two CPE makers in the room have dismissed both fq and sce as > being too difficult to implement. They did say that dualpi was > actually implemented in software, not hardware. > > I would certainly like them to benchmark what they plan to offer in L4S > vs what is already available in the edgerouter X, as one low end > example among thousands. > > I also have to note, at higher speeds, all the buffering moves into > the wifi and the results are currently ugly. I imagine > they are exploring how to fix their wifi stacks also? I wish more folk > were using RVR + latency benchmarks like this one: > > > http://flent-newark.bufferbloat.net/~d/Airtime%20based%20queue%20limit%20= for%20FQ_CoDel%20in%20wireless%20interface.pdf > > > Same goes for the LTE folk. > > E) Andrew mcgregor mentioned how great it would be for a closeted musicia= n > to > be able to play in real time with someone across town. that has been my > goal > for nearly 30 years now!! And although I rather enjoyed his participation > in > my last talk on the subject ( > > https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-but-its-not-o= ver-yet/ > > ) conflating > a need for ecn and l4s signalling for low latency audio applications > with what I actually said in that talk, kind of hurt. I achieved > "my 2ms fiber based guitarist to fiber based drummer dream" 4+ years > back with fq_codel and diffserv, no ecn required, > no changes to the specs, no mandating packets be undroppable" and > would like to rip the opus codec out of that mix one day. > > F) I agree with jana that changing the definition of RFC3168 to suit > the RED algorithm (which is not pi or anything fancy) often present in > network switches, > today to suit dctcp, works. But you should say "configuring red to > have l4s marking style" and document that. > > Sometimes I try to point out many switches have a form of DRR in them, > and it's helpful to use that in conjunction with whatever diffserv > markings you trust in your network. > > To this day I wish someone would publish how much they use DCTCP style > signalling on a dc network relative to their other traffic. > > To this day I keep hoping that someone will publish a suitable > set of RED parameters for a wide variety of switches and routers - > for the most common switches and ethernet chips, for correct DCTCP usage. > > Mellonox's example: > ( > https://community.mellanox.com/s/article/howto-configure-ecn-on-mellanox-= ethernet-switches--spectrum-x > > ) is not dctcp specific. > > many switches have a form of DRR in them, and it's helpful to use that > in conjunction with whatever diffserv markings you trust in your > network, > and, as per the above example, segregate two red queues that way. From > what I see > above there is no way to differentiate ECT(0) from ECT(1) in that switch. > (?) > > I do keep trying to point out the size of the end user ecn enabled > deployment, starting with the data I have from free.fr > . > Are we > building a network for AIs or people? > > G) Jana also made a point about 2 queues "being enough" (I might be > mis-remembering the exact point). Mellonoxes ethernet chips at 10Gig expo= se > 64 hardware queues, some new intel hardware exposes 2000+. How do these > queues work relative to these algorithms? > > We have generally found hw mq to be far less of a benefit than the > manufacturers think, especially as regard to > lower latency or reduced cpu usage (as cache crossing is a bear). > There is a lot of software work in this area left to be done, however > they are needed to match queues to cpus (and tenants) > > Until sch_pie gained timestamping support recently, the rate estimator > did not work correctly in a hw mq environment. Haven't looked over > dualpi in this respect. > > > > > > -- > Make Music, Not War > > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > > Tel: 1-831-435-0729 > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > > > --0000000000002c126105a45e810e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Jake,

Thanks for the notes. Very useful.
The other i= ssue with the meeting was that the virtual mic queue control channel was th= e WebEx Meeting chat that does not exist in WebEx Teams. So, I had to switc= h to Meetings and lost some pieces of the discussion.=C2=A0

Yes there might be a termino= logy difference. Elastic traffic is usually used in the sense of bandwidth = sharing not just to define variable bit rates.

The point is th= at there are incentives to cheat in L4S.

There is a priority=C2=A0queue tha= t my application can enter by providing=C2=A0as input ECT(1).=C2=A0
Applications suc= h as on-line meetings will have a relatively low and highly paced rate.

This traffic is conformant to dualQ L queue but is unresponsive t= o congestion notifications. =C2=A0

This is especially true fo= r FEC streams which could be used to ameliorate the media quality in presen= ce of losses(e.g. Wi-Fi)
or increased jitter.


<= div class=3D"gmail_default" style=3D"font-family:monospace" dir=3D"auto">Th= at was one more point on why using ECT(1) as input assumes trust or a black= list after being caught.

In both cases the ECT(1) as input i= s DoSable.



On= Tue, Apr 28, 2020 at 7:12 PM Holland, Jake <jholland@akamai.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft:1px solid rgb(204,204,204);padding-left:1ex">

Hi Luca,

=C2=A0

To your point about the discussion being difficult t= o follow: I tried to capture the intent of everyone who commented while tak= ing notes:

https://etherpad.ietf.org:9009/= p/notes-ietf-interim-2020-tsvwg-03

=C2=A0

I think this was intended to take the place of a nee= d for everyone to re-send the same points to the list, but of course some o= f the most crucial points could probably use fleshing out with on-list foll= ow up.

=C2=A0

It got a bit rough in places because I was disconnec= ted a few times and had to cut over to a local text file, and I may have fa= iled to correctly understand or summarize some of the comments, so there=E2= =80=99s chances I might have missed something, but I did my best to capture them all.

=C2=A0

I encourage people to review comments and check whet= her they came out more or less correct, and to offer formatting and cleanup= suggestions if there=E2=80=99s a good way to make it easier to follow.<= /u>

=C2=A0

I had timestamps at the beginning of each main point= of discussion, with the intent that after the video is published it would = be easier to go back and check precisely what was said. It looks like someo= ne has been making cleanup edits that removed the first half of those so far, but my local text file still has m= ost of those and I can go back and re-insert them if it seems useful.

=C2=A0

@Luca: during your comments in particular I think th= ere might have been a disruption--I had a =E2=80=9Cfirst comment missed, pl= ease check video=E2=80=9D placeholder and I may have misunderstood the part= about video elasticity, but my interpretation at the time was that Stuart was claiming that video was elastic in that it wo= uld adjust downward to avoid overflowing a loaded link, and I thought you w= ere claiming that it was not elastic in that it would not exceed a maximum = rate, which I summarized as perhaps a semantic disagreement, but if you=E2=80=99d like to help clean that up, = it might be useful.

=C2=A0

From this message, it sounds like the key point you = were making was that it also will not go below a certain rate, and perhaps = that quality can stay relatively good in spite of high network loss?=

=C2=A0

Best regards,

Jake

=C2=A0

From: = Luca Muscariello <= muscariello@ieee.= org>
Date: Tuesday, April 28, 2020 at 1:54 AM
To: Dave Taht <dave.taht@gmail.com>
Cc: tsvwg IETF list <tsvwg@ietf.org>, bloat <bloat@lists.bufferbloat.net>
Subject: Re: [Bloat] my backlogged comments on the ECT(1) interim ca= ll

=C2=A0

= Hi Dave and list members,

= =C2=A0

= It was difficult to follow the discussion at the meeting yesterday.<= u>

= Who=C2=A0 said what in the first place.

= =C2=A0

= There have been a lot of non-technical comments such as: this solution

= is better than another in my opinion. "better" has often been use= d

= as when evaluating the taste of an ice cream: White chocolate vs black choc= olate.

= This has taken a significant amount=C2=A0of time at the meeting. I haven= 9;t learned

= much from that kind of discussion and I do not think that helped to make=C2= =A0

= much progress.

= =C2=A0

= If people can re-make their points in the list it would help the debate.=

= =C2=A0

= Another point that a few raised is that we have to make a decision as fast = as=C2=A0possible.

= I dismissed entirely that argument. Trading off latency with resilience of = the Internet

= is entirely against the design principle of the Internet architecture itsel= f.

= Risk analysis is something that we should keep=C2=A0in mind even when deplo= ying=C2=A0any experiment

= and should be a substantial part of it.=C2=A0

= =C2=A0

= Someone=C2=A0claimed that on-line meeting traffic is elastic. This is not t= rue, I tried to

= clarify this. These applications (WebEx/Zoom) are low rate, a typical maxim= um upstream

= rate is 2Mbps and is not elastic. These applications have often a stand-alo= ne app

= that is not using the browser WebRTC stack (the standalone app typically wo= rks better).

= =C2=A0

= A client sends upstream one or two video qualities unless the video camera = is switched off.=C2=A0

= In presence of losses, FEC is used but it is still non elastic.

= Someone claimed (at yesterday's meeting) that fairness is not an issue = (who cares, I heard!)

= Well, fairness can constitute a differentiation advantage=C2=A0between two = companies that are=C2=A0

= commercializing on-line meetings products. Unless at the IETF we accept=C2= =A0

= "law-of-the-jungle" behaviours from Internet applications develop= ers, we should be careful

= about making such claims.

= Any opportunity to cheat, that brings a business advantage WILL be used.=

= =C2=A0

= /Luca

= =C2=A0

= TL;DR

= To Dave: you asked several times what=C2=A0 Cisco does on latency reduction= in

= network equipment. I tend to be very shy when replying on these questions

= as this is not vendor neutral. If chairs think this is not appropriate for<= u>

= the list, please say it and I'll reply privately only.

= =C2=A0

= What I write below can be found in Cisco products data sheets and is not=

= trade secret. There are very good blog posts explaining details.<= /u>

= Not surprisingly=C2=A0Cisco implements the state of the art on the topic=

= and it is totally feasible to do-the-right-thing in software and hardware.<= u>

= =C2=A0

= Cisco implements AFD (one queue + a flow table) accompanied by a priority q= ueue for=C2=A0

= flows that have a certain profile in rate and size. The concept is well kno= wn and well

= studied in the literature. AFD is safe and can well serve a complex traffic= mix when=C2=A0

= accompanied by a priority queue. This prio-queue should not be confused wit= h a strict

= priority queue (e.g. EF in diffserv). There are subtleties related to the D= OCSIS

= shared medium which would be too long to describe=C2=A0here.<= /span>

= =C2=A0

= This is available in Cisco CMTS for the DOCSIS segment. Bottleneck traffic<= u>

= does not negatively=C2=A0impact non-bottlenecked-traffic such as an on-line= meeting like

= the WebEx call we had yesterday. It is safe from a network neutrality point= -of-view

= and no applications get hurt.=C2=A0

= =C2=A0

= Cisco implements AFD+prio also for some DC switches such as the Nexus 9k. T= here

= is a blog post written by Tom Edsal online that explains pretty well how th= at works.

= This includes mechanisms such as p-fabric to approximate SRPT (shortest rem= aining processing time)

= and minimize flow completion time for many DC workloads. The mix of the two=

= brings FCT minimization AND latency minimization. This is silicon and scale= s at any speed.

= For those who are not familiar with these concepts, please search the resea= rch work of Balaji=C2=A0

= Prabhakar and Ron Pang at Stanford.

= =C2=A0

= Wi-Fi: Cisco does airtime fairness in Aironet but I think in the Meraki ser= ies too.

= The concept is similar to what described above but there are several queues= , one per STA.

= Packets are enqueued in the access (category) queue at dequeue time from th= e air-time

= packet scheduler.=C2=A0

=C2=A0

On Mon, Apr 27, 2020 at 9:24 PM Dave Taht <dave.taht@gmail.com&= gt; wrote:

It looks like the majority of what I say below is no= t related to the
fate of the "bit". The push to take the bit was
strong with this one, and me... can't we deploy more of what we
already got in places where it matters?

...

so: A) PLEA: From 10 years now, of me working on bufferbloat, working
on real end-user and wifi traffic and real networks....

I would like folk here to stop benchmarking two flows that run for a long t= ime
and in one direction only... and thus exclusively in tcp congestion
avoidance mode.

Please. just. stop. Real traffic looks nothing like that. The internet
looks nothing like that.
The netops folk I know just roll their eyes up at benchmarks like this
that prove nothing and tell me to go to ripe meetings instead.
When y'all talk about "not looking foolish for not mandating ecn n= ow",
you've already lost that audience with benchmarks like these.

Sure, setup a background flow(s)=C2=A0 like that, but then hit the result with a mix of
far more normal traffic? Please? networks are never used unidirectionally and both directions congesting is frequent. To illustrate that problem...
I have a really robust benchmark that we have used throughout the bufferblo= at
project that I would like everyone to run in their environments, the flent<= br> "rrul" test. Everybody on both sides has big enough testbeds setu= p that a few
hours spent on doing that - and please add in asymmetric networks especiall= y -
and perusing the results ought to be enlightening to everyone as to the kin= d
of problems real people have, on real networks.

Can the L4S and SCE folk run the rrul test some day soon? Please?

I rather liked this benchmark that tested another traffic mix,

( https://www.cablelabs.com/wp-content/uploads/2014/06/DOCSIS-AQM_May2014.pdf= )

although it had many flaws (like not doing dns lookups), I wish it
could be dusted off and used to compare this
new fangled ecn enabled stuff with the kind of results you can merely get with packet loss and rtt awareness. It would be so great to be able
to directly compare all these new algorithms against this benchmark.

Adding in a non ecn'd udp based routing protocol on heavily
oversubscribed 100mbit link is also enlightening.

I'd rather like to see that benchmark improved for a more modernized home traffic mix
where it is projected there may be 30 devices on the network on average, in a few years.

If there is any one thing y'all can do to reduce my blood pressure and<= br> keep me engaged here whilst you
debate the end of the internet as I understand it, it would be to run
the rrul test as part of all your benchmarks.

thank you.

B) Stuart Cheshire regaled us with several anecdotes - one concerning
his problems
with comcast's 1Gbit/35mbit service being unusable, under load, for
videoconferencing. This is true. The overbuffering at the CMTSes
still, has to be seen to be believed, at all rates. At lower rates
it's possible to shape this, with another device (which is what
the entire SQM deployment does in self defense and why cake has a
specific docsis ingress mode), but it is cpu intensive
and requires x86 hardware to do well at rates above 500Mbits, presently.
So I wish CMTS makers (Arris and Cisco) were in this room. are they?

(Stuart, if you'd like a box that can make your comcast link pleasurabl= e
under all workloads, whenever you get back to los gatos, I've got a few=
lying around. Was so happy to get a few ietfers this past week to apply
what's off the shelf for end users today. :)

C) I am glad bob said the L4S is finally looking at asymmetric
networks, and starting to tackle ack-filtering and accecn issues
there.

But... I would have *started there*. Asymmetric access is the predominate f= orm
of all edge technologies.

I would love to see flent rrul test results for 1gig/35mbit, 100/10, 200/10=
services, in particular. (from SCE also!). "lifeline" service (11= /2)
would be good
to have results on. It would be especially good to have baseline
comparison data from the measured, current deployment
of the CMTSes at these rates, to start with, with no queue management in play, then pie on the uplink, then fq_codel on the uplink, and then
this ecn stuff, and so on.

D) The two CPE makers in the room have dismissed both fq and sce as
being too difficult to implement. They did say that dualpi was
actually implemented in software, not hardware.

I would certainly like them to benchmark what they plan to offer in L4S
vs what is already available in the edgerouter X, as one low end
example among thousands.

I also have to note, at higher speeds, all the buffering moves into
the wifi and the results are currently ugly. I imagine
they are exploring how to fix their wifi stacks also? I wish more folk
were using RVR + latency benchmarks like this one:

http://flent-newark.= bufferbloat.net/~d/Airtime%20based%20queue%20limit%20for%20FQ_CoDel%20in%20= wireless%20interface.pdf

Same goes for the LTE folk.

E) Andrew mcgregor mentioned how great it would be for a closeted musician = to
be able to play in real time with someone across town. that has been my goa= l
for nearly 30 years now!! And although I rather enjoyed his participation i= n
my last talk on the subject (
https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-but= -its-not-over-yet/
) conflating
a need for ecn and l4s signalling for low latency audio applications
with what I actually said in that talk, kind of hurt. I achieved
"my 2ms fiber based guitarist to fiber based drummer dream" 4+ ye= ars
back with fq_codel and diffserv, no ecn required,
no changes to the specs, no mandating packets be undroppable" and
would like to rip the opus codec out of that mix one day.

F) I agree with jana that changing the definition of RFC3168 to suit
the RED algorithm (which is not pi or anything fancy) often present in
network switches,
today to suit dctcp, works. But you should say "configuring red to
have l4s marking style" and document that.

Sometimes I try to point out many switches have a form of DRR in them,
and it's helpful to use that in conjunction with whatever diffserv
markings you trust in your network.

To this day I wish someone would publish how much they use DCTCP style
signalling on a dc network relative to their other traffic.

To this day I keep hoping that someone will publish a suitable
set of RED parameters for a wide variety of switches and routers -
for the most common switches and ethernet chips, for correct DCTCP usage.
Mellonox's example:
( https://community.mellanox.com/s/article/howto-configure-ecn-on-mellanox-et= hernet-switches--spectrum-x
) is not dctcp specific.

many switches have a form of DRR in them, and it's helpful to use that<= br> in conjunction with whatever diffserv markings you trust in your
network,
and, as per the above example, segregate two red queues that way. From
what I see
above there is no way to differentiate ECT(0) from ECT(1) in that switch. (= ?)

I do keep trying to point out the size of the end user ecn enabled
deployment, starting with the data I have from free.fr. Are we
building a network for AIs or people?

G) Jana also made a point about 2 queues "being enough" (I might = be
mis-remembering the exact point). Mellonoxes ethernet chips at 10Gig expose=
64 hardware queues, some new intel hardware exposes 2000+. How do these
queues work relative to these algorithms?

We have generally found hw mq to be far less of a benefit than the
manufacturers think, especially as regard to
lower latency or reduced cpu usage (as cache crossing is a bear).
There is a lot of software work in this area left to be done, however
they are needed to match queues to cpus (and tenants)

Until sch_pie gained timestamping support recently, the rate estimator
did not work correctly in a hw mq environment. Haven't looked over
dualpi in this respect.





--
Make Music, Not War

Dave T=C3=A4ht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-435-0729
_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listinfo/blo= at

--0000000000002c126105a45e810e--