[Bloat] AQM & Net Neutrality

[Bloat] AQM & Net Neutrality

[Livingood, Jason]

[-- Attachment #1: Type: text/plain, Size: 669 bytes --]

I’m looking for opinions here re bloat-busting techniques like AQM in the context of network neutrality (NN). The worry I have is whether some non-technical people will misunderstand how AQM works & conclude that implementing it may violate NN because it would make interactive traffic perform better than it does today. That is true of course – it’s a design goal of AQM, but non-interactive traffic performs as well as it always has – it is not disadvantaged.

Maybe the worries I have heard just points out the need for more education/awareness about what delay is and why things like AQM are not prioritization/QoS? I appreciate any thoughts.

Jason


[-- Attachment #2: Type: text/html, Size: 2622 bytes --]

Re: [Bloat] AQM & Net Neutrality

[Jonathan Morton]

> On 24 May, 2021, at 4:09 pm, Livingood, Jason via Bloat <bloat@lists.bufferbloat.net> wrote:
> 
> I’m looking for opinions here re bloat-busting techniques like AQM in the context of network neutrality (NN). The worry I have is whether some non-technical people will misunderstand how AQM works & conclude that implementing it may violate NN because it would make interactive traffic perform better than it does today. That is true of course – it’s a design goal of AQM, but non-interactive traffic performs as well as it always has – it is not disadvantaged.
>  
> Maybe the worries I have heard just points out the need for more education/awareness about what delay is and why things like AQM are not prioritization/QoS? I appreciate any thoughts.

I'm pleased to help with education in this area.  The short and simplistic answer would be that AQM treats all traffic going through it equally; the non-interactive traffic *also* sees a reduction in latency; though many people won't viscerally notice this, they can observe it if they look closely.  More importantly, it's not necessary for traffic to make any sort of business or authentication arrangement in order to benefit from AQM, only comply with existing, well-established specifications as they already do.

If the traffic supports ECN, the AQM can use that instead of packet drops for signalling, which tends to actually *reduce* packet loss in bulk transfers, compared to simply bouncing off the tail end of a dumb FIFO.  Reduced latency would already make recovering from these losses easier for the transport, but eliminating them entirely means that the application receives a completely smooth delivery, with no sudden pauses and jumps caused by the recovery process.

It's worth digging into the details a bit to solidify the message for a broader range of audiences.  You might start with my recently published Informational draft discussing different types of latency:

	https://datatracker.ietf.org/doc/html/draft-morton-tsvwg-interflow-intraflow-delays-00

I'll note in passing that AQM can be used as part of a system that would, in fact, violate Net Neutrality.  It's important to distinguish the effects caused by different parts of the system, so that a Net-Neutral system can be obtained by deleting just the parts that are incompatible with it.  Current indications, for example, are that L4S would fall under that definition, since it gives a strong throughput priority to traffic carrying its identifier.

Without an AQM or FQ system at the bottleneck, interactive traffic is at the mercy of any competing traffic as to how much delay it will suffer in the queue.  Non-interactive traffic seeks to keep that queue full in order to maximise throughput, with latency being considered unimportant.  If the queue size is chosen carefully, the damage can be limited to some extent, but there is a limit to how much a buffer can be shrunk before it starts reducing throughput as well.  In short, it is not possible to treat both types of traffic equally well with a dumb FIFO; you must favour one or the other, and historically it was throughput-sensitive applications that won that debate.  I think there is a little bit more awareness of right-sizing buffers these days, but that can still easily lead to hundreds of milliseconds of unnecessary delay to interactive traffic, which is difficult to tolerate.

By contrast, when you implement both AQM and FQ at the bottleneck, interactive traffic is no longer affected by competing traffic at all, as long as they use less than their "fair share" of available throughput capacity.  When that threshold is exceeded, the AQM will start working to inform them that they're sending too fast, and the FQ will regulate the flow so that it *is* a fair share that it consumes.  This "fair share" metric has a number of different reasonable definitions, but it should not be confused with a so-called "fair usage policy"; it only kicks in when all the capacity on the link is already in use.

You can also implement *just* AQM at the bottleneck.  In this case the benefit seen by interactive traffic is somewhat diluted, because all traffic goes through the same queue in FIFO order.  The AQM simply tells traffic to slow down if the queue shows signs of filling up.  This leaves the queue still able to handle bursts of traffic (which are the main concern for throughput), while minimising the delay since the queue does not sit constantly full.  An "Approximate Fairness" AQM also dynamically steers AQM signalling towards traffic that contributes the most to congestion, leaving lighter and interactive traffic alone.

Alternatively, you could implement *just* FQ, although this is usually seen as the more difficult component of the two to implement at high speed and/or large scale.  This would simply hold bulk traffic to its "fair share", and keep it out of the way of interactive traffic, without also reducing the delay to the bulk traffic flows.  I would suggest that if you implement FQ, you can also usually implement AQM on top with little difficulty.

Please do ask for further clarification if that would be helpful.

 - Jonathan Morton

Re: [Bloat] AQM & Net Neutrality

[Jonathan Morton]

>> Maybe the worries I have heard just points out the need for more education/awareness about what delay is and why things like AQM are not prioritization/QoS? I appreciate any thoughts.
> 
> I'm pleased to help with education in this area.  The short and simplistic answer would be that AQM treats all traffic going through it equally; the non-interactive traffic *also* sees a reduction in latency; though many people won't viscerally notice this, they can observe it if they look closely.  More importantly, it's not necessary for traffic to make any sort of business or authentication arrangement in order to benefit from AQM, only comply with existing, well-established specifications as they already do.

There is one more point I'd like to touch on up front.  Net Neutrality first became a concern with file-sharing "swarm" protocols, and then with video-on-demand services.  The common feature of these from a technical perspective, is high utilisation of throughput capacity, to the detriment of other users sharing the same back-end and head-end ISP infrastructure.

Implementing AF-AQM or FQ-AQM within the backhaul and head-end equipment, not to distinguish individual 5-tuple flows but merely traffic associated with different subscribers, would fairly share out back-end and head-end capacity between subscribers.  This would reduce the pressure on the ISP to implement policies and techniques that violate Net Neutrality and/or are otherwise unpopular with consumers, such as data caps.  This assumes (as I believe has been represented in some official forums) that these measures are due to technical needs rather than financial greed.

I'm aware of some reasonably fast equipment that already implements AF-AQM commercially.  My understanding is that similar functionality can also be added to many recent cable head-ends by a firmware upgrade.

 - Jonathan Morton

Re: [Bloat] AQM & Net Neutrality

[Stuart Cheshire]

On 24 May 2021, at 06:09, Livingood, Jason via Bloat <bloat@lists.bufferbloat.net> wrote:

> I’m looking for opinions here re bloat-busting techniques like AQM in the context of network neutrality (NN). The worry I have is whether some non-technical people will misunderstand how AQM works & conclude that implementing it may violate NN because it would make interactive traffic perform better than it does today. That is true of course – it’s a design goal of AQM, but non-interactive traffic performs as well as it always has – it is not disadvantaged.

There’s a faulty assumption buried in this, a common misunderstanding that we need to correct.

Consistently lower delay benefits *all* applications. The lower the round-trip time, the better TCP fast-retransmit works. TCP connection setup is faster. The TLS handshake is faster. The less buffering in the network, the less application-layer buffering is required for streaming video, giving quicker startup times, and quicker random access. If you try to make a list of applications that *don’t* benefit from lower delays, you’ll probably end up with an empty list.

This is why I’ve been advocating for making low delay available for *any* traffic that chooses to opt-in to this smarter queue management, not selectively for just some privileged traffic. I’m not making any subjective value judgement that video conferencing traffic is more important or more deserving that streaming video traffic, or weather forecasts, or driving directions, or software downloads. I do not support traffic prioritization schemes that privilege some traffic types over others. I support making low delay available to *all* traffic that agrees to behave properly and cooperate to keep the queues short -- meaning packet pacing, responding appropriately to congestion signals, etc.

Delay reduction is not an either/or choice. In order for some traffic to benefit other traffic doesn’t have to suffer. It’s not a zero-sum game. Eliminating standing queues in network buffers benefits all traffic. This can be hard to communicate because it seems counter to human intuition. It sounds too good to be true. In normal human life this is uncommon. When first class passengers board the plane first, all economy passengers wait a little bit longer as a result. Computer network queueing doesn’t operate like that, which makes it hard to explain by analogy to everyday experiences that most people understand.

I talked about this six years ago in my presentation at the Apple developer conference:
<https://developer.apple.com/videos/play/wwdc2015/719/?time=2702>

There’s also a neat demo a little earlier in that same video:
<https://developer.apple.com/videos/play/wwdc2015/719/?time=2520>

Stuart Cheshire

Re: [Bloat] AQM & Net Neutrality

[Michael Richardson]

[-- Attachment #1: Type: text/plain, Size: 1949 bytes --]


Jonathan Morton <chromatix99@gmail.com> wrote:
    > I'm pleased to help with education in this area.  The short and
    > simplistic answer would be that AQM treats all traffic going through it
    > equally; the non-interactive traffic *also* sees a reduction in
    > latency; though many people won't viscerally notice this, they can
    > observe it if they look closely.  More importantly, it's not necessary
    > for traffic to make any sort of business or authentication arrangement
    > in order to benefit from AQM, only comply with existing,
    > well-established specifications as they already do.

    > If the traffic supports ECN, the AQM can use that instead of packet
    > drops for signalling, which tends to actually *reduce* packet loss in
    > bulk transfers, compared to simply bouncing off the tail end of a dumb
    > FIFO.  Reduced latency would already make recovering from these losses
    > easier for the transport, but eliminating them entirely means that the
    > application receives a completely smooth delivery, with no sudden
    > pauses and jumps caused by the recovery process.

So it seems that it be fair to say that by reducing the latency, and finding a more
reasonable bandwidth*delay value, the AQM allows non-interactive traffic to
make do with less buffering.  This improves memory utilitization on the
receiver (the viewer), but perhaps more importantly, might it also improve
memory ultilization on the sender by allowing sent data to be purged faster.

Could this reduce the cost of OTT streaming services?
Maybe the savings is insignificant for a small service, but could it be
substantial for larger services?  (there is a media spin to avoid here)

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [Bloat] AQM & Net Neutrality

[Michael Richardson]

Stuart Cheshire via Bloat <bloat@lists.bufferbloat.net> wrote:
    > When first class passengers board the plane first,
    > all economy passengers wait a little bit longer as a result. Computer
    > network queueing doesn’t operate like that, which makes it hard to
    > explain by analogy to everyday experiences that most people understand.

I wonder if airplane boarding is a place make an elevator pitch analogy.

What pisses everyone off when getting the plane is that we queue at multiple
points.   Hurry up and line up so that we can check your boarding card, and
then make you wait four more times on the jetway, and then in the aisles.
It we all could just walk a lot slower, then it might feel less annoying.

(So apparently, it's the overhead bin accesses that cause most actual delays)

    > I talked about this six years ago in my presentation at the Apple
    > developer conference:
    > <https://developer.apple.com/videos/play/wwdc2015/719/?time=2702>

    > There’s also a neat demo a little earlier in that same video:
    > <https://developer.apple.com/videos/play/wwdc2015/719/?time=2520>

A demo worth watching!

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [

Re: [Bloat] AQM & Net Neutrality

[Jonathan Morton]

> On 24 May, 2021, at 10:18 pm, Stuart Cheshire via Bloat <bloat@lists.bufferbloat.net> wrote:
> 
> When first class passengers board the plane first, all economy passengers wait a little bit longer as a result.

Technically, they all get to the runway at the same time anyway; the first-class pax just get out of the terminal to sit in their airline seats waiting for longer, while the more congested cattle-class cabin sorts itself out.  If the latter process were optimised better, the first-class passengers might actually end up waiting less, and pretty much everyone would benefit accordingly.

Where first-class passengers *do* have an advantage is in priority lanes at check-in and security.  It means they can turn up at the airport later to catch the same flight, without fear of missing it and without having to spend unnecessary hours in duty-free hell.  They also get posher waiting lounges with "free" food.  It is that sort of atmosphere that Net Neutrality advocates object to in computer networking.

I believe NN advocates will respond positively to concrete signs of improvement in perceived consumer fairness and reduction of costs to consumers.  I also believe that implementing AQM well is a key enabler towards those improvements.  That is probably the right perspective for "selling" AQM to them.

 - Jonathan Morton

Re: [Bloat] AQM & Net Neutrality

[Dave Taht]

I made the most articulate pleas I could to both sides of the debate on this:

https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-but-its-not-over-yet/

I do think that now that the scandal here has reached a peak:
https://ag.ny.gov/sites/default/files/oag-fakecommentsreport.pdf

that perhaps an honest appraisal of both AQM and fq+aqm technologies can be had
in public, again. I'd be willing to reach across the isle on this, and
patiently explain stuff to lawmakers where their thinking is
incorrect, in order to finally fix bufferbloat.

Re: [Bloat] AQM & Net Neutrality

[Dave Taht]

oops, wrong link for the that plea to both sides of the debate. The
correct link was:

http://blog.cerowrt.org/post/net_neutrality_customers/

On Wed, May 26, 2021 at 11:32 AM Dave Taht <dave.taht@gmail.com> wrote:
>
> I made the most articulate pleas I could to both sides of the debate on this:
>
> https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-but-its-not-over-yet/
>
> I do think that now that the scandal here has reached a peak:
> https://ag.ny.gov/sites/default/files/oag-fakecommentsreport.pdf
>
> that perhaps an honest appraisal of both AQM and fq+aqm technologies can be had
> in public, again. I'd be willing to reach across the isle on this, and
> patiently explain stuff to lawmakers where their thinking is
> incorrect, in order to finally fix bufferbloat.



-- 
Latest Podcast:
https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/

Dave Täht CTO, TekLibre, LLC

Re: [Bloat] AQM & Net Neutrality

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 1983 bytes --]

I think one of the big advantages that AQM has is that it doesn't know, or
care, who the flow is.  It can't, itself, violate NN concerns because it
has no knowledge with which to do so.

Instead, it punishes the greed flows that try to use more than their fair
share of the available bandwidth.  It doesn't really favor one protocol, or
provider, or site, or anything else, because it's not aware of them, as
such.  Instead it just stops the kid that's trying to take all the candy
from the candy bowl, to make sure there's enough for everyone else in line.

The working could perhaps be softened from "punished", perhaps, depending
on the audience.

"Everyone gets an equal slice, up to their fill" is another way of looking
at it, I think.

On Wed, May 26, 2021 at 11:36 AM Dave Taht <dave.taht@gmail.com> wrote:

> oops, wrong link for the that plea to both sides of the debate. The
> correct link was:
>
> http://blog.cerowrt.org/post/net_neutrality_customers/
>
> On Wed, May 26, 2021 at 11:32 AM Dave Taht <dave.taht@gmail.com> wrote:
> >
> > I made the most articulate pleas I could to both sides of the debate on
> this:
> >
> >
> https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-but-its-not-over-yet/
> >
> > I do think that now that the scandal here has reached a peak:
> > https://ag.ny.gov/sites/default/files/oag-fakecommentsreport.pdf
> >
> > that perhaps an honest appraisal of both AQM and fq+aqm technologies can
> be had
> > in public, again. I'd be willing to reach across the isle on this, and
> > patiently explain stuff to lawmakers where their thinking is
> > incorrect, in order to finally fix bufferbloat.
>
>
>
> --
> Latest Podcast:
> https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/
>
> Dave Täht CTO, TekLibre, LLC
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 3184 bytes --]

Re: [Bloat] AQM & Net Neutrality

[Sebastian Moeller]

Hi Aaron,

I understand your argument more as an argument for equitable sharing/FQ than for AQM, even though both complement each other quite well.
The biggest point for FQ in my opinion is, while this is certainly not the optimal capacity share regime for all traffic mixes, but without an oracle at the bottleneck (or robust un-gameable information about relative priority encoded in the packets themselves) it is the one sharing regime that will never be pessimal, and should always be good enough and also per definitionem compatible with net neutrality rules, since no flow gets an advantage over another one. That said FQ does not solve all pathologies and is for example gameable by splitting a transfer into many concurrent flows, but single queue AQMs and even a dumb FIFO behave similarly the same pathology already, so FQ does not make matters worse.

Regards
	Sebastian

> On May 29, 2021, at 00:28, Aaron Wood <woody77@gmail.com> wrote:
> 
> I think one of the big advantages that AQM has is that it doesn't know, or care, who the flow is.  It can't, itself, violate NN concerns because it has no knowledge with which to do so.
> 
> Instead, it punishes the greed flows that try to use more than their fair share of the available bandwidth.  It doesn't really favor one protocol, or provider, or site, or anything else, because it's not aware of them, as such.  Instead it just stops the kid that's trying to take all the candy from the candy bowl, to make sure there's enough for everyone else in line.
> 
> The working could perhaps be softened from "punished", perhaps, depending on the audience.
> 
> "Everyone gets an equal slice, up to their fill" is another way of looking at it, I think.
> 
> On Wed, May 26, 2021 at 11:36 AM Dave Taht <dave.taht@gmail.com> wrote:
> oops, wrong link for the that plea to both sides of the debate. The
> correct link was:
> 
> http://blog.cerowrt.org/post/net_neutrality_customers/
> 
> On Wed, May 26, 2021 at 11:32 AM Dave Taht <dave.taht@gmail.com> wrote:
> >
> > I made the most articulate pleas I could to both sides of the debate on this:
> >
> > https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-but-its-not-over-yet/
> >
> > I do think that now that the scandal here has reached a peak:
> > https://ag.ny.gov/sites/default/files/oag-fakecommentsreport.pdf
> >
> > that perhaps an honest appraisal of both AQM and fq+aqm technologies can be had
> > in public, again. I'd be willing to reach across the isle on this, and
> > patiently explain stuff to lawmakers where their thinking is
> > incorrect, in order to finally fix bufferbloat.
> 
> 
> 
> -- 
> Latest Podcast:
> https://www.linkedin.com/feed/update/urn:li:activity:6791014284936785920/
> 
> Dave Täht CTO, TekLibre, LLC
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] AQM & Net Neutrality

[Holland, Jake]

Hi Stuart,

On 05-24, 12:18 PM, "Stuart Cheshire via Bloat" <bloat@lists.bufferbloat.net> wrote:
> Delay reduction is not an either/or choice. In order for some traffic to benefit other traffic doesn’t have to suffer. It’s not a zero-sum game. Eliminating standing queues in network buffers benefits all traffic. This can be hard to communicate because it seems counter to human intuition. It sounds too good to be true. In normal human life this is uncommon. When first class passengers board the plane first, all economy passengers wait a little bit longer as a result. Computer network queueing doesn’t operate like that, which makes it hard to explain by analogy to everyday experiences that most people understand.

This is a great point and maybe worth expanding on.  It made me
think of a few such analogies in the real world that maybe could be
easier for people to intuitively grasp.

For example, there's some good videos on traffic handling like this
one that can show the difference between 2 intersecting 4-lane roads
with an intersection having a traffic flow of 191 (at the start), vs.
2 intersecting 4-lane roads with a stack interchange doing an almost
6x better 1099 (at the 6-minute mark):
https://www.youtube.com/watch?v=yITr127KZtQ

I feel like if you told people that's kinda like the difference
between using a router with and without AQM, they'd have a useful
model that would make some sense to them, even though the dynamics
aren't quite the same.  The point is that by setting up a nicely
optimized infrastructure, it prevents people from getting in each
others' way and makes it better and faster for everyone.

But I don't think that's the only example, there's other real-world
scenarios people can maybe follow, like comparing the Steffan airplane
boarding method to back-to-front.  There's some short and sweet videos
that can make the point, perhaps in a way that non-experts can more or
less follow and see how it helps:
https://www.youtube.com/watch?v=Y7RXo20jTM4
(Even when there's a first class section with special treatment, this
could still help in coach, if only there were enough luggage space...)

Likewise with some other IRL queue-tuning strategies like how to
set up checkout lines at stores, there's some videos that do a good
job explaining why it helps and how it works (though IIRC this trades
off maximum delay vs. average delay, so it's not quite the same
"everybody just purely wins" point):
https://www.youtube.com/watch?v=9nczHfj-Oh8

But maybe some of these are too far away from networking, I dunno.
So another also-ran on "accessible explanation" is this adorable video
from RIPE in 2016, which is nice because it's actually talking about
the right thing.  It says the same thing about helping everyone, but
only quickly and in-passing.  I feel like something along these lines
that harps on the "helps everyone" point a bit more might be the right
kind of thing:
https://www.youtube.com/watch?v=3eAVGF70HjY

Anyway, it's an excellent point that this non-zero-sum aspect might be
a barrier to understanding, and I bet it's worthwhile to try to get
the "everybody benefits from a good AQM" point across more explicitly
and more often, especially if people are mistaking this for priority/
net neutrality issue.

And just a quick side note:

> This is why I’ve been advocating for making low delay available for *any* traffic that chooses to opt-in to this smarter queue management, not selectively for just some privileged traffic. 

AQM at a network bottleneck is not an opt-in thing, it just applies
to all the traffic passing through.  (ECN helps a little at the app
layer from loss avoidance, but usually not as much as the lower queue
target by far.)


Best regards,
Jake