[Bloat] Bloat done correctly?

[Benjamin Cronce]

> Hi Benjamin,
>
> On Jun 12, 2015, at 17:33 , Benjamin Cronce <bcronce at gmail.com> wrote:
>
> > > On Fri, Jun 12, 2015 at 4:08 AM, Sebastian Moeller wrote:
> > > Hi Benjamin,
> > >
> > > To go off onto a tangent:
> > >
> > > On Jun 12, 2015, at 06:45 , Benjamin Cronce wrote:
> > >
> > > > [...]
> > > > Under load while doing P2P(About 80Mb down and 20Mb up just as I
started the test)
> > > > HFSC: P2P in 20% queue and 80/443/8080 in 40% queue with ACKs going
to a 20% realtime queue
> > > > http://www.dslreports.com/speedtest/622452
> > >
> > >         I know this is not really your question, but I think the ACKs
should go into the same queue as the matching data packets. Think about it
that way, if the data is delayed due to congestion it does not make too
much sense to tell the sender to send more faster (which essentially is
what ACK prioritization does) as that will not really reduce the congestion
but rather increase it.
> > >         There is one caveat though: when ECN is used it might make
sense to send out the ACK that will signal the congestion state back to the
sender faster… So if you prioritize ACKs only select those with an ECN-Echo
flag ;)
> > >         @bloat : What do you all think about this refined ACK
prioritization scheme?
> > >
> > > Best Regards
> > >         Sebastian
> >
> > Here's a very real problem for many users. If you have a highly
asymmetrical connection like what many DSL and cable users have, doing even
mild uploading can consume enough bandwidth to affect your ability to
upload ACKs, which can negatively affect your downloads.
> > A regular offender to many is P2P. If you're uploading while
downloading on a 30/3 connection, you may not be able to ACK data fast
enough. Of course this is in of itself mostly an issue caused by
bufferbloat and/or lack of fair queueing.
>
> I agree, but I think reducing buffer bloat and using fair queueing is
superior to ACK prioritization, that’s all.

I wholly agree. A separate ACK queue could help in some very specific cases
where a trade off is needed, but letting an AQM do its job is probably
better because it should "just work".
I will do away with my ACK queue once fq_codel comes to PFSense, but until
then, I want to make sure my ACKs are not getting stuck behind some burst
of traffic.

>
> >
> > I guess the real question is, should ACKs be prioritized in a system
that does not exhibit bufferbloat or has fair queuing that allows the ACKs
to not feel the affect of other bandwidth being consumed. With no data to
back this up, my first guess would be it will not matter. If you have no
bufferbloat or you have fair queuing, the ACKs should effectively be sent
nearly immediately.
>
> Actually fq_codel mildly boosts sparse flows, and since ACK typically are
sparse they usually do not suffer. But all sparse or starting flows get the
same treatment, so nothing ACK specific just something that typically works
well.
>
> > One case where it could make a difference if where ACKs make up a
sizable portion of the upload bandwidth if download is saturated. An
example would be a hyper-asymmetrical connection like 60/3.
>
> Okay, I will bite, at one ACK every 2 full MSS send, ACKs take up roughly
2% of the reverse traffic, so at 60 the ACK will cost 60*0.02 = 1.2 or
100*1.2/3 = 40%, and I agree that is most likely will cause problems for
people using smaller priority queues. It is also one reason why I have read
recommendations to size priority classes equally (percentage wise) for up-
and download, and make sure ACKs get the same priority as the reverse
data...
>
> > In this case, having the ACKs in a separate queue would actually do the
opposite, it would put an upper bound on how much bandwidth ACKs could
consume unless you gave your ACK queue nearly all of the bandwidth.
>
> Well, as stated above if your incoming traffic class allows 100%
bandwidth use the corresponding outgoing class would better allow the same
percentage… So i still think not putting ACKs in higher priorities gives
better overall system balance. BUT I have no experience with P2P traffic,
so things could be different there. I would hope to be able to tell my P2P
apps to mark their packets as background priority, so the data and ACKs
would move out of the way of more urgent traffic. I understand that you
have actual P2P experience so take my input with a grain of salt, please...
>
> Best Regards
> Sebastian

My P2P experience is limited at best. I'm on a dedicated symmetrical
connection and my ISP already has anti-bufferbloat stuff implemented. Most
of what I hear about using separate ACK queues is entirely because of
bufferbloat, you don't want your ACKs stuck behind 1sec of bloat or getting
dropped.
Until bufferbloat is mostly solved, there probably won't be enough user
stories to make a case one way or the other about how modern AQMs will
affect ACKs under a wide range of loads, technologies, topologies, and
bandwidths. Don't fix what ain't broken and KISS. If you have access to a
reliable AQM, don't use an ACK queue unless you have proof that it'll help.

>
> >
> > In the two examples that I could quickly think of, either it made
little difference or it did the opposite of what your proposed. Of course
it depends on the configuration of the shaping.
> >
> > Minor side tangent about bloat and ACKs that doesn't really need a
discussion
> >
> > A few months back I was downloading some Linux torrents when I noticed
I was getting packetloss. I pretty much never get packetloss. I also
noticed that I was receiving a fluctuating 100Mb/s-103Mb/s of ingress on my
WAN but only seeing about 80Mb/s of egress on my LAN. I did a packet
capture and saw something funny. My WAN was sending out a lot of Dup ACK
packets.
> > I grabbed a few of the dest IPs that my firewall was sending to and did
a trace route. Nice low pings for most of the path, then suddenly about 2-3
hops before the seeder in case their ISP's network, I started to see huge
jitter and pings, in the 1sec-3sec range. Cox and Comcast were the two main
offenders but they do represent a sizable portion of the population.
> > My conclusion was bufferbloat was so bad on their networks that the
seeders where not getting ACKs from me in a timely fashion, so they resent
the segments assuming they were lost. This issue was so bad that about
20Mb/s of the 100Mb/s was duplicate data segments. I was being DDOS'd by
bufferbloat.
> >
> > P.S. I removed the emails because I was not absolutely sure if they
would be scrubbed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20150612/a3d892fe/attachment-0003.html>