Re: [Cake] [Bloat] benefits of ack filtering

Re: [Cake] [Bloat] benefits of ack filtering

[Dave Taht]

On Wed, Nov 29, 2017 at 10:21 AM, Juliusz Chroboczek <jch@irif.fr> wrote:
>> The better solution would of course be to have the TCP peeps change the
>> way TCP works so that it sends fewer ACKs.
>
> Which tends to perturb the way the TCP self-clocking feedback loop works,
> and to break Nagle.

Linux TCP is no longer particularly ack-clocked. In the post pacing,
post sch_fq world, packets are released (currently) on a 1ms schedule.
Support was recently released for modifying that schedule on a per
driver basis, which turns out to be helpful for wifi.

see: https://www.spinics.net/lists/netdev/msg466312.html

>
>> In the TCP implementations I tcpdump regularily, it seems they send one
>> ACK per 2 downstream packets.
>
> That's the delack algorithm.  One of the stupidest algorithms I've had the
> displeasure of looking at (a fixed 500ms timeout, sheesh).

Nagle would probably agree. He once told me he wished for 1 ack per
data packet...

We were young then.

>
> And yes, it breaks Nagle.
>
>> I don't want middle boxes making "smart" decisions

Ironically, it was dave reed's (co-author of the end to end argument)
50x1 ratio network connection that was an impetus to look harder at
this, and what I modeled in
http://blog.cerowrt.org/post/ack_filtering/

(I note there is discussion and way more tests landing on the cake mailing list)

The astounding number was that we were able to drop 70% of all packets
(and 90+% of acks) without doing any visible harm on the tests.

>
> I agree, especially if they use transport-layer data to make their
> decisions.

I'm not particularly fond of the idea myself! But I didn't invent
severe network asymmetry, or cpus that can't context switch worth a
damn.

>> Since this ACK reduction is done on probably hundreds of millions of
>> fixed-line subscriber lines today,

What I'd started with was wanting to create impairments for netem that
matched common ack-filtering schemes in the field already.

> what arguments do designers of TCP have
>> to keep sending one ACK per 2 received TCP packets?

this would be a good list to have.

I note osx does stretch acks by default.

>
> I think it's about growing the TCP congestion window fast enough.  Recall
> that that AIMD counts received ACKs, not ACKed bytes.

the cake code has a specific optimization to preserve slow start. It
can be improved.

>
> (And not breaking Nagle.)
>
> -- Juliusz



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] [Bloat] benefits of ack filtering

[Dave Taht]

On Wed, Nov 29, 2017 at 10:28 AM, Juliusz Chroboczek <jch@irif.fr> wrote:
>> Recently Ryan Mounce added ack filtering cabilities to the cake qdisc.
>> The benefits were pretty impressive at a 50x1 Down/Up ratio:
>
> If I read this posting right, you're only measuring bulk performance.
> What about interactive traffic, when there's only one or two data segments in
> flight at a given time

In this design, you can only filter out an ack when you have a queue of them.

I am thinking saying "filter" has been misleading. Tho plenty
stateless ack filters exist.

ack-queue-compression?

>> I'd rather like to have a compelling list of reasons why not to do
>> this!
>
> I haven't looked at Cake in detail, and I haven't put much thought into
> ACK filtering, but off the top of my head:
>
>   - not risking breaking any of the TCP-related algorithms that depend on
>     ACKs arriving in a timely manner (AIMD, Nagle, Eifel, etc.),
>     especially in the case of just one segment in flight;
>   - not contributing to the ossification of the Internet by giving an
>     unfair advantage to TCP over other protocols;
>   - limiting the amount of knowledge that middleboxes have of the
>     transport-layer protocols, which leads to further ossification;
>   - avoiding complexity in middleboxes, which leads to a more brittle
>     Internet;
>   - not encouraging ISPs to deploy highly asymmetric links.

I'll add these to my list!

>
> This is not my area of expertise, and therefore I don't feel competent to
> have an opinion, but I think that before you deploy ACK filtering, you
> really should consider the worries expressed above and whatever other
> worries more competent people might have.

been worrying ever since I touched the wet paint!

> -- Juliusz



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] [Bloat] benefits of ack filtering

[Steinar H. Gunderson]

On Wed, Nov 29, 2017 at 10:41:41AM -0800, Dave Taht wrote:
> Linux TCP is no longer particularly ack-clocked. In the post pacing,
> post sch_fq world, packets are released (currently) on a 1ms schedule.

Most Linux hosts don't run sch_fq, though. I mean, it's not even default in a
4.15 kernel.

/* Steinar */
-- 
Homepage: https://www.sesse.net/

Re: [Cake] [Bloat] benefits of ack filtering

[Stephen Hemminger]

On Wed, 29 Nov 2017 10:41:41 -0800
Dave Taht <dave.taht@gmail.com> wrote:

> On Wed, Nov 29, 2017 at 10:21 AM, Juliusz Chroboczek <jch@irif.fr> wrote:
> >> The better solution would of course be to have the TCP peeps change the
> >> way TCP works so that it sends fewer ACKs.  
> >
> > Which tends to perturb the way the TCP self-clocking feedback loop works,
> > and to break Nagle.  
> 
> Linux TCP is no longer particularly ack-clocked. In the post pacing,
> post sch_fq world, packets are released (currently) on a 1ms schedule.
> Support was recently released for modifying that schedule on a per
> driver basis, which turns out to be helpful for wifi.
> 
> see: https://www.spinics.net/lists/netdev/msg466312.html

Also TCP BBR has lost its initial luster since it is unfair and ignores
losses and ECN (see recent netdev paper).

Re: [Cake] [Bloat] benefits of ack filtering

[Eric Dumazet]

On Wed, 2017-11-29 at 15:59 -0800, Stephen Hemminger wrote:
> On Wed, 29 Nov 2017 10:41:41 -0800
> Dave Taht <dave.taht@gmail.com> wrote:
> 
> > On Wed, Nov 29, 2017 at 10:21 AM, Juliusz Chroboczek <jch@irif.fr>
> > wrote:
> > > > The better solution would of course be to have the TCP peeps
> > > > change the
> > > > way TCP works so that it sends fewer ACKs.  
> > > 
> > > Which tends to perturb the way the TCP self-clocking feedback
> > > loop works,
> > > and to break Nagle.  
> > 
> > Linux TCP is no longer particularly ack-clocked. In the post
> > pacing,
> > post sch_fq world, packets are released (currently) on a 1ms
> > schedule.
> > Support was recently released for modifying that schedule on a per
> > driver basis, which turns out to be helpful for wifi.
> > 
> > see: https://www.spinics.net/lists/netdev/msg466312.html
> 
> Also TCP BBR has lost its initial luster since it is unfair and
> ignores
> losses and ECN (see recent netdev paper).

Recent netdev paper (from Larry) mentioned that fq_codel is used.

fq_codel is stochastic, so not a fairness champion with many flows.
There is a reason we use fq [1] instead ;)

We asked Larry how to reproduce his (surprising) results, because we
suspect some setup error or bias. He has to update his github trees.

netem can be tricky to use properly.

[1] Although the choice of packet scheduler is no longer an issue with
BBR now TCP can fallback to internal pacing implementation.

About ECN : We do not enable ECN for edge communications, so BBR runs
without ECN being negotiated/accepted.

We will probably take care of this point soon, but we had more urgent
problems.