[Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 93 bytes --]

The folks over at Meraki had a paper at this year's IMC talking about
TCP over WiFi.

-Toke


[-- Attachment #2: Type: message/rfc822, Size: 10553 bytes --]

[-- Attachment #2.1.1.1: Type: text/plain, Size: 2020 bytes --]

We’ve been doing some studies and work in this area at Meraki.

https://conferences.sigcomm.org/imc/2017/papers/imc17-final203.pdf

Simon


> On Nov 29, 2017, at 7:17 AM, Aaron Falk <aaron.falk@gmail.com> wrote:
> 
> Hey Mark-
> 
> Didn't you do some work on TCP over wifi a while ago?
> 
> --aaron
> 
> On Wed, Nov 8, 2017 at 1:33 PM Michael Welzl <michawe@ifi.uio.no <mailto:michawe@ifi.uio.no>> wrote:
> I would think that 1) there are probably pointers, and 2) the people who have them should be on the ICCRG list, which I’m cc’ing.
> 
> I suggest for this to be the last email that includes tsvarea so that the thread entirely moves to ICCRG.
> 
> 
> > On Nov 8, 2017, at 6:42 PM, Toerless Eckert <tte+ietf@cs.fau.de <mailto:tte%2Bietf@cs.fau.de>> wrote:
> >
> > Any pointers to work analyzing the differences in behavior when TCP is run
> > across WiFi as opposed to wired ? Especially with WiFi in the home ?
> >
> > I am primarily thinking that there could be a higher demand for
> > TCP (end-to-end) retransmissions when using WiFi because the L2/WiFi
> > local retransmissions are insufficient. And if so, what the characteristics
> > of those end-to-end retransmissions is (would assume they would be larger
> > than N msec, where N is whatever the L2/wifi protection window is, which
> > unfortunately i don't know).
> >
> > Asking because we've got the poor "must-sit-in-back-of-the-bus" traffic
> > called IP multicast that is not protected by L2/wifi retransmissions at
> > all and now we're wondering if carrying it over TCP as a workaround
> > could help, and therefore trying to educate myself on specific known
> > issue left when running traffic over TCP over WiFi.
> >
> > If any other TSV or other WG mailing list might be a better place to
> > ask. pls. let me know.
> >
> > Thank!
> >    Toerless
> >
> 
> _______________________________________________
> iccrg mailing list
> iccrg@irtf.org
> https://www.irtf.org/mailman/listinfo/iccrg


[-- Attachment #2.1.1.2: Type: text/html, Size: 3534 bytes --]

[-- Attachment #2.1.2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
iccrg mailing list
iccrg@irtf.org
https://www.irtf.org/mailman/listinfo/iccrg

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Dave Taht]

wow. *median* of 7 interfering APs at 2.4ghz. 10% with more than 29 interferers.

On Wed, Nov 29, 2017 at 11:39 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> The folks over at Meraki had a paper at this year's IMC talking about
> TCP over WiFi.
>
> -Toke
>
>
>
> ---------- Forwarded message ----------
> From: Simon Barber <simon@superduper.net>
> To: Aaron Falk <aaron.falk@gmail.com>
> Cc: Toerless Eckert <tte+ietf@cs.fau.de>, iccrg@irtf.org, tsv-area@ietf.org, Michael Welzl <michawe@ifi.uio.no>, Mark Allman <mallman@icir.org>
> Bcc:
> Date: Wed, 29 Nov 2017 16:41:04 -0800
> Subject: Re: [iccrg] TCP behavior across WiFi pointers ?
> We’ve been doing some studies and work in this area at Meraki.
>
> https://conferences.sigcomm.org/imc/2017/papers/imc17-final203.pdf
>
> Simon
>
>
> On Nov 29, 2017, at 7:17 AM, Aaron Falk <aaron.falk@gmail.com> wrote:
>
> Hey Mark-
>
> Didn't you do some work on TCP over wifi a while ago?
>
> --aaron
>
> On Wed, Nov 8, 2017 at 1:33 PM Michael Welzl <michawe@ifi.uio.no> wrote:
>>
>> I would think that 1) there are probably pointers, and 2) the people who have them should be on the ICCRG list, which I’m cc’ing.
>>
>> I suggest for this to be the last email that includes tsvarea so that the thread entirely moves to ICCRG.
>>
>>
>> > On Nov 8, 2017, at 6:42 PM, Toerless Eckert <tte+ietf@cs.fau.de> wrote:
>> >
>> > Any pointers to work analyzing the differences in behavior when TCP is run
>> > across WiFi as opposed to wired ? Especially with WiFi in the home ?
>> >
>> > I am primarily thinking that there could be a higher demand for
>> > TCP (end-to-end) retransmissions when using WiFi because the L2/WiFi
>> > local retransmissions are insufficient. And if so, what the characteristics
>> > of those end-to-end retransmissions is (would assume they would be larger
>> > than N msec, where N is whatever the L2/wifi protection window is, which
>> > unfortunately i don't know).
>> >
>> > Asking because we've got the poor "must-sit-in-back-of-the-bus" traffic
>> > called IP multicast that is not protected by L2/wifi retransmissions at
>> > all and now we're wondering if carrying it over TCP as a workaround
>> > could help, and therefore trying to educate myself on specific known
>> > issue left when running traffic over TCP over WiFi.
>> >
>> > If any other TSV or other WG mailing list might be a better place to
>> > ask. pls. let me know.
>> >
>> > Thank!
>> >    Toerless
>> >
>>
> _______________________________________________
> iccrg mailing list
> iccrg@irtf.org
> https://www.irtf.org/mailman/listinfo/iccrg
>
>
>
> _______________________________________________
> iccrg mailing list
> iccrg@irtf.org
> https://www.irtf.org/mailman/listinfo/iccrg
>
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Dave Taht]

"We propose that receiving of an 802.11 ACK is a reliable hint that
the corresponding TCP segments will eventually be processed by the
receiver’s transport layer. On receipt of an 802.11 ACK from the
wireless client, the AP will proactively generate a fake TCP ACK on
behalf of the TCP receiver, forwarding it to the TCP sender. This
results in the TCP ACKs arriving sooner at the sender, eliminating the
delay variation induced by medium contention at the receiver. Since
the receiver will still send a TCP ACK, these now-duplicate TCP ACKs
should be suppressed by the AP. The TCP sender is thus shielded in the
revers..."



On Thu, Nov 30, 2017 at 12:04 AM, Dave Taht <dave.taht@gmail.com> wrote:
> wow. *median* of 7 interfering APs at 2.4ghz. 10% with more than 29 interferers.
>
> On Wed, Nov 29, 2017 at 11:39 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> The folks over at Meraki had a paper at this year's IMC talking about
>> TCP over WiFi.
>>
>> -Toke
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Simon Barber <simon@superduper.net>
>> To: Aaron Falk <aaron.falk@gmail.com>
>> Cc: Toerless Eckert <tte+ietf@cs.fau.de>, iccrg@irtf.org, tsv-area@ietf.org, Michael Welzl <michawe@ifi.uio.no>, Mark Allman <mallman@icir.org>
>> Bcc:
>> Date: Wed, 29 Nov 2017 16:41:04 -0800
>> Subject: Re: [iccrg] TCP behavior across WiFi pointers ?
>> We’ve been doing some studies and work in this area at Meraki.
>>
>> https://conferences.sigcomm.org/imc/2017/papers/imc17-final203.pdf
>>
>> Simon
>>
>>
>> On Nov 29, 2017, at 7:17 AM, Aaron Falk <aaron.falk@gmail.com> wrote:
>>
>> Hey Mark-
>>
>> Didn't you do some work on TCP over wifi a while ago?
>>
>> --aaron
>>
>> On Wed, Nov 8, 2017 at 1:33 PM Michael Welzl <michawe@ifi.uio.no> wrote:
>>>
>>> I would think that 1) there are probably pointers, and 2) the people who have them should be on the ICCRG list, which I’m cc’ing.
>>>
>>> I suggest for this to be the last email that includes tsvarea so that the thread entirely moves to ICCRG.
>>>
>>>
>>> > On Nov 8, 2017, at 6:42 PM, Toerless Eckert <tte+ietf@cs.fau.de> wrote:
>>> >
>>> > Any pointers to work analyzing the differences in behavior when TCP is run
>>> > across WiFi as opposed to wired ? Especially with WiFi in the home ?
>>> >
>>> > I am primarily thinking that there could be a higher demand for
>>> > TCP (end-to-end) retransmissions when using WiFi because the L2/WiFi
>>> > local retransmissions are insufficient. And if so, what the characteristics
>>> > of those end-to-end retransmissions is (would assume they would be larger
>>> > than N msec, where N is whatever the L2/wifi protection window is, which
>>> > unfortunately i don't know).
>>> >
>>> > Asking because we've got the poor "must-sit-in-back-of-the-bus" traffic
>>> > called IP multicast that is not protected by L2/wifi retransmissions at
>>> > all and now we're wondering if carrying it over TCP as a workaround
>>> > could help, and therefore trying to educate myself on specific known
>>> > issue left when running traffic over TCP over WiFi.
>>> >
>>> > If any other TSV or other WG mailing list might be a better place to
>>> > ask. pls. let me know.
>>> >
>>> > Thank!
>>> >    Toerless
>>> >
>>>
>> _______________________________________________
>> iccrg mailing list
>> iccrg@irtf.org
>> https://www.irtf.org/mailman/listinfo/iccrg
>>
>>
>>
>> _______________________________________________
>> iccrg mailing list
>> iccrg@irtf.org
>> https://www.irtf.org/mailman/listinfo/iccrg
>>
>> _______________________________________________
>> Make-wifi-fast mailing list
>> Make-wifi-fast@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/make-wifi-fast
>
>
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 176 bytes --]

A hint, but not a guarantee.  If you generate a fake ack, you'd better be
prepared to retransmit the packet if, in fact, it gets lost deeper in the
network.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 223 bytes --]

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Toke Høiland-Jørgensen]

Jonathan Morton <chromatix99@gmail.com> writes:

> A hint, but not a guarantee.  If you generate a fake ack, you'd better be
> prepared to retransmit the packet if, in fact, it gets lost deeper in the
> network.

I think their use case here is an AP that (they presume) is talking
directly to the client. In which case it's probably a fair assumption...
Most operating systems don't drop packets between the WiFi card and the
TCP stack :)

Of course, a client may be forwarding the packet somewhere, in which
case this would probably break in interesting ways...

-Toke

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Sebastian Moeller]

> On Nov 30, 2017, at 09:07, Dave Taht <dave.taht@gmail.com> wrote:
> 
> "We propose that receiving of an 802.11 ACK is a reliable hint that
> the corresponding TCP segments will eventually be processed by the
> receiver’s transport layer. On receipt of an 802.11 ACK from the
> wireless client, the AP will proactively generate a fake TCP ACK on
> behalf of the TCP receiver, forwarding it to the TCP sender. This
> results in the TCP ACKs arriving sooner at the sender, eliminating the
> delay variation induced by medium contention at the receiver. Since
> the receiver will still send a TCP ACK, these now-duplicate TCP ACKs
> should be suppressed by the AP. The TCP sender is thus shielded in the
> revers..."


Now, I admit I am no expert here, but that is crazy talk, making ack thinning look sane (note, I believe that thinning is _sane_ if explicitly opted-in by the end-user, less so if transparently and proactively performed by some hop along the path not under control of the end-users; I have no real opinion on its sanity as a measure against actual congestion.). But this is an active (crazy?) bet on the rest of the rest of the link being error free. In other words it is a gamble which will probably improve the average while also causing more/more extreme extremes, I wonder what measures they use to support their . 

Mildly funny side-note fast in german literally means "almost" in that sense the moniker "fastACK" looks actually quite fitting.

QUESTION:
"The receipt of an 802.11 ACK at the AP triggers it to send the corresponding fast ACK to the TCP sender, thereby moving it past the current sequence number. The client on the other hand would send a duplicate ACK for this lost segment. This is a problem since the TCP sender might not hold that packet anymore in its outgoing bu er, thereby disrupting the TCP session. To avoid this, the FastACK agent inserts each data packet into its local cache before forwarding it downstream. A duplicate ACK from the client then triggers local retransmissions from this cache, on behalf of the TCP sender. Second, local retransmissions are cheaper (and low-overhead) as compared to the end-to-end retransmissions. Further, it also avoids lowering the congestion window of the TCP sender, and maintaining the high end-to-end data ow."

Does that mean that the AP now needs to allow up to a full window of buffering for each TCP flow it fakeACKs?


I have a hunch that the _real_ solution would lie in making the 802.11 MAC have less per packet overhead, so that the need for excessive aggregation would go away, because in the end fastACK aims at lying to the endpoints that there is a relative stable link with nicely bound latency and latency variance when in reality the actual link does not show that, but what do I know...
Then again, since the wifi MAC seems unlikely to change this might be as good as it gets...


> 
> 
> On Thu, Nov 30, 2017 at 12:04 AM, Dave Taht <dave.taht@gmail.com> wrote:
>> wow. *median* of 7 interfering APs at 2.4ghz. 10% with more than 29 interferers.
>> 
>> On Wed, Nov 29, 2017 at 11:39 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>> The folks over at Meraki had a paper at this year's IMC talking about
>>> TCP over WiFi.
>>> 
>>> -Toke
>>> 
>>> 
>>> 
>>> ---------- Forwarded message ----------
>>> From: Simon Barber <simon@superduper.net>
>>> To: Aaron Falk <aaron.falk@gmail.com>
>>> Cc: Toerless Eckert <tte+ietf@cs.fau.de>, iccrg@irtf.org, tsv-area@ietf.org, Michael Welzl <michawe@ifi.uio.no>, Mark Allman <mallman@icir.org>
>>> Bcc:
>>> Date: Wed, 29 Nov 2017 16:41:04 -0800
>>> Subject: Re: [iccrg] TCP behavior across WiFi pointers ?
>>> We’ve been doing some studies and work in this area at Meraki.
>>> 
>>> https://conferences.sigcomm.org/imc/2017/papers/imc17-final203.pdf
>>> 
>>> Simon
>>> 
>>> 
>>> On Nov 29, 2017, at 7:17 AM, Aaron Falk <aaron.falk@gmail.com> wrote:
>>> 
>>> Hey Mark-
>>> 
>>> Didn't you do some work on TCP over wifi a while ago?
>>> 
>>> --aaron
>>> 
>>> On Wed, Nov 8, 2017 at 1:33 PM Michael Welzl <michawe@ifi.uio.no> wrote:
>>>> 
>>>> I would think that 1) there are probably pointers, and 2) the people who have them should be on the ICCRG list, which I’m cc’ing.
>>>> 
>>>> I suggest for this to be the last email that includes tsvarea so that the thread entirely moves to ICCRG.
>>>> 
>>>> 
>>>>> On Nov 8, 2017, at 6:42 PM, Toerless Eckert <tte+ietf@cs.fau.de> wrote:
>>>>> 
>>>>> Any pointers to work analyzing the differences in behavior when TCP is run
>>>>> across WiFi as opposed to wired ? Especially with WiFi in the home ?
>>>>> 
>>>>> I am primarily thinking that there could be a higher demand for
>>>>> TCP (end-to-end) retransmissions when using WiFi because the L2/WiFi
>>>>> local retransmissions are insufficient. And if so, what the characteristics
>>>>> of those end-to-end retransmissions is (would assume they would be larger
>>>>> than N msec, where N is whatever the L2/wifi protection window is, which
>>>>> unfortunately i don't know).
>>>>> 
>>>>> Asking because we've got the poor "must-sit-in-back-of-the-bus" traffic
>>>>> called IP multicast that is not protected by L2/wifi retransmissions at
>>>>> all and now we're wondering if carrying it over TCP as a workaround
>>>>> could help, and therefore trying to educate myself on specific known
>>>>> issue left when running traffic over TCP over WiFi.
>>>>> 
>>>>> If any other TSV or other WG mailing list might be a better place to
>>>>> ask. pls. let me know.
>>>>> 
>>>>> Thank!
>>>>>   Toerless
>>>>> 
>>>> 
>>> _______________________________________________
>>> iccrg mailing list
>>> iccrg@irtf.org
>>> https://www.irtf.org/mailman/listinfo/iccrg
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> iccrg mailing list
>>> iccrg@irtf.org
>>> https://www.irtf.org/mailman/listinfo/iccrg
>>> 
>>> _______________________________________________
>>> Make-wifi-fast mailing list
>>> Make-wifi-fast@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/make-wifi-fast
>> 
>> 
>> 
>> --
>> 
>> Dave Täht
>> CEO, TekLibre, LLC
>> http://www.teklibre.com
>> Tel: 1-669-226-2619
> 
> 
> 
> -- 
> 
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Simon Barber]

This was our assumption (client = endpoint, 802.11 ACK = TCP delivery), but we discovered that this is not always the case, even where the client is the endpoint. FastACK keeps the data around and retransmits it in this case.

Simon

> On Nov 30, 2017, at 12:52 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Jonathan Morton <chromatix99@gmail.com> writes:
> 
>> A hint, but not a guarantee.  If you generate a fake ack, you'd better be
>> prepared to retransmit the packet if, in fact, it gets lost deeper in the
>> network.
> 
> I think their use case here is an AP that (they presume) is talking
> directly to the client. In which case it's probably a fair assumption...
> Most operating systems don't drop packets between the WiFi card and the
> TCP stack :)
> 
> Of course, a client may be forwarding the packet somewhere, in which
> case this would probably break in interesting ways...
> 
> -Toke
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Toke Høiland-Jørgensen]

On 30 November 2017 23:35:33 CET, Simon Barber <simon@superduper.net> wrote:
>This was our assumption (client = endpoint, 802.11 ACK = TCP delivery),
>but we discovered that this is not always the case, even where the
>client is the endpoint. FastACK keeps the data around and retransmits
>it in this case.

Ah. So that means you have something very close to a full TCP
state machine in the AP?

-Toke

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Simon Barber]

Certain things, yes, others - not so much. No congestion control for example. It’s very much the common case that the wireless client is the TCP endpoint, and it’s rare to see drops, but they do happen. FastACK keeps the TCP ACK rate controlled by the wireless bandwidth, unlike a proxy. This is the key innovation here.

Simon

> On Nov 30, 2017, at 2:50 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> 
> 
> On 30 November 2017 23:35:33 CET, Simon Barber <simon@superduper.net> wrote:
>> This was our assumption (client = endpoint, 802.11 ACK = TCP delivery),
>> but we discovered that this is not always the case, even where the
>> client is the endpoint. FastACK keeps the data around and retransmits
>> it in this case.
> 
> Ah. So that means you have something very close to a full TCP
> state machine in the AP?
> 
> -Toke

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Toke Høiland-Jørgensen]

Simon Barber <simon@superduper.net> writes:

> Certain things, yes, others - not so much. No congestion control for
> example. It’s very much the common case that the wireless client is
> the TCP endpoint, and it’s rare to see drops, but they do happen.
> FastACK keeps the TCP ACK rate controlled by the wireless bandwidth,
> unlike a proxy. This is the key innovation here.

Right, actually went and read the paper now. Seems clear enough; a few
questions, though:

- From figure 12 it looks like you are also implicitly doing ACK
  compression? I.e., if you get two packets with seq n and n+1 you will
  only send an ACK back to the sender for n+1?

- Did you measure the latency impact of FastACK? You mention you will do
  that once it's deployed, but does that mean you didn't measure this at
  all in the testbed?

-Toke

Re: [Make-wifi-fast] Fwd: Re: [iccrg] TCP behavior across WiFi pointers ?

[Dave Taht]

On Fri, Dec 1, 2017 at 4:48 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Simon Barber <simon@superduper.net> writes:
>
>> Certain things, yes, others - not so much. No congestion control for
>> example. It’s very much the common case that the wireless client is
>> the TCP endpoint, and it’s rare to see drops, but they do happen.
>> FastACK keeps the TCP ACK rate controlled by the wireless bandwidth,
>> unlike a proxy. This is the key innovation here.
>
> Right, actually went and read the paper now. Seems clear enough; a few
> questions, though:
>
> - From figure 12 it looks like you are also implicitly doing ACK
>   compression? I.e., if you get two packets with seq n and n+1 you will
>   only send an ACK back to the sender for n+1?

https://github.com/dtaht/sch_cake/blob/cobalt/sch_cake.c#L902 might be
worthy of review.

> - Did you measure the latency impact of FastACK? You mention you will do
>   that once it's deployed, but does that mean you didn't measure this at
>   all in the testbed?
> -Toke



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619