[Bloat] My (controversial) position paper on TCP

[Bloat] My (controversial) position paper on TCP

[Dave Taht]

I'm sure this would be controversial, and at the moment I'm focused on
testing some sce.h +fq_codel code for freebsd. I'll slam it into the
ecn-sane website at some point.

...

TCP is done. It's baked. It's finished. There is very little left we
can do to improve it, and we should move on to improving new
transports such as QUIC which have option space left. [1]

Ever since https://tools.ietf.org/html/rfc6013 failed in favor of tcp
fast open, I'd given up on tcp. It was a lousy rfc in that it didn't
make clear its best use case was in giving dns servers a safe and fast
way to use tcp, which would have helped reduce the amount of DDOS and
reflection attacks, speed things up, and so on. It wasn't until I had
a long discussion with paul vixie about this use case and worldwide
problem with dns that I understood it's intent to add a good stable
3way handshake to dns was so good.... and by then it was too late.

Instead, tcp fast open was standardized for a limited (IMHO) use case
of making web traffic better. Web traffic has a terrible interaction
with TCP, in that it tends to start up 6 or more simultaneous
connections and slam the link with stuff in slow start simultaneously.
Others standards that I opposed, like IW10 [2], also got adopted, and
we (as part of the cake project) tried to get an AQM (cobalt) that
responded faster to stuff in slow start. Which we succeeded at, and
that paper is progress, but it's still not good enough.

It makes me really crazy that all the other TCP researchers in the
world tend to focus on improving TCP behavior in congestion avoidance
mode - because the statistics are easy to measure! - instead of
focusing on the 95% of flows that never manage to get out of slow
start. Yea, it's hard to look at slow start. That's why we've been
looking at it hard for 5+ years in the bufferbloat project - trying to
get linux, flent, irtt, to be able to look in detail at sub 4ms
intervals, among other things.

There are so many other problems with TCP as a transport - it requires
a stateful firewall for ipv4 + nat, and more stuff than I have time to
go into today...

One item off that long list:

QUIC and Wireguard have a really nice 0 RTT reconnect over crypto
time. I like it a lot. I have not had time to poke much into the DOH
working group at the ietf, but my take on it was that we needed to
make dns better, not replace it.

[1] Up until about 6 months ago, I really felt that we couldn't
improve tcp anymore. DCTCP was a dead end. However the SCE idea makes
it possible to have selectable behaviors on the receiver side -
notably, a low priority background transport application (for
backups/bittorrent) can merely overreact to SCE markings by sending
back ECE to the tcp sender thus getting them to back off faster and be
invisible to other applications. Or something more complicated (in
slow start phase) could be used. ACCUECN also seemed feasible. And
dctcp like approaches to another transport than tcp seemed very
feasible.

But to me, the idea was that we'd improve low latency applications
such as gaming and videoconferencing and VR/AR with SCE, not "fix"
tcp, overall. Goal in life was to have 0 latency for all flows - if it
cost a little bandwidth, fine - 0 latency. The world is evolving to
"enough" bandwidth for everything, but still has too much latency. The
whole l4s thing conflating the benefits low latency with an
ecn-enabled tcp has makes me crazy because it isn't true, as loss is
just fine on most paths - lordy I don't want to go into that here,
today. loss hurts gaming and videoconferencing more.

Another ietf idea that makes me crazy is the motto of "no host
changes" in homenet, and "dumb endpoints" - when we live in an age
where we have quad cores and AI coprocessors in everybody's hands.

The whole QUIC experiment shows what can be done when you have smart
endpoints, along with a network that is as dumb as possible, but no
dumber.

[2] https://tools.ietf.org/html/draft-gettys-iw10-considered-harmful-00

I would prefer folk wrote a position paper for ecn-sane rather than
endlessly discuss this over email. that said, I needed to get this out
of my system.

-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Bloat] My (controversial) position paper on TCP

[David Collier-Brown]

[-- Attachment #1: Type: text/plain, Size: 4930 bytes --]

Hey Dave, are you available for consulting gigs in Canada?

In my latest incarnation, I'm doing on-line auctions in < 120 
milliseconds, with at least one round trip to ~10 bidders, and I suspect 
we never get out of slow start.

I wonder if I can make a case that this is significant, and if you can 
suggest a consulting gig to fix it??? Centos on Intel, in this case.

--dave

On 2019-03-18 6:04 a.m., Dave Taht wrote:
> I'm sure this would be controversial, and at the moment I'm focused on
> testing some sce.h +fq_codel code for freebsd. I'll slam it into the
> ecn-sane website at some point.
>
> ...
>
> TCP is done. It's baked. It's finished. There is very little left we
> can do to improve it, and we should move on to improving new
> transports such as QUIC which have option space left. [1]
>
> Ever since https://tools.ietf.org/html/rfc6013 failed in favor of tcp
> fast open, I'd given up on tcp. It was a lousy rfc in that it didn't
> make clear its best use case was in giving dns servers a safe and fast
> way to use tcp, which would have helped reduce the amount of DDOS and
> reflection attacks, speed things up, and so on. It wasn't until I had
> a long discussion with paul vixie about this use case and worldwide
> problem with dns that I understood it's intent to add a good stable
> 3way handshake to dns was so good.... and by then it was too late.
>
> Instead, tcp fast open was standardized for a limited (IMHO) use case
> of making web traffic better. Web traffic has a terrible interaction
> with TCP, in that it tends to start up 6 or more simultaneous
> connections and slam the link with stuff in slow start simultaneously.
> Others standards that I opposed, like IW10 [2], also got adopted, and
> we (as part of the cake project) tried to get an AQM (cobalt) that
> responded faster to stuff in slow start. Which we succeeded at, and
> that paper is progress, but it's still not good enough.
>
> It makes me really crazy that all the other TCP researchers in the
> world tend to focus on improving TCP behavior in congestion avoidance
> mode - because the statistics are easy to measure! - instead of
> focusing on the 95% of flows that never manage to get out of slow
> start. Yea, it's hard to look at slow start. That's why we've been
> looking at it hard for 5+ years in the bufferbloat project - trying to
> get linux, flent, irtt, to be able to look in detail at sub 4ms
> intervals, among other things.
>
> There are so many other problems with TCP as a transport - it requires
> a stateful firewall for ipv4 + nat, and more stuff than I have time to
> go into today...
>
> One item off that long list:
>
> QUIC and Wireguard have a really nice 0 RTT reconnect over crypto
> time. I like it a lot. I have not had time to poke much into the DOH
> working group at the ietf, but my take on it was that we needed to
> make dns better, not replace it.
>
> [1] Up until about 6 months ago, I really felt that we couldn't
> improve tcp anymore. DCTCP was a dead end. However the SCE idea makes
> it possible to have selectable behaviors on the receiver side -
> notably, a low priority background transport application (for
> backups/bittorrent) can merely overreact to SCE markings by sending
> back ECE to the tcp sender thus getting them to back off faster and be
> invisible to other applications. Or something more complicated (in
> slow start phase) could be used. ACCUECN also seemed feasible. And
> dctcp like approaches to another transport than tcp seemed very
> feasible.
>
> But to me, the idea was that we'd improve low latency applications
> such as gaming and videoconferencing and VR/AR with SCE, not "fix"
> tcp, overall. Goal in life was to have 0 latency for all flows - if it
> cost a little bandwidth, fine - 0 latency. The world is evolving to
> "enough" bandwidth for everything, but still has too much latency. The
> whole l4s thing conflating the benefits low latency with an
> ecn-enabled tcp has makes me crazy because it isn't true, as loss is
> just fine on most paths - lordy I don't want to go into that here,
> today. loss hurts gaming and videoconferencing more.
>
> Another ietf idea that makes me crazy is the motto of "no host
> changes" in homenet, and "dumb endpoints" - when we live in an age
> where we have quad cores and AI coprocessors in everybody's hands.
>
> The whole QUIC experiment shows what can be done when you have smart
> endpoints, along with a network that is as dumb as possible, but no
> dumber.
>
> [2] https://tools.ietf.org/html/draft-gettys-iw10-considered-harmful-00
>
> I would prefer folk wrote a position paper for ecn-sane rather than
> endlessly discuss this over email. that said, I needed to get this out
> of my system.
>
-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb@spamcop.net           |                      -- Mark Twain


[-- Attachment #2: Type: text/html, Size: 5666 bytes --]

Re: [Bloat] [Ecn-sane] My (controversial) position paper on TCP

[Mikael Abrahamsson]

On Mon, 18 Mar 2019, Dave Taht wrote:

> Another ietf idea that makes me crazy is the motto of "no host changes" 
> in homenet, and "dumb endpoints" - when we live in an age where we have 
> quad cores and AI coprocessors in everybody's hands.

This isn't a resource problem, it's a code problem. The IETF wants 10-15 
year old hosts to be able to connect to a network and perform basic 
networking. It might not be very optimized, but the basic function should 
be there. New functionality can optimize for different factors, but making 
older host stop working is frowned upon.

If the endpoints are going to be smarter (and they will, question is how 
fast), how do we keep the smarts updated with new functionality?

TCP does the same thing, it wants to be backwards compatible and that's 
why QUIC has been more free to innovate in some fashions. I also believe 
it's perhaps time to cut TCP off and come up with something new, the bad 
part is that it seems all innovation then has to be done over UDP which 
has its own drawbacks (because of NATs).

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] [Ecn-sane] My (controversial) position paper on TCP

[David Collier-Brown]

[-- Attachment #1: Type: text/plain, Size: 1572 bytes --]

On 2019-03-20 10:28 a.m., Mikael Abrahamsson wrote:
> On Mon, 18 Mar 2019, Dave Taht wrote:
>
>> Another ietf idea that makes me crazy is the motto of "no host 
>> changes" in homenet, and "dumb endpoints" - when we live in an age 
>> where we have quad cores and AI coprocessors in everybody's hands.

Dumb endpoints is a huge change from the design of the internet (and 
ARPAnet), with a dumb middle and smart ends:  I suspect the original 
commentator was talking about mere /relative/ dumbness...

>
> This isn't a resource problem, it's a code problem. The IETF wants 
> 10-15 year old hosts to be able to connect to a network and perform 
> basic networking. It might not be very optimized, but the basic 
> function should be there. New functionality can optimize for different 
> factors, but making older host stop working is frowned upon.

Fortunately this is a solved problem in capacity planning: you replace 
machines often enough that they're not constantly out of service being 
repaired. 10 to 15 human-years is the equivalent of 70 to 105 of the 
dog-years we use in this silly business (;-))

The oldest machine I use is a dps8-m, and I'll be slightly amazed if we 
can provide native TCP. Right now I ssh into the virtual machine that's 
running it, all so I can continue to fiddle around with Multics.

--dave (DavidBrown.TSDC@HI-Multics.ARPA) c-b

-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb@spamcop.net           |                      -- Mark Twain


[-- Attachment #2: Type: text/html, Size: 2435 bytes --]

Re: [Bloat] [Ecn-sane] My (controversial) position paper on TCP

[David Lang]

[-- Attachment #1: Type: text/plain, Size: 896 bytes --]

On Wed, 20 Mar 2019, David Collier-Brown wrote:

> On 2019-03-20 10:28 a.m., Mikael Abrahamsson wrote:
>> 
>> This isn't a resource problem, it's a code problem. The IETF wants 10-15 
>> year old hosts to be able to connect to a network and perform basic 
>> networking. It might not be very optimized, but the basic function should 
>> be there. New functionality can optimize for different factors, but making 
>> older host stop working is frowned upon.
>
> Fortunately this is a solved problem in capacity planning: you replace 
> machines often enough that they're not constantly out of service being 
> repaired. 10 to 15 human-years is the equivalent of 70 to 105 of the 
> dog-years we use in this silly business (;-))

I have quite a number of consumer devices from 2000 or earlier still running, 
consumer endpoints (aka IoT devices) do not get updated very much, if at all.

David Lang

[-- Attachment #2: Type: text/plain, Size: 140 bytes --]

_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] [Ecn-sane] My (controversial) position paper on TCP

[David Collier-Brown]

[-- Attachment #1: Type: text/plain, Size: 1765 bytes --]

On 2019-03-20 4:29 p.m., David Lang wrote:

> On Wed, 20 Mar 2019, David Collier-Brown wrote:
>
>> On 2019-03-20 10:28 a.m., Mikael Abrahamsson wrote:
>>>
>>> This isn't a resource problem, it's a code problem. The IETF wants 
>>> 10-15 year old hosts to be able to connect to a network and perform 
>>> basic networking. It might not be very optimized, but the basic 
>>> function should be there. New functionality can optimize for 
>>> different factors, but making older host stop working is frowned upon.
>>
>> Fortunately this is a solved problem in capacity planning: you 
>> replace machines often enough that they're not constantly out of 
>> service being repaired. 10 to 15 human-years is the equivalent of 70 
>> to 105 of the dog-years we use in this silly business (;-))
>
> I have quite a number of consumer devices from 2000 or earlier still 
> running, consumer endpoints (aka IoT devices) do not get updated very 
> much, if at all.
>
> David Lang

Interesting thought: I wasn't expecting consumer devices from 14 years 
ago! What do you have?

In our house,

  * the cable modem is about a year old,
  * it's predecessor was about 5 when it died,
  * the old printer was 3 years old
  * the new one is about 4
  * my homemade PVR is about 6, and is starting to look elderly, and
  * the old cable box was about 3,
  * the new one about one
  * and my netbook is older than the PVR by maybe a year or so (;-))

--dave
(I intentionally skipped IOT devices, as I expect they could 
change/pivot hugely about the time the market starts to mature,

-- 
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
davecb@spamcop.net           |                      -- Mark Twain


[-- Attachment #2: Type: text/html, Size: 2695 bytes --]

Re: [Bloat] [Ecn-sane] My (controversial) position paper on TCP

[Jonathan Morton]

I still have a functioning "UFO"-style Apple Airport Base Station from about 2000.  I no longer have a convenient means of reconfiguring it (relies on a real Mac within a certain range of vintages), let alone updating its firmware, if Apple bothered to do that any more.

A lot of those died early due to a bad batch of capacitors that were going around at the time.  I repaired mine, and it hasn't gone wrong since.  Only trouble is, it only supports 802.11b and 10baseT half duplex.  And an analogue modem which, again, I can no longer command conveniently.

I also have a considerably better 2007 model.

Both recently retired in favour of the IQrouter.

 - Jonathan Morton

Re: [Bloat] [Ecn-sane] My (controversial) position paper on TCP

[David Lang]

On Wed, 20 Mar 2019, David Collier-Brown wrote:

> On 2019-03-20 4:29 p.m., David Lang wrote:
>
>> On Wed, 20 Mar 2019, David Collier-Brown wrote:
>> 
>>> On 2019-03-20 10:28 a.m., Mikael Abrahamsson wrote:
>>>> 
>>>> This isn't a resource problem, it's a code problem. The IETF wants 10-15 
>>>> year old hosts to be able to connect to a network and perform basic 
>>>> networking. It might not be very optimized, but the basic function should 
>>>> be there. New functionality can optimize for different factors, but 
>>>> making older host stop working is frowned upon.
>>> 
>>> Fortunately this is a solved problem in capacity planning: you replace 
>>> machines often enough that they're not constantly out of service being 
>>> repaired. 10 to 15 human-years is the equivalent of 70 to 105 of the 
>>> dog-years we use in this silly business (;-))
>> 
>> I have quite a number of consumer devices from 2000 or earlier still 
>> running, consumer endpoints (aka IoT devices) do not get updated very much, 
>> if at all.
>> 
>> David Lang
>
> Interesting thought: I wasn't expecting consumer devices from 14 years ago! 
> What do you have?

I've got 4 Tivo's that have been in operation since 2000, they have add-on 
netork cards instead of the original dial-up.

At the Scale conference earlier this month, I used a couple of dozen wndr3800 
APs that were sold in 2010. They are actually still used extensively (I have 
updated software on them). I've seen a lot of people who still use the stock 
AP/router their ISP gave them when they signed up for service many years ago.

Technology refreshes happen at (some) companies, many homes tend to run things 
untilthey break.

Even at some companies, I've seen them running 7+ year old hardware in 
mission-critical production situations.

David Lang

> In our house,
>
> * the cable modem is about a year old,
> * it's predecessor was about 5 when it died,
> * the old printer was 3 years old
> * the new one is about 4
> * my homemade PVR is about 6, and is starting to look elderly, and
> * the old cable box was about 3,
> * the new one about one
> * and my netbook is older than the PVR by maybe a year or so (;-))
>
> --dave
> (I intentionally skipped IOT devices, as I expect they could change/pivot 
> hugely about the time the market starts to mature,
>
>

Re: [Bloat] [Ecn-sane] My (controversial) position paper on TCP

[Dave Collier-Brown]

Super, much appreciated!

My thoughts about "effective lifetime" just got a long tail, possibly
sufficient to wag the dog (:-))

Do you think we should be shifting to v6, and keeping v4 nearly
invariant to address the long tail?

--dave



On 2019-03-20 6:28 p.m., Jonathan Morton wrote:
> I still have a functioning "UFO"-style Apple Airport Base Station from about 2000.  I no longer have a convenient means of reconfiguring it (relies on a real Mac within a certain range of vintages), let alone updating its firmware, if Apple bothered to do that any more.
>
> A lot of those died early due to a bad batch of capacitors that were going around at the time.  I repaired mine, and it hasn't gone wrong since.  Only trouble is, it only supports 802.11b and 10baseT half duplex.  And an analogue modem which, again, I can no longer command conveniently.
>
> I also have a considerably better 2007 model.
>
> Both recently retired in favour of the IQrouter.
>
>   - Jonathan Morton
>
>
--
David Collier-Brown,         | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
dave.collier-brown@indexexchange.com |              -- Mark Twain


CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including any and all attachments, contains confidential information intended only for the person(s) to whom it is addressed. Any dissemination, distribution, copying or disclosure is strictly prohibited and is not a waiver of confidentiality. If you have received this telecommunication in error, please notify the sender immediately by return electronic mail and delete the message from your inbox and deleted items folders. This telecommunication does not constitute an express or implied agreement to conduct transactions by electronic means, nor does it constitute a contract offer, a contract amendment or an acceptance of a contract offer. Contract terms contained in this telecommunication are subject to legal review and the completion of formal documentation and are not binding until same is confirmed in writing and has been signed by an authorized signatory.