From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp64.iad3a.emailsrvr.com (smtp64.iad3a.emailsrvr.com [173.203.187.64]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 9587E3B29E for ; Sun, 14 Jun 2020 17:04:06 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=g001.emailsrvr.com; s=20190322-9u7zjiwi; t=1592168646; bh=36NAAeH+a90/Zg3kqRpHTMLkH3h3eoow/pTNAaOgRjg=; h=Date:Subject:From:To:From; b=xVWZSLgoMgIHMUGqPoNiQ9QGuHOst+ubZKAyCfK3j89IcsVSVKAJO8NL0j8Bp21uh p5hbcKNBBaJuvN+73rW2VIPFyrdE1TZXVbxeWfMcynmynClZIIej0uPhQsKjCK1llX e/RWICkPAKYYNQSN4sMUeZ6VT5ipnpo6KNGIWf4g= Received: from app8.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp1.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id F0251175B; Sun, 14 Jun 2020 17:04:05 -0400 (EDT) X-Sender-Id: dpreed@deepplum.com Received: from app8.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by 0.0.0.0:25 (trex/5.7.12); Sun, 14 Jun 2020 17:04:06 -0400 Received: from deepplum.com (localhost.localdomain [127.0.0.1]) by app8.wa-webapps.iad3a (Postfix) with ESMTP id DA302E063C; Sun, 14 Jun 2020 17:04:05 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@deepplum.com, from: dpreed@deepplum.com) with HTTP; Sun, 14 Jun 2020 17:04:05 -0400 (EDT) X-Auth-ID: dpreed@deepplum.com Date: Sun, 14 Jun 2020 17:04:05 -0400 (EDT) From: "David P. Reed" To: "Michael Richardson" Cc: "David Lang" , "Jonathan Morton" , "bloat" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20200614170405000000_65370" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: <23070.1592150246@localhost> References: <1591891396.41838464@apps.rackspace.com> <1591901205.85717618@apps.rackspace.com> <1591902977.45963161@apps.rackspace.com> <12673.1591976376@localhost> <20571.1592095016@localhost> <1592149254.956212731@apps.rackspace.com> <23070.1592150246@localhost> Message-ID: <1592168645.890914226@apps.rackspace.com> X-Mailer: webmail/17.3.12-RC X-Classification-ID: 59c16ed9-24a6-4656-8646-37949bfb7701-1-1 Subject: Re: [Bloat] FW: [Dewayne-Net] Ajit Pai caves to SpaceX but is still skeptical of Musk's latency claims X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jun 2020 21:04:06 -0000 ------=_20200614170405000000_65370 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0AI have no problem with WebRTC based videoconferencing. In fact, I think = it is pretty good for 2-4 endpoints. But when you have lower cost laptops, = they drag down the conferencing because of the compositing and mixing and m= ultiple stream transmission load, even with 2-3 other participants.=0A =0A[= What do you think I and others fought for datagrams back in 1976 for? If it= hadn't been for our little cabal, you would have TCP virtual circuits. Per= iod. And it took a lot of arguing - a whole year of nearly losing - until w= e forced the creation of the IP datagram layer separate from TCP. It was ea= sier to disentangle Telnet from TCP - the initial TCP was going to have "br= eak" and "formatting" characters defined in it and "line at a time" options= , etc. Because it was thought that Telnet was the primary use of the Intern= et, and if you wanted binary octets, you could just quote them.]=0A =0ASo, = WebRTC is fine, and it works as well as it works. However, the engineering = thought needed to make peer-to-peer *conferencing* scale well (up to intera= ctive webinars with meeting rooms that can be split off, etc) is simplified= by having resources other than just "peers" to do that work. Simplifying s= calability is a good thing.=0A =0AI don't think forcing everyone to do medi= a over WebRTC is any better than forcing everyone to use centralized server= s. Each approach has tradeoffs. The point of the end-to-end argument was to= enable those tradeoffs.=0A =0AAs far as billing by packets, no, I hope it = never happens. 70% of the opex of the Bell System was billing-related, beca= use of micro-specific billing.=0ATo bill, you need auditability of the bill= s. It's not simple at all - you can't just send billing packets to accounts= associated with endpoints in an Internet. Every packet on every link is po= tentially (and likely) billed differently.=0A =0AIf you want the amount you= pay your access provider to be proportional to traffic you generate or rec= eive, that may be feasible. Desirable? That's another issue not relevant he= re. I was talking about billing every packet back to "some responsible part= y" from every link to reimburse the investor who built that link, bit by bi= t. That's what the Bell System actually did (outside a local exchange, and = between states). And that's why 70% of opex was billing, which would have s= oared if it were by packet rather than by "call".=0A =0AYou have the Intern= et ONLY because of Carterfone and other decisions that forced interfaces in= to the Bell System that they didn't want, because it wrecked their monopoly= business model. Data was charged at an incredibly high rate, separate from= voice, because to bill for it cost more. The UK got the Internet only beca= use they had to follow the US (rather than billing outrageously). Europe go= t it last, because PTT's were essentially government revenue generators, so= the government hated the idea of losing the money.=0A =0A =0AOn Sunday, Ju= ne 14, 2020 11:57am, "Michael Richardson" said:=0A=0A=0A= =0A> =0A> David P. Reed wrote:=0A> >> > The lockdown = has shown that actual low-latency e2e communication=0A> matters.=0A> =0A> >= > > The gaming community has known this for awhile.=0A> >>=0A> >> how has t= he lockdown shown this? video conferencing is seldom e2e=0A> =0A> > Well, i= t's seldom peer-to-peer (and for good reason, the number of=0A> > streams t= o each endpoint would grow linearly in complexity, if not=0A> > bandwidth, = in peer-to-peer implementations of conferencing, and quickly=0A> > become u= nusable. In principle, one could have the video/audio sources=0A> > transmi= t multiple resolution versions of their camera/mic capture to=0A> > each de= stination, and each destination could composite screens and mix=0A> > audio= itself, with a tightly coupled "decentralized" control=0A> > algorithm.)= =0A> =0A> JITSI, whereby, and bluejeans are all p2p for example. There are = n+1 webrtc=0A> streams (+1 because server). It's a significantly better exp= erience.=0A> Yes, it doesn't scale to large groups. So what?=0A> =0A> My Ka= rate class of ~15 people uses Zoom ... it is TERRIBLE in so many ways.=0A> = All that command and control, yet my Sensei can't take a question while=0A>= demonstrating.=0A> With all the other services, at least I can lock my vie= w on him.=0A> =0A> My nieces and my mom and I are not a 100 person conferen= ce.=0A> It's more secure, lower latency, more resilient and does not requir= e quite so=0A> much management BS to operate.=0A> =0A> > But, nonetheless, = the application server architecture of Zoom and WebEx=0A> > are pretty dist= ributed on the conference-server end, though it=0A> > definitely needs high= er capacity than each endpoint, And it *is*=0A> > end-to-end at the network= level. It would be relatively simple to=0A> > scatter this load out into m= any more conference-server endpoints,=0A> > because of the basic e2e argume= nt that separates the IP layer from the=0A> > application layer. Blizzard E= ntertainment pioneered this kind of=0A> > solution - scattering its gaming = servers out close to the edge, and did=0A> > so in an "end-to-end" way.=0A>= =0A> Yup.=0A> =0A> > With a system like starlink it seems important to me = to distinguish=0A> > peer-to-peer from end-to-end, something I have had a h= ard time=0A> > explaining to people since 1978 when the first end-to-end ar= guments had=0A> > their impact on the Internet design. Yes, I'm a big fan o= f moving=0A> > function to the human-located endpoints where possible. But = I also=0A> > fought against multicasting in the routers/switches, because v= ery few=0A> > applications benefit from multi-casting of packets alone by t= he=0A> > network. Instead, almost all multi-endpoint systems need to coordi= nate,=0A> > and that coordination is often best done (most scalably) by a n= etwork=0A> > of endpoints that do the coordinated functions needed for a=0A= > > system.=0A> =0A> I see your point. You jump from e2e vs p2p to multicas= t, and I think that=0A> there might be an intermediate part of the argument= that I've missed.=0A> =0A> > However, deciding what those functions must b= e, to put them in=0A> > the basic routers seems basically wrong - it blocks= evolution of the=0A> > application functionality, and puts lots of crap in= the transport=0A> > network that is at best suboptimal, ,and at worst gets= actively in the=0A> > way. (Billing by the packet in each link being the c= lassic example of a=0A> > "feature" that destroyed the Bell System architec= ture as a useful=0A> > technology).=0A> =0A> I'd like to go the other way: = while I don't want to bring back the Bell=0A> System architecture, where on= ly the network could innovate, I do think that=0A> being able to bill by th= e packet is an important feature that I think we now=0A> have the crypto an= d CPU power to do right.=0A> Consider the affect on spam and DDoS that such= a thing would have.=0A> We don't even have to bill for good packets :-)=0A= > There could be a bounty that every packet comes from, and if it is reject= ed,=0A> then the bounty is collected.=0A> =0A> >> and starlink will do very= well with e2e communications, but the=0A> potential=0A> >> bottlenecks (an= d therefor potential buffering) aren't going to show=0A> up in e2e=0A> >> c= ommunications, they will show up where lots of endpoints are pulling=0A> da= ta from=0A> >> servers not directly connected to starlink.=0A> =0A> > I hop= e neither Starlink or the applications using it choose to=0A> > "optimize" = themselves for their first usage. That would be suicidal -=0A> > it's what = killed Iridium, which could ONLY carry 14.4 kb/sec per=0A> > connection, by= design. Optimized for compressed voice only. That's why=0A> > Negroponte a= nd Papert and I couldn't use it to build 2B1, and went with=0A> > Tachyon, = despite Iridium being available for firesale prices and=0A> > Nicholas's be= ing on Motorola's board. Of course 2B1 was way too early=0A> > in the satel= lite game, back in the 1990's. Interesting story there.=0A> =0A> I agree: t= hey need to have the ability to support a variety of services,=0A> particul= arly ones that we have no clue about.=0A> =0A> --=0A> ] Never tell me the o= dds! | ipv6 mesh networks [=0A> ] Michael Richardson, Sandelman Software Wo= rks | IoT architect [=0A> ] mcr@sandelman.ca http://www.sandelman.ca/ | rub= y on rails [=0A> =0A> ------=_20200614170405000000_65370 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I have no problem with= WebRTC based videoconferencing. In fact, I think it is pretty good for 2-4= endpoints. But when you have lower cost laptops, they drag down the confer= encing because of the compositing and mixing and multiple stream transmissi= on load, even with 2-3 other participants.

=0A

 = ;

=0A

[What do you think I and others fought for dat= agrams back in 1976 for? If it hadn't been for our little cabal, you would = have TCP virtual circuits. Period. And it took a lot of arguing - a whole y= ear of nearly losing - until we forced the creation of the IP datagram laye= r separate from TCP. It was easier to disentangle Telnet from TCP - the ini= tial TCP was going to have "break" and "formatting" characters defined in i= t and "line at a time" options, etc. Because it was thought that Telnet was= the primary use of the Internet, and if you wanted binary octets, you coul= d just quote them.]

=0A

 

=0A

So, WebRTC is fine, and it works as well as it works. However, the en= gineering thought needed to make peer-to-peer *conferencing* scale well (up= to interactive webinars with meeting rooms that can be split off, etc) is = simplified by having resources other than just "peers" to do that work. Sim= plifying scalability is a good thing.

=0A

 

= =0A

I don't think forcing everyone to do media over Web= RTC is any better than forcing everyone to use centralized servers. Each ap= proach has tradeoffs. The point of the end-to-end argument was to enable th= ose tradeoffs.

=0A

 

=0A

As far as billing by packets, no, I hope it never happens. 70% of the opex= of the Bell System was billing-related, because of micro-specific billing.=

=0A

To bill, you need auditability of the bills. It= 's not simple at all - you can't just send billing packets to accounts asso= ciated with endpoints in an Internet. Every packet on every link is potenti= ally (and likely) billed differently.

=0A

 

= =0A

If you want the amount you pay your access provider= to be proportional to traffic you generate or receive, that may be feasibl= e. Desirable? That's another issue not relevant here. I was talking about b= illing every packet back to "some responsible party" from every link to rei= mburse the investor who built that link, bit by bit. That's what the Bell S= ystem actually did (outside a local exchange, and between states). And that= 's why 70% of opex was billing, which would have soared if it were by packe= t rather than by "call".

=0A

 

=0A

You have the Internet ONLY because of Carterfone and other dec= isions that forced interfaces into the Bell System that they didn't want, b= ecause it wrecked their monopoly business model. Data was charged at an inc= redibly high rate, separate from voice, because to bill for it cost more. T= he UK got the Internet only because they had to follow the US (rather than = billing outrageously). Europe got it last, because PTT's were essentially g= overnment revenue generators, so the government hated the idea of losing th= e money.

=0A

 

=0A

 = ;

=0A

On Sunday, June 14, 2020 11:57am, "Michael Ric= hardson" <mcr@sandelman.ca> said:

=0A
=0A

>
> David P. Reed <d= preed@deepplum.com> wrote:
> >> > The lockdown has show= n that actual low-latency e2e communication
> matters.
> > >> > The gaming community has known this for awhile.
> >>
> >> how has the lockdown shown this? video co= nferencing is seldom e2e
>
> > Well, it's seldom peer-t= o-peer (and for good reason, the number of
> > streams to each e= ndpoint would grow linearly in complexity, if not
> > bandwidth,= in peer-to-peer implementations of conferencing, and quickly
> >= ; become unusable. In principle, one could have the video/audio sources
> > transmit multiple resolution versions of their camera/mic captu= re to
> > each destination, and each destination could composite= screens and mix
> > audio itself, with a tightly coupled "decen= tralized" control
> > algorithm.)
>
> JITSI, wh= ereby, and bluejeans are all p2p for example. There are n+1 webrtc
>= ; streams (+1 because server). It's a significantly better experience.
> Yes, it doesn't scale to large groups. So what?
>
> = My Karate class of ~15 people uses Zoom ... it is TERRIBLE in so many ways.=
> All that command and control, yet my Sensei can't take a questio= n while
> demonstrating.
> With all the other services, at = least I can lock my view on him.
>
> My nieces and my mom = and I are not a 100 person conference.
> It's more secure, lower la= tency, more resilient and does not require quite so
> much manageme= nt BS to operate.
>
> > But, nonetheless, the applicati= on server architecture of Zoom and WebEx
> > are pretty distribu= ted on the conference-server end, though it
> > definitely needs= higher capacity than each endpoint, And it *is*
> > end-to-end = at the network level. It would be relatively simple to
> > scatt= er this load out into many more conference-server endpoints,
> >= because of the basic e2e argument that separates the IP layer from the
> > application layer. Blizzard Entertainment pioneered this kind o= f
> > solution - scattering its gaming servers out close to the = edge, and did
> > so in an "end-to-end" way.
>
>= ; Yup.
>
> > With a system like starlink it seems impor= tant to me to distinguish
> > peer-to-peer from end-to-end, some= thing I have had a hard time
> > explaining to people since 1978= when the first end-to-end arguments had
> > their impact on the= Internet design. Yes, I'm a big fan of moving
> > function to t= he human-located endpoints where possible. But I also
> > fought= against multicasting in the routers/switches, because very few
> &= gt; applications benefit from multi-casting of packets alone by the
&g= t; > network. Instead, almost all multi-endpoint systems need to coordin= ate,
> > and that coordination is often best done (most scalably= ) by a network
> > of endpoints that do the coordinated function= s needed for a
> > system.
>
> I see your point= . You jump from e2e vs p2p to multicast, and I think that
> there m= ight be an intermediate part of the argument that I've missed.
> > > However, deciding what those functions must be, to put them i= n
> > the basic routers seems basically wrong - it blocks evolut= ion of the
> > application functionality, and puts lots of crap = in the transport
> > network that is at best suboptimal, ,and at= worst gets actively in the
> > way. (Billing by the packet in e= ach link being the classic example of a
> > "feature" that destr= oyed the Bell System architecture as a useful
> > technology).>
> I'd like to go the other way: while I don't want to bri= ng back the Bell
> System architecture, where only the network coul= d innovate, I do think that
> being able to bill by the packet is a= n important feature that I think we now
> have the crypto and CPU p= ower to do right.
> Consider the affect on spam and DDoS that such = a thing would have.
> We don't even have to bill for good packets := -)
> There could be a bounty that every packet comes from, and if i= t is rejected,
> then the bounty is collected.
>
>= >> and starlink will do very well with e2e communications, but the> potential
> >> bottlenecks (and therefor potential b= uffering) aren't going to show
> up in e2e
> >> commu= nications, they will show up where lots of endpoints are pulling
> = data from
> >> servers not directly connected to starlink.>
> > I hope neither Starlink or the applications using i= t choose to
> > "optimize" themselves for their first usage. Tha= t would be suicidal -
> > it's what killed Iridium, which could = ONLY carry 14.4 kb/sec per
> > connection, by design. Optimized = for compressed voice only. That's why
> > Negroponte and Papert = and I couldn't use it to build 2B1, and went with
> > Tachyon, d= espite Iridium being available for firesale prices and
> > Nicho= las's being on Motorola's board. Of course 2B1 was way too early
> = > in the satellite game, back in the 1990's. Interesting story there.>
> I agree: they need to have the ability to support a vari= ety of services,
> particularly ones that we have no clue about.>
> --
> ] Never tell me the odds! | ipv6 mesh netwo= rks [
> ] Michael Richardson, Sandelman Software Works | IoT archit= ect [
> ] mcr@sandelman.ca http://www.sandelman.ca/ | ruby on rails= [
>
>

=0A
 ------=_20200614170405000000_65370--