From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yk0-x232.google.com (mail-yk0-x232.google.com [IPv6:2607:f8b0:4002:c07::232]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 58B4D21F409; Fri, 12 Jun 2015 22:37:05 -0700 (PDT) Received: by ykar6 with SMTP id r6so652514yka.2; Fri, 12 Jun 2015 22:37:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=EzUPsxH03Wf7uxl4XPqbUee2BSYaFVAr9jc6zwpKisE=; b=Dnypwoos1/Dcv/zR/CPHf1HZdHKDs6dk0vHtXSf1TdKJ030YGzXiIlmm5A0UDFF1Gp ni7VXj6KVbC9ETskB3B9jbI81bAVgN6Iymi4UwhvMgXH2sC6ikU3h/KSO7hHwU1HJGo+ TJwhe+gGT5O0b5VuExTDdSMR+O/D/GpIrwH1X/Iyjx9G6UaVrD9cozswwGPSmteZDbkk Yygsj4G5clBqPGxHU5B8lRAHTpZL0j66i8yBR803Km94fZGpByn+tAEXSY6x2d5sSyxh 1tSsNpod1DsTOHVYEsVlvDf180iBON4JN46a1qh5dak7X66hpvd6XKLAo2v8TuIwa0o9 ztng== MIME-Version: 1.0 X-Received: by 10.170.35.207 with SMTP id 198mr22222477ykd.121.1434173824060; Fri, 12 Jun 2015 22:37:04 -0700 (PDT) Received: by 10.129.148.194 with HTTP; Fri, 12 Jun 2015 22:37:03 -0700 (PDT) In-Reply-To: References: <5578DEE8.6090209@gmail.com> <2ACEC68A-3795-4F7C-BA0F-FBDBA3732566@gmx.de> Date: Sat, 13 Jun 2015 00:37:03 -0500 Message-ID: From: Benjamin Cronce To: David Lang Content-Type: multipart/alternative; boundary=001a1137b69854549c05185f9d32 Cc: cake@lists.bufferbloat.net, Daniel Havey , "cerowrt-devel@lists.bufferbloat.net" , bloat Subject: Re: [Cerowrt-devel] [Cake] [Bloat] active sensing queue management X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jun 2015 05:37:34 -0000 --001a1137b69854549c05185f9d32 Content-Type: text/plain; charset=UTF-8 > On Fri, 12 Jun 2015, Benjamin Cronce wrote: > > >> On 12/06/15 02:44, David Lang wrote: > >>> On Thu, 11 Jun 2015, Sebastian Moeller wrote: > >>> > >>>> > >>>> On Jun 11, 2015, at 03:05 , Alan Jenkins > >>>> wrote: > >>>> > >>>>> On 10/06/15 21:54, Sebastian Moeller wrote: > >>>>>> > >>>>> One solution would be if ISPs made sure upload is 100% provisioned. > >>>>> Could be cheaper than for (the higher rate) download. > >>>> > >>>> Not going to happen, in my opinion, as economically unfeasible > >>>> for a publicly traded ISP. I would settle for that approach as long > >>>> as the ISP is willing to fix its provisioning so that > >>>> oversubscription episodes are reasonable rare, though. > >>> > >>> not going to happen on any network, publicly traded or not. > > > > Sorry if this is a tangent from where the current discussion has gone, but > > I wanted to correct someone saying something is "impossible". > > > > > > > I guess I went off on this tangent because "Not going to happen, in my > > opinion, as economically unfeasible" and "not going to happen on any > > network, publicly traded or not." are too absolute. It can be done, it is > > being done, it is being done for cheap, and being done with "business > > class" professionalism. Charter Comm is 1/2 the download speed for the same > > price and they don't even have symmetrical or dedicated. > > not being oversubscribed includes the trunk. Who cares if there is no congestion > within the ISP if you reach the trunk and everything comes to a screeching halt. > > The reason I used the word "imossible" is because we only have the ability to > make links so fast. Right now we have 10G common, 40G in places, and research > into 100G, if you go back a few years, 1G was the limit. While the fastest > connections have increased by a factor of 100, the home connections have > increased by close to a factor of 1000 during that time (1.5Mb theoretical DSL > vs 1Gb fiber), and 10G is getting cheap enough to be used for the corporate > networks. 100Gb ports only cost about $5k and you can purchase, for unlisted prices, 36Tb/s mutliplexers over a single fiber with 1200km ranges. That's enough bandwidth for 360 100Gb ports. Congestion on trunks are easy because of the large number of flows, you can have strong guarantees via statistical multiplexing. The last mile is the main issue because that's where a single person can make a difference. > > so the ratio between the fastest link that's possible and the subscribers has > dropped from 1000:1 to 100:1 with 10:1 uncomfortably close. > > some of this can be covered up by deploying more lines, but that only goes so > far. > > If you are trying to guarantee no bandwidth limits on your network under any > conditions, you need that sort of bandwith between all possible sets of > customers as well, which means that as you scale to more customers, your > bandwidth requirements are going up O(n^2). O(n^2) is only needed if you desire a fully connected graph. When I have a 100Mb connection, it doesn't mean I get a 100Mb connection to every customer at the same time, I get 100Mb total. Otherwise with 1000 customer, that would mean I would have to have 100Gb link at home. You only need O(n) scaling. Anyway, I don't think you fully appreciate modern 1 petabit(500Tb/s ingress+egress) line rate core routers. This is all commercially available hardware, but would of course be incredibly expensive. 1 petabit router, with 1,000 500Gb/s ports, each 500Gb port plugs into a fiber chassis with 500 1Gb ports. Now you have 500,000 people all with fully dedicated 1Gb/s connections. Of course you do not have any internet access with this setup because all of the router's ports are used for connecting customers. Cut the customers in half, 250k customer, that frees up half of the ports. Now you have 500 500Gb ports freed up. Take those 500Gb ports, plug those into 36Tb/s multiplexers, get 14 strands of fiber, 7 up and 7 down, and run them to your local exchange points. Mind you this system would have 10x more bandwidth than the entire world wide internet peak data usage. That includes all CDNs, trunks, everything. Of course the average user consumes less than 10Mb/s during peak hours, even if they have a 1Gb connection. All that matters is peak usage. Everything about society is based on actual usage, not theoreticals that are about as likely as a duplicate Earth appearing next to us in space. Yes, it is possible, just not likely. Our electrical infrastructure could not handle every user at once, neither can out water, roads, phones, hospitals, stores, etc. > And then the pricing comes into it. 1G fiber to the home is <$100/month (if it > exists at all) isn't going to pay for that sort of idle bandwidth. > > But the good thing is, you don't actually need that much bandwidth to keep your > customer's happy either. > > If the 'guaranteed bandwidth' is written into the contracts properly, there is a > penalty to the company if they can't provide you the bandwidth. That leaves them > the possibility of not actually building out the O(n^2) network, just keeping > ahead of actual reqirements, and occasionally paying a penalty if they don't. > > David Lang --001a1137b69854549c05185f9d32 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
> On Fri, 12 Jun 2015, Benjamin Cronce wrote:
>=C2=A0
> >> On 12/06/15 02:44, David Lang wrot= e:
> >>> On Thu, 11 Jun 2015, Sebastian Moeller wrote= :
> >>>
> >>>>
>= ; >>>> On Jun 11, 2015, at 03:05 , Alan Jenkins
> = >>>> <alan.christopher.jenkins at gmail.com> wrote:
> >>>>
> = >>>>> On 10/06/15 21:54, Sebastian Moeller wrote:
= > >>>>>>
> >>>>> One solut= ion would be if ISPs made sure upload is 100% provisioned.
> &= gt;>>>> Could be cheaper than for (the higher rate) download.
> >>>>
> >>>> =C2=A0 =C2= =A0 Not going to happen, in my opinion, as economically unfeasible
> >>>> for a publicly traded ISP. I would settle for that = approach as long
> >>>> as the ISP is willing to f= ix its provisioning so that
> >>>> oversubscriptio= n episodes are reasonable rare, though.
> >>>
> >>> not going to happen on any network, publicly traded or= not.
> >
> > Sorry if this is a tangent fr= om where the current discussion has gone, but
> > I wanted = to correct someone saying something is "impossible".
&g= t; >
> <snip>
> >
> >= I guess I went off on this tangent because "Not going to happen, in m= y
> > opinion, as economically unfeasible" and "n= ot going to happen on any
> > network, publicly traded or n= ot." are too absolute. It can be done, it is
> > being= done, it is being done for cheap, and being done with "business
=
> > class" professionalism. Charter Comm is 1/2 the downloa= d speed for the same
> > price and they don't even have= symmetrical or dedicated.
>=C2=A0
> not being ov= ersubscribed includes the trunk. Who cares if there is no congestion=C2=A0<= /div>
> within the ISP if you reach the trunk and everything comes t= o a screeching halt.
>=C2=A0
> The reason I used = the word "imossible" is because we only have the ability to=C2=A0=
> make links so fast. Right now we have 10G common, 40G in pl= aces, and research=C2=A0
> into 100G, if you go back a few yea= rs, 1G was the limit. While the fastest=C2=A0
> connections ha= ve increased by a factor of 100, the home connections have=C2=A0
= > increased by close to a factor of 1000 during that time (1.5Mb theoret= ical DSL=C2=A0
> vs 1Gb fiber), and 10G is getting cheap enoug= h to be used for the corporate=C2=A0
> networks.
100Gb ports only cost about $5k and you can purchase, for unlis= ted prices, 36Tb/s mutliplexers over a single fiber with 1200km ranges. Tha= t's enough bandwidth for 360 100Gb ports. Congestion on trunks are easy= because of the large number of flows, you can have strong guarantees via s= tatistical multiplexing. The last mile is the main issue because that's= where a single person can make a difference.

>= =C2=A0
> so the ratio between the fastest link that's poss= ible and the subscribers has=C2=A0
> dropped from 1000:1 to 10= 0:1 with 10:1 uncomfortably close.
>=C2=A0
> some= of this can be covered up by deploying more lines, but that only goes so= =C2=A0
> far.
>=C2=A0
> If you are t= rying to guarantee no bandwidth limits on your network under any=C2=A0
> conditions, you need that sort of bandwith between all possible = sets of=C2=A0
> customers as well, which means that as you sca= le to more customers, your=C2=A0
> bandwidth requirements are = going up O(n^2).

O(n^2) is only needed if you desi= re a fully connected graph. When I have a 100Mb connection, it doesn't = mean I get a 100Mb connection to every customer at the same time, I get 100= Mb total. Otherwise with 1000 customer, that would mean I would have to hav= e 100Gb link at home. You only need O(n) scaling. Anyway, I don't think= you fully appreciate modern 1 petabit(500Tb/s ingress+egress) line rate co= re routers.=C2=A0

This is all commercially availab= le hardware, but would of course be incredibly expensive. 1 petabit router,= with 1,000 500Gb/s ports, each 500Gb port plugs into a fiber chassis with = 500 1Gb ports. Now you have 500,000 people all with fully dedicated 1Gb/s c= onnections. Of course you do not have any internet access with this setup b= ecause all of the router's ports are used for connecting customers. Cut= the customers in half, 250k customer, that frees up half of the ports. Now= you have 500 500Gb ports freed up. Take those 500Gb ports, plug those into= 36Tb/s multiplexers, get 14 strands of fiber, 7 up and 7 down, and run the= m to your local exchange points. Mind you this system would have 10x more b= andwidth than the entire world wide internet peak data usage. That includes= all CDNs, trunks, everything.

Of course the avera= ge user consumes less than 10Mb/s during peak hours, even if they have a 1G= b connection. All that matters is peak usage. Everything about society is b= ased on actual usage, not theoreticals that are about as likely as a duplic= ate Earth appearing next to us in space. Yes, it is possible, just not like= ly. Our electrical infrastructure could not handle every user at once, neit= her can out water, roads, phones, hospitals, stores, etc.

> And then the pricing comes into it. 1G fiber to the home is &l= t;$100/month (if it=C2=A0
> exists at all) isn't going to = pay for that sort of idle bandwidth.
>=C2=A0
> Bu= t the good thing is, you don't actually need that much bandwidth to kee= p your=C2=A0
> customer's happy either.
>=C2= =A0
> If the 'guaranteed bandwidth' is written into th= e contracts properly, there is a=C2=A0
> penalty to the compan= y if they can't provide you the bandwidth. That leaves them=C2=A0
=
> the possibility of not actually building out the O(n^2) network, = just keeping=C2=A0
> ahead of actual reqirements, and occasion= ally paying a penalty if they don't.
>=C2=A0
>= ; David Lang
--001a1137b69854549c05185f9d32--