From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 91DC33CB39 for ; Sun, 15 Oct 2023 12:45:52 -0400 (EDT) Received: by mail-pj1-x1035.google.com with SMTP id 98e67ed59e1d1-27d3ede72f6so1905857a91.1 for ; Sun, 15 Oct 2023 09:45:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697388351; x=1697993151; darn=lists.bufferbloat.net; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=T7memSEkEtrtmAAe+ZxsXPhr2SbqCWmFT8dBrj6K9oQ=; b=X9rv11EeC8C6zxWdme6gJ3j+3xs7HD9nPPTnROzN34j9O/svO+szRPNE6rk7HO/Eji iLGzLPW1iKysXlwQM33/YtU3MJ8lKzULy+o+0PayG4aKD2iHhhNiL9KGC6catklrD/fQ Oe6iQrwEy8ePElSnpeflXB9EQhvm9pjXGathgpHBiWVp9oB7WVyp6Wo9jPPwqTB9hvAK zu0zRqi1WXnK36NuILz4ONLBWqVseP/BAAFSUZh/cAdEtBwV6ptf59A4Nebc1FypwvBX RJ1GbfQj/Fqsm5T08Che8Q2IwzrYNizGXmh7oUx5B1pB1RVaYYBKjYzQ9Y14tEdZS27D tjuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697388351; x=1697993151; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T7memSEkEtrtmAAe+ZxsXPhr2SbqCWmFT8dBrj6K9oQ=; b=KLicEOOXgevoVVlKem/tEL445g9HZkor4sBXjQmcGb1rgLcErKg0uRgz2iohp4A6H1 mJMWuaESuYL5DAqkz1n8e2XRDxtO7IZGEmnDsp3TL8/0aNFXH+0QfE6o6/dfPxjK1lqF hwnL7cowd3vzdsH+jXW9XFBHpnZCNT1/3aVo2n6ZivLiJNNl0UjVBc4KCwn7wwPzxcXr YktvQw1/3GyiOrxA4JIOVJjjf0ydkFqSIHoP8Ahgh30eUhKsnGfDrXL6oj7/pj4aVxmB kb8RuypvzHgdMD+POLAi1oSpbuOwEjJ+Qwl8eVphR2ZSaH4HBYBwksJw4/kyyT2pnbsI NHnQ== X-Gm-Message-State: AOJu0YykdGIvHCbQY55Ckqtn1ZALQ0C82fqfbTgpox5mbEX+2lEFrkQM cvtAkb8Fsx08dzCeeXd5xpQNjjFZVRGpKfOpQAQ= X-Google-Smtp-Source: AGHT+IFuJ087RSFzCH6RVkNjLPOUcRP3Pyt0UGIhgGsSqXcO6w+ST0AN89AtzwWCNAZjUwte7cQoM+raQpYk2WtIgm4= X-Received: by 2002:a17:90b:23d4:b0:27d:b3d:5c33 with SMTP id md20-20020a17090b23d400b0027d0b3d5c33mr12977871pjb.28.1697388351313; Sun, 15 Oct 2023 09:45:51 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Sun, 15 Oct 2023 09:45:25 -0700 Message-ID: To: Tom Beecher Cc: Tim Burke , =?UTF-8?Q?Network_Neutrality_is_back=21_Let=C2=B4s_make_the_technical_aspect?= =?UTF-8?Q?s_heard_this_time=21?= , NANOG Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [NNagain] transit and peering costs projections X-BeenThere: nnagain@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: =?utf-8?q?Network_Neutrality_is_back!_Let=C2=B4s_make_the_technical_aspects_heard_this_time!?= List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Oct 2023 16:45:52 -0000 For starters I would like to apologize for cc-ing both nanog and my new nn list. (I will add sender filters) A bit more below. On Sun, Oct 15, 2023 at 9:32=E2=80=AFAM Tom Beecher wr= ote: >> >> So for now, we'll keep paying for transit to get to the others (since it= =E2=80=99s about as much as transporting IXP from Dallas), and hoping someo= ne at Google finally sees Houston as more than a third rate city hanging of= f of Dallas. Or=E2=80=A6 someone finally brings a worthwhile IX to Houston = that gets us more than peering to Kansas City. Yeah, I think the former is = more likely. =F0=9F=98=8A > > > There is often a chicken/egg scenario here with the economics. As an eyeb= all network, your costs to build out and connect to Dallas are greater than= your transit cost, so you do that. Totally fair. > > However think about it from the content side. Say I want to build into to= Houston. I have to put routers in, and a bunch of cache servers, so I have= capital outlay , plus opex for space, power, IX/backhaul/transit costs. Th= at's not cheap, so there's a lot of calculations that go into it. Is there = enough total eyeball traffic there to make it worth it? Is saving 8-10ms en= ough of a performance boost to justify the spend? What are the long term tr= ends in that market? These answers are of course different for a company ru= nning their own CDN vs the commercial CDNs. > > I don't work for Google and obviously don't speak for them, but I would s= uspect that they're happy to eat a 8-10ms performance hit to serve from Dal= las , versus the amount of capital outlay to build out there right now. The three forms of traffic I care most about are voip, gaming, and videoconferencing, which are rewarding to have at lower latencies. When I was a kid, we had switched phone networks, and while the sound quality was poorer than today, the voice latency cross-town was just like "being there". Nowadays we see 500+ms latencies for this kind of traffic. As to how to make calls across town work that well again, cost-wise, I do not know, but the volume of traffic that would be better served by these interconnects quite low, respective to the overall gains in lower latency experiences for them. > > On Sat, Oct 14, 2023 at 11:47=E2=80=AFPM Tim Burke wrote: >> >> I would say that a 1Gbit IP transit in a carrier neutral DC can be had f= or a good bit less than $900 on the wholesale market. >> >> Sadly, IXP=E2=80=99s are seemingly turning into a pay to play game, with= rates almost costing as much as transit in many cases after you factor in = loop costs. >> >> For example, in the Houston market (one of the largest and fastest growi= ng regions in the US!), we do not have a major IX, so to get up to Dallas i= t=E2=80=99s several thousand for a 100g wave, plus several thousand for a 1= 00g port on one of those major IXes. Or, a better option, we can get a 100g= flat internet transit for just a little bit more. >> >> Fortunately, for us as an eyeball network, there are a good number of ma= jor content networks that are allowing for private peering in markets like = Houston for just the cost of a cross connect and a QSFP if you=E2=80=99re i= n the right DC, with Google and some others being the outliers. >> >> So for now, we'll keep paying for transit to get to the others (since it= =E2=80=99s about as much as transporting IXP from Dallas), and hoping someo= ne at Google finally sees Houston as more than a third rate city hanging of= f of Dallas. Or=E2=80=A6 someone finally brings a worthwhile IX to Houston = that gets us more than peering to Kansas City. Yeah, I think the former is = more likely. =F0=9F=98=8A >> >> See y=E2=80=99all in San Diego this week, >> Tim >> >> On Oct 14, 2023, at 18:04, Dave Taht wrote: >> > >> > =EF=BB=BFThis set of trendlines was very interesting. Unfortunately th= e data >> > stops in 2015. Does anyone have more recent data? >> > >> > https://drpeering.net/white-papers/Internet-Transit-Pricing-Historical= -And-Projected.php >> > >> > I believe a gbit circuit that an ISP can resell still runs at about >> > $900 - $1.4k (?) in the usa? How about elsewhere? >> > >> > ... >> > >> > I am under the impression that many IXPs remain very successful, >> > states without them suffer, and I also find the concept of doing micro >> > IXPs at the city level, appealing, and now achievable with cheap gear. >> > Finer grained cross connects between telco and ISP and IXP would lower >> > latencies across town quite hugely... >> > >> > PS I hear ARIN is planning on dropping the price for, and bundling 3 >> > BGP AS numbers at a time, as of the end of this year, also. >> > >> > >> > >> > -- >> > Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bo= f.html >> > Dave T=C3=A4ht CSO, LibreQos --=20 Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.htm= l Dave T=C3=A4ht CSO, LibreQos