From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DCD033B29D for ; Wed, 15 Apr 2020 17:35:22 -0400 (EDT) Received: by mail-io1-xd32.google.com with SMTP id o127so18885743iof.0 for ; Wed, 15 Apr 2020 14:35:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=zBDkwmX8n35cE1HuwAWzWUNxbWPJOo4xL/DR0mHifaY=; b=H3li/1cB/qG4+UNSvjnroE6YhI69SH3mgdWXjuRl/APr2qBnysEDD87lXQE9Hqq2yF vn2659Yr8Cbb7BPQ8uePRMmzG2yGC3uBW2X1o+4TWpZuyEOgchPTYlOkM9ggmNtSEhJl avXYliMxz/5Lgh1qa/SsY3g8g3kYY44CVrg8v9IBGojfBnfkfb2cSFjOQJRe/wG7sZMK Y6rCa7n8MAeW4NAvLGNNnhWckBoqr1h28B3jndlhoGHMkncHdF9mZXkHf/lOsNfx7C9j 5uUtgOxSVNQiB4lvH/jkM3VGDEhqiSPihqtn4MtgxjNkWMWzPE6XGNQJiDm8o6GZrVtj qWKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=zBDkwmX8n35cE1HuwAWzWUNxbWPJOo4xL/DR0mHifaY=; b=unFmVW6qqdYHbNuVfhIIdW16Y9WRmrgFKQS9NkWfd4M29AMHDMVuA83agOnI6BHBda uA99rYkHrXOQoM075qKUUhXp4tRRgyYlNMqH+NIbtartxOCzjV9TBTp9FYPJKMm5jrzP C6T/P6Jlj56pZxvTzB5S+o6ziGvPG0Evv4r31oTaH7EKWVzffrh8UOu1dtE1MlX0zWuR RbTc8HTGnlZHzcin0+x1eu4v+oD+unKWzOSbI3P3mWTZo9yZT9pUm6VxrOMwVkpgpKIA mqj8mOkZ/s+7NW6GU9bvq1V4liWEw9rzs15pjo037Uwyq+ay/sW3eAHKRrhAZmeXUeFG l/tg== X-Gm-Message-State: AGi0PuY+G3jQy6aDC5PFqReyQ4jpClXCLvrZnWiy+8NSVTwIzbX2jyWp z2Dq65n3DKqyn+mMVXtBHDkpWnHUzgdjd/08T1Q= X-Google-Smtp-Source: APiQypJGQMv5WCuWizWHnCudZkmYzxS+rnYdf+iwl+j7TWW9/R90KzCqFEuSxo/3emZA//UrDGFfh5UWCD6jLd/8QMQ= X-Received: by 2002:a5d:93cd:: with SMTP id j13mr28281898ioo.27.1586986522282; Wed, 15 Apr 2020 14:35:22 -0700 (PDT) MIME-Version: 1.0 References: <1177.1586984260@localhost> In-Reply-To: From: Dave Taht Date: Wed, 15 Apr 2020 14:35:10 -0700 Message-ID: To: =?UTF-8?Q?Joel_Wir=C4=81mu_Pauling?= Cc: Michael Richardson , cerowrt-devel Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cerowrt-devel] 800gige X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Apr 2020 21:35:23 -0000 I've always kind of wanted a guestimate and cost breakdown (politics/fiber cost/trenching) as to how much it costs to run, oh, 16km of quality 100GBit fiber from los gatos to me. I know, month to month that would kind of cost a lot to fill.... I costed out what it would take to trench the whole community once upon a time, and instead of that I've been patiently awaiting my first starlink terminals.... https://www.google.com/maps/place/20600+Aldercroft+Heights+Rd,+Los+Gatos,+C= A+95033/@37.1701322,-121.9806674,17z/data=3D!3m1!4b1!4m5!3m4!1s0x808e37ced6= 0da4fd:0x189086a00c73ad37!8m2!3d37.1701322!4d-121.9784787 On Wed, Apr 15, 2020 at 2:09 PM Joel Wir=C4=81mu Pauling wrote: > > Another neat thing about 400 and 800GE is that you can get MPO optics tha= t allow splitting a single 4x100 or 8x100 into individual 100G feeds. Good = for port density and/or adding capacity to processing/Edge/Appliances > > Now there are decent ER optics for 100G you can now do 40-70KM runs of ea= ch 100G link without additional active electronics on the path or going to = and optical transport route. > > On Thu, 16 Apr 2020 at 08:57, Michael Richardson wrote= : >> >> >> Mikael Abrahamsson via Cerowrt-devel wrote: >> > Backbone ISPs today are built with lots of parallel links (20x100G= E for >> > instance) and then we do L4 hashing for flows across these. This m= eans >> >> got it. inverse multiplexing of flows across *links* >> >> > We're now going for 100 gigabit/s per lane (it's been going up fro= m 4x2.5G >> > for 10GE to 1x10G, then we went for lane speeds of 10G, 25G, 50G a= nd now >> > we're at 100G per lane), and it seems the 800GE in your link has 8= lanes of >> > that. This means a single L4 flow can be 800GE even though it's in= reality >> > 8x100G lanes, as a single packet bits are being sprayed across all= the >> > lanes. >> >> Here you talk about *lanes*, and inverse multiplexing of a single frame = across *lanes*. >> Your allusion to PCI-E is well taken, but if I am completing the analogy= , and >> the reference to DWDM, I'm thinking that you are talking about 100 gigab= it/s >> per lambda, with a single frame being inverse multiplexed across lambdas= (as lanes). >> >> Did I understand this correctly? >> >> I understand a bit of "because we can". >> I also understand that 20 x 800GE parallel links is better than 20 x 100= GE >> parallel links across the same long-haul (dark) fiber. >> >> But, what is the reason among ISPs to desire enabling a single L4 flow t= o use more >> than 100GE? Given that it seems that being able to L3 switch 800GE is h= arder >> than switching 8x flows of already L4 ordered 100GE. (Flowlabel!), why p= ay >> the extra price here? >> >> While I can see L2VPN use cases, I can also see that L2VPNs could genera= te >> multiple flows themselves if they wanted. >> >> -- >> ] Never tell me the odds! | ipv6 mesh netw= orks [ >> ] Michael Richardson, Sandelman Software Works | IoT archite= ct [ >> ] mcr@sandelman.ca http://www.sandelman.ca/ | ruby on rail= s [ >> >> _______________________________________________ >> Cerowrt-devel mailing list >> Cerowrt-devel@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cerowrt-devel > > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel --=20 Make Music, Not War Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-435-0729