From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yb1-xb36.google.com (mail-yb1-xb36.google.com [IPv6:2607:f8b0:4864:20::b36]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 91AFB3B29D for ; Wed, 15 Apr 2020 17:08:57 -0400 (EDT) Received: by mail-yb1-xb36.google.com with SMTP id l5so793479ybf.5 for ; Wed, 15 Apr 2020 14:08:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aenertia.net; s=dkimaenertianet; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hhTRrT9CjEbZLNOXnkCszMaR2HrL2MgVOApromNaA1s=; b=Hy0rx2e34mlY7fjqOsUSoldPWs34E/Fir+1cp4KSWYPXUzwSS81Yl9/ninJSO9ID0t MzhnETEZo8RNRh4WQ88cPfSKbPJv/QgjkzWyM2ljOUza78CvttkkfLbfPQ7Y1ceorY15 TTHPC5H98nwjmkwEHzvPJQtKibOpVfy/17l40= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hhTRrT9CjEbZLNOXnkCszMaR2HrL2MgVOApromNaA1s=; b=cFiH9FRLDHX8dNG+k14ppXxjNk7I35DPqXlt+WI3ACU0PNd5MozlAjEJWi+NvEroum RiWkPXx/pO/Vso4des9ZFjn1jqX//aSmnnNI3jjoCfe0eqbLySxkfzxBVwsZEmlhE68J DyFjOEATpNcJ5usrfXJq80Vt4ggQ94SKxb9hRYH4aeMtNiWAThJ1ZV9f4JaLk0wYuJqA k+pf32tnDZvgCtwcLwEmQWWClpSZZwMxsMPfBrd4dcifVZUNUPy2L+RliOPrKaaXGE8J Bmbts8u3bVIfWXPAcPFpKqRvG465WhYrWBxy2LlKQlwX4wePtryy7bfl1NAmvyPDdeKr 8N7Q== X-Gm-Message-State: AGi0PuYP4ezJ/gDrDg4aJMPI1l+81tigCAf3K1o5nWSlvbzExW76mKZ/ wMAzHIu/US2ATKdSnwqrIQj1cTb4LjOubohaqWInT6GN X-Google-Smtp-Source: APiQypIYcGPRVjwHwnRadNfjyej3mfBwXFuQVb+sElPsVHitZndJ4ZDxEfatgmR+/f/X7dgZyLiB9IFBlcHjD/CpdTg= X-Received: by 2002:a25:d702:: with SMTP id o2mr11698118ybg.256.1586984936936; Wed, 15 Apr 2020 14:08:56 -0700 (PDT) MIME-Version: 1.0 References: <1177.1586984260@localhost> In-Reply-To: <1177.1586984260@localhost> From: =?UTF-8?Q?Joel_Wir=C4=81mu_Pauling?= Date: Thu, 16 Apr 2020 09:08:45 +1200 Message-ID: To: Michael Richardson Cc: Mikael Abrahamsson , cerowrt-devel Content-Type: multipart/alternative; boundary="0000000000006e27e105a35ab946" Subject: Re: [Cerowrt-devel] 800gige X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Apr 2020 21:08:57 -0000 --0000000000006e27e105a35ab946 Content-Type: text/plain; charset="UTF-8" Another neat thing about 400 and 800GE is that you can get MPO optics that allow splitting a single 4x100 or 8x100 into individual 100G feeds. Good for port density and/or adding capacity to processing/Edge/Appliances Now there are decent ER optics for 100G you can now do 40-70KM runs of each 100G link without additional active electronics on the path or going to and optical transport route. On Thu, 16 Apr 2020 at 08:57, Michael Richardson wrote: > > Mikael Abrahamsson via Cerowrt-devel wrote: > > Backbone ISPs today are built with lots of parallel links (20x100GE > for > > instance) and then we do L4 hashing for flows across these. This > means > > got it. inverse multiplexing of flows across *links* > > > We're now going for 100 gigabit/s per lane (it's been going up from > 4x2.5G > > for 10GE to 1x10G, then we went for lane speeds of 10G, 25G, 50G and > now > > we're at 100G per lane), and it seems the 800GE in your link has 8 > lanes of > > that. This means a single L4 flow can be 800GE even though it's in > reality > > 8x100G lanes, as a single packet bits are being sprayed across all > the > > lanes. > > Here you talk about *lanes*, and inverse multiplexing of a single frame > across *lanes*. > Your allusion to PCI-E is well taken, but if I am completing the analogy, > and > the reference to DWDM, I'm thinking that you are talking about 100 > gigabit/s > per lambda, with a single frame being inverse multiplexed across lambdas > (as lanes). > > Did I understand this correctly? > > I understand a bit of "because we can". > I also understand that 20 x 800GE parallel links is better than 20 x 100GE > parallel links across the same long-haul (dark) fiber. > > But, what is the reason among ISPs to desire enabling a single L4 flow to > use more > than 100GE? Given that it seems that being able to L3 switch 800GE is > harder > than switching 8x flows of already L4 ordered 100GE. (Flowlabel!), why pay > the extra price here? > > While I can see L2VPN use cases, I can also see that L2VPNs could generate > multiple flows themselves if they wanted. > > -- > ] Never tell me the odds! | ipv6 mesh > networks [ > ] Michael Richardson, Sandelman Software Works | IoT > architect [ > ] mcr@sandelman.ca http://www.sandelman.ca/ | ruby on > rails [ > > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel > --0000000000006e27e105a35ab946 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Another neat thing about 400 and 800GE is that you can get MPO = optics that allow splitting a single 4x100 or 8x100 into individual 100G fe= eds. Good for port density and/or adding capacity to processing/Edge/Applia= nces

Now there are decent ER optics for 100G you can now do 40-70K= M runs of each 100G link without additional active electronics on the path = or going to and optical transport route.

On Thu, 16 Apr 2020 at 08= :57, Michael Richardson <mcr@sandelm= an.ca> wrote:

Mikael Abrahamsson via Cerowrt-devel wrote:
=C2=A0 =C2=A0 > Backbone ISPs today are built with lots of parallel link= s (20x100GE for
=C2=A0 =C2=A0 > instance) and then we do L4 hashing for flows across the= se. This means

got it. inverse multiplexing of flows across *links*

=C2=A0 =C2=A0 > We're now going for 100 gigabit/s per lane (it's= been going up from 4x2.5G
=C2=A0 =C2=A0 > for 10GE to 1x10G, then we went for lane speeds of 10G, = 25G, 50G and now
=C2=A0 =C2=A0 > we're at 100G per lane), and it seems the 800GE in y= our link has 8 lanes of
=C2=A0 =C2=A0 > that. This means a single L4 flow can be 800GE even thou= gh it's in reality
=C2=A0 =C2=A0 > 8x100G lanes, as a single packet bits are being sprayed = across all the
=C2=A0 =C2=A0 > lanes.

Here you talk about *lanes*, and inverse multiplexing of a single frame acr= oss *lanes*.
Your allusion to PCI-E is well taken, but if I am completing the analogy, a= nd
the reference to DWDM, I'm thinking that you are talking about 100 giga= bit/s
per lambda, with a single frame being inverse multiplexed across lambdas (a= s lanes).

Did I understand this correctly?

I understand a bit of "because we can".
I also understand that 20 x 800GE parallel links is better than 20 x 100GE<= br> parallel links across the same long-haul (dark) fiber.

But, what is the reason among ISPs to desire enabling a single L4 flow to u= se more
than 100GE?=C2=A0 Given that it seems that being able to L3 switch 800GE is= harder
than switching 8x flows of already L4 ordered 100GE. (Flowlabel!), why pay<= br> the extra price here?

While I can see L2VPN use cases, I can also see that L2VPNs could generate<= br> multiple flows themselves if they wanted.

--
]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Never tell me the o= dds!=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| ipv6 me= sh networks [
]=C2=A0 =C2=A0Michael Richardson, Sandelman Software Works=C2=A0 =C2=A0 =C2= =A0 =C2=A0 |=C2=A0 =C2=A0 IoT architect=C2=A0 =C2=A0[
]=C2=A0 =C2=A0 =C2=A0= mcr@sandelman.ca=C2=A0 http://www.sandelman.ca/=C2=A0 =C2=A0 =C2=A0 = =C2=A0 |=C2=A0 =C2=A0ruby on rails=C2=A0 =C2=A0 [

_______________________________________________
Cerowrt-devel mailing list
Ce= rowrt-devel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-d= evel
--0000000000006e27e105a35ab946--