[Bloat] vyatta in AT&T 5G gear

[Bloat] vyatta in AT&T 5G gear

[Dave Taht]

Vyos (the open source fork of vyatta) was one of the first to add
fq_codel support... I wonder....

http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/

-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Bloat] vyatta in AT&T 5G gear

[Mikael Abrahamsson]

On Mon, 15 Oct 2018, Dave Taht wrote:

> Vyos (the open source fork of vyatta) was one of the first to add
> fq_codel support... I wonder....
>
> http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/

Isn't Vyos just running the Linux kernel for forwarding? So they received 
fq_codel for free when the Linux kernel got support for it? They just had 
to make it configurable?

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] vyatta in AT&T 5G gear

[Stefan Alfredsson]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

On 2018-10-16 11:31, Mikael Abrahamsson wrote:

> On Mon, 15 Oct 2018, Dave Taht wrote:
>
>> Vyos (the open source fork of vyatta) was one of the first to add
>> fq_codel support... I wonder....
>>
>> http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/ 
>>
>
> Isn't Vyos just running the Linux kernel for forwarding? So they 
> received fq_codel for free when the Linux kernel got support for it? 
> They just had to make it configurable?
>

Yes, according to this blog post, 
http://www.five-ten-sg.com/mapper/blog/Bufferbloat%20solved%20with%20Vyos

"Now that Vyos "helium" is available with a Linux 3.13 kernel, the 
fq_codel queueing discipline can be used to solve many bufferbloat 
issues. The nightly "lithium" builds contain my patches that allow 
fq_codel to be used via the native Vyos configuration system."

Anyway it's nice to see the Vyatta heritage living on in it's various 
forms (the AT&T "production hardened" Vyatta, to the Ubiquity EdgeMax 
and some UniFi devices, to the VyOS open version and now the future 
plans with dNOS -> DANOS.

/Stefan




[-- Attachment #2: Type: text/html, Size: 2475 bytes --]

Re: [Bloat] vyatta in AT&T 5G gear

[Stephen Hemminger]

On Tue, 16 Oct 2018 11:59:18 +0200
Stefan Alfredsson <stefan.alfredsson@kau.se> wrote:

> On 2018-10-16 11:31, Mikael Abrahamsson wrote:
> 
> > On Mon, 15 Oct 2018, Dave Taht wrote:
> >  
> >> Vyos (the open source fork of vyatta) was one of the first to add
> >> fq_codel support... I wonder....
> >>
> >> http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/ 
> >>  
> >
> > Isn't Vyos just running the Linux kernel for forwarding? So they 
> > received fq_codel for free when the Linux kernel got support for it? 
> > They just had to make it configurable?
> >  
> 
> Yes, according to this blog post, 
> http://www.five-ten-sg.com/mapper/blog/Bufferbloat%20solved%20with%20Vyos
> 
> "Now that Vyos "helium" is available with a Linux 3.13 kernel, the 
> fq_codel queueing discipline can be used to solve many bufferbloat 
> issues. The nightly "lithium" builds contain my patches that allow 
> fq_codel to be used via the native Vyos configuration system."
> 
> Anyway it's nice to see the Vyatta heritage living on in it's various 
> forms (the AT&T "production hardened" Vyatta, to the Ubiquity EdgeMax 
> and some UniFi devices, to the VyOS open version and now the future 
> plans with dNOS -> DANOS.
> 
> /Stefan
> 
> 
> 

There are two basic components to network OS, the control plane and the
data plane.  VyOs has the old V1 which is filesystem based control plane
and kernel dataplane. DaNoS has yang/netconf database based control plane
(in Go) and DPDK (or switch offload?) based dataplane.  Ubiquity redid
the control plane as well, and uses their own hardware for dataplane.

So more of "my grandfather's ax"...

Re: [Bloat] vyatta in AT&T 5G gear

[Dave Taht]

On Tue, Oct 16, 2018 at 8:06 AM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> On Tue, 16 Oct 2018 11:59:18 +0200
> Stefan Alfredsson <stefan.alfredsson@kau.se> wrote:
>
> > On 2018-10-16 11:31, Mikael Abrahamsson wrote:
> >
> > > On Mon, 15 Oct 2018, Dave Taht wrote:
> > >
> > >> Vyos (the open source fork of vyatta) was one of the first to add
> > >> fq_codel support... I wonder....
> > >>
> > >> http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/
> > >>
> > >
> > > Isn't Vyos just running the Linux kernel for forwarding? So they
> > > received fq_codel for free when the Linux kernel got support for it?
> > > They just had to make it configurable?
> > >
> >
> > Yes, according to this blog post,
> > http://www.five-ten-sg.com/mapper/blog/Bufferbloat%20solved%20with%20Vyos
> >
> > "Now that Vyos "helium" is available with a Linux 3.13 kernel, the
> > fq_codel queueing discipline can be used to solve many bufferbloat
> > issues. The nightly "lithium" builds contain my patches that allow
> > fq_codel to be used via the native Vyos configuration system."
> >
> > Anyway it's nice to see the Vyatta heritage living on in it's various
> > forms (the AT&T "production hardened" Vyatta, to the Ubiquity EdgeMax
> > and some UniFi devices, to the VyOS open version and now the future
> > plans with dNOS -> DANOS.
> >
> > /Stefan
> >
> >
> >
>
> There are two basic components to network OS, the control plane and the
> data plane.  VyOs has the old V1 which is filesystem based control plane
> and kernel dataplane. DaNoS has yang/netconf database based control plane
> (in Go) and DPDK (or switch offload?) based dataplane.  Ubiquity redid
> the control plane as well, and uses their own hardware for dataplane.
>
> So more of "my grandfather's ax"...

so in other words we should have done a dpdk version long ago. And even then,
this white box brags of the "deep buffering" in the switch chip.

... and I have an initial report of 2 seconds of buffering on one of
the first 5G devices.

Sigh.

And what was so wrong with the "everything as a file" model??

double sigh.

> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Bloat] vyatta in AT&T 5G gear

[Jan Ceuleers]

On 16/10/2018 17:14, Dave Taht wrote:
> And what was so wrong with the "everything as a file" model??

On a single box - not a lot.

On a 5G network, which might consist of 10^3 - 10^5 boxes you need
aggregation and abstraction layers.

Re: [Bloat] vyatta in AT&T 5G gear

[Stephen Hemminger]

On Tue, 16 Oct 2018 08:14:36 -0700
Dave Taht <dave.taht@gmail.com> wrote:

> On Tue, Oct 16, 2018 at 8:06 AM Stephen Hemminger
> <stephen@networkplumber.org> wrote:
> >
> > On Tue, 16 Oct 2018 11:59:18 +0200
> > Stefan Alfredsson <stefan.alfredsson@kau.se> wrote:
> >  
> > > On 2018-10-16 11:31, Mikael Abrahamsson wrote:
> > >  
> > > > On Mon, 15 Oct 2018, Dave Taht wrote:
> > > >  
> > > >> Vyos (the open source fork of vyatta) was one of the first to add
> > > >> fq_codel support... I wonder....
> > > >>
> > > >> http://linuxgizmos.com/att-releases-white-box-spec-for-its-linux-based-5g-routers/
> > > >>  
> > > >
> > > > Isn't Vyos just running the Linux kernel for forwarding? So they
> > > > received fq_codel for free when the Linux kernel got support for it?
> > > > They just had to make it configurable?
> > > >  
> > >
> > > Yes, according to this blog post,
> > > http://www.five-ten-sg.com/mapper/blog/Bufferbloat%20solved%20with%20Vyos
> > >
> > > "Now that Vyos "helium" is available with a Linux 3.13 kernel, the
> > > fq_codel queueing discipline can be used to solve many bufferbloat
> > > issues. The nightly "lithium" builds contain my patches that allow
> > > fq_codel to be used via the native Vyos configuration system."
> > >
> > > Anyway it's nice to see the Vyatta heritage living on in it's various
> > > forms (the AT&T "production hardened" Vyatta, to the Ubiquity EdgeMax
> > > and some UniFi devices, to the VyOS open version and now the future
> > > plans with dNOS -> DANOS.
> > >
> > > /Stefan
> > >
> > >
> > >  
> >
> > There are two basic components to network OS, the control plane and the
> > data plane.  VyOs has the old V1 which is filesystem based control plane
> > and kernel dataplane. DaNoS has yang/netconf database based control plane
> > (in Go) and DPDK (or switch offload?) based dataplane.  Ubiquity redid
> > the control plane as well, and uses their own hardware for dataplane.
> >
> > So more of "my grandfather's ax"...  
> 
> so in other words we should have done a dpdk version long ago. And even then,
> this white box brags of the "deep buffering" in the switch chip.
> 
> ... and I have an initial report of 2 seconds of buffering on one of
> the first 5G devices.
> 
> Sigh.
> 
> And what was so wrong with the "everything as a file" model??

Two things were an issue. At scale, the filesystem was a bottleneck and it
would have been harder to implement netconf/yang model as required by
the big boy market.

Filesystem works as toy. (see plan 9) 

Re: [Bloat] vyatta in AT&T 5G gear

[Dave Taht]

Jan Ceuleers <jan.ceuleers@gmail.com> writes:

> On 16/10/2018 17:14, Dave Taht wrote:
>> And what was so wrong with the "everything as a file" model??
>
> On a single box - not a lot.
>
> On a 5G network, which might consist of 10^3 - 10^5 boxes you need
> aggregation and abstraction layers.

I was mostly just venting. Still... I'd hoped that the next generation
fios gear would get bufferbloat right... and with all the 5G boasting
about the 1ms mac, that they'd get queuing delay below 20ms also.

I was also hoping the DOCSIS 3.1 line cards would get it right...

It has been a disappointing day.

Re: [Bloat] vyatta in AT&T 5G gear

[Stephen Hemminger]

On Tue, 16 Oct 2018 17:46:44 -0700
Dave Taht <dave@taht.net> wrote:

> Jan Ceuleers <jan.ceuleers@gmail.com> writes:
> 
> > On 16/10/2018 17:14, Dave Taht wrote:  
> >> And what was so wrong with the "everything as a file" model??  
> >
> > On a single box - not a lot.
> >
> > On a 5G network, which might consist of 10^3 - 10^5 boxes you need
> > aggregation and abstraction layers.  
> 
> I was mostly just venting. Still... I'd hoped that the next generation
> fios gear would get bufferbloat right... and with all the 5G boasting
> about the 1ms mac, that they'd get queuing delay below 20ms also.
> 
> I was also hoping the DOCSIS 3.1 line cards would get it right...
> 
> It has been a disappointing day.

At scale, you need a usable programming model; on box CLI or files don't work well.
Netconf/yang was built for that.

Re: [Bloat] vyatta in AT&T 5G gear

[Jonas Mårtensson]

[-- Attachment #1: Type: text/plain, Size: 297 bytes --]

Hi Stephen,

DaNoS has yang/netconf database based control plane
> (in Go) and DPDK (or switch offload?) based dataplane.
>

Do you have more information about DANOS and using Go for the control
plane? The AT&T design uses a Broadcom Qumran-AX switching chip for the
dataplane so no DPDK.

/Jonas

[-- Attachment #2: Type: text/html, Size: 575 bytes --]

Re: [Bloat] vyatta in AT&T 5G gear

[Stephen Hemminger]

On Sun, 21 Oct 2018 11:36:43 +0200
Jonas Mårtensson <martensson.jonas@gmail.com> wrote:

> Hi Stephen,
> 
> DaNoS has yang/netconf database based control plane
> > (in Go) and DPDK (or switch offload?) based dataplane.
> >  
> 
> Do you have more information about DANOS and using Go for the control
> plane? The AT&T design uses a Broadcom Qumran-AX switching chip for the
> dataplane so no DPDK.
> 
> /Jonas

Linux Foundation announced Danos, but no code is yet available.

Re: [Bloat] vyatta in AT&T 5G gear

[Dave Taht]

On Mon, Oct 22, 2018 at 9:42 AM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> On Sun, 21 Oct 2018 11:36:43 +0200
> Jonas Mårtensson <martensson.jonas@gmail.com> wrote:
>
> > Hi Stephen,
> >
> > DaNoS has yang/netconf database based control plane
> > > (in Go) and DPDK (or switch offload?) based dataplane.
> > >
> >
> > Do you have more information about DANOS and using Go for the control
> > plane? The AT&T design uses a Broadcom Qumran-AX switching chip for the
> > dataplane so no DPDK.
> >
> > /Jonas
>
> Linux Foundation announced Danos, but no code is yet available.

Not even which license seems to be available.

Still... I guess this is going to be "a thing", and we should probably
get involved.

I'd like to see P4 evolve a bit more "our way".
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740

Re: [Bloat] vyatta in AT&T 5G gear

[Toke Høiland-Jørgensen]

Dave Taht <dave.taht@gmail.com> writes:

> On Mon, Oct 22, 2018 at 9:42 AM Stephen Hemminger
> <stephen@networkplumber.org> wrote:
>>
>> On Sun, 21 Oct 2018 11:36:43 +0200
>> Jonas Mårtensson <martensson.jonas@gmail.com> wrote:
>>
>> > Hi Stephen,
>> >
>> > DaNoS has yang/netconf database based control plane
>> > > (in Go) and DPDK (or switch offload?) based dataplane.
>> > >
>> >
>> > Do you have more information about DANOS and using Go for the control
>> > plane? The AT&T design uses a Broadcom Qumran-AX switching chip for the
>> > dataplane so no DPDK.
>> >
>> > /Jonas
>>
>> Linux Foundation announced Danos, but no code is yet available.
>
> Not even which license seems to be available.
>
> Still... I guess this is going to be "a thing", and we should probably
> get involved.
>
> I'd like to see P4 evolve a bit more "our way".

There's a fairly new traffic management working group in the P4 org.
They're trying to define an extension to the P4 language to describe a
programmable interface to packet scheduling and traffic shaping.

You have to be a member of the P4 consortium to take part (most of the
lists on http://lists.p4.org/mailman/listinfo are members only). It's
free for individuals, though; but you do have to sign a CLA. See
https://p4.org/join/

-Toke

Re: [Bloat] vyatta in AT&T 5G gear

[Jonas Mårtensson]

[-- Attachment #1: Type: text/plain, Size: 223 bytes --]

>
> Linux Foundation announced Danos, but no code is yet available.
>

Yeah I know, but you seemed to know it is written in Go, which I can't find
any information about, so I thought maybe you had more information.

/Jonas

[-- Attachment #2: Type: text/html, Size: 443 bytes --]

Re: [Bloat] vyatta in AT&T 5G gear

[Stephen Hemminger]

On Tue, 23 Oct 2018 09:38:10 +0200
Jonas Mårtensson <martensson.jonas@gmail.com> wrote:

> >
> > Linux Foundation announced Danos, but no code is yet available.
> >  
> 
> Yeah I know, but you seemed to know it is written in Go, which I can't find
> any information about, so I thought maybe you had more information.
> 
> /Jonas

I used to work at Brocade. Didn't work directly on the control handler but did
write lots of yang models for it. Yes it was in Go at the time.