[Cerowrt-devel] 10gige and 2.5gige

[Cerowrt-devel] 10gige and 2.5gige

[Dave Taht]

has really got cheap.

https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter

On the other hand users are reporting issues with actually using
2.5ghz cable with this router in particular, halving the achieved rate
by negotiating 2.5gbit vs negotiating 1gbit.

https://forum.mikrotik.com/viewtopic.php?t=179145#p897836


-- 
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC

Re: [Cerowrt-devel] 10gige and 2.5gige

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 2172 bytes --]


Yes, it's very cheap and getting cheaper.
 
Since its price fell to the point I thought was cheap, my home has a 10 GigE fiber backbone, 2 switches in my main centers of computers, lots of 10 GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3 external adapter for my primary desktop, which is a Skull Canyon NUC.
 
I strongly recommend people use fiber and sfp+ DAC cabling because twisted pair, while cheaper, actually is problematic at speeds above 1 Gig - mostly due to power and heat.
 
BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and USB-C connectors and cables can carry Thunderbolt at higher rates.  Those adapters are REALLY CHEAP. There's nothing inherently different about the electronics, if anything, USB 3.1 is more complicate logic than the ethernet MAC.
 
So the reason 10 GigE is still far more expensive than USB 3.1 is mainly market volume - if 10 GigE were a consumer product, not a datacenter product, you'd think it would already be as cheap as USB 3.1 in computers and switches.
 
Since DOCSIS can support up to 5 Gb/s, I think, when will Internet Access Providers start offering "Cable Modems" that support customers who want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out there, it's just a configuration change. 
 
So when will consumer "routers" support 5 Gig, 10 Gig?
 
On Thursday, December 16, 2021 11:20am, "Dave Taht" <dave.taht@gmail.com> said:



> has really got cheap.
> 
> https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter
> 
> On the other hand users are reporting issues with actually using
> 2.5ghz cable with this router in particular, halving the achieved rate
> by negotiating 2.5gbit vs negotiating 1gbit.
> 
> https://forum.mikrotik.com/viewtopic.php?t=179145#p897836
> 
> 
> --
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> 
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

[-- Attachment #2: Type: text/html, Size: 4014 bytes --]

Re: [Cerowrt-devel] 10gige and 2.5gige

[Joel Wirāmu Pauling]

[-- Attachment #1: Type: text/plain, Size: 3221 bytes --]

Heat issues you mention with UTP are gone; with the 803.bz stuff (i.e
Base-N).

It was mostly due to the 10G-Base-T spec being old and out of line with the
SFP+ spec ; which led to higher power consumption than SFP+ cages were
rated to draw and aforementioned heat problems; this is not a problem with
newer kit.
It went away with the move to smaller silicon processes and now UTP based
10G in the home devices are more common and don't suffer from the fragility
issues of the earlier copper based 10G spec. The AQC chipsets were the
first to introduce it but most other vendors have finally picked it up
after 5 years or feet dragging.



On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <dpreed@deepplum.com> wrote:

> Yes, it's very cheap and getting cheaper.
>
>
>
> Since its price fell to the point I thought was cheap, my home has a 10
> GigE fiber backbone, 2 switches in my main centers of computers, lots of 10
> GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3
> external adapter for my primary desktop, which is a Skull Canyon NUC.
>
>
>
> I strongly recommend people use fiber and sfp+ DAC cabling because twisted
> pair, while cheaper, actually is problematic at speeds above 1 Gig - mostly
> due to power and heat.
>
>
>
> BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and
> USB-C connectors and cables can carry Thunderbolt at higher rates.  Those
> adapters are REALLY CHEAP. There's nothing inherently different about the
> electronics, if anything, USB 3.1 is more complicate logic than the
> ethernet MAC.
>
>
>
> So the reason 10 GigE is still far more expensive than USB 3.1 is mainly
> market volume - if 10 GigE were a consumer product, not a datacenter
> product, you'd think it would already be as cheap as USB 3.1 in computers
> and switches.
>
>
>
> Since DOCSIS can support up to 5 Gb/s, I think, when will Internet Access
> Providers start offering "Cable Modems" that support customers who want
> more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out
> there, it's just a configuration change.
>
>
>
> So when will consumer "routers" support 5 Gig, 10 Gig?
>
>
>
> On Thursday, December 16, 2021 11:20am, "Dave Taht" <dave.taht@gmail.com>
> said:
>
> > has really got cheap.
> >
> > https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter
> >
> > On the other hand users are reporting issues with actually using
> > 2.5ghz cable with this router in particular, halving the achieved rate
> > by negotiating 2.5gbit vs negotiating 1gbit.
> >
> > https://forum.mikrotik.com/viewtopic.php?t=179145#p897836
> >
> >
> > --
> > I tried to build a better future, a few times:
> > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >
> > Dave Täht CEO, TekLibre, LLC
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> >
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

[-- Attachment #2: Type: text/html, Size: 5916 bytes --]

Re: [Cerowrt-devel] 10gige and 2.5gige

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 3891 bytes --]


Thanks, That's good to know...The whole SFP+ adapter concept has seemed to me to be a "tweener" in hardware design space. Too many failure points. That said, I like fiber's properties as a medium for distances.
 
 
On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling" <joel@aenertia.net> said:




Heat issues you mention with UTP are gone; with the [ 803.bz ]( http://803.bz ) stuff (i.e Base-N). 
It was mostly due to the 10G-Base-T spec being old and out of line with the SFP+ spec ; which led to higher power consumption than SFP+ cages were rated to draw and aforementioned heat problems; this is not a problem with newer kit.
It went away with the move to smaller silicon processes and now UTP based 10G in the home devices are more common and don't suffer from the fragility issues of the earlier copper based 10G spec. The AQC chipsets were the first to introduce it but most other vendors have finally picked it up after 5 years or feet dragging. 


On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com ]( mailto:dpreed@deepplum.com )> wrote:
Yes, it's very cheap and getting cheaper.
 
Since its price fell to the point I thought was cheap, my home has a 10 GigE fiber backbone, 2 switches in my main centers of computers, lots of 10 GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3 external adapter for my primary desktop, which is a Skull Canyon NUC.
 
I strongly recommend people use fiber and sfp+ DAC cabling because twisted pair, while cheaper, actually is problematic at speeds above 1 Gig - mostly due to power and heat.
 
BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and USB-C connectors and cables can carry Thunderbolt at higher rates.  Those adapters are REALLY CHEAP. There's nothing inherently different about the electronics, if anything, USB 3.1 is more complicate logic than the ethernet MAC.
 
So the reason 10 GigE is still far more expensive than USB 3.1 is mainly market volume - if 10 GigE were a consumer product, not a datacenter product, you'd think it would already be as cheap as USB 3.1 in computers and switches.
 
Since DOCSIS can support up to 5 Gb/s, I think, when will Internet Access Providers start offering "Cable Modems" that support customers who want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out there, it's just a configuration change. 
 
So when will consumer "routers" support 5 Gig, 10 Gig?
 
On Thursday, December 16, 2021 11:20am, "Dave Taht" <[ dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:



> has really got cheap.
> 
> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter ]( https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
> 
> On the other hand users are reporting issues with actually using
> 2.5ghz cable with this router in particular, halving the achieved rate
> by negotiating 2.5gbit vs negotiating 1gbit.
> 
> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ]( https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
> 
> 
> --
> I tried to build a better future, a few times:
> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org ]( https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
> 
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> Cerowrt-devel mailing list
> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )
>_______________________________________________
 Cerowrt-devel mailing list
[ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
[ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )

[-- Attachment #2: Type: text/html, Size: 6907 bytes --]

Re: [Cerowrt-devel] 10gige and 2.5gige

[David Lang]

[-- Attachment #1: Type: text/plain, Size: 4244 bytes --]

another valuable featur of fiber for home use is that fiber can't contribute to 
ground loops the way that copper cables can.

and for the paranoid (like me :-) ) fiber also means that any electrical 
disaster that happens to one end won't propgate through and fry other equipment

David Lang

On Thu, 16 Dec 2021, David P. Reed wrote:

> Thanks, That's good to know...The whole SFP+ adapter concept has seemed to me to be a "tweener" in hardware design space. Too many failure points. That said, I like fiber's properties as a medium for distances.
> 
> 
> On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling" <joel@aenertia.net> said:
>
>
>
>
> Heat issues you mention with UTP are gone; with the [ 803.bz ]( http://803.bz ) stuff (i.e Base-N). 
> It was mostly due to the 10G-Base-T spec being old and out of line with the SFP+ spec ; which led to higher power consumption than SFP+ cages were rated to draw and aforementioned heat problems; this is not a problem with newer kit.
> It went away with the move to smaller silicon processes and now UTP based 10G in the home devices are more common and don't suffer from the fragility issues of the earlier copper based 10G spec. The AQC chipsets were the first to introduce it but most other vendors have finally picked it up after 5 years or feet dragging. 
>
>
> On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com ]( mailto:dpreed@deepplum.com )> wrote:
> Yes, it's very cheap and getting cheaper.
> 
> Since its price fell to the point I thought was cheap, my home has a 10 GigE fiber backbone, 2 switches in my main centers of computers, lots of 10 GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3 external adapter for my primary desktop, which is a Skull Canyon NUC.
> 
> I strongly recommend people use fiber and sfp+ DAC cabling because twisted pair, while cheaper, actually is problematic at speeds above 1 Gig - mostly due to power and heat.
> 
> BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and USB-C connectors and cables can carry Thunderbolt at higher rates.  Those adapters are REALLY CHEAP. There's nothing inherently different about the electronics, if anything, USB 3.1 is more complicate logic than the ethernet MAC.
> 
> So the reason 10 GigE is still far more expensive than USB 3.1 is mainly market volume - if 10 GigE were a consumer product, not a datacenter product, you'd think it would already be as cheap as USB 3.1 in computers and switches.
> 
> Since DOCSIS can support up to 5 Gb/s, I think, when will Internet Access Providers start offering "Cable Modems" that support customers who want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out there, it's just a configuration change. 
> 
> So when will consumer "routers" support 5 Gig, 10 Gig?
> 
> On Thursday, December 16, 2021 11:20am, "Dave Taht" <[ dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:
>
>
>
>> has really got cheap.
>> 
>> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter ]( https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
>> 
>> On the other hand users are reporting issues with actually using
>> 2.5ghz cable with this router in particular, halving the achieved rate
>> by negotiating 2.5gbit vs negotiating 1gbit.
>> 
>> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ]( https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
>> 
>> 
>> --
>> I tried to build a better future, a few times:
>> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org ]( https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
>> 
>> Dave Täht CEO, TekLibre, LLC
>> _______________________________________________
>> Cerowrt-devel mailing list
>> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
>> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )
>> _______________________________________________
> Cerowrt-devel mailing list
> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )

[-- Attachment #2: Type: text/plain, Size: 164 bytes --]

_______________________________________________
Cerowrt-devel mailing list
Cerowrt-devel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] 10gige and 2.5gige

[Joel Wirāmu Pauling]

[-- Attachment #1: Type: text/plain, Size: 4967 bytes --]

Yes but as much as I like fibre; it's too fragile for the average household
structured cabling real world use case. Not to mention nothing consumwe
comes with SFP+ in the home space.

On Fri, 17 Dec 2021, 10:43 am David Lang, <david@lang.hm> wrote:

> another valuable featur of fiber for home use is that fiber can't
> contribute to
> ground loops the way that copper cables can.
>
> and for the paranoid (like me :-) ) fiber also means that any electrical
> disaster that happens to one end won't propgate through and fry other
> equipment
>
> David Lang
>
> On Thu, 16 Dec 2021, David P. Reed wrote:
>
> > Thanks, That's good to know...The whole SFP+ adapter concept has seemed
> to me to be a "tweener" in hardware design space. Too many failure points.
> That said, I like fiber's properties as a medium for distances.
> >
> >
> > On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling" <
> joel@aenertia.net> said:
> >
> >
> >
> >
> > Heat issues you mention with UTP are gone; with the [ 803.bz ](
> http://803.bz ) stuff (i.e Base-N).
> > It was mostly due to the 10G-Base-T spec being old and out of line with
> the SFP+ spec ; which led to higher power consumption than SFP+ cages were
> rated to draw and aforementioned heat problems; this is not a problem with
> newer kit.
> > It went away with the move to smaller silicon processes and now UTP
> based 10G in the home devices are more common and don't suffer from the
> fragility issues of the earlier copper based 10G spec. The AQC chipsets
> were the first to introduce it but most other vendors have finally picked
> it up after 5 years or feet dragging.
> >
> >
> > On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com ](
> mailto:dpreed@deepplum.com )> wrote:
> > Yes, it's very cheap and getting cheaper.
> >
> > Since its price fell to the point I thought was cheap, my home has a 10
> GigE fiber backbone, 2 switches in my main centers of computers, lots of 10
> GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3
> external adapter for my primary desktop, which is a Skull Canyon NUC.
> >
> > I strongly recommend people use fiber and sfp+ DAC cabling because
> twisted pair, while cheaper, actually is problematic at speeds above 1 Gig
> - mostly due to power and heat.
> >
> > BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and
> USB-C connectors and cables can carry Thunderbolt at higher rates.  Those
> adapters are REALLY CHEAP. There's nothing inherently different about the
> electronics, if anything, USB 3.1 is more complicate logic than the
> ethernet MAC.
> >
> > So the reason 10 GigE is still far more expensive than USB 3.1 is mainly
> market volume - if 10 GigE were a consumer product, not a datacenter
> product, you'd think it would already be as cheap as USB 3.1 in computers
> and switches.
> >
> > Since DOCSIS can support up to 5 Gb/s, I think, when will Internet
> Access Providers start offering "Cable Modems" that support customers who
> want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out
> there, it's just a configuration change.
> >
> > So when will consumer "routers" support 5 Gig, 10 Gig?
> >
> > On Thursday, December 16, 2021 11:20am, "Dave Taht" <[
> dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:
> >
> >
> >
> >> has really got cheap.
> >>
> >> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter ](
> https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
> >>
> >> On the other hand users are reporting issues with actually using
> >> 2.5ghz cable with this router in particular, halving the achieved rate
> >> by negotiating 2.5gbit vs negotiating 1gbit.
> >>
> >> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ](
> https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
> >>
> >>
> >> --
> >> I tried to build a better future, a few times:
> >> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org ](
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
> >>
> >> Dave Täht CEO, TekLibre, LLC
> >> _______________________________________________
> >> Cerowrt-devel mailing list
> >> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:
> Cerowrt-devel@lists.bufferbloat.net )
> >> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ](
> https://lists.bufferbloat.net/listinfo/cerowrt-devel )
> >> _______________________________________________
> > Cerowrt-devel mailing list
> > [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:
> Cerowrt-devel@lists.bufferbloat.net )
> > [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ](
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> )_______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

[-- Attachment #2: Type: text/html, Size: 7855 bytes --]

Re: [Cerowrt-devel] 10gige and 2.5gige

[Sebastian Moeller]

To add to Joel's point,

I can do my own catX cable runs and connect sockets/plugs to the cables, but I lack the tools for fiber-splicing... as cool as that would be it is going to be hard to justify multi-100s EUR for a splicer.. That still leaves short distance in the main computing area of an appartment/house, but I doubt that many consumers have a concentration high enough to justify the costs even there.

What I do see over here in Europe, with FTTH-roll out speeding up, is CPE that offer SFP/SFP+ cages for the WAN side though, SFP+ becoming more common since ISPs started to deploy XGS-PON (gross 10Gpbs bidirectionally, after FEC ~8.5 Gbps).


Regards
	Sebastian

P.S.: I have not started jumping on the 2.5 Gbps or higher train just yet, none of my devices seems massively underserved with just 1Gbps yet (with the potential exception of a single link where >= 2Gbps would be nice since I am one cabe short and >2Gbps would allow to multiplex two 1Gbps connections over that cable).


> On Dec 16, 2021, at 22:57, Joel Wirāmu Pauling <joel@aenertia.net> wrote:
> 
> Yes but as much as I like fibre; it's too fragile for the average household structured cabling real world use case. Not to mention nothing consumwe comes with SFP+ in the home space.
> 
> On Fri, 17 Dec 2021, 10:43 am David Lang, <david@lang.hm> wrote:
> another valuable featur of fiber for home use is that fiber can't contribute to 
> ground loops the way that copper cables can.
> 
> and for the paranoid (like me :-) ) fiber also means that any electrical 
> disaster that happens to one end won't propgate through and fry other equipment
> 
> David Lang
> 
> On Thu, 16 Dec 2021, David P. Reed wrote:
> 
> > Thanks, That's good to know...The whole SFP+ adapter concept has seemed to me to be a "tweener" in hardware design space. Too many failure points. That said, I like fiber's properties as a medium for distances.
> > 
> > 
> > On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling" <joel@aenertia.net> said:
> >
> >
> >
> >
> > Heat issues you mention with UTP are gone; with the [ 803.bz ]( http://803.bz ) stuff (i.e Base-N). 
> > It was mostly due to the 10G-Base-T spec being old and out of line with the SFP+ spec ; which led to higher power consumption than SFP+ cages were rated to draw and aforementioned heat problems; this is not a problem with newer kit.
> > It went away with the move to smaller silicon processes and now UTP based 10G in the home devices are more common and don't suffer from the fragility issues of the earlier copper based 10G spec. The AQC chipsets were the first to introduce it but most other vendors have finally picked it up after 5 years or feet dragging. 
> >
> >
> > On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com ]( mailto:dpreed@deepplum.com )> wrote:
> > Yes, it's very cheap and getting cheaper.
> > 
> > Since its price fell to the point I thought was cheap, my home has a 10 GigE fiber backbone, 2 switches in my main centers of computers, lots of 10 GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3 external adapter for my primary desktop, which is a Skull Canyon NUC.
> > 
> > I strongly recommend people use fiber and sfp+ DAC cabling because twisted pair, while cheaper, actually is problematic at speeds above 1 Gig - mostly due to power and heat.
> > 
> > BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and USB-C connectors and cables can carry Thunderbolt at higher rates.  Those adapters are REALLY CHEAP. There's nothing inherently different about the electronics, if anything, USB 3.1 is more complicate logic than the ethernet MAC.
> > 
> > So the reason 10 GigE is still far more expensive than USB 3.1 is mainly market volume - if 10 GigE were a consumer product, not a datacenter product, you'd think it would already be as cheap as USB 3.1 in computers and switches.
> > 
> > Since DOCSIS can support up to 5 Gb/s, I think, when will Internet Access Providers start offering "Cable Modems" that support customers who want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out there, it's just a configuration change. 
> > 
> > So when will consumer "routers" support 5 Gig, 10 Gig?
> > 
> > On Thursday, December 16, 2021 11:20am, "Dave Taht" <[ dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:
> >
> >
> >
> >> has really got cheap.
> >> 
> >> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter ]( https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
> >> 
> >> On the other hand users are reporting issues with actually using
> >> 2.5ghz cable with this router in particular, halving the achieved rate
> >> by negotiating 2.5gbit vs negotiating 1gbit.
> >> 
> >> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ]( https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
> >> 
> >> 
> >> --
> >> I tried to build a better future, a few times:
> >> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org ]( https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
> >> 
> >> Dave Täht CEO, TekLibre, LLC
> >> _______________________________________________
> >> Cerowrt-devel mailing list
> >> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> >> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )
> >> _______________________________________________
> > Cerowrt-devel mailing list
> > [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> > [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )_______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] 10gige and 2.5gige

[Joel Wirāmu Pauling]

[-- Attachment #1: Type: text/plain, Size: 7208 bytes --]

The XG PON ONT units from Nokia/Huawei are coming with only 10G NbaseT
(usually singular port) only in the consumer access space. No SFP+

We have rolled out XG PON on the PON side to 70% of the country here  (NZ)
over the last 2 years. Only a small % of that are actually making use of
the XGPON on the consumer side and retailers vary in offering it as a
service mainly due to having to truck roll a new ONT and lack of in home
10G kit on the market. But the access network is there.

Similar stories in other regions I know of that offer XGPon - lack of
consumer demand, lack of ONTs in the market that are suitable for
residential use.







On Fri, 17 Dec 2021, 9:18 pm Sebastian Moeller, <moeller0@gmx.de> wrote:

> To add to Joel's point,
>
> I can do my own catX cable runs and connect sockets/plugs to the cables,
> but I lack the tools for fiber-splicing... as cool as that would be it is
> going to be hard to justify multi-100s EUR for a splicer.. That still
> leaves short distance in the main computing area of an appartment/house,
> but I doubt that many consumers have a concentration high enough to justify
> the costs even there.
>
> What I do see over here in Europe, with FTTH-roll out speeding up, is CPE
> that offer SFP/SFP+ cages for the WAN side though, SFP+ becoming more
> common since ISPs started to deploy XGS-PON (gross 10Gpbs bidirectionally,
> after FEC ~8.5 Gbps).
>
>
> Regards
>         Sebastian
>
> P.S.: I have not started jumping on the 2.5 Gbps or higher train just yet,
> none of my devices seems massively underserved with just 1Gbps yet (with
> the potential exception of a single link where >= 2Gbps would be nice since
> I am one cabe short and >2Gbps would allow to multiplex two 1Gbps
> connections over that cable).
>
>
> > On Dec 16, 2021, at 22:57, Joel Wirāmu Pauling <joel@aenertia.net>
> wrote:
> >
> > Yes but as much as I like fibre; it's too fragile for the average
> household structured cabling real world use case. Not to mention nothing
> consumwe comes with SFP+ in the home space.
> >
> > On Fri, 17 Dec 2021, 10:43 am David Lang, <david@lang.hm> wrote:
> > another valuable featur of fiber for home use is that fiber can't
> contribute to
> > ground loops the way that copper cables can.
> >
> > and for the paranoid (like me :-) ) fiber also means that any electrical
> > disaster that happens to one end won't propgate through and fry other
> equipment
> >
> > David Lang
> >
> > On Thu, 16 Dec 2021, David P. Reed wrote:
> >
> > > Thanks, That's good to know...The whole SFP+ adapter concept has
> seemed to me to be a "tweener" in hardware design space. Too many failure
> points. That said, I like fiber's properties as a medium for distances.
> > >
> > >
> > > On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling" <
> joel@aenertia.net> said:
> > >
> > >
> > >
> > >
> > > Heat issues you mention with UTP are gone; with the [ 803.bz ](
> http://803.bz ) stuff (i.e Base-N).
> > > It was mostly due to the 10G-Base-T spec being old and out of line
> with the SFP+ spec ; which led to higher power consumption than SFP+ cages
> were rated to draw and aforementioned heat problems; this is not a problem
> with newer kit.
> > > It went away with the move to smaller silicon processes and now UTP
> based 10G in the home devices are more common and don't suffer from the
> fragility issues of the earlier copper based 10G spec. The AQC chipsets
> were the first to introduce it but most other vendors have finally picked
> it up after 5 years or feet dragging.
> > >
> > >
> > > On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com
> ]( mailto:dpreed@deepplum.com )> wrote:
> > > Yes, it's very cheap and getting cheaper.
> > >
> > > Since its price fell to the point I thought was cheap, my home has a
> 10 GigE fiber backbone, 2 switches in my main centers of computers, lots of
> 10 GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3
> external adapter for my primary desktop, which is a Skull Canyon NUC.
> > >
> > > I strongly recommend people use fiber and sfp+ DAC cabling because
> twisted pair, while cheaper, actually is problematic at speeds above 1 Gig
> - mostly due to power and heat.
> > >
> > > BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too,
> and USB-C connectors and cables can carry Thunderbolt at higher rates.
> Those adapters are REALLY CHEAP. There's nothing inherently different about
> the electronics, if anything, USB 3.1 is more complicate logic than the
> ethernet MAC.
> > >
> > > So the reason 10 GigE is still far more expensive than USB 3.1 is
> mainly market volume - if 10 GigE were a consumer product, not a datacenter
> product, you'd think it would already be as cheap as USB 3.1 in computers
> and switches.
> > >
> > > Since DOCSIS can support up to 5 Gb/s, I think, when will Internet
> Access Providers start offering "Cable Modems" that support customers who
> want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out
> there, it's just a configuration change.
> > >
> > > So when will consumer "routers" support 5 Gig, 10 Gig?
> > >
> > > On Thursday, December 16, 2021 11:20am, "Dave Taht" <[
> dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:
> > >
> > >
> > >
> > >> has really got cheap.
> > >>
> > >> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter
> ]( https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
> > >>
> > >> On the other hand users are reporting issues with actually using
> > >> 2.5ghz cable with this router in particular, halving the achieved rate
> > >> by negotiating 2.5gbit vs negotiating 1gbit.
> > >>
> > >> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ](
> https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
> > >>
> > >>
> > >> --
> > >> I tried to build a better future, a few times:
> > >> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org ](
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
> > >>
> > >> Dave Täht CEO, TekLibre, LLC
> > >> _______________________________________________
> > >> Cerowrt-devel mailing list
> > >> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:
> Cerowrt-devel@lists.bufferbloat.net )
> > >> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ](
> https://lists.bufferbloat.net/listinfo/cerowrt-devel )
> > >> _______________________________________________
> > > Cerowrt-devel mailing list
> > > [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:
> Cerowrt-devel@lists.bufferbloat.net )
> > > [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ](
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> )_______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>

[-- Attachment #2: Type: text/html, Size: 10941 bytes --]

Re: [Cerowrt-devel] 10gige and 2.5gige

[Joel Wirāmu Pauling]

[-- Attachment #1: Type: text/plain, Size: 7708 bytes --]

Completely aside I have never got Cake SQM to work with connection's beyond
about a gigabit biderectional ; without loosing gigabits of throughput even
when running on beefy hardware. Has been a problem here for some time  now.

On Fri, 17 Dec 2021, 9:36 pm Joel Wirāmu Pauling, <joel@aenertia.net> wrote:

> The XG PON ONT units from Nokia/Huawei are coming with only 10G NbaseT
> (usually singular port) only in the consumer access space. No SFP+
>
> We have rolled out XG PON on the PON side to 70% of the country here  (NZ)
> over the last 2 years. Only a small % of that are actually making use of
> the XGPON on the consumer side and retailers vary in offering it as a
> service mainly due to having to truck roll a new ONT and lack of in home
> 10G kit on the market. But the access network is there.
>
> Similar stories in other regions I know of that offer XGPon - lack of
> consumer demand, lack of ONTs in the market that are suitable for
> residential use.
>
>
>
>
>
>
>
> On Fri, 17 Dec 2021, 9:18 pm Sebastian Moeller, <moeller0@gmx.de> wrote:
>
>> To add to Joel's point,
>>
>> I can do my own catX cable runs and connect sockets/plugs to the cables,
>> but I lack the tools for fiber-splicing... as cool as that would be it is
>> going to be hard to justify multi-100s EUR for a splicer.. That still
>> leaves short distance in the main computing area of an appartment/house,
>> but I doubt that many consumers have a concentration high enough to justify
>> the costs even there.
>>
>> What I do see over here in Europe, with FTTH-roll out speeding up, is CPE
>> that offer SFP/SFP+ cages for the WAN side though, SFP+ becoming more
>> common since ISPs started to deploy XGS-PON (gross 10Gpbs bidirectionally,
>> after FEC ~8.5 Gbps).
>>
>>
>> Regards
>>         Sebastian
>>
>> P.S.: I have not started jumping on the 2.5 Gbps or higher train just
>> yet, none of my devices seems massively underserved with just 1Gbps yet
>> (with the potential exception of a single link where >= 2Gbps would be nice
>> since I am one cabe short and >2Gbps would allow to multiplex two 1Gbps
>> connections over that cable).
>>
>>
>> > On Dec 16, 2021, at 22:57, Joel Wirāmu Pauling <joel@aenertia.net>
>> wrote:
>> >
>> > Yes but as much as I like fibre; it's too fragile for the average
>> household structured cabling real world use case. Not to mention nothing
>> consumwe comes with SFP+ in the home space.
>> >
>> > On Fri, 17 Dec 2021, 10:43 am David Lang, <david@lang.hm> wrote:
>> > another valuable featur of fiber for home use is that fiber can't
>> contribute to
>> > ground loops the way that copper cables can.
>> >
>> > and for the paranoid (like me :-) ) fiber also means that any
>> electrical
>> > disaster that happens to one end won't propgate through and fry other
>> equipment
>> >
>> > David Lang
>> >
>> > On Thu, 16 Dec 2021, David P. Reed wrote:
>> >
>> > > Thanks, That's good to know...The whole SFP+ adapter concept has
>> seemed to me to be a "tweener" in hardware design space. Too many failure
>> points. That said, I like fiber's properties as a medium for distances.
>> > >
>> > >
>> > > On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling" <
>> joel@aenertia.net> said:
>> > >
>> > >
>> > >
>> > >
>> > > Heat issues you mention with UTP are gone; with the [ 803.bz ](
>> http://803.bz ) stuff (i.e Base-N).
>> > > It was mostly due to the 10G-Base-T spec being old and out of line
>> with the SFP+ spec ; which led to higher power consumption than SFP+ cages
>> were rated to draw and aforementioned heat problems; this is not a problem
>> with newer kit.
>> > > It went away with the move to smaller silicon processes and now UTP
>> based 10G in the home devices are more common and don't suffer from the
>> fragility issues of the earlier copper based 10G spec. The AQC chipsets
>> were the first to introduce it but most other vendors have finally picked
>> it up after 5 years or feet dragging.
>> > >
>> > >
>> > > On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com
>> ]( mailto:dpreed@deepplum.com )> wrote:
>> > > Yes, it's very cheap and getting cheaper.
>> > >
>> > > Since its price fell to the point I thought was cheap, my home has a
>> 10 GigE fiber backbone, 2 switches in my main centers of computers, lots of
>> 10 GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3
>> external adapter for my primary desktop, which is a Skull Canyon NUC.
>> > >
>> > > I strongly recommend people use fiber and sfp+ DAC cabling because
>> twisted pair, while cheaper, actually is problematic at speeds above 1 Gig
>> - mostly due to power and heat.
>> > >
>> > > BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too,
>> and USB-C connectors and cables can carry Thunderbolt at higher rates.
>> Those adapters are REALLY CHEAP. There's nothing inherently different about
>> the electronics, if anything, USB 3.1 is more complicate logic than the
>> ethernet MAC.
>> > >
>> > > So the reason 10 GigE is still far more expensive than USB 3.1 is
>> mainly market volume - if 10 GigE were a consumer product, not a datacenter
>> product, you'd think it would already be as cheap as USB 3.1 in computers
>> and switches.
>> > >
>> > > Since DOCSIS can support up to 5 Gb/s, I think, when will Internet
>> Access Providers start offering "Cable Modems" that support customers who
>> want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out
>> there, it's just a configuration change.
>> > >
>> > > So when will consumer "routers" support 5 Gig, 10 Gig?
>> > >
>> > > On Thursday, December 16, 2021 11:20am, "Dave Taht" <[
>> dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:
>> > >
>> > >
>> > >
>> > >> has really got cheap.
>> > >>
>> > >> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter
>> ]( https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
>> > >>
>> > >> On the other hand users are reporting issues with actually using
>> > >> 2.5ghz cable with this router in particular, halving the achieved
>> rate
>> > >> by negotiating 2.5gbit vs negotiating 1gbit.
>> > >>
>> > >> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ](
>> https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
>> > >>
>> > >>
>> > >> --
>> > >> I tried to build a better future, a few times:
>> > >> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>> ]( https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
>> > >>
>> > >> Dave Täht CEO, TekLibre, LLC
>> > >> _______________________________________________
>> > >> Cerowrt-devel mailing list
>> > >> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:
>> Cerowrt-devel@lists.bufferbloat.net )
>> > >> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ](
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel )
>> > >> _______________________________________________
>> > > Cerowrt-devel mailing list
>> > > [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:
>> Cerowrt-devel@lists.bufferbloat.net )
>> > > [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ](
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> )_______________________________________________
>> > Cerowrt-devel mailing list
>> > Cerowrt-devel@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> > _______________________________________________
>> > Cerowrt-devel mailing list
>> > Cerowrt-devel@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>
>>

[-- Attachment #2: Type: text/html, Size: 11843 bytes --]

Re: [Cerowrt-devel] 10gige and 2.5gige

[Sebastian Moeller]

Mmmh, I guess our approach at traffic shaping does not scale well at those speeds. Maybe this could be fixed with larger batching?

I think it might be worth trying to switch to simple.qos/fq_codel and set a somewhat larger burst/quantum duration in defaults.sh, then disable BQL on the NIC and configure a beefy txqueuelen on the interface. This might help making SQM limp along to higher rates. If that would actually work, we could try to see whether we can make cake learn coarser batching ("bursts" and/or quantum) at high rates (but I did not check what cake does internally, it might already do this, @jonathan?)....

Regards
	Sebastian


> On Dec 17, 2021, at 09:39, Joel Wirāmu Pauling <joel@aenertia.net> wrote:
> 
> Completely aside I have never got Cake SQM to work with connection's beyond about a gigabit biderectional ; without loosing gigabits of throughput even when running on beefy hardware. Has been a problem here for some time  now.
> 
> On Fri, 17 Dec 2021, 9:36 pm Joel Wirāmu Pauling, <joel@aenertia.net> wrote:
> The XG PON ONT units from Nokia/Huawei are coming with only 10G NbaseT (usually singular port) only in the consumer access space. No SFP+ 
> 
> We have rolled out XG PON on the PON side to 70% of the country here  (NZ) over the last 2 years. Only a small % of that are actually making use of the XGPON on the consumer side and retailers vary in offering it as a service mainly due to having to truck roll a new ONT and lack of in home 10G kit on the market. But the access network is there.
> 
> Similar stories in other regions I know of that offer XGPon - lack of consumer demand, lack of ONTs in the market that are suitable for residential use.
> 
> 
> 
> 
> 
> 
> 
> On Fri, 17 Dec 2021, 9:18 pm Sebastian Moeller, <moeller0@gmx.de> wrote:
> To add to Joel's point,
> 
> I can do my own catX cable runs and connect sockets/plugs to the cables, but I lack the tools for fiber-splicing... as cool as that would be it is going to be hard to justify multi-100s EUR for a splicer.. That still leaves short distance in the main computing area of an appartment/house, but I doubt that many consumers have a concentration high enough to justify the costs even there.
> 
> What I do see over here in Europe, with FTTH-roll out speeding up, is CPE that offer SFP/SFP+ cages for the WAN side though, SFP+ becoming more common since ISPs started to deploy XGS-PON (gross 10Gpbs bidirectionally, after FEC ~8.5 Gbps).
> 
> 
> Regards
>         Sebastian
> 
> P.S.: I have not started jumping on the 2.5 Gbps or higher train just yet, none of my devices seems massively underserved with just 1Gbps yet (with the potential exception of a single link where >= 2Gbps would be nice since I am one cabe short and >2Gbps would allow to multiplex two 1Gbps connections over that cable).
> 
> 
> > On Dec 16, 2021, at 22:57, Joel Wirāmu Pauling <joel@aenertia.net> wrote:
> > 
> > Yes but as much as I like fibre; it's too fragile for the average household structured cabling real world use case. Not to mention nothing consumwe comes with SFP+ in the home space.
> > 
> > On Fri, 17 Dec 2021, 10:43 am David Lang, <david@lang.hm> wrote:
> > another valuable featur of fiber for home use is that fiber can't contribute to 
> > ground loops the way that copper cables can.
> > 
> > and for the paranoid (like me :-) ) fiber also means that any electrical 
> > disaster that happens to one end won't propgate through and fry other equipment
> > 
> > David Lang
> > 
> > On Thu, 16 Dec 2021, David P. Reed wrote:
> > 
> > > Thanks, That's good to know...The whole SFP+ adapter concept has seemed to me to be a "tweener" in hardware design space. Too many failure points. That said, I like fiber's properties as a medium for distances.
> > > 
> > > 
> > > On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling" <joel@aenertia.net> said:
> > >
> > >
> > >
> > >
> > > Heat issues you mention with UTP are gone; with the [ 803.bz ]( http://803.bz ) stuff (i.e Base-N). 
> > > It was mostly due to the 10G-Base-T spec being old and out of line with the SFP+ spec ; which led to higher power consumption than SFP+ cages were rated to draw and aforementioned heat problems; this is not a problem with newer kit.
> > > It went away with the move to smaller silicon processes and now UTP based 10G in the home devices are more common and don't suffer from the fragility issues of the earlier copper based 10G spec. The AQC chipsets were the first to introduce it but most other vendors have finally picked it up after 5 years or feet dragging. 
> > >
> > >
> > > On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com ]( mailto:dpreed@deepplum.com )> wrote:
> > > Yes, it's very cheap and getting cheaper.
> > > 
> > > Since its price fell to the point I thought was cheap, my home has a 10 GigE fiber backbone, 2 switches in my main centers of computers, lots of 10 GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3 external adapter for my primary desktop, which is a Skull Canyon NUC.
> > > 
> > > I strongly recommend people use fiber and sfp+ DAC cabling because twisted pair, while cheaper, actually is problematic at speeds above 1 Gig - mostly due to power and heat.
> > > 
> > > BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and USB-C connectors and cables can carry Thunderbolt at higher rates.  Those adapters are REALLY CHEAP. There's nothing inherently different about the electronics, if anything, USB 3.1 is more complicate logic than the ethernet MAC.
> > > 
> > > So the reason 10 GigE is still far more expensive than USB 3.1 is mainly market volume - if 10 GigE were a consumer product, not a datacenter product, you'd think it would already be as cheap as USB 3.1 in computers and switches.
> > > 
> > > Since DOCSIS can support up to 5 Gb/s, I think, when will Internet Access Providers start offering "Cable Modems" that support customers who want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out there, it's just a configuration change. 
> > > 
> > > So when will consumer "routers" support 5 Gig, 10 Gig?
> > > 
> > > On Thursday, December 16, 2021 11:20am, "Dave Taht" <[ dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:
> > >
> > >
> > >
> > >> has really got cheap.
> > >> 
> > >> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter ]( https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
> > >> 
> > >> On the other hand users are reporting issues with actually using
> > >> 2.5ghz cable with this router in particular, halving the achieved rate
> > >> by negotiating 2.5gbit vs negotiating 1gbit.
> > >> 
> > >> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ]( https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
> > >> 
> > >> 
> > >> --
> > >> I tried to build a better future, a few times:
> > >> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org ]( https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
> > >> 
> > >> Dave Täht CEO, TekLibre, LLC
> > >> _______________________________________________
> > >> Cerowrt-devel mailing list
> > >> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> > >> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )
> > >> _______________________________________________
> > > Cerowrt-devel mailing list
> > > [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> > > [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )_______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

Re: [Cerowrt-devel] 10gige and 2.5gige

[Toke Høiland-Jørgensen]

Sebastian Moeller <moeller0@gmx.de> writes:

> Mmmh, I guess our approach at traffic shaping does not scale well at those speeds. Maybe this could be fixed with larger batching?
>
> I think it might be worth trying to switch to simple.qos/fq_codel and
> set a somewhat larger burst/quantum duration in defaults.sh, then
> disable BQL on the NIC and configure a beefy txqueuelen on the
> interface. This might help making SQM limp along to higher rates. If
> that would actually work, we could try to see whether we can make cake
> learn coarser batching ("bursts" and/or quantum) at high rates (but I
> did not check what cake does internally, it might already do this,
> @jonathan?)....

I fear the problem is rather the lack of multithreading. I have a fairly
beefy 8-core ARM box for my main router these days, and even that can't
forward at a gigabit on a single core. There was a bug in the HW
configuration so all traffic was sent to a single core, which resulted
in 50+ms of bloat and traffic capping out way short of a gigabit. Now
that it's fixed and traffic is mixed over all eight cores I have smooth
sailing and no bloat. Thankfully I don't need to shape, so I'm just
running straight mq+fq_codel on the physical interface....

-Toke

Re: [Cerowrt-devel] 10gige and 2.5gige

[Sebastian Moeller]

Hi Joel,


> On Dec 17, 2021, at 09:36, Joel Wirāmu Pauling <joel@aenertia.net> wrote:
> 
> The XG PON ONT units from Nokia/Huawei are coming with only 10G NbaseT (usually singular port) only in the consumer access space. No SFP+ 

Yes, ISPs are not that keen on distributing SFP/SFP+ modules, but there are some CPE that already come with SFP ports (AVM's Fritzbox 5530 Fiber, Telekom's Speedport Pro), IMHO only a question of time until these or their successors will sport SFP+ cages. I probably should add, that Germany (in unusual leadership in consumer-focus) actually has a "router-freedom" rule on the books, requiring ISPs to allow end-users to use any (compatible) router they like, and there is a current discussion about extending this right to choose your own also to ONTs; and at least in the enthusiasts' circles there seems to be a desire to use SFP-type ONTs to directly add to the router. (Personally I have no strong opinion on this, and will probably happily use my ISP's ONT once FTTH comes to my home, but I clearly see others already jumping through hoops to use SFP-GPON-ONTs).


> 
> We have rolled out XG PON on the PON side to 70% of the country here  (NZ) over the last 2 years. Only a small % of that are actually making use of the XGPON on the consumer side

	So these are OLT's then that allow simultaneous GPON and XGPON? Any idea how much more expensive such OLTs are compared to either pure GPON or pure XGPON OLT's and do you know whether XGPON capability is something an ISP needs to pay for per connected customer (so some sort of per-user license)?


> and retailers vary in offering it as a service mainly due to having to truck roll a new ONT and lack of in home 10G kit on the market. But the access network is there.

	Sweet! Sees like a reasonably forward looking deployment! That is something probably going to happen here as well, but since almost all CPE are explicitly rented out for an additional fee from the ISP (and shipping is charged extra) I do not think that the "truck-roll" part is going to be a big hurdle (not that any ISP here actually offers XGPON, but at least they are testing it).

> 
> Similar stories in other regions I know of that offer XGPon - lack of consumer demand, lack of ONTs in the market that are suitable for residential use.

	Yah, I am not surprised, you need to invest heavily into the home network before > 1 Gbps access results in any noticeable improvement, and IMHO there are diminishing returns for adding more "bandwidth" to a link. E.g. I am currently on a 100/40 plan, so a far cry from GPON's local max of 1000/200 let alone XGSPONs yet unkown rate-plans, but I rarely think "if I only had faster internet access" (I will still switch to FTTH ASAP, since I would like not having to bother/monitor the DSL link parameters to check for errors and changes in sync rates).


Best Regards
	Sebastian


> 
> 
> On Fri, 17 Dec 2021, 9:18 pm Sebastian Moeller, <moeller0@gmx.de> wrote:
> To add to Joel's point,
> 
> I can do my own catX cable runs and connect sockets/plugs to the cables, but I lack the tools for fiber-splicing... as cool as that would be it is going to be hard to justify multi-100s EUR for a splicer.. That still leaves short distance in the main computing area of an appartment/house, but I doubt that many consumers have a concentration high enough to justify the costs even there.
> 
> What I do see over here in Europe, with FTTH-roll out speeding up, is CPE that offer SFP/SFP+ cages for the WAN side though, SFP+ becoming more common since ISPs started to deploy XGS-PON (gross 10Gpbs bidirectionally, after FEC ~8.5 Gbps).
> 
> 
> Regards
>         Sebastian
> 
> P.S.: I have not started jumping on the 2.5 Gbps or higher train just yet, none of my devices seems massively underserved with just 1Gbps yet (with the potential exception of a single link where >= 2Gbps would be nice since I am one cabe short and >2Gbps would allow to multiplex two 1Gbps connections over that cable).
> 
> 
> > On Dec 16, 2021, at 22:57, Joel Wirāmu Pauling <joel@aenertia.net> wrote:
> > 
> > Yes but as much as I like fibre; it's too fragile for the average household structured cabling real world use case. Not to mention nothing consumwe comes with SFP+ in the home space.
> > 
> > On Fri, 17 Dec 2021, 10:43 am David Lang, <david@lang.hm> wrote:
> > another valuable featur of fiber for home use is that fiber can't contribute to 
> > ground loops the way that copper cables can.
> > 
> > and for the paranoid (like me :-) ) fiber also means that any electrical 
> > disaster that happens to one end won't propgate through and fry other equipment
> > 
> > David Lang
> > 
> > On Thu, 16 Dec 2021, David P. Reed wrote:
> > 
> > > Thanks, That's good to know...The whole SFP+ adapter concept has seemed to me to be a "tweener" in hardware design space. Too many failure points. That said, I like fiber's properties as a medium for distances.
> > > 
> > > 
> > > On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling" <joel@aenertia.net> said:
> > >
> > >
> > >
> > >
> > > Heat issues you mention with UTP are gone; with the [ 803.bz ]( http://803.bz ) stuff (i.e Base-N). 
> > > It was mostly due to the 10G-Base-T spec being old and out of line with the SFP+ spec ; which led to higher power consumption than SFP+ cages were rated to draw and aforementioned heat problems; this is not a problem with newer kit.
> > > It went away with the move to smaller silicon processes and now UTP based 10G in the home devices are more common and don't suffer from the fragility issues of the earlier copper based 10G spec. The AQC chipsets were the first to introduce it but most other vendors have finally picked it up after 5 years or feet dragging. 
> > >
> > >
> > > On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com ]( mailto:dpreed@deepplum.com )> wrote:
> > > Yes, it's very cheap and getting cheaper.
> > > 
> > > Since its price fell to the point I thought was cheap, my home has a 10 GigE fiber backbone, 2 switches in my main centers of computers, lots of 10 GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3 external adapter for my primary desktop, which is a Skull Canyon NUC.
> > > 
> > > I strongly recommend people use fiber and sfp+ DAC cabling because twisted pair, while cheaper, actually is problematic at speeds above 1 Gig - mostly due to power and heat.
> > > 
> > > BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and USB-C connectors and cables can carry Thunderbolt at higher rates.  Those adapters are REALLY CHEAP. There's nothing inherently different about the electronics, if anything, USB 3.1 is more complicate logic than the ethernet MAC.
> > > 
> > > So the reason 10 GigE is still far more expensive than USB 3.1 is mainly market volume - if 10 GigE were a consumer product, not a datacenter product, you'd think it would already be as cheap as USB 3.1 in computers and switches.
> > > 
> > > Since DOCSIS can support up to 5 Gb/s, I think, when will Internet Access Providers start offering "Cable Modems" that support customers who want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out there, it's just a configuration change. 
> > > 
> > > So when will consumer "routers" support 5 Gig, 10 Gig?
> > > 
> > > On Thursday, December 16, 2021 11:20am, "Dave Taht" <[ dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:
> > >
> > >
> > >
> > >> has really got cheap.
> > >> 
> > >> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter ]( https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
> > >> 
> > >> On the other hand users are reporting issues with actually using
> > >> 2.5ghz cable with this router in particular, halving the achieved rate
> > >> by negotiating 2.5gbit vs negotiating 1gbit.
> > >> 
> > >> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ]( https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
> > >> 
> > >> 
> > >> --
> > >> I tried to build a better future, a few times:
> > >> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org ]( https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
> > >> 
> > >> Dave Täht CEO, TekLibre, LLC
> > >> _______________________________________________
> > >> Cerowrt-devel mailing list
> > >> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> > >> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )
> > >> _______________________________________________
> > > Cerowrt-devel mailing list
> > > [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> > > [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )_______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

Re: [Cerowrt-devel] 10gige and 2.5gige

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 7128 bytes --]


Leviton has wallplates for fiber, and the tools for fiber are cheaper than the tools for CAT6.
Pulling fiber through walls hasn't been a problem for me. No more than pulling CAT6.
 
I know I shouldn't kink or pull fiber hard. In the worst case, I pull light flexible conduit through walls with pull strings so I can add arbitrary numbers of fibers. This is good practice, anyway (for wires or fibers).
 
 
On Friday, December 17, 2021 3:18am, "Sebastian Moeller" <moeller0@gmx.de> said:



> To add to Joel's point,
> 
> I can do my own catX cable runs and connect sockets/plugs to the cables, but I
> lack the tools for fiber-splicing... as cool as that would be it is going to be
> hard to justify multi-100s EUR for a splicer.. That still leaves short distance in
> the main computing area of an appartment/house, but I doubt that many consumers
> have a concentration high enough to justify the costs even there.
> 
> What I do see over here in Europe, with FTTH-roll out speeding up, is CPE that
> offer SFP/SFP+ cages for the WAN side though, SFP+ becoming more common since ISPs
> started to deploy XGS-PON (gross 10Gpbs bidirectionally, after FEC ~8.5 Gbps).
> 
> 
> Regards
> Sebastian
> 
> P.S.: I have not started jumping on the 2.5 Gbps or higher train just yet, none of
> my devices seems massively underserved with just 1Gbps yet (with the potential
> exception of a single link where >= 2Gbps would be nice since I am one cabe
> short and >2Gbps would allow to multiplex two 1Gbps connections over that
> cable).
> 
> 
> > On Dec 16, 2021, at 22:57, Joel Wirāmu Pauling <joel@aenertia.net>
> wrote:
> >
> > Yes but as much as I like fibre; it's too fragile for the average household
> structured cabling real world use case. Not to mention nothing consumwe comes with
> SFP+ in the home space.
> >
> > On Fri, 17 Dec 2021, 10:43 am David Lang, <david@lang.hm> wrote:
> > another valuable featur of fiber for home use is that fiber can't contribute
> to
> > ground loops the way that copper cables can.
> >
> > and for the paranoid (like me :-) ) fiber also means that any electrical
> > disaster that happens to one end won't propgate through and fry other
> equipment
> >
> > David Lang
> >
> > On Thu, 16 Dec 2021, David P. Reed wrote:
> >
> > > Thanks, That's good to know...The whole SFP+ adapter concept has seemed
> to me to be a "tweener" in hardware design space. Too many failure points. That
> said, I like fiber's properties as a medium for distances.
> > >
> > >
> > > On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling"
> <joel@aenertia.net> said:
> > >
> > >
> > >
> > >
> > > Heat issues you mention with UTP are gone; with the [ 803.bz ](
> http://803.bz ) stuff (i.e Base-N).
> > > It was mostly due to the 10G-Base-T spec being old and out of line with
> the SFP+ spec ; which led to higher power consumption than SFP+ cages were rated
> to draw and aforementioned heat problems; this is not a problem with newer kit.
> > > It went away with the move to smaller silicon processes and now UTP
> based 10G in the home devices are more common and don't suffer from the fragility
> issues of the earlier copper based 10G spec. The AQC chipsets were the first to
> introduce it but most other vendors have finally picked it up after 5 years or
> feet dragging.
> > >
> > >
> > > On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com
> ]( mailto:dpreed@deepplum.com )> wrote:
> > > Yes, it's very cheap and getting cheaper.
> > >
> > > Since its price fell to the point I thought was cheap, my home has a 10
> GigE fiber backbone, 2 switches in my main centers of computers, lots of 10 GigE
> NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3 external
> adapter for my primary desktop, which is a Skull Canyon NUC.
> > >
> > > I strongly recommend people use fiber and sfp+ DAC cabling because
> twisted pair, while cheaper, actually is problematic at speeds above 1 Gig -
> mostly due to power and heat.
> > >
> > > BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and
> USB-C connectors and cables can carry Thunderbolt at higher rates. Those adapters
> are REALLY CHEAP. There's nothing inherently different about the electronics, if
> anything, USB 3.1 is more complicate logic than the ethernet MAC.
> > >
> > > So the reason 10 GigE is still far more expensive than USB 3.1 is mainly
> market volume - if 10 GigE were a consumer product, not a datacenter product,
> you'd think it would already be as cheap as USB 3.1 in computers and switches.
> > >
> > > Since DOCSIS can support up to 5 Gb/s, I think, when will Internet
> Access Providers start offering "Cable Modems" that support customers who want
> more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out there, it's
> just a configuration change.
> > >
> > > So when will consumer "routers" support 5 Gig, 10 Gig?
> > >
> > > On Thursday, December 16, 2021 11:20am, "Dave Taht" <[
> dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:
> > >
> > >
> > >
> > >> has really got cheap.
> > >>
> > >> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter
> ]( https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
> > >>
> > >> On the other hand users are reporting issues with actually using
> > >> 2.5ghz cable with this router in particular, halving the achieved
> rate
> > >> by negotiating 2.5gbit vs negotiating 1gbit.
> > >>
> > >> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ](
> https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
> > >>
> > >>
> > >> --
> > >> I tried to build a better future, a few times:
> > >> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org ](
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
> > >>
> > >> Dave Täht CEO, TekLibre, LLC
> > >> _______________________________________________
> > >> Cerowrt-devel mailing list
> > >> [ Cerowrt-devel@lists.bufferbloat.net ](
> mailto:Cerowrt-devel@lists.bufferbloat.net )
> > >> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ](
> https://lists.bufferbloat.net/listinfo/cerowrt-devel )
> > >> _______________________________________________
> > > Cerowrt-devel mailing list
> > > [ Cerowrt-devel@lists.bufferbloat.net ](
> mailto:Cerowrt-devel@lists.bufferbloat.net )
> > > [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ](
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> )_______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

[-- Attachment #2: Type: text/html, Size: 9546 bytes --]

Re: [Cerowrt-devel] 10gige and 2.5gige

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 675 bytes --]

On Sun, Dec 19, 2021 at 10:07 AM David P. Reed <dpreed@deepplum.com> wrote:

> I know I shouldn't kink or pull fiber hard. In the worst case, I pull
> light flexible conduit through walls with pull strings so I can add
> arbitrary numbers of fibers. This is good practice, anyway (for wires or
> fibers).
>

This is what I recommend to my friends/family when they talk about remodels
and having the walls open:  Run 1" flexible conduit to any place that you
think you'd want to run a cable to, while the walls are open, it's cheap
and easy then, and a massive reduction in headaches later on, if you find
you want to replace coax with cat6, or cat5 with fiber, etc..

-Aaron

[-- Attachment #2: Type: text/html, Size: 1174 bytes --]