Re: [Cake] [Cerowrt-devel] 10gige and 2.5gige

Re: [Cake] [Cerowrt-devel] 10gige and 2.5gige

[Sebastian Moeller]

Mmmh, I guess our approach at traffic shaping does not scale well at those speeds. Maybe this could be fixed with larger batching?

I think it might be worth trying to switch to simple.qos/fq_codel and set a somewhat larger burst/quantum duration in defaults.sh, then disable BQL on the NIC and configure a beefy txqueuelen on the interface. This might help making SQM limp along to higher rates. If that would actually work, we could try to see whether we can make cake learn coarser batching ("bursts" and/or quantum) at high rates (but I did not check what cake does internally, it might already do this, @jonathan?)....

Regards
	Sebastian


> On Dec 17, 2021, at 09:39, Joel Wirāmu Pauling <joel@aenertia.net> wrote:
> 
> Completely aside I have never got Cake SQM to work with connection's beyond about a gigabit biderectional ; without loosing gigabits of throughput even when running on beefy hardware. Has been a problem here for some time  now.
> 
> On Fri, 17 Dec 2021, 9:36 pm Joel Wirāmu Pauling, <joel@aenertia.net> wrote:
> The XG PON ONT units from Nokia/Huawei are coming with only 10G NbaseT (usually singular port) only in the consumer access space. No SFP+ 
> 
> We have rolled out XG PON on the PON side to 70% of the country here  (NZ) over the last 2 years. Only a small % of that are actually making use of the XGPON on the consumer side and retailers vary in offering it as a service mainly due to having to truck roll a new ONT and lack of in home 10G kit on the market. But the access network is there.
> 
> Similar stories in other regions I know of that offer XGPon - lack of consumer demand, lack of ONTs in the market that are suitable for residential use.
> 
> 
> 
> 
> 
> 
> 
> On Fri, 17 Dec 2021, 9:18 pm Sebastian Moeller, <moeller0@gmx.de> wrote:
> To add to Joel's point,
> 
> I can do my own catX cable runs and connect sockets/plugs to the cables, but I lack the tools for fiber-splicing... as cool as that would be it is going to be hard to justify multi-100s EUR for a splicer.. That still leaves short distance in the main computing area of an appartment/house, but I doubt that many consumers have a concentration high enough to justify the costs even there.
> 
> What I do see over here in Europe, with FTTH-roll out speeding up, is CPE that offer SFP/SFP+ cages for the WAN side though, SFP+ becoming more common since ISPs started to deploy XGS-PON (gross 10Gpbs bidirectionally, after FEC ~8.5 Gbps).
> 
> 
> Regards
>         Sebastian
> 
> P.S.: I have not started jumping on the 2.5 Gbps or higher train just yet, none of my devices seems massively underserved with just 1Gbps yet (with the potential exception of a single link where >= 2Gbps would be nice since I am one cabe short and >2Gbps would allow to multiplex two 1Gbps connections over that cable).
> 
> 
> > On Dec 16, 2021, at 22:57, Joel Wirāmu Pauling <joel@aenertia.net> wrote:
> > 
> > Yes but as much as I like fibre; it's too fragile for the average household structured cabling real world use case. Not to mention nothing consumwe comes with SFP+ in the home space.
> > 
> > On Fri, 17 Dec 2021, 10:43 am David Lang, <david@lang.hm> wrote:
> > another valuable featur of fiber for home use is that fiber can't contribute to 
> > ground loops the way that copper cables can.
> > 
> > and for the paranoid (like me :-) ) fiber also means that any electrical 
> > disaster that happens to one end won't propgate through and fry other equipment
> > 
> > David Lang
> > 
> > On Thu, 16 Dec 2021, David P. Reed wrote:
> > 
> > > Thanks, That's good to know...The whole SFP+ adapter concept has seemed to me to be a "tweener" in hardware design space. Too many failure points. That said, I like fiber's properties as a medium for distances.
> > > 
> > > 
> > > On Thursday, December 16, 2021 2:31pm, "Joel Wirāmu Pauling" <joel@aenertia.net> said:
> > >
> > >
> > >
> > >
> > > Heat issues you mention with UTP are gone; with the [ 803.bz ]( http://803.bz ) stuff (i.e Base-N). 
> > > It was mostly due to the 10G-Base-T spec being old and out of line with the SFP+ spec ; which led to higher power consumption than SFP+ cages were rated to draw and aforementioned heat problems; this is not a problem with newer kit.
> > > It went away with the move to smaller silicon processes and now UTP based 10G in the home devices are more common and don't suffer from the fragility issues of the earlier copper based 10G spec. The AQC chipsets were the first to introduce it but most other vendors have finally picked it up after 5 years or feet dragging. 
> > >
> > >
> > > On Fri, Dec 17, 2021 at 7:16 AM David P. Reed <[ dpreed@deepplum.com ]( mailto:dpreed@deepplum.com )> wrote:
> > > Yes, it's very cheap and getting cheaper.
> > > 
> > > Since its price fell to the point I thought was cheap, my home has a 10 GigE fiber backbone, 2 switches in my main centers of computers, lots of 10 GigE NICs in servers, and even dual 10 GigE adapters in a Thunderbolt 3 external adapter for my primary desktop, which is a Skull Canyon NUC.
> > > 
> > > I strongly recommend people use fiber and sfp+ DAC cabling because twisted pair, while cheaper, actually is problematic at speeds above 1 Gig - mostly due to power and heat.
> > > 
> > > BTW, it's worth pointing out that USB 3.1 can handle 10 Gb/sec, too, and USB-C connectors and cables can carry Thunderbolt at higher rates.  Those adapters are REALLY CHEAP. There's nothing inherently different about the electronics, if anything, USB 3.1 is more complicate logic than the ethernet MAC.
> > > 
> > > So the reason 10 GigE is still far more expensive than USB 3.1 is mainly market volume - if 10 GigE were a consumer product, not a datacenter product, you'd think it would already be as cheap as USB 3.1 in computers and switches.
> > > 
> > > Since DOCSIS can support up to 5 Gb/s, I think, when will Internet Access Providers start offering "Cable Modems" that support customers who want more than "a full Gig"? Given all the current DOCSIS 3 CMTS's etc. out there, it's just a configuration change. 
> > > 
> > > So when will consumer "routers" support 5 Gig, 10 Gig?
> > > 
> > > On Thursday, December 16, 2021 11:20am, "Dave Taht" <[ dave.taht@gmail.com ]( mailto:dave.taht@gmail.com )> said:
> > >
> > >
> > >
> > >> has really got cheap.
> > >> 
> > >> [ https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter ]( https://www.tomshardware.com/news/innodisk-m2-2280-10gbe-adapter )
> > >> 
> > >> On the other hand users are reporting issues with actually using
> > >> 2.5ghz cable with this router in particular, halving the achieved rate
> > >> by negotiating 2.5gbit vs negotiating 1gbit.
> > >> 
> > >> [ https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 ]( https://forum.mikrotik.com/viewtopic.php?t=179145#p897836 )
> > >> 
> > >> 
> > >> --
> > >> I tried to build a better future, a few times:
> > >> [ https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org ]( https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org )
> > >> 
> > >> Dave Täht CEO, TekLibre, LLC
> > >> _______________________________________________
> > >> Cerowrt-devel mailing list
> > >> [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> > >> [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )
> > >> _______________________________________________
> > > Cerowrt-devel mailing list
> > > [ Cerowrt-devel@lists.bufferbloat.net ]( mailto:Cerowrt-devel@lists.bufferbloat.net )
> > > [ https://lists.bufferbloat.net/listinfo/cerowrt-devel ]( https://lists.bufferbloat.net/listinfo/cerowrt-devel )_______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

Re: [Cake] [Cerowrt-devel] 10gige and 2.5gige

[Toke Høiland-Jørgensen]

Sebastian Moeller <moeller0@gmx.de> writes:

> Mmmh, I guess our approach at traffic shaping does not scale well at those speeds. Maybe this could be fixed with larger batching?
>
> I think it might be worth trying to switch to simple.qos/fq_codel and
> set a somewhat larger burst/quantum duration in defaults.sh, then
> disable BQL on the NIC and configure a beefy txqueuelen on the
> interface. This might help making SQM limp along to higher rates. If
> that would actually work, we could try to see whether we can make cake
> learn coarser batching ("bursts" and/or quantum) at high rates (but I
> did not check what cake does internally, it might already do this,
> @jonathan?)....

I fear the problem is rather the lack of multithreading. I have a fairly
beefy 8-core ARM box for my main router these days, and even that can't
forward at a gigabit on a single core. There was a bug in the HW
configuration so all traffic was sent to a single core, which resulted
in 50+ms of bloat and traffic capping out way short of a gigabit. Now
that it's fixed and traffic is mixed over all eight cores I have smooth
sailing and no bloat. Thankfully I don't need to shape, so I'm just
running straight mq+fq_codel on the physical interface....

-Toke