Development issues regarding the cerowrt test router project
 help / color / mirror / Atom feed
* [Cerowrt-devel] tp-link request for SQM
@ 2021-12-02 18:48 Dave Taht
  2021-12-03 10:10 ` [Cerowrt-devel] [Bloat] " Kenneth Porter
  2021-12-03 11:59 ` Luca Muscariello
  0 siblings, 2 replies; 15+ messages in thread
From: Dave Taht @ 2021-12-02 18:48 UTC (permalink / raw)
  To: bloat, cerowrt-devel

tp-link, is, so far as I know, the last major home router vendor NOT
shipping a SQM system. Perhaps this could be modded up with someones
with accounts?

https://community.tp-link.com/us/home/forum/topic/511156


-- 
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-02 18:48 [Cerowrt-devel] tp-link request for SQM Dave Taht
@ 2021-12-03 10:10 ` Kenneth Porter
  2021-12-03 10:40   ` Sebastian Moeller
  2021-12-03 11:59 ` Luca Muscariello
  1 sibling, 1 reply; 15+ messages in thread
From: Kenneth Porter @ 2021-12-03 10:10 UTC (permalink / raw)
  To: bloat, cerowrt-devel

--On Thursday, December 02, 2021 10:48 AM -0800 Dave Taht 
<dave.taht@gmail.com> wrote:

> tp-link, is, so far as I know, the last major home router vendor NOT
> shipping a SQM system. Perhaps this could be modded up with someones
> with accounts?
>
> https://community.tp-link.com/us/home/forum/topic/511156

I just signed up an account and will add my vote.

I just bought an Archer 20 to replace my old 2016 Zyxel running OpenWrt. 
I'd found it by looking at various reviews of "best OpenWrt router for 
2021". I just updated my Zyxel firmware from v18 to v20 firmware. I get 
about 280 Mbps from Xfinity. I turned on cake and it dropped by 50%! So I 
think the old router's CPU isn't up to it. I'll be swapping in the TP-Link 
soon so I can turn on cake without the big performance hit.



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-03 10:10 ` [Cerowrt-devel] [Bloat] " Kenneth Porter
@ 2021-12-03 10:40   ` Sebastian Moeller
  2021-12-03 12:12     ` Wheelock, Ian
  2021-12-03 12:38     ` Kenneth Porter
  0 siblings, 2 replies; 15+ messages in thread
From: Sebastian Moeller @ 2021-12-03 10:40 UTC (permalink / raw)
  To: Kenneth Porter; +Cc: bloat, cerowrt-devel



> On Dec 3, 2021, at 11:10, Kenneth Porter <shiva@sewingwitch.com> wrote:
> 
> --On Thursday, December 02, 2021 10:48 AM -0800 Dave Taht <dave.taht@gmail.com> wrote:
> 
>> tp-link, is, so far as I know, the last major home router vendor NOT
>> shipping a SQM system. Perhaps this could be modded up with someones
>> with accounts?
>> 
>> https://community.tp-link.com/us/home/forum/topic/511156
> 
> I just signed up an account and will add my vote.
> 
> I just bought an Archer 20 to replace my old 2016 Zyxel running OpenWrt. I'd found it by looking at various reviews of "best OpenWrt router for 2021". I just updated my Zyxel firmware from v18 to v20 firmware. I get about 280 Mbps from Xfinity. I turned on cake and it dropped by 50%! So I think the old router's CPU isn't up to it. I'll be swapping in the TP-Link soon so I can turn on cake without the big performance hit.

	Getting low latency traffic shaping to work in a robust and reliable way above say ~100Mbps is still a challenge even for relatively recent router SoCs. Modern multicore SoCs upped the ante in the things-to-look-out for area by adding CPU power-saving (especially frequency scaling) and load distribution over CPUs to the mix... Now even something like a raspberry pi 4B with an additional well-selected USB3 gigabit ethernet dongle (costing less than 100 EUR all in all) will allow cake up to 1/1 Gbps but still requires careful configuration to do so. No idea whether an archer 20 will do (not even sure what model that is, here in Germany I see either an C20 or an AX20 but no plain unadorned 20). If you should try OpenWrt on that thing, the OpenWrt forum is a good place to ask for configuration advice for specific models (will obviously not help if you stick to the manufacturer's firmware).

Regards
	Sebastian



> 
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Bloat] tp-link request for SQM
  2021-12-02 18:48 [Cerowrt-devel] tp-link request for SQM Dave Taht
  2021-12-03 10:10 ` [Cerowrt-devel] [Bloat] " Kenneth Porter
@ 2021-12-03 11:59 ` Luca Muscariello
  2021-12-03 14:18   ` [Cerowrt-devel] " Dave Taht
  1 sibling, 1 reply; 15+ messages in thread
From: Luca Muscariello @ 2021-12-03 11:59 UTC (permalink / raw)
  To: Dave Taht; +Cc: bloat, cerowrt-devel

[-- Attachment #1: Type: text/plain, Size: 748 bytes --]

Test using a tp-link AP EAP 245

https://www.waveform.com/tools/bufferbloat?test-id=bbcc5ef5-e677-4f27-aa04-1849db81d0f5



On Thu, Dec 2, 2021 at 7:48 PM Dave Taht <dave.taht@gmail.com> wrote:

> tp-link, is, so far as I know, the last major home router vendor NOT
> shipping a SQM system. Perhaps this could be modded up with someones
> with accounts?
>
> https://community.tp-link.com/us/home/forum/topic/511156
>
>
> --
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 1875 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-03 10:40   ` Sebastian Moeller
@ 2021-12-03 12:12     ` Wheelock, Ian
  2021-12-03 12:43       ` Kenneth Porter
       [not found]       ` <A5CB878868B38B2F0660B051@192.168.1.16>
  2021-12-03 12:38     ` Kenneth Porter
  1 sibling, 2 replies; 15+ messages in thread
From: Wheelock, Ian @ 2021-12-03 12:12 UTC (permalink / raw)
  To: Sebastian Moeller, Kenneth Porter; +Cc: cerowrt-devel, bloat

[-- Attachment #1: Type: text/plain, Size: 4787 bytes --]

Hi
280Mbps service with Comcast is likely the 300/10 package offering… In that case the US is limited to 10Mbps

I understand there might be some issue if 280Mbps was being processed – but in the US direction, we are not talking >>100Mbps – its about 10Mbps US, I would have thought running cake on 10Mbps US cake would not have triggered a 50% loss in performance even on this platform. Now if cake is applied to both inbound and outbound traffic then having to deal with ~280Mbps might be tough. In the case of DOCSIS AQM, PIE runs in the GW only on outbound traffic.



From: Bloat <bloat-bounces@lists.bufferbloat.net> on behalf of Sebastian Moeller <moeller0@gmx.de>
Date: Friday 3 December 2021 at 10:40
To: Kenneth Porter <shiva@sewingwitch.com>
Cc: cerowrt-devel <cerowrt-devel@lists.bufferbloat.net>, bloat <bloat@lists.bufferbloat.net>
Subject: Re: [Bloat] tp-link request for SQM

> On Dec 3, 2021, at 11:10, Kenneth Porter wrote: > > --On Thursday, December 02, 2021 10:48 AM -0800 Dave Taht wrote: > >> tp-link, is, so far as I know,
External (moeller0@gmx.de<mailto:moeller0@gmx.de>)
  Report This Email<https://protection.inkyphishfence.com/report?id=Y29tbXNjb3BlL2lhbi53aGVlbG9ja0Bjb21tc2NvcGUuY29tLzg5ODdjNTNmZWEyNTcwMzY0NzhkMzQxNjY1YmE3YjRmLzE2Mzg1MjgwNDkuODQ=#key=a3e94f965af0f0f0acc4bf74d04b930a>  FAQ<https://www.inky.com/banner-faq/>  Protection by INKY<https://www.inky.com>






> On Dec 3, 2021, at 11:10, Kenneth Porter <shiva@sewingwitch.com> wrote:

>

> --On Thursday, December 02, 2021 10:48 AM -0800 Dave Taht <dave.taht@gmail.com> wrote:

>

>> tp-link, is, so far as I know, the last major home router vendor NOT

>> shipping a SQM system. Perhaps this could be modded up with someones

>> with accounts?

>>

>> https://secure-web.cisco.com/11xOUC1zY509JqNeDmjflwH4rSaowAKbPpsMSSKsLaElYT5bKwUqpCvIP8pSIJN0xDSludBbilZxiCWeGyX9QBVXQIYGg4aeJlOIvbdBnAe-AGULG-eanWnKwFWAYCjRhnT-GbDz2UCRsr4YTmHkHQVjfSafvUq4KtMTkOBAhmvbMckt7aS8HDyLXxNQDxuTyJebXHYNYvNKVXNoeU562qbXBhB-ud2slR8bxmp5SrW2zaA0BTM-dK0S4ryLxffAxnZ4JjFcLRpX1I4xL1vIy1r3HCDpAwFidpPnHP-jznaHho4VzKVmEn5T4fAgYwSA-_7WEx2qqWgizSp8HE-kOmQ/https%3A%2F%2Fcommunity.tp-link.com%2Fus%2Fhome%2Fforum%2Ftopic%2F511156

>

> I just signed up an account and will add my vote.

>

> I just bought an Archer 20 to replace my old 2016 Zyxel running OpenWrt. I'd found it by looking at various reviews of "best OpenWrt router for 2021". I just updated my Zyxel firmware from v18 to v20 firmware. I get about 280 Mbps from Xfinity. I turned on cake and it dropped by 50%! So I think the old router's CPU isn't up to it. I'll be swapping in the TP-Link soon so I can turn on cake without the big performance hit.



        Getting low latency traffic shaping to work in a robust and reliable way above say ~100Mbps is still a challenge even for relatively recent router SoCs. Modern multicore SoCs upped the ante in the things-to-look-out for area by adding CPU power-saving (especially frequency scaling) and load distribution over CPUs to the mix... Now even something like a raspberry pi 4B with an additional well-selected USB3 gigabit ethernet dongle (costing less than 100 EUR all in all) will allow cake up to 1/1 Gbps but still requires careful configuration to do so. No idea whether an archer 20 will do (not even sure what model that is, here in Germany I see either an C20 or an AX20 but no plain unadorned 20). If you should try OpenWrt on that thing, the OpenWrt forum is a good place to ask for configuration advice for specific models (will obviously not help if you stick to the manufacturer's firmware).



Regards

        Sebastian







>

>

> _______________________________________________

> Bloat mailing list

> Bloat@lists.bufferbloat.net

> https://secure-web.cisco.com/1Z2y8NhzIa813o53EO2ZaEd_c84yx8078Yt5QdXENdHS1zmXuEmxrBiiVcTtdPd4aSueQwqb1LNjPUC-JPnSHnunQLYhMUg22HKM9c9d41a17ITcIQIU9SxTj9fXsDo6olgiCfxxZgURhHnL8RQdjc486i3YRR9svzSz_KqH8OqaQhVOQhIREqb0yVIX8pWWTdwbNIA5EXM7uey0j1C8KlC12QVTerBPM9DQTHbebjUnM0iShNbvfDEl4lZFVkCtwIDrSdcbbb0r5c-sfl-dDCDlAwQuB9wgWpJO8j41FeDE1ckiZJtbNpKTZo1MJuEEFpP2DYfe-iskti5Znw5LRtA/https%3A%2F%2Flists.bufferbloat.net%2Flistinfo%2Fbloat



_______________________________________________

Bloat mailing list

Bloat@lists.bufferbloat.net

https://secure-web.cisco.com/1Z2y8NhzIa813o53EO2ZaEd_c84yx8078Yt5QdXENdHS1zmXuEmxrBiiVcTtdPd4aSueQwqb1LNjPUC-JPnSHnunQLYhMUg22HKM9c9d41a17ITcIQIU9SxTj9fXsDo6olgiCfxxZgURhHnL8RQdjc486i3YRR9svzSz_KqH8OqaQhVOQhIREqb0yVIX8pWWTdwbNIA5EXM7uey0j1C8KlC12QVTerBPM9DQTHbebjUnM0iShNbvfDEl4lZFVkCtwIDrSdcbbb0r5c-sfl-dDCDlAwQuB9wgWpJO8j41FeDE1ckiZJtbNpKTZo1MJuEEFpP2DYfe-iskti5Znw5LRtA/https%3A%2F%2Flists.bufferbloat.net%2Flistinfo%2Fbloat

[-- Attachment #2: Type: text/html, Size: 9811 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-03 10:40   ` Sebastian Moeller
  2021-12-03 12:12     ` Wheelock, Ian
@ 2021-12-03 12:38     ` Kenneth Porter
  1 sibling, 0 replies; 15+ messages in thread
From: Kenneth Porter @ 2021-12-03 12:38 UTC (permalink / raw)
  To: cerowrt-devel, bloat

--On Friday, December 03, 2021 11:40 AM +0100 Sebastian Moeller 
<moeller0@gmx.de> wrote:

> No idea whether an archer 20 will do (not even sure what model that is,
> here in Germany I see either an C20 or an AX20 but no plain unadorned
> 20). If you should try OpenWrt on that thing, the OpenWrt forum is a good
> place to ask for configuration advice for specific models (will obviously
> not help if you stick to the manufacturer's firmware).

Sorry, that was the AX 20. I'd misremembered it. (I find router model 
numbers quite confusing.)

<https://www.amazon.com/gp/product/B085288G3M/>


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-03 12:12     ` Wheelock, Ian
@ 2021-12-03 12:43       ` Kenneth Porter
       [not found]       ` <A5CB878868B38B2F0660B051@192.168.1.16>
  1 sibling, 0 replies; 15+ messages in thread
From: Kenneth Porter @ 2021-12-03 12:43 UTC (permalink / raw)
  To: cerowrt-devel, bloat

--On Friday, December 03, 2021 12:12 PM +0000 "Wheelock, Ian" 
<ian.wheelock@commscope.com> wrote:

> 280Mbps service with Comcast is likely the 300/10 package offering… In
> that case the US is limited to 10Mbps
>
> I understand there might be some issue if 280Mbps was being processed –
> but in the US direction, we are not talking >>100Mbps – its about
> 10Mbps US, I would have thought running cake on 10Mbps US cake would not
> have triggered a 50% loss in performance even on this platform. Now if
> cake is applied to both inbound and outbound traffic then having to deal
> with ~280Mbps might be tough. In the case of DOCSIS AQM, PIE runs in the
> GW only on outbound traffic.

That's indeed the package. I was seeing 270-280 down and 12 up.

"Good" tests without cake:
<https://www.waveform.com/tools/bufferbloat?test-id=22cc9c3e-d3bd-4893-8b68-aba088e199e1>
<https://www.waveform.com/tools/bufferbloat?test-id=90bfc919-0ccc-47d6-88e6-dee10a4f1b16>

It got slammed down to 49 here:
<https://www.waveform.com/tools/bufferbloat?test-id=a670b084-0e90-434b-aedb-e82b7ea7f7ab>

I disabled cake and it slowly recovered:
<https://www.waveform.com/tools/bufferbloat?test-id=eb5e874a-b5a4-436a-bb9b-b3a90ae14930>
<https://www.waveform.com/tools/bufferbloat?test-id=6afcc17b-8d6a-47ee-9a78-ad5b65b74205>

(I wish the tests had a timestamp so I could be sure I'm ordering them 
correctly.)



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-03 11:59 ` Luca Muscariello
@ 2021-12-03 14:18   ` Dave Taht
  2021-12-03 14:35     ` Sebastian Moeller
  0 siblings, 1 reply; 15+ messages in thread
From: Dave Taht @ 2021-12-03 14:18 UTC (permalink / raw)
  To: Luca Muscariello; +Cc: bloat, cerowrt-devel

On Fri, Dec 3, 2021 at 4:00 AM Luca Muscariello <muscariello@ieee.org> wrote:
>
> Test using a tp-link AP EAP 245
>
> https://www.waveform.com/tools/bufferbloat?test-id=bbcc5ef5-e677-4f27-aa04-1849db81d0f5

Nice.

A kvetch is that I really wish they also tested up and down at the same time.

Another kvetch is I think the test needs to run longer at these speeds.

Another another kvetch is they factor in baseline latency to determine
if the link is suitable for gaming or not.

We had them participating on this list at some point....

>
>
> On Thu, Dec 2, 2021 at 7:48 PM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> tp-link, is, so far as I know, the last major home router vendor NOT
>> shipping a SQM system. Perhaps this could be modded up with someones
>> with accounts?
>>
>> https://community.tp-link.com/us/home/forum/topic/511156
>>
>>
>> --
>> I tried to build a better future, a few times:
>> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>>
>> Dave Täht CEO, TekLibre, LLC
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat



-- 
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
       [not found]       ` <A5CB878868B38B2F0660B051@192.168.1.16>
@ 2021-12-03 14:23         ` Dave Taht
  0 siblings, 0 replies; 15+ messages in thread
From: Dave Taht @ 2021-12-03 14:23 UTC (permalink / raw)
  To: Kenneth Porter; +Cc: cerowrt-devel, bloat

On Fri, Dec 3, 2021 at 4:44 AM Kenneth Porter <shiva@sewingwitch.com> wrote:
>
> --On Friday, December 03, 2021 12:12 PM +0000 "Wheelock, Ian"
> <ian.wheelock@commscope.com> wrote:
>
> > 280Mbps service with Comcast is likely the 300/10 package offering… In
> > that case the US is limited to 10Mbps
> >
> > I understand there might be some issue if 280Mbps was being processed –
> > but in the US direction, we are not talking >>100Mbps – its about
> > 10Mbps US, I would have thought running cake on 10Mbps US cake would not
> > have triggered a 50% loss in performance even on this platform. Now if
> > cake is applied to both inbound and outbound traffic then having to deal
> > with ~280Mbps might be tough. In the case of DOCSIS AQM, PIE runs in the
> > GW only on outbound traffic.
>
> That's indeed the package. I was seeing 270-280 down and 12 up.
>
> "Good" tests without cake:
> <https://www.waveform.com/tools/bufferbloat?test-id=22cc9c3e-d3bd-4893-8b68-aba088e199e1>
> <https://www.waveform.com/tools/bufferbloat?test-id=90bfc919-0ccc-47d6-88e6-dee10a4f1b16>

I'm confused, was cake on on the upload? The whole "upload under load
+0" thing I am finding a bit worrisome,
certainly at higher rates with fq on it's going to be close to 0, but
in actually finding one ONT with a mere 5ms
buffer (so far), I am fearing that a solution entering the field is
really short buffers.

Could you take a packet capture of a test without cake?

> It got slammed down to 49 here:
> <https://www.waveform.com/tools/bufferbloat?test-id=a670b084-0e90-434b-aedb-e82b7ea7f7ab>

Ouch. And your uplink went to heck, also. The other worrisome thing
has been powersave and interacting with the
request/grant processes badly.

>
> I disabled cake and it slowly recovered:
> <https://www.waveform.com/tools/bufferbloat?test-id=eb5e874a-b5a4-436a-bb9b-b3a90ae14930>
> <https://www.waveform.com/tools/bufferbloat?test-id=6afcc17b-8d6a-47ee-9a78-ad5b65b74205>
>
> (I wish the tests had a timestamp so I could be sure I'm ordering them
> correctly.)
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-03 14:18   ` [Cerowrt-devel] " Dave Taht
@ 2021-12-03 14:35     ` Sebastian Moeller
  2021-12-03 14:58       ` Luca Muscariello
  0 siblings, 1 reply; 15+ messages in thread
From: Sebastian Moeller @ 2021-12-03 14:35 UTC (permalink / raw)
  To: Dave Täht; +Cc: Luca Muscariello, cerowrt-devel, bloat

Hi Dave,


> On Dec 3, 2021, at 15:18, Dave Taht <dave.taht@gmail.com> wrote:
> 
> On Fri, Dec 3, 2021 at 4:00 AM Luca Muscariello <muscariello@ieee.org> wrote:
>> 
>> Test using a tp-link AP EAP 245
>> 
>> https://www.waveform.com/tools/bufferbloat?test-id=bbcc5ef5-e677-4f27-aa04-1849db81d0f5
> 
> Nice.
> 
> A kvetch is that I really wish they also tested up and down at the same time.
> 
> Another kvetch is I think the test needs to run longer at these speeds.
> 
> Another another kvetch is they factor in baseline latency to determine
> if the link is suitable for gaming or not.

	Which is the sane thing to do... IMHO. For any "twitch"-type reation-time gated gaming all players need to be in an acceptable range of "RTTs" to and from the server so that all perceive the world similarly and nobody has an unfair advantage/disadvantage, so absolute RTT does seem to matter. I would agree that jitter is nastier in that is will cause "randomish" variations of the RTT, but then the known solution against jitter is additional buffering (large enough to simply even out the unequal jittered packet arrival times) which in turn just increases the "RTT", no? (I guess no game really does this enough, so jitter stays the constant problem for internet game-play).
> 
> We had them participating on this list at some point....
> 
>> 
>> 
>> On Thu, Dec 2, 2021 at 7:48 PM Dave Taht <dave.taht@gmail.com> wrote:
>>> 
>>> tp-link, is, so far as I know, the last major home router vendor NOT
>>> shipping a SQM system. Perhaps this could be modded up with someones
>>> with accounts?
>>> 
>>> https://community.tp-link.com/us/home/forum/topic/511156
>>> 
>>> 
>>> --
>>> I tried to build a better future, a few times:
>>> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>>> 
>>> Dave Täht CEO, TekLibre, LLC
>>> _______________________________________________
>>> Bloat mailing list
>>> Bloat@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
> 
> 
> 
> -- 
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> 
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-03 14:35     ` Sebastian Moeller
@ 2021-12-03 14:58       ` Luca Muscariello
  2021-12-09 16:09         ` Sebastian Moeller
  0 siblings, 1 reply; 15+ messages in thread
From: Luca Muscariello @ 2021-12-03 14:58 UTC (permalink / raw)
  To: Sebastian Moeller; +Cc: Dave Täht, cerowrt-devel, bloat

[-- Attachment #1: Type: text/plain, Size: 2825 bytes --]

On Fri, Dec 3, 2021 at 3:35 PM Sebastian Moeller <moeller0@gmx.de> wrote:

> Hi Dave,
>
>
> > On Dec 3, 2021, at 15:18, Dave Taht <dave.taht@gmail.com> wrote:
> >
> > On Fri, Dec 3, 2021 at 4:00 AM Luca Muscariello <muscariello@ieee.org>
> wrote:
> >>
> >> Test using a tp-link AP EAP 245
> >>
> >>
> https://www.waveform.com/tools/bufferbloat?test-id=bbcc5ef5-e677-4f27-aa04-1849db81d0f5
> >
> > Nice.
> >
> > A kvetch is that I really wish they also tested up and down at the same
> time.
> >
> > Another kvetch is I think the test needs to run longer at these speeds.
> >
> > Another another kvetch is they factor in baseline latency to determine
> > if the link is suitable for gaming or not.
>
>         Which is the sane thing to do... IMHO. For any "twitch"-type
> reation-time gated gaming all players need to be in an acceptable range of
> "RTTs" to and from the server so that all perceive the world similarly and
> nobody has an unfair advantage/disadvantage, so absolute RTT does seem to
> matter. I would agree that jitter is nastier in that is will cause
> "randomish" variations of the RTT, but then the known solution against
> jitter is additional buffering (large enough to simply even out the unequal
> jittered packet arrival times) which in turn just increases the "RTT", no?
> (I guess no game really does this enough, so jitter stays the constant
> problem for internet game-play).
> >


Mobile multiplayer competitive games, like PvP, 5v5 such as CoDM, Critical
Ops, LoL Wild Rift may have very different ways to face/conceal network
conditions based on the experience the studio wants to offer to the gamer.



>
> > We had them participating on this list at some point....
> >
> >>
> >>
> >> On Thu, Dec 2, 2021 at 7:48 PM Dave Taht <dave.taht@gmail.com> wrote:
> >>>
> >>> tp-link, is, so far as I know, the last major home router vendor NOT
> >>> shipping a SQM system. Perhaps this could be modded up with someones
> >>> with accounts?
> >>>
> >>> https://community.tp-link.com/us/home/forum/topic/511156
> >>>
> >>>
> >>> --
> >>> I tried to build a better future, a few times:
> >>> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >>>
> >>> Dave Täht CEO, TekLibre, LLC
> >>> _______________________________________________
> >>> Bloat mailing list
> >>> Bloat@lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/bloat
> >
> >
> >
> > --
> > I tried to build a better future, a few times:
> > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >
> > Dave Täht CEO, TekLibre, LLC
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>

[-- Attachment #2: Type: text/html, Size: 4824 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-03 14:58       ` Luca Muscariello
@ 2021-12-09 16:09         ` Sebastian Moeller
  2021-12-09 17:38           ` Luca Muscariello
  0 siblings, 1 reply; 15+ messages in thread
From: Sebastian Moeller @ 2021-12-09 16:09 UTC (permalink / raw)
  To: Luca Muscariello; +Cc: Dave Täht, cerowrt-devel, bloat

Hi Luca

> On Dec 3, 2021, at 15:58, Luca Muscariello <muscariello@ieee.org> wrote:
> 
> 
> 
> On Fri, Dec 3, 2021 at 3:35 PM Sebastian Moeller <moeller0@gmx.de> wrote:
> Hi Dave,
> 
> 
> > On Dec 3, 2021, at 15:18, Dave Taht <dave.taht@gmail.com> wrote:
> > 
> > On Fri, Dec 3, 2021 at 4:00 AM Luca Muscariello <muscariello@ieee.org> wrote:
> >> 
> >> Test using a tp-link AP EAP 245
> >> 
> >> https://www.waveform.com/tools/bufferbloat?test-id=bbcc5ef5-e677-4f27-aa04-1849db81d0f5
> > 
> > Nice.
> > 
> > A kvetch is that I really wish they also tested up and down at the same time.
> > 
> > Another kvetch is I think the test needs to run longer at these speeds.
> > 
> > Another another kvetch is they factor in baseline latency to determine
> > if the link is suitable for gaming or not.
> 
>         Which is the sane thing to do... IMHO. For any "twitch"-type reation-time gated gaming all players need to be in an acceptable range of "RTTs" to and from the server so that all perceive the world similarly and nobody has an unfair advantage/disadvantage, so absolute RTT does seem to matter. I would agree that jitter is nastier in that is will cause "randomish" variations of the RTT, but then the known solution against jitter is additional buffering (large enough to simply even out the unequal jittered packet arrival times) which in turn just increases the "RTT", no? (I guess no game really does this enough, so jitter stays the constant problem for internet game-play).
> >
> 
> Mobile multiplayer competitive games, like PvP, 5v5 such as CoDM, Critical Ops, LoL Wild Rift may have very different ways to face/conceal network conditions based on the experience the studio wants to offer to the gamer.

	Care to elaborate, please? As far as I can tell what I describe above is pretty universal iff fair "coordination" between different player's actions is required. How do you conceal issues about causal ordering of events that ideally are identical from all perspectives? 

Regards
	Sebastian


> 
>  
> 
> > We had them participating on this list at some point....
> > 
> >> 
> >> 
> >> On Thu, Dec 2, 2021 at 7:48 PM Dave Taht <dave.taht@gmail.com> wrote:
> >>> 
> >>> tp-link, is, so far as I know, the last major home router vendor NOT
> >>> shipping a SQM system. Perhaps this could be modded up with someones
> >>> with accounts?
> >>> 
> >>> https://community.tp-link.com/us/home/forum/topic/511156
> >>> 
> >>> 
> >>> --
> >>> I tried to build a better future, a few times:
> >>> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >>> 
> >>> Dave Täht CEO, TekLibre, LLC
> >>> _______________________________________________
> >>> Bloat mailing list
> >>> Bloat@lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/bloat
> > 
> > 
> > 
> > -- 
> > I tried to build a better future, a few times:
> > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> > 
> > Dave Täht CEO, TekLibre, LLC
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-09 16:09         ` Sebastian Moeller
@ 2021-12-09 17:38           ` Luca Muscariello
  2021-12-09 18:23             ` Sebastian Moeller
  0 siblings, 1 reply; 15+ messages in thread
From: Luca Muscariello @ 2021-12-09 17:38 UTC (permalink / raw)
  To: Sebastian Moeller; +Cc: Dave Täht, cerowrt-devel, bloat

[-- Attachment #1: Type: text/plain, Size: 4303 bytes --]

Hi Sebastian

On Thu, Dec 9, 2021, 17:09 Sebastian Moeller <moeller0@gmx.de> wrote:

> Hi Luca
>
> > On Dec 3, 2021, at 15:58, Luca Muscariello <muscariello@ieee.org> wrote:
> >
> >
> >
> > On Fri, Dec 3, 2021 at 3:35 PM Sebastian Moeller <moeller0@gmx.de>
> wrote:
> > Hi Dave,
> >
> >
> > > On Dec 3, 2021, at 15:18, Dave Taht <dave.taht@gmail.com> wrote:
> > >
> > > On Fri, Dec 3, 2021 at 4:00 AM Luca Muscariello <muscariello@ieee.org>
> wrote:
> > >>
> > >> Test using a tp-link AP EAP 245
> > >>
> > >>
> https://www.waveform.com/tools/bufferbloat?test-id=bbcc5ef5-e677-4f27-aa04-1849db81d0f5
> > >
> > > Nice.
> > >
> > > A kvetch is that I really wish they also tested up and down at the
> same time.
> > >
> > > Another kvetch is I think the test needs to run longer at these speeds.
> > >
> > > Another another kvetch is they factor in baseline latency to determine
> > > if the link is suitable for gaming or not.
> >
> >         Which is the sane thing to do... IMHO. For any "twitch"-type
> reation-time gated gaming all players need to be in an acceptable range of
> "RTTs" to and from the server so that all perceive the world similarly and
> nobody has an unfair advantage/disadvantage, so absolute RTT does seem to
> matter. I would agree that jitter is nastier in that is will cause
> "randomish" variations of the RTT, but then the known solution against
> jitter is additional buffering (large enough to simply even out the unequal
> jittered packet arrival times) which in turn just increases the "RTT", no?
> (I guess no game really does this enough, so jitter stays the constant
> problem for internet game-play).
> > >
> >
> > Mobile multiplayer competitive games, like PvP, 5v5 such as CoDM,
> Critical Ops, LoL Wild Rift may have very different ways to face/conceal
> network conditions based on the experience the studio wants to offer to the
> gamer.
>
>         Care to elaborate, please? As far as I can tell what I describe
> above is pretty universal iff fair "coordination" between different
> player's actions is required. How do you conceal issues about causal
> ordering of events that ideally are identical from all perspectives?


Not all games use deterministic lockstep because it requires to compensate
latency from the different players. Of course if latency is zero it is
easier. But that assumption or that objective is one of the fallacies of
distributed computing.

Predictive lockstep or even rollback netcode are examples where you fix
update  accuracy reactively.

Beyond specific netcode techniques, some mobile games can give you the
illusion that you're playing PvP while at some point reality is that you
can continue to play almost offline against bots.

There are games where physics inaccuracies can be concealed. Physics is
very important for first person shooter games while an ARPG may conceal
inaccuracies in the middle of the mess that is rendered on screen, think of
League of Legends Wild Rift.


>
> Regards
>         Sebastian
>
>
> >
> >
> >
> > > We had them participating on this list at some point....
> > >
> > >>
> > >>
> > >> On Thu, Dec 2, 2021 at 7:48 PM Dave Taht <dave.taht@gmail.com> wrote:
> > >>>
> > >>> tp-link, is, so far as I know, the last major home router vendor NOT
> > >>> shipping a SQM system. Perhaps this could be modded up with someones
> > >>> with accounts?
> > >>>
> > >>> https://community.tp-link.com/us/home/forum/topic/511156
> > >>>
> > >>>
> > >>> --
> > >>> I tried to build a better future, a few times:
> > >>> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> > >>>
> > >>> Dave Täht CEO, TekLibre, LLC
> > >>> _______________________________________________
> > >>> Bloat mailing list
> > >>> Bloat@lists.bufferbloat.net
> > >>> https://lists.bufferbloat.net/listinfo/bloat
> > >
> > >
> > >
> > > --
> > > I tried to build a better future, a few times:
> > > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> > >
> > > Dave Täht CEO, TekLibre, LLC
> > > _______________________________________________
> > > Cerowrt-devel mailing list
> > > Cerowrt-devel@lists.bufferbloat.net
> > > https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>

[-- Attachment #2: Type: text/html, Size: 7078 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-09 17:38           ` Luca Muscariello
@ 2021-12-09 18:23             ` Sebastian Moeller
  2021-12-09 18:54               ` Luca Muscariello
  0 siblings, 1 reply; 15+ messages in thread
From: Sebastian Moeller @ 2021-12-09 18:23 UTC (permalink / raw)
  To: Luca Muscariello; +Cc: Dave Täht, cerowrt-devel, bloat

Hi Luca,


> On Dec 9, 2021, at 18:38, Luca Muscariello <muscariello@ieee.org> wrote:
> 
> Hi Sebastian
> 
> On Thu, Dec 9, 2021, 17:09 Sebastian Moeller <moeller0@gmx.de> wrote:
> Hi Luca
> 
> > On Dec 3, 2021, at 15:58, Luca Muscariello <muscariello@ieee.org> wrote:
> > 
> > 
> > 
> > On Fri, Dec 3, 2021 at 3:35 PM Sebastian Moeller <moeller0@gmx.de> wrote:
> > Hi Dave,
> > 
> > 
> > > On Dec 3, 2021, at 15:18, Dave Taht <dave.taht@gmail.com> wrote:
> > > 
> > > On Fri, Dec 3, 2021 at 4:00 AM Luca Muscariello <muscariello@ieee.org> wrote:
> > >> 
> > >> Test using a tp-link AP EAP 245
> > >> 
> > >> https://www.waveform.com/tools/bufferbloat?test-id=bbcc5ef5-e677-4f27-aa04-1849db81d0f5
> > > 
> > > Nice.
> > > 
> > > A kvetch is that I really wish they also tested up and down at the same time.
> > > 
> > > Another kvetch is I think the test needs to run longer at these speeds.
> > > 
> > > Another another kvetch is they factor in baseline latency to determine
> > > if the link is suitable for gaming or not.
> > 
> >         Which is the sane thing to do... IMHO. For any "twitch"-type reation-time gated gaming all players need to be in an acceptable range of "RTTs" to and from the server so that all perceive the world similarly and nobody has an unfair advantage/disadvantage, so absolute RTT does seem to matter. I would agree that jitter is nastier in that is will cause "randomish" variations of the RTT, but then the known solution against jitter is additional buffering (large enough to simply even out the unequal jittered packet arrival times) which in turn just increases the "RTT", no? (I guess no game really does this enough, so jitter stays the constant problem for internet game-play).
> > >
> > 
> > Mobile multiplayer competitive games, like PvP, 5v5 such as CoDM, Critical Ops, LoL Wild Rift may have very different ways to face/conceal network conditions based on the experience the studio wants to offer to the gamer.
> 
>         Care to elaborate, please? As far as I can tell what I describe above is pretty universal iff fair "coordination" between different player's actions is required. How do you conceal issues about causal ordering of events that ideally are identical from all perspectives?
> 
> Not all games use deterministic lockstep because it requires to compensate latency from the different players. Of course if latency is zero it is easier. But that assumption or that objective is one of the fallacies of distributed computing. 
> 
> Predictive lockstep or even rollback netcode are examples where you fix update  accuracy reactively. 

	Ah, sure, I was talking core principle not implementation. I fully agree that predictive/speculative techniques help to make the causality problems of different information propagation delays appear less often. In retrospect that is what "conceal network conditions" alstready covered ;)


> Beyond specific netcode techniques, some mobile games can give you the illusion that you're playing PvP while at some point reality is that you can continue to play almost offline against bots.  

	I don't want to sound harsh, but that feels like "cheating".

> 
> There are games where physics inaccuracies can be concealed. Physics is very important for first person shooter games while an ARPG may conceal inaccuracies in the middle of the mess that is rendered on screen, think of League of Legends Wild Rift. 

	Ah, okay. Didn't know at all about the last two ;)

Thanks!

Regards
	Sebastan


> 
> 
> 
> Regards
>         Sebastian
> 
> 
> > 
> >  
> > 
> > > We had them participating on this list at some point....
> > > 
> > >> 
> > >> 
> > >> On Thu, Dec 2, 2021 at 7:48 PM Dave Taht <dave.taht@gmail.com> wrote:
> > >>> 
> > >>> tp-link, is, so far as I know, the last major home router vendor NOT
> > >>> shipping a SQM system. Perhaps this could be modded up with someones
> > >>> with accounts?
> > >>> 
> > >>> https://community.tp-link.com/us/home/forum/topic/511156
> > >>> 
> > >>> 
> > >>> --
> > >>> I tried to build a better future, a few times:
> > >>> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> > >>> 
> > >>> Dave Täht CEO, TekLibre, LLC
> > >>> _______________________________________________
> > >>> Bloat mailing list
> > >>> Bloat@lists.bufferbloat.net
> > >>> https://lists.bufferbloat.net/listinfo/bloat
> > > 
> > > 
> > > 
> > > -- 
> > > I tried to build a better future, a few times:
> > > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> > > 
> > > Dave Täht CEO, TekLibre, LLC
> > > _______________________________________________
> > > Cerowrt-devel mailing list
> > > Cerowrt-devel@lists.bufferbloat.net
> > > https://lists.bufferbloat.net/listinfo/cerowrt-devel


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [Cerowrt-devel] [Bloat] tp-link request for SQM
  2021-12-09 18:23             ` Sebastian Moeller
@ 2021-12-09 18:54               ` Luca Muscariello
  0 siblings, 0 replies; 15+ messages in thread
From: Luca Muscariello @ 2021-12-09 18:54 UTC (permalink / raw)
  To: Sebastian Moeller; +Cc: Dave Täht, cerowrt-devel, bloat

[-- Attachment #1: Type: text/plain, Size: 5736 bytes --]

On Thu, Dec 9, 2021, 19:23 Sebastian Moeller <moeller0@gmx.de> wrote:

> Hi Luca,
>
>
> > On Dec 9, 2021, at 18:38, Luca Muscariello <muscariello@ieee.org> wrote:
> >
> > Hi Sebastian
> >
> > On Thu, Dec 9, 2021, 17:09 Sebastian Moeller <moeller0@gmx.de> wrote:
> > Hi Luca
> >
> > > On Dec 3, 2021, at 15:58, Luca Muscariello <muscariello@ieee.org>
> wrote:
> > >
> > >
> > >
> > > On Fri, Dec 3, 2021 at 3:35 PM Sebastian Moeller <moeller0@gmx.de>
> wrote:
> > > Hi Dave,
> > >
> > >
> > > > On Dec 3, 2021, at 15:18, Dave Taht <dave.taht@gmail.com> wrote:
> > > >
> > > > On Fri, Dec 3, 2021 at 4:00 AM Luca Muscariello <
> muscariello@ieee.org> wrote:
> > > >>
> > > >> Test using a tp-link AP EAP 245
> > > >>
> > > >>
> https://www.waveform.com/tools/bufferbloat?test-id=bbcc5ef5-e677-4f27-aa04-1849db81d0f5
> > > >
> > > > Nice.
> > > >
> > > > A kvetch is that I really wish they also tested up and down at the
> same time.
> > > >
> > > > Another kvetch is I think the test needs to run longer at these
> speeds.
> > > >
> > > > Another another kvetch is they factor in baseline latency to
> determine
> > > > if the link is suitable for gaming or not.
> > >
> > >         Which is the sane thing to do... IMHO. For any "twitch"-type
> reation-time gated gaming all players need to be in an acceptable range of
> "RTTs" to and from the server so that all perceive the world similarly and
> nobody has an unfair advantage/disadvantage, so absolute RTT does seem to
> matter. I would agree that jitter is nastier in that is will cause
> "randomish" variations of the RTT, but then the known solution against
> jitter is additional buffering (large enough to simply even out the unequal
> jittered packet arrival times) which in turn just increases the "RTT", no?
> (I guess no game really does this enough, so jitter stays the constant
> problem for internet game-play).
> > > >
> > >
> > > Mobile multiplayer competitive games, like PvP, 5v5 such as CoDM,
> Critical Ops, LoL Wild Rift may have very different ways to face/conceal
> network conditions based on the experience the studio wants to offer to the
> gamer.
> >
> >         Care to elaborate, please? As far as I can tell what I describe
> above is pretty universal iff fair "coordination" between different
> player's actions is required. How do you conceal issues about causal
> ordering of events that ideally are identical from all perspectives?
> >
> > Not all games use deterministic lockstep because it requires to
> compensate latency from the different players. Of course if latency is zero
> it is easier. But that assumption or that objective is one of the fallacies
> of distributed computing.
> >
> > Predictive lockstep or even rollback netcode are examples where you fix
> update  accuracy reactively.
>
>         Ah, sure, I was talking core principle not implementation. I fully
> agree that predictive/speculative techniques help to make the causality
> problems of different information propagation delays appear less often. In
> retrospect that is what "conceal network conditions" alstready covered ;)
>
>
> > Beyond specific netcode techniques, some mobile games can give you the
> illusion that you're playing PvP while at some point reality is that you
> can continue to play almost offline against bots.
>
>         I don't want to sound harsh, but that feels like "cheating".
>

That's what I was referring to when I wrote that the game studio works on
the experience they want to provide. If the game is a competitive game with
esport competitions where accuracy is crucial that has implications on the
way the netcode is written.

If the studio is looking for gamer retention it's important the gamer with
little experience is able to learn, get more skilled and have fun along the
way before they get to the competitive level.

BTW, it is not cheating. It is fishing in some sense.


> >
> > There are games where physics inaccuracies can be concealed. Physics is
> very important for first person shooter games while an ARPG may conceal
> inaccuracies in the middle of the mess that is rendered on screen, think of
> League of Legends Wild Rift.
>
>         Ah, okay. Didn't know at all about the last two ;)
>
> Thanks!
>
> Regards
>         Sebastan
>
>
> >
> >
> >
> > Regards
> >         Sebastian
> >
> >
> > >
> > >
> > >
> > > > We had them participating on this list at some point....
> > > >
> > > >>
> > > >>
> > > >> On Thu, Dec 2, 2021 at 7:48 PM Dave Taht <dave.taht@gmail.com>
> wrote:
> > > >>>
> > > >>> tp-link, is, so far as I know, the last major home router vendor
> NOT
> > > >>> shipping a SQM system. Perhaps this could be modded up with
> someones
> > > >>> with accounts?
> > > >>>
> > > >>> https://community.tp-link.com/us/home/forum/topic/511156
> > > >>>
> > > >>>
> > > >>> --
> > > >>> I tried to build a better future, a few times:
> > > >>> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> > > >>>
> > > >>> Dave Täht CEO, TekLibre, LLC
> > > >>> _______________________________________________
> > > >>> Bloat mailing list
> > > >>> Bloat@lists.bufferbloat.net
> > > >>> https://lists.bufferbloat.net/listinfo/bloat
> > > >
> > > >
> > > >
> > > > --
> > > > I tried to build a better future, a few times:
> > > > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> > > >
> > > > Dave Täht CEO, TekLibre, LLC
> > > > _______________________________________________
> > > > Cerowrt-devel mailing list
> > > > Cerowrt-devel@lists.bufferbloat.net
> > > > https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>

[-- Attachment #2: Type: text/html, Size: 8806 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2021-12-09 18:54 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-12-02 18:48 [Cerowrt-devel] tp-link request for SQM Dave Taht
2021-12-03 10:10 ` [Cerowrt-devel] [Bloat] " Kenneth Porter
2021-12-03 10:40   ` Sebastian Moeller
2021-12-03 12:12     ` Wheelock, Ian
2021-12-03 12:43       ` Kenneth Porter
     [not found]       ` <A5CB878868B38B2F0660B051@192.168.1.16>
2021-12-03 14:23         ` Dave Taht
2021-12-03 12:38     ` Kenneth Porter
2021-12-03 11:59 ` Luca Muscariello
2021-12-03 14:18   ` [Cerowrt-devel] " Dave Taht
2021-12-03 14:35     ` Sebastian Moeller
2021-12-03 14:58       ` Luca Muscariello
2021-12-09 16:09         ` Sebastian Moeller
2021-12-09 17:38           ` Luca Muscariello
2021-12-09 18:23             ` Sebastian Moeller
2021-12-09 18:54               ` Luca Muscariello

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox