[LibreQoS] ookla speedtest results?

[LibreQoS] ookla speedtest results?

[Dave Taht]

I am curious if anyone here has speedtest results with libreqos on and
off, and can send a screen shot?

-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

Re: [LibreQoS] ookla speedtest results?

[Herbert Wolverson]

[-- Attachment #1: Type: text/plain, Size: 1226 bytes --]

Are you looking for a shaped result? That'll show "whatever bandwidth
is left on the sector" without Libre, and around the customer's paid-for
target with Libre (hopefully!).

One thing Preseem does well is that their onboarding process includes
"leave it in monitoring-only mode" for a week, gathering data before you
pull the switch (although most ISPs I've talked to just pull the switch...)
It's quite enlightening to see a before/after, even with only FQ_CODEL
as the shaper.

We could probably do something similar with a "monitor only" mode that
doesn't enable any queues (is there a TC equivalent to "dummy" on
FreeBSD?).

On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
libreqos@lists.bufferbloat.net> wrote:

> I am curious if anyone here has speedtest results with libreqos on and
> off, and can send a screen shot?
>
> --
> This song goes out to all the folk that thought Stadia would work:
>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>

[-- Attachment #2: Type: text/html, Size: 2024 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Robert Chacón]

[-- Attachment #1: Type: text/plain, Size: 2324 bytes --]

I can run an Ookla test in the middle of the night with LibreQoS off for a
bit.
From what I recall - without Libre we see download bloat of 300ms or so.
With Libre, it's gone. On waveform we see 0ms added bloat each direction
for most clients.

And I like the "monitor only" mode idea.
Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
Technically even HTB could reduce latency by itself on overloaded APs , so
it wouldn't be true passive monitoring, but it would allow us to compare
the before/after of CAKE while still having qdiscs we can point
cpumap-pping to.

Alternatively, maybe we could use eBPF PPing just for this passive
monitoring period?


On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <
libreqos@lists.bufferbloat.net> wrote:

> Are you looking for a shaped result? That'll show "whatever bandwidth
> is left on the sector" without Libre, and around the customer's paid-for
> target with Libre (hopefully!).
>
> One thing Preseem does well is that their onboarding process includes
> "leave it in monitoring-only mode" for a week, gathering data before you
> pull the switch (although most ISPs I've talked to just pull the switch...)
> It's quite enlightening to see a before/after, even with only FQ_CODEL
> as the shaper.
>
> We could probably do something similar with a "monitor only" mode that
> doesn't enable any queues (is there a TC equivalent to "dummy" on
> FreeBSD?).
>
> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
>
>> I am curious if anyone here has speedtest results with libreqos on and
>> off, and can send a screen shot?
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC
>> _______________________________________________
>> LibreQoS mailing list
>> LibreQoS@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/libreqos
>>
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>


-- 
Robert Chacón
CEO | JackRabbit Wireless LLC <http://jackrabbitwireless.com>

[-- Attachment #2: Type: text/html, Size: 3765 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
<libreqos@lists.bufferbloat.net> wrote:
>
> I can run an Ookla test in the middle of the night with LibreQoS off for a bit.

Groovy.

> From what I recall - without Libre we see download bloat of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each direction for most clients.

The numbers report by speedtest bounce around a lot, and they tend to
pick a lower number as a final result than I'd
like. If there's a way to capture a movie of it...
>
> And I like the "monitor only" mode idea.
> Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
> Technically even HTB could reduce latency by itself on overloaded APs , so it wouldn't be true passive monitoring, but it would allow us to compare the before/after of CAKE while still having qdiscs we can point cpumap-pping to.
>
> Alternatively, maybe we could use eBPF PPing just for this passive monitoring period?
>
>
> On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>
>> Are you looking for a shaped result? That'll show "whatever bandwidth
>> is left on the sector" without Libre, and around the customer's paid-for
>> target with Libre (hopefully!).
>>
>> One thing Preseem does well is that their onboarding process includes
>> "leave it in monitoring-only mode" for a week, gathering data before you
>> pull the switch (although most ISPs I've talked to just pull the switch...)
>> It's quite enlightening to see a before/after, even with only FQ_CODEL
>> as the shaper.
>>
>> We could probably do something similar with a "monitor only" mode that
>> doesn't enable any queues (is there a TC equivalent to "dummy" on
>> FreeBSD?).
>>
>> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>>
>>> I am curious if anyone here has speedtest results with libreqos on and
>>> off, and can send a screen shot?
>>>
>>> --
>>> This song goes out to all the folk that thought Stadia would work:
>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> Dave Täht CEO, TekLibre, LLC
>>> _______________________________________________
>>> LibreQoS mailing list
>>> LibreQoS@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/libreqos
>>
>> _______________________________________________
>> LibreQoS mailing list
>> LibreQoS@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/libreqos
>
>
>
> --
> Robert Chacón
> CEO | JackRabbit Wireless LLC
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos



-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

Re: [LibreQoS] ookla speedtest results?

[dan]

[-- Attachment #1: Type: text/plain, Size: 3555 bytes --]

Dave, I think that 'bouncing' is more CPU time in the browser.  I don't
think it's indicative over what's actually happening.


On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
libreqos@lists.bufferbloat.net> wrote:

> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
> <libreqos@lists.bufferbloat.net> wrote:
> >
> > I can run an Ookla test in the middle of the night with LibreQoS off for
> a bit.
>
> Groovy.
>
> > From what I recall - without Libre we see download bloat of 300ms or so.
> With Libre, it's gone. On waveform we see 0ms added bloat each direction
> for most clients.
>
> The numbers report by speedtest bounce around a lot, and they tend to
> pick a lower number as a final result than I'd
> like. If there's a way to capture a movie of it...
> >
> > And I like the "monitor only" mode idea.
> > Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
> > Technically even HTB could reduce latency by itself on overloaded APs ,
> so it wouldn't be true passive monitoring, but it would allow us to compare
> the before/after of CAKE while still having qdiscs we can point
> cpumap-pping to.
> >
> > Alternatively, maybe we could use eBPF PPing just for this passive
> monitoring period?
> >
> >
> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>
> >> Are you looking for a shaped result? That'll show "whatever bandwidth
> >> is left on the sector" without Libre, and around the customer's paid-for
> >> target with Libre (hopefully!).
> >>
> >> One thing Preseem does well is that their onboarding process includes
> >> "leave it in monitoring-only mode" for a week, gathering data before you
> >> pull the switch (although most ISPs I've talked to just pull the
> switch...)
> >> It's quite enlightening to see a before/after, even with only FQ_CODEL
> >> as the shaper.
> >>
> >> We could probably do something similar with a "monitor only" mode that
> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
> >> FreeBSD?).
> >>
> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>>
> >>> I am curious if anyone here has speedtest results with libreqos on and
> >>> off, and can send a screen shot?
> >>>
> >>> --
> >>> This song goes out to all the folk that thought Stadia would work:
> >>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >>> Dave Täht CEO, TekLibre, LLC
> >>> _______________________________________________
> >>> LibreQoS mailing list
> >>> LibreQoS@lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >>
> >> _______________________________________________
> >> LibreQoS mailing list
> >> LibreQoS@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/libreqos
> >
> >
> >
> > --
> > Robert Chacón
> > CEO | JackRabbit Wireless LLC
> > _______________________________________________
> > LibreQoS mailing list
> > LibreQoS@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/libreqos
>
>
>
> --
> This song goes out to all the folk that thought Stadia would work:
>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>

[-- Attachment #2: Type: text/html, Size: 5520 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>
> Dave, I think that 'bouncing' is more CPU time in the browser.  I don't think it's indicative over what's actually happening.

A simultaneous packet capture and wireshark plot of the RTTs would
also be helpful.

>
> On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>
>> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>> <libreqos@lists.bufferbloat.net> wrote:
>> >
>> > I can run an Ookla test in the middle of the night with LibreQoS off for a bit.
>>
>> Groovy.
>>
>> > From what I recall - without Libre we see download bloat of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each direction for most clients.
>>
>> The numbers report by speedtest bounce around a lot, and they tend to
>> pick a lower number as a final result than I'd
>> like. If there's a way to capture a movie of it...
>> >
>> > And I like the "monitor only" mode idea.
>> > Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
>> > Technically even HTB could reduce latency by itself on overloaded APs , so it wouldn't be true passive monitoring, but it would allow us to compare the before/after of CAKE while still having qdiscs we can point cpumap-pping to.
>> >
>> > Alternatively, maybe we could use eBPF PPing just for this passive monitoring period?
>> >
>> >
>> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>
>> >> Are you looking for a shaped result? That'll show "whatever bandwidth
>> >> is left on the sector" without Libre, and around the customer's paid-for
>> >> target with Libre (hopefully!).
>> >>
>> >> One thing Preseem does well is that their onboarding process includes
>> >> "leave it in monitoring-only mode" for a week, gathering data before you
>> >> pull the switch (although most ISPs I've talked to just pull the switch...)
>> >> It's quite enlightening to see a before/after, even with only FQ_CODEL
>> >> as the shaper.
>> >>
>> >> We could probably do something similar with a "monitor only" mode that
>> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
>> >> FreeBSD?).
>> >>
>> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>>
>> >>> I am curious if anyone here has speedtest results with libreqos on and
>> >>> off, and can send a screen shot?
>> >>>
>> >>> --
>> >>> This song goes out to all the folk that thought Stadia would work:
>> >>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>> Dave Täht CEO, TekLibre, LLC
>> >>> _______________________________________________
>> >>> LibreQoS mailing list
>> >>> LibreQoS@lists.bufferbloat.net
>> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >>
>> >> _______________________________________________
>> >> LibreQoS mailing list
>> >> LibreQoS@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >
>> >
>> >
>> > --
>> > Robert Chacón
>> > CEO | JackRabbit Wireless LLC
>> > _______________________________________________
>> > LibreQoS mailing list
>> > LibreQoS@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/libreqos
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC
>> _______________________________________________
>> LibreQoS mailing list
>> LibreQoS@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/libreqos



-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

Re: [LibreQoS] ookla speedtest results?

[dan]

[-- Attachment #1: Type: text/plain, Size: 4359 bytes --]

sniff DNS for those speed test lookups and trigger a packet capture for x
duration?

On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:

> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
> >
> > Dave, I think that 'bouncing' is more CPU time in the browser.  I don't
> think it's indicative over what's actually happening.
>
> A simultaneous packet capture and wireshark plot of the RTTs would
> also be helpful.
>
> >
> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>
> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
> >> <libreqos@lists.bufferbloat.net> wrote:
> >> >
> >> > I can run an Ookla test in the middle of the night with LibreQoS off
> for a bit.
> >>
> >> Groovy.
> >>
> >> > From what I recall - without Libre we see download bloat of 300ms or
> so. With Libre, it's gone. On waveform we see 0ms added bloat each
> direction for most clients.
> >>
> >> The numbers report by speedtest bounce around a lot, and they tend to
> >> pick a lower number as a final result than I'd
> >> like. If there's a way to capture a movie of it...
> >> >
> >> > And I like the "monitor only" mode idea.
> >> > Perhaps we could create the HTB tree, but just not attach the CAKE
> qdisc?
> >> > Technically even HTB could reduce latency by itself on overloaded APs
> , so it wouldn't be true passive monitoring, but it would allow us to
> compare the before/after of CAKE while still having qdiscs we can point
> cpumap-pping to.
> >> >
> >> > Alternatively, maybe we could use eBPF PPing just for this passive
> monitoring period?
> >> >
> >> >
> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >>
> >> >> Are you looking for a shaped result? That'll show "whatever bandwidth
> >> >> is left on the sector" without Libre, and around the customer's
> paid-for
> >> >> target with Libre (hopefully!).
> >> >>
> >> >> One thing Preseem does well is that their onboarding process includes
> >> >> "leave it in monitoring-only mode" for a week, gathering data before
> you
> >> >> pull the switch (although most ISPs I've talked to just pull the
> switch...)
> >> >> It's quite enlightening to see a before/after, even with only
> FQ_CODEL
> >> >> as the shaper.
> >> >>
> >> >> We could probably do something similar with a "monitor only" mode
> that
> >> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
> >> >> FreeBSD?).
> >> >>
> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >>>
> >> >>> I am curious if anyone here has speedtest results with libreqos on
> and
> >> >>> off, and can send a screen shot?
> >> >>>
> >> >>> --
> >> >>> This song goes out to all the folk that thought Stadia would work:
> >> >>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> >>> Dave Täht CEO, TekLibre, LLC
> >> >>> _______________________________________________
> >> >>> LibreQoS mailing list
> >> >>> LibreQoS@lists.bufferbloat.net
> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >> >>
> >> >> _______________________________________________
> >> >> LibreQoS mailing list
> >> >> LibreQoS@lists.bufferbloat.net
> >> >> https://lists.bufferbloat.net/listinfo/libreqos
> >> >
> >> >
> >> >
> >> > --
> >> > Robert Chacón
> >> > CEO | JackRabbit Wireless LLC
> >> > _______________________________________________
> >> > LibreQoS mailing list
> >> > LibreQoS@lists.bufferbloat.net
> >> > https://lists.bufferbloat.net/listinfo/libreqos
> >>
> >>
> >>
> >> --
> >> This song goes out to all the folk that thought Stadia would work:
> >>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> Dave Täht CEO, TekLibre, LLC
> >> _______________________________________________
> >> LibreQoS mailing list
> >> LibreQoS@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/libreqos
>
>
>
> --
> This song goes out to all the folk that thought Stadia would work:
>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
>

[-- Attachment #2: Type: text/html, Size: 7048 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Herbert Wolverson]

[-- Attachment #1: Type: text/plain, Size: 5131 bytes --]

> sniff DNS for those speed test lookups and trigger a packet capture for x
duration?

That's a good idea (I think there's a service somewhere for finding
speedtest
servers?). We're discussing testing the ack-filter feature right now, and
something
similar would work well for that. (We have a bunch of 25/5 and similarly
asymmetric
connections, in theory it'll help...)

On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <
libreqos@lists.bufferbloat.net> wrote:

> sniff DNS for those speed test lookups and trigger a packet capture for x
> duration?
>
> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>
>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>> >
>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I don't
>> think it's indicative over what's actually happening.
>>
>> A simultaneous packet capture and wireshark plot of the RTTs would
>> also be helpful.
>>
>> >
>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >>
>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>> >> <libreqos@lists.bufferbloat.net> wrote:
>> >> >
>> >> > I can run an Ookla test in the middle of the night with LibreQoS off
>> for a bit.
>> >>
>> >> Groovy.
>> >>
>> >> > From what I recall - without Libre we see download bloat of 300ms or
>> so. With Libre, it's gone. On waveform we see 0ms added bloat each
>> direction for most clients.
>> >>
>> >> The numbers report by speedtest bounce around a lot, and they tend to
>> >> pick a lower number as a final result than I'd
>> >> like. If there's a way to capture a movie of it...
>> >> >
>> >> > And I like the "monitor only" mode idea.
>> >> > Perhaps we could create the HTB tree, but just not attach the CAKE
>> qdisc?
>> >> > Technically even HTB could reduce latency by itself on overloaded
>> APs , so it wouldn't be true passive monitoring, but it would allow us to
>> compare the before/after of CAKE while still having qdiscs we can point
>> cpumap-pping to.
>> >> >
>> >> > Alternatively, maybe we could use eBPF PPing just for this passive
>> monitoring period?
>> >> >
>> >> >
>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >> >>
>> >> >> Are you looking for a shaped result? That'll show "whatever
>> bandwidth
>> >> >> is left on the sector" without Libre, and around the customer's
>> paid-for
>> >> >> target with Libre (hopefully!).
>> >> >>
>> >> >> One thing Preseem does well is that their onboarding process
>> includes
>> >> >> "leave it in monitoring-only mode" for a week, gathering data
>> before you
>> >> >> pull the switch (although most ISPs I've talked to just pull the
>> switch...)
>> >> >> It's quite enlightening to see a before/after, even with only
>> FQ_CODEL
>> >> >> as the shaper.
>> >> >>
>> >> >> We could probably do something similar with a "monitor only" mode
>> that
>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
>> >> >> FreeBSD?).
>> >> >>
>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >> >>>
>> >> >>> I am curious if anyone here has speedtest results with libreqos on
>> and
>> >> >>> off, and can send a screen shot?
>> >> >>>
>> >> >>> --
>> >> >>> This song goes out to all the folk that thought Stadia would work:
>> >> >>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >>> Dave Täht CEO, TekLibre, LLC
>> >> >>> _______________________________________________
>> >> >>> LibreQoS mailing list
>> >> >>> LibreQoS@lists.bufferbloat.net
>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >>
>> >> >> _______________________________________________
>> >> >> LibreQoS mailing list
>> >> >> LibreQoS@lists.bufferbloat.net
>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Robert Chacón
>> >> > CEO | JackRabbit Wireless LLC
>> >> > _______________________________________________
>> >> > LibreQoS mailing list
>> >> > LibreQoS@lists.bufferbloat.net
>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >>
>> >>
>> >>
>> >> --
>> >> This song goes out to all the folk that thought Stadia would work:
>> >>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> Dave Täht CEO, TekLibre, LLC
>> >> _______________________________________________
>> >> LibreQoS mailing list
>> >> LibreQoS@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/libreqos
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC
>>
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>

[-- Attachment #2: Type: text/html, Size: 8244 bytes --]

Re: [LibreQoS] ookla speedtest results?

[dan]

[-- Attachment #1: Type: text/plain, Size: 5762 bytes --]

we also have a bunch of 25/5.  It would be nice to get some more detailed
numbers from ookla tests.  We also have a lot of Eero units out there, so
watching for the eero speed test would also be great.

On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <
libreqos@lists.bufferbloat.net> wrote:

> > sniff DNS for those speed test lookups and trigger a packet capture for
> x duration?
>
> That's a good idea (I think there's a service somewhere for finding
> speedtest
> servers?). We're discussing testing the ack-filter feature right now, and
> something
> similar would work well for that. (We have a bunch of 25/5 and similarly
> asymmetric
> connections, in theory it'll help...)
>
> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
>
>> sniff DNS for those speed test lookups and trigger a packet capture for x
>> duration?
>>
>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>>> >
>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I
>>> don't think it's indicative over what's actually happening.
>>>
>>> A simultaneous packet capture and wireshark plot of the RTTs would
>>> also be helpful.
>>>
>>> >
>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
>>> libreqos@lists.bufferbloat.net> wrote:
>>> >>
>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>>> >> <libreqos@lists.bufferbloat.net> wrote:
>>> >> >
>>> >> > I can run an Ookla test in the middle of the night with LibreQoS
>>> off for a bit.
>>> >>
>>> >> Groovy.
>>> >>
>>> >> > From what I recall - without Libre we see download bloat of 300ms
>>> or so. With Libre, it's gone. On waveform we see 0ms added bloat each
>>> direction for most clients.
>>> >>
>>> >> The numbers report by speedtest bounce around a lot, and they tend to
>>> >> pick a lower number as a final result than I'd
>>> >> like. If there's a way to capture a movie of it...
>>> >> >
>>> >> > And I like the "monitor only" mode idea.
>>> >> > Perhaps we could create the HTB tree, but just not attach the CAKE
>>> qdisc?
>>> >> > Technically even HTB could reduce latency by itself on overloaded
>>> APs , so it wouldn't be true passive monitoring, but it would allow us to
>>> compare the before/after of CAKE while still having qdiscs we can point
>>> cpumap-pping to.
>>> >> >
>>> >> > Alternatively, maybe we could use eBPF PPing just for this passive
>>> monitoring period?
>>> >> >
>>> >> >
>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <
>>> libreqos@lists.bufferbloat.net> wrote:
>>> >> >>
>>> >> >> Are you looking for a shaped result? That'll show "whatever
>>> bandwidth
>>> >> >> is left on the sector" without Libre, and around the customer's
>>> paid-for
>>> >> >> target with Libre (hopefully!).
>>> >> >>
>>> >> >> One thing Preseem does well is that their onboarding process
>>> includes
>>> >> >> "leave it in monitoring-only mode" for a week, gathering data
>>> before you
>>> >> >> pull the switch (although most ISPs I've talked to just pull the
>>> switch...)
>>> >> >> It's quite enlightening to see a before/after, even with only
>>> FQ_CODEL
>>> >> >> as the shaper.
>>> >> >>
>>> >> >> We could probably do something similar with a "monitor only" mode
>>> that
>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
>>> >> >> FreeBSD?).
>>> >> >>
>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
>>> libreqos@lists.bufferbloat.net> wrote:
>>> >> >>>
>>> >> >>> I am curious if anyone here has speedtest results with libreqos
>>> on and
>>> >> >>> off, and can send a screen shot?
>>> >> >>>
>>> >> >>> --
>>> >> >>> This song goes out to all the folk that thought Stadia would work:
>>> >> >>>
>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> >> >>> Dave Täht CEO, TekLibre, LLC
>>> >> >>> _______________________________________________
>>> >> >>> LibreQoS mailing list
>>> >> >>> LibreQoS@lists.bufferbloat.net
>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >>
>>> >> >> _______________________________________________
>>> >> >> LibreQoS mailing list
>>> >> >> LibreQoS@lists.bufferbloat.net
>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Robert Chacón
>>> >> > CEO | JackRabbit Wireless LLC
>>> >> > _______________________________________________
>>> >> > LibreQoS mailing list
>>> >> > LibreQoS@lists.bufferbloat.net
>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> This song goes out to all the folk that thought Stadia would work:
>>> >>
>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> >> Dave Täht CEO, TekLibre, LLC
>>> >> _______________________________________________
>>> >> LibreQoS mailing list
>>> >> LibreQoS@lists.bufferbloat.net
>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>>>
>>>
>>>
>>> --
>>> This song goes out to all the folk that thought Stadia would work:
>>>
>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> Dave Täht CEO, TekLibre, LLC
>>>
>> _______________________________________________
>> LibreQoS mailing list
>> LibreQoS@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/libreqos
>>
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>

[-- Attachment #2: Type: text/html, Size: 9192 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

Some of the context of my request for libreqos on vs off is from here,
presentation in the wee hours, wed morning.

https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/

On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
<libreqos@lists.bufferbloat.net> wrote:
>
> we also have a bunch of 25/5.  It would be nice to get some more detailed numbers from ookla tests.

+1! We've argued methods with many a provider, and ookla and samknows
are new to this game.

As one example I don't know if they use decimal or binary mbits!

...

ack-filtering in the libreqos case can and will drop acks in both
directions, due to the structure of how htb
works in this scenario, at any bandwidth ratio. I *think* it does very
little harm but I would use the ack-filter
not the aggressive ack filter out of caution.

Secondly if you are gathering drop statistics from the higher level
htb categories, total drops will go WAY up,
and it is best to use the cake drop, ack_drop outputs statistics directly.

ack_drops have nothing to do with congestion control and should be
ignored in most cases.

Also I keep hoping more will pull out ecn marks separately. Public use
is way up by e2e transports, and I worry that a high ecn ratio is a
e2e network error, which might show up as a big backlog.

Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
which is a good idea, however perhaps drops, marks and drops+marks
would be better.

An inverse log scale is needed to plot marks, drops, and ack_drops.

I'll always be of the opinion that it's best to run cake on egress at
the cpe, first, doing all this work before it hits the wire.
Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?

>We also have a lot of Eero units out there, so watching for the eero speed test would also be great.

eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
buggy fq_codel offload:

https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
> On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>
>> > sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>>
>> That's a good idea (I think there's a service somewhere for finding speedtest
>> servers?). We're discussing testing the ack-filter feature right now, and something
>> similar would work well for that. (We have a bunch of 25/5 and similarly asymmetric
>> connections, in theory it'll help...)
>>
>> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>>
>>> sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>>>
>>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>>>>
>>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>>>> >
>>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I don't think it's indicative over what's actually happening.
>>>>
>>>> A simultaneous packet capture and wireshark plot of the RTTs would
>>>> also be helpful.
>>>>
>>>> >
>>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>>> >>
>>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>>>> >> <libreqos@lists.bufferbloat.net> wrote:
>>>> >> >
>>>> >> > I can run an Ookla test in the middle of the night with LibreQoS off for a bit.
>>>> >>
>>>> >> Groovy.
>>>> >>
>>>> >> > From what I recall - without Libre we see download bloat of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each direction for most clients.
>>>> >>
>>>> >> The numbers report by speedtest bounce around a lot, and they tend to
>>>> >> pick a lower number as a final result than I'd
>>>> >> like. If there's a way to capture a movie of it...
>>>> >> >
>>>> >> > And I like the "monitor only" mode idea.
>>>> >> > Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
>>>> >> > Technically even HTB could reduce latency by itself on overloaded APs , so it wouldn't be true passive monitoring, but it would allow us to compare the before/after of CAKE while still having qdiscs we can point cpumap-pping to.
>>>> >> >
>>>> >> > Alternatively, maybe we could use eBPF PPing just for this passive monitoring period?
>>>> >> >
>>>> >> >
>>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>>> >> >>
>>>> >> >> Are you looking for a shaped result? That'll show "whatever bandwidth
>>>> >> >> is left on the sector" without Libre, and around the customer's paid-for
>>>> >> >> target with Libre (hopefully!).
>>>> >> >>
>>>> >> >> One thing Preseem does well is that their onboarding process includes
>>>> >> >> "leave it in monitoring-only mode" for a week, gathering data before you
>>>> >> >> pull the switch (although most ISPs I've talked to just pull the switch...)
>>>> >> >> It's quite enlightening to see a before/after, even with only FQ_CODEL
>>>> >> >> as the shaper.
>>>> >> >>
>>>> >> >> We could probably do something similar with a "monitor only" mode that
>>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
>>>> >> >> FreeBSD?).
>>>> >> >>
>>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>>> >> >>>
>>>> >> >>> I am curious if anyone here has speedtest results with libreqos on and
>>>> >> >>> off, and can send a screen shot?
>>>> >> >>>
>>>> >> >>> --
>>>> >> >>> This song goes out to all the folk that thought Stadia would work:
>>>> >> >>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>>> >> >>> Dave Täht CEO, TekLibre, LLC
>>>> >> >>> _______________________________________________
>>>> >> >>> LibreQoS mailing list
>>>> >> >>> LibreQoS@lists.bufferbloat.net
>>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>>>> >> >>
>>>> >> >> _______________________________________________
>>>> >> >> LibreQoS mailing list
>>>> >> >> LibreQoS@lists.bufferbloat.net
>>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > --
>>>> >> > Robert Chacón
>>>> >> > CEO | JackRabbit Wireless LLC
>>>> >> > _______________________________________________
>>>> >> > LibreQoS mailing list
>>>> >> > LibreQoS@lists.bufferbloat.net
>>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> This song goes out to all the folk that thought Stadia would work:
>>>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>>> >> Dave Täht CEO, TekLibre, LLC
>>>> >> _______________________________________________
>>>> >> LibreQoS mailing list
>>>> >> LibreQoS@lists.bufferbloat.net
>>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>>>>
>>>>
>>>>
>>>> --
>>>> This song goes out to all the folk that thought Stadia would work:
>>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>>> Dave Täht CEO, TekLibre, LLC
>>>
>>> _______________________________________________
>>> LibreQoS mailing list
>>> LibreQoS@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/libreqos
>>
>> _______________________________________________
>> LibreQoS mailing list
>> LibreQoS@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/libreqos
>
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos



-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

Re: [LibreQoS] ookla speedtest results?

[dan]

[-- Attachment #1: Type: text/plain, Size: 9354 bytes --]

eero's traffic shaper is problematic because it's based on previous speed
tests.  It gets super erratic.  I have well over 1000 eeros in the field
now and have dealt with people turning it on and their service going to sht
usually because of the late night speed test sharing bandwidth with some
game update or something and updating the shaper.

I don't believe there is a realistic way to handle client side shaping
right now.  Too many vendors, too many use cases, would likely require an
operator to add a demarc router to every customer to shape there.    Should
be noted that I have asked mikrotik to make me a GPeR inspired ARM based
PoE pass through router but got pretty mixed and mostly useless comments in
the forums.    The purpose being a cake shaper AND DHCP option 82 insertion
device with some ability to do packet capture etc on prem.

On Mon, Oct 24, 2022 at 11:14 AM Dave Taht <dave.taht@gmail.com> wrote:

> Some of the context of my request for libreqos on vs off is from here,
> presentation in the wee hours, wed morning.
>
> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>
> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
> <libreqos@lists.bufferbloat.net> wrote:
> >
> > we also have a bunch of 25/5.  It would be nice to get some more
> detailed numbers from ookla tests.
>
> +1! We've argued methods with many a provider, and ookla and samknows
> are new to this game.
>
> As one example I don't know if they use decimal or binary mbits!
>
> ...
>
> ack-filtering in the libreqos case can and will drop acks in both
> directions, due to the structure of how htb
> works in this scenario, at any bandwidth ratio. I *think* it does very
> little harm but I would use the ack-filter
> not the aggressive ack filter out of caution.
>
> Secondly if you are gathering drop statistics from the higher level
> htb categories, total drops will go WAY up,
> and it is best to use the cake drop, ack_drop outputs statistics directly.
>
> ack_drops have nothing to do with congestion control and should be
> ignored in most cases.
>
> Also I keep hoping more will pull out ecn marks separately. Public use
> is way up by e2e transports, and I worry that a high ecn ratio is a
> e2e network error, which might show up as a big backlog.
>
> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
> which is a good idea, however perhaps drops, marks and drops+marks
> would be better.
>
> An inverse log scale is needed to plot marks, drops, and ack_drops.
>
> I'll always be of the opinion that it's best to run cake on egress at
> the cpe, first, doing all this work before it hits the wire.
> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>
> >We also have a lot of Eero units out there, so watching for the eero
> speed test would also be great.
>
> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
> buggy fq_codel offload:
>
>
> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>
> >> > sniff DNS for those speed test lookups and trigger a packet capture
> for x duration?
> >>
> >> That's a good idea (I think there's a service somewhere for finding
> speedtest
> >> servers?). We're discussing testing the ack-filter feature right now,
> and something
> >> similar would work well for that. (We have a bunch of 25/5 and
> similarly asymmetric
> >> connections, in theory it'll help...)
> >>
> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>>
> >>> sniff DNS for those speed test lookups and trigger a packet capture
> for x duration?
> >>>
> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
> >>>>
> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
> >>>> >
> >>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I
> don't think it's indicative over what's actually happening.
> >>>>
> >>>> A simultaneous packet capture and wireshark plot of the RTTs would
> >>>> also be helpful.
> >>>>
> >>>> >
> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>>> >>
> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
> >>>> >> >
> >>>> >> > I can run an Ookla test in the middle of the night with LibreQoS
> off for a bit.
> >>>> >>
> >>>> >> Groovy.
> >>>> >>
> >>>> >> > From what I recall - without Libre we see download bloat of
> 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each
> direction for most clients.
> >>>> >>
> >>>> >> The numbers report by speedtest bounce around a lot, and they tend
> to
> >>>> >> pick a lower number as a final result than I'd
> >>>> >> like. If there's a way to capture a movie of it...
> >>>> >> >
> >>>> >> > And I like the "monitor only" mode idea.
> >>>> >> > Perhaps we could create the HTB tree, but just not attach the
> CAKE qdisc?
> >>>> >> > Technically even HTB could reduce latency by itself on
> overloaded APs , so it wouldn't be true passive monitoring, but it would
> allow us to compare the before/after of CAKE while still having qdiscs we
> can point cpumap-pping to.
> >>>> >> >
> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this
> passive monitoring period?
> >>>> >> >
> >>>> >> >
> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>>> >> >>
> >>>> >> >> Are you looking for a shaped result? That'll show "whatever
> bandwidth
> >>>> >> >> is left on the sector" without Libre, and around the customer's
> paid-for
> >>>> >> >> target with Libre (hopefully!).
> >>>> >> >>
> >>>> >> >> One thing Preseem does well is that their onboarding process
> includes
> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering data
> before you
> >>>> >> >> pull the switch (although most ISPs I've talked to just pull
> the switch...)
> >>>> >> >> It's quite enlightening to see a before/after, even with only
> FQ_CODEL
> >>>> >> >> as the shaper.
> >>>> >> >>
> >>>> >> >> We could probably do something similar with a "monitor only"
> mode that
> >>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy"
> on
> >>>> >> >> FreeBSD?).
> >>>> >> >>
> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>>> >> >>>
> >>>> >> >>> I am curious if anyone here has speedtest results with
> libreqos on and
> >>>> >> >>> off, and can send a screen shot?
> >>>> >> >>>
> >>>> >> >>> --
> >>>> >> >>> This song goes out to all the folk that thought Stadia would
> work:
> >>>> >> >>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
> >>>> >> >>> _______________________________________________
> >>>> >> >>> LibreQoS mailing list
> >>>> >> >>> LibreQoS@lists.bufferbloat.net
> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >>>> >> >>
> >>>> >> >> _______________________________________________
> >>>> >> >> LibreQoS mailing list
> >>>> >> >> LibreQoS@lists.bufferbloat.net
> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
> >>>> >> >
> >>>> >> >
> >>>> >> >
> >>>> >> > --
> >>>> >> > Robert Chacón
> >>>> >> > CEO | JackRabbit Wireless LLC
> >>>> >> > _______________________________________________
> >>>> >> > LibreQoS mailing list
> >>>> >> > LibreQoS@lists.bufferbloat.net
> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> --
> >>>> >> This song goes out to all the folk that thought Stadia would work:
> >>>> >>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >>>> >> Dave Täht CEO, TekLibre, LLC
> >>>> >> _______________________________________________
> >>>> >> LibreQoS mailing list
> >>>> >> LibreQoS@lists.bufferbloat.net
> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> This song goes out to all the folk that thought Stadia would work:
> >>>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >>>> Dave Täht CEO, TekLibre, LLC
> >>>
> >>> _______________________________________________
> >>> LibreQoS mailing list
> >>> LibreQoS@lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >>
> >> _______________________________________________
> >> LibreQoS mailing list
> >> LibreQoS@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/libreqos
> >
> > _______________________________________________
> > LibreQoS mailing list
> > LibreQoS@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/libreqos
>
>
>
> --
> This song goes out to all the folk that thought Stadia would work:
>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
>

[-- Attachment #2: Type: text/html, Size: 14714 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

On Mon, Oct 24, 2022 at 10:30 AM dan <dandenson@gmail.com> wrote:
>
> eero's traffic shaper is problematic because it's based on previous speed tests.  It gets super erratic.  I have well over 1000 eeros in the field now and have dealt with people turning it on and their service going to sht usually because of the late night speed test sharing bandwidth with some game update or something and updating the shaper.

I begged them to let people configure it manually.

>
> I don't believe there is a realistic way to handle client side shaping right now.  Too many vendors, too many use cases, would likely require an operator to add a demarc router to every customer to shape there.

Standardized pppoe and/or dhcp messages would help. Or telling
customers how to configure things manually.

I'd also be a big fan of telling them how to use dscp correctly in zoom, etc.

> Should be noted that I have asked mikrotik to make me a GPeR inspired ARM based PoE pass through router but got pretty mixed and mostly useless comments in the forums.    The purpose being a cake shaper AND DHCP option 82 insertion device with some ability to do packet capture etc on prem.
>
> On Mon, Oct 24, 2022 at 11:14 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> Some of the context of my request for libreqos on vs off is from here,
>> presentation in the wee hours, wed morning.
>>
>> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>>
>> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
>> <libreqos@lists.bufferbloat.net> wrote:
>> >
>> > we also have a bunch of 25/5.  It would be nice to get some more detailed numbers from ookla tests.
>>
>> +1! We've argued methods with many a provider, and ookla and samknows
>> are new to this game.
>>
>> As one example I don't know if they use decimal or binary mbits!
>>
>> ...
>>
>> ack-filtering in the libreqos case can and will drop acks in both
>> directions, due to the structure of how htb
>> works in this scenario, at any bandwidth ratio. I *think* it does very
>> little harm but I would use the ack-filter
>> not the aggressive ack filter out of caution.
>>
>> Secondly if you are gathering drop statistics from the higher level
>> htb categories, total drops will go WAY up,
>> and it is best to use the cake drop, ack_drop outputs statistics directly.
>>
>> ack_drops have nothing to do with congestion control and should be
>> ignored in most cases.
>>
>> Also I keep hoping more will pull out ecn marks separately. Public use
>> is way up by e2e transports, and I worry that a high ecn ratio is a
>> e2e network error, which might show up as a big backlog.
>>
>> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>> which is a good idea, however perhaps drops, marks and drops+marks
>> would be better.
>>
>> An inverse log scale is needed to plot marks, drops, and ack_drops.
>>
>> I'll always be of the opinion that it's best to run cake on egress at
>> the cpe, first, doing all this work before it hits the wire.
>> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>>
>> >We also have a lot of Eero units out there, so watching for the eero speed test would also be great.
>>
>> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
>> buggy fq_codel offload:
>>
>> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
>> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>
>> >> > sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>> >>
>> >> That's a good idea (I think there's a service somewhere for finding speedtest
>> >> servers?). We're discussing testing the ack-filter feature right now, and something
>> >> similar would work well for that. (We have a bunch of 25/5 and similarly asymmetric
>> >> connections, in theory it'll help...)
>> >>
>> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>>
>> >>> sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>> >>>
>> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >>>>
>> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>> >>>> >
>> >>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I don't think it's indicative over what's actually happening.
>> >>>>
>> >>>> A simultaneous packet capture and wireshark plot of the RTTs would
>> >>>> also be helpful.
>> >>>>
>> >>>> >
>> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>>> >>
>> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
>> >>>> >> >
>> >>>> >> > I can run an Ookla test in the middle of the night with LibreQoS off for a bit.
>> >>>> >>
>> >>>> >> Groovy.
>> >>>> >>
>> >>>> >> > From what I recall - without Libre we see download bloat of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each direction for most clients.
>> >>>> >>
>> >>>> >> The numbers report by speedtest bounce around a lot, and they tend to
>> >>>> >> pick a lower number as a final result than I'd
>> >>>> >> like. If there's a way to capture a movie of it...
>> >>>> >> >
>> >>>> >> > And I like the "monitor only" mode idea.
>> >>>> >> > Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
>> >>>> >> > Technically even HTB could reduce latency by itself on overloaded APs , so it wouldn't be true passive monitoring, but it would allow us to compare the before/after of CAKE while still having qdiscs we can point cpumap-pping to.
>> >>>> >> >
>> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this passive monitoring period?
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>>> >> >>
>> >>>> >> >> Are you looking for a shaped result? That'll show "whatever bandwidth
>> >>>> >> >> is left on the sector" without Libre, and around the customer's paid-for
>> >>>> >> >> target with Libre (hopefully!).
>> >>>> >> >>
>> >>>> >> >> One thing Preseem does well is that their onboarding process includes
>> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering data before you
>> >>>> >> >> pull the switch (although most ISPs I've talked to just pull the switch...)
>> >>>> >> >> It's quite enlightening to see a before/after, even with only FQ_CODEL
>> >>>> >> >> as the shaper.
>> >>>> >> >>
>> >>>> >> >> We could probably do something similar with a "monitor only" mode that
>> >>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
>> >>>> >> >> FreeBSD?).
>> >>>> >> >>
>> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>>> >> >>>
>> >>>> >> >>> I am curious if anyone here has speedtest results with libreqos on and
>> >>>> >> >>> off, and can send a screen shot?
>> >>>> >> >>>
>> >>>> >> >>> --
>> >>>> >> >>> This song goes out to all the folk that thought Stadia would work:
>> >>>> >> >>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
>> >>>> >> >>> _______________________________________________
>> >>>> >> >>> LibreQoS mailing list
>> >>>> >> >>> LibreQoS@lists.bufferbloat.net
>> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >>>> >> >>
>> >>>> >> >> _______________________________________________
>> >>>> >> >> LibreQoS mailing list
>> >>>> >> >> LibreQoS@lists.bufferbloat.net
>> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >>>> >> >
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > --
>> >>>> >> > Robert Chacón
>> >>>> >> > CEO | JackRabbit Wireless LLC
>> >>>> >> > _______________________________________________
>> >>>> >> > LibreQoS mailing list
>> >>>> >> > LibreQoS@lists.bufferbloat.net
>> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> --
>> >>>> >> This song goes out to all the folk that thought Stadia would work:
>> >>>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>>> >> Dave Täht CEO, TekLibre, LLC
>> >>>> >> _______________________________________________
>> >>>> >> LibreQoS mailing list
>> >>>> >> LibreQoS@lists.bufferbloat.net
>> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> This song goes out to all the folk that thought Stadia would work:
>> >>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>>> Dave Täht CEO, TekLibre, LLC
>> >>>
>> >>> _______________________________________________
>> >>> LibreQoS mailing list
>> >>> LibreQoS@lists.bufferbloat.net
>> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >>
>> >> _______________________________________________
>> >> LibreQoS mailing list
>> >> LibreQoS@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >
>> > _______________________________________________
>> > LibreQoS mailing list
>> > LibreQoS@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/libreqos
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC



-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

Re: [LibreQoS] ookla speedtest results?

[Robert Chacón]

[-- Attachment #1: Type: text/plain, Size: 9673 bytes --]

> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
which is a good idea, however perhaps drops, marks and drops+marks
would be better.

Yup, in InfluxDB for the stats by Tin (diffserv4) we display true drops as
(trueDrops = ecn_mark + drops - ack_drops). But we don't do it by sub quite
yet.
And ok, drops, marks and drops+marks can be shown instead - by subscriber
or by Node (AP/Site).
Dave - is it ok if we graph this data as a percentage of packets sent? So
for drops, drop / sent_packets ? I just imagine that  would make comparing
different subscribers' connections easier.

Question for all - is having drops / marks/ drops+marks by
subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we
just show those stats by AP/Node?
If preseem currently offers such stats at a subscriber level we can do the
same, just wanna be sure I'm not overtaxing CPU.

On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <
libreqos@lists.bufferbloat.net> wrote:

> Some of the context of my request for libreqos on vs off is from here,
> presentation in the wee hours, wed morning.
>
> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>
> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
> <libreqos@lists.bufferbloat.net> wrote:
> >
> > we also have a bunch of 25/5.  It would be nice to get some more
> detailed numbers from ookla tests.
>
> +1! We've argued methods with many a provider, and ookla and samknows
> are new to this game.
>
> As one example I don't know if they use decimal or binary mbits!
>
> ...
>
> ack-filtering in the libreqos case can and will drop acks in both
> directions, due to the structure of how htb
> works in this scenario, at any bandwidth ratio. I *think* it does very
> little harm but I would use the ack-filter
> not the aggressive ack filter out of caution.
>
> Secondly if you are gathering drop statistics from the higher level
> htb categories, total drops will go WAY up,
> and it is best to use the cake drop, ack_drop outputs statistics directly.
>
> ack_drops have nothing to do with congestion control and should be
> ignored in most cases.
>
> Also I keep hoping more will pull out ecn marks separately. Public use
> is way up by e2e transports, and I worry that a high ecn ratio is a
> e2e network error, which might show up as a big backlog.
>
> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
> which is a good idea, however perhaps drops, marks and drops+marks
> would be better.
>
> An inverse log scale is needed to plot marks, drops, and ack_drops.
>
> I'll always be of the opinion that it's best to run cake on egress at
> the cpe, first, doing all this work before it hits the wire.
> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>
> >We also have a lot of Eero units out there, so watching for the eero
> speed test would also be great.
>
> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
> buggy fq_codel offload:
>
>
> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>
> >> > sniff DNS for those speed test lookups and trigger a packet capture
> for x duration?
> >>
> >> That's a good idea (I think there's a service somewhere for finding
> speedtest
> >> servers?). We're discussing testing the ack-filter feature right now,
> and something
> >> similar would work well for that. (We have a bunch of 25/5 and
> similarly asymmetric
> >> connections, in theory it'll help...)
> >>
> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>>
> >>> sniff DNS for those speed test lookups and trigger a packet capture
> for x duration?
> >>>
> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
> >>>>
> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
> >>>> >
> >>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I
> don't think it's indicative over what's actually happening.
> >>>>
> >>>> A simultaneous packet capture and wireshark plot of the RTTs would
> >>>> also be helpful.
> >>>>
> >>>> >
> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>>> >>
> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
> >>>> >> >
> >>>> >> > I can run an Ookla test in the middle of the night with LibreQoS
> off for a bit.
> >>>> >>
> >>>> >> Groovy.
> >>>> >>
> >>>> >> > From what I recall - without Libre we see download bloat of
> 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each
> direction for most clients.
> >>>> >>
> >>>> >> The numbers report by speedtest bounce around a lot, and they tend
> to
> >>>> >> pick a lower number as a final result than I'd
> >>>> >> like. If there's a way to capture a movie of it...
> >>>> >> >
> >>>> >> > And I like the "monitor only" mode idea.
> >>>> >> > Perhaps we could create the HTB tree, but just not attach the
> CAKE qdisc?
> >>>> >> > Technically even HTB could reduce latency by itself on
> overloaded APs , so it wouldn't be true passive monitoring, but it would
> allow us to compare the before/after of CAKE while still having qdiscs we
> can point cpumap-pping to.
> >>>> >> >
> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this
> passive monitoring period?
> >>>> >> >
> >>>> >> >
> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>>> >> >>
> >>>> >> >> Are you looking for a shaped result? That'll show "whatever
> bandwidth
> >>>> >> >> is left on the sector" without Libre, and around the customer's
> paid-for
> >>>> >> >> target with Libre (hopefully!).
> >>>> >> >>
> >>>> >> >> One thing Preseem does well is that their onboarding process
> includes
> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering data
> before you
> >>>> >> >> pull the switch (although most ISPs I've talked to just pull
> the switch...)
> >>>> >> >> It's quite enlightening to see a before/after, even with only
> FQ_CODEL
> >>>> >> >> as the shaper.
> >>>> >> >>
> >>>> >> >> We could probably do something similar with a "monitor only"
> mode that
> >>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy"
> on
> >>>> >> >> FreeBSD?).
> >>>> >> >>
> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>>> >> >>>
> >>>> >> >>> I am curious if anyone here has speedtest results with
> libreqos on and
> >>>> >> >>> off, and can send a screen shot?
> >>>> >> >>>
> >>>> >> >>> --
> >>>> >> >>> This song goes out to all the folk that thought Stadia would
> work:
> >>>> >> >>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
> >>>> >> >>> _______________________________________________
> >>>> >> >>> LibreQoS mailing list
> >>>> >> >>> LibreQoS@lists.bufferbloat.net
> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >>>> >> >>
> >>>> >> >> _______________________________________________
> >>>> >> >> LibreQoS mailing list
> >>>> >> >> LibreQoS@lists.bufferbloat.net
> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
> >>>> >> >
> >>>> >> >
> >>>> >> >
> >>>> >> > --
> >>>> >> > Robert Chacón
> >>>> >> > CEO | JackRabbit Wireless LLC
> >>>> >> > _______________________________________________
> >>>> >> > LibreQoS mailing list
> >>>> >> > LibreQoS@lists.bufferbloat.net
> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> --
> >>>> >> This song goes out to all the folk that thought Stadia would work:
> >>>> >>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >>>> >> Dave Täht CEO, TekLibre, LLC
> >>>> >> _______________________________________________
> >>>> >> LibreQoS mailing list
> >>>> >> LibreQoS@lists.bufferbloat.net
> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> This song goes out to all the folk that thought Stadia would work:
> >>>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >>>> Dave Täht CEO, TekLibre, LLC
> >>>
> >>> _______________________________________________
> >>> LibreQoS mailing list
> >>> LibreQoS@lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >>
> >> _______________________________________________
> >> LibreQoS mailing list
> >> LibreQoS@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/libreqos
> >
> > _______________________________________________
> > LibreQoS mailing list
> > LibreQoS@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/libreqos
>
>
>
> --
> This song goes out to all the folk that thought Stadia would work:
>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>


-- 
Robert Chacón
CEO | JackRabbit Wireless LLC <http://jackrabbitwireless.com>

[-- Attachment #2: Type: text/html, Size: 15439 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Herbert Wolverson]

[-- Attachment #1: Type: text/plain, Size: 11332 bytes --]

Preseem is a little funky in actually showing any queue statistics, because
it also pounds
your devices with SNMP requests and gathers things like TCP
drops/retransmits from
there (and uses it to try and figure out their hierarchy).

It was actually one of the major problems we had with it - we have a few
EdgeRouters
around, and they have a bug that they utilize ram disk space on a
successful SNMPv2 query.
They don't leak a lot of RAM, just a few bytes per request - but Preseem
pounds the
living daylights of out them, and we had to implement a "reboot all the
EdgeRouters within
30 days" policy to avoid the things silently locking up. Ugh.

I'd love to be able to provide cake at the end-point, but it's not very
practical. Our
customers use a wide range of devices (and CPEs, even a few old M2 devices
still up!), and a "use the router we provide" policy isn't going to fly for
quite
a few of them - they just really don't like that.

I've had UI issues with displaying "drops". Calling it that terrifies our
support
people. I'd love to have another name that conveys the meaning!

On Mon, Oct 24, 2022 at 12:35 PM Robert Chacón via LibreQoS <
libreqos@lists.bufferbloat.net> wrote:

> > Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
> which is a good idea, however perhaps drops, marks and drops+marks
> would be better.
>
> Yup, in InfluxDB for the stats by Tin (diffserv4) we display true drops as
> (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by sub quite
> yet.
> And ok, drops, marks and drops+marks can be shown instead - by subscriber
> or by Node (AP/Site).
> Dave - is it ok if we graph this data as a percentage of packets sent? So
> for drops, drop / sent_packets ? I just imagine that  would make comparing
> different subscribers' connections easier.
>
> Question for all - is having drops / marks/ drops+marks by
> subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we
> just show those stats by AP/Node?
> If preseem currently offers such stats at a subscriber level we can do the
> same, just wanna be sure I'm not overtaxing CPU.
>
> On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
>
>> Some of the context of my request for libreqos on vs off is from here,
>> presentation in the wee hours, wed morning.
>>
>> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>>
>> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
>> <libreqos@lists.bufferbloat.net> wrote:
>> >
>> > we also have a bunch of 25/5.  It would be nice to get some more
>> detailed numbers from ookla tests.
>>
>> +1! We've argued methods with many a provider, and ookla and samknows
>> are new to this game.
>>
>> As one example I don't know if they use decimal or binary mbits!
>>
>> ...
>>
>> ack-filtering in the libreqos case can and will drop acks in both
>> directions, due to the structure of how htb
>> works in this scenario, at any bandwidth ratio. I *think* it does very
>> little harm but I would use the ack-filter
>> not the aggressive ack filter out of caution.
>>
>> Secondly if you are gathering drop statistics from the higher level
>> htb categories, total drops will go WAY up,
>> and it is best to use the cake drop, ack_drop outputs statistics directly.
>>
>> ack_drops have nothing to do with congestion control and should be
>> ignored in most cases.
>>
>> Also I keep hoping more will pull out ecn marks separately. Public use
>> is way up by e2e transports, and I worry that a high ecn ratio is a
>> e2e network error, which might show up as a big backlog.
>>
>> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>> which is a good idea, however perhaps drops, marks and drops+marks
>> would be better.
>>
>> An inverse log scale is needed to plot marks, drops, and ack_drops.
>>
>> I'll always be of the opinion that it's best to run cake on egress at
>> the cpe, first, doing all this work before it hits the wire.
>> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>>
>> >We also have a lot of Eero units out there, so watching for the eero
>> speed test would also be great.
>>
>> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
>> buggy fq_codel offload:
>>
>>
>> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
>> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >>
>> >> > sniff DNS for those speed test lookups and trigger a packet capture
>> for x duration?
>> >>
>> >> That's a good idea (I think there's a service somewhere for finding
>> speedtest
>> >> servers?). We're discussing testing the ack-filter feature right now,
>> and something
>> >> similar would work well for that. (We have a bunch of 25/5 and
>> similarly asymmetric
>> >> connections, in theory it'll help...)
>> >>
>> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >>>
>> >>> sniff DNS for those speed test lookups and trigger a packet capture
>> for x duration?
>> >>>
>> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com>
>> wrote:
>> >>>>
>> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>> >>>> >
>> >>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I
>> don't think it's indicative over what's actually happening.
>> >>>>
>> >>>> A simultaneous packet capture and wireshark plot of the RTTs would
>> >>>> also be helpful.
>> >>>>
>> >>>> >
>> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >>>> >>
>> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
>> >>>> >> >
>> >>>> >> > I can run an Ookla test in the middle of the night with
>> LibreQoS off for a bit.
>> >>>> >>
>> >>>> >> Groovy.
>> >>>> >>
>> >>>> >> > From what I recall - without Libre we see download bloat of
>> 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each
>> direction for most clients.
>> >>>> >>
>> >>>> >> The numbers report by speedtest bounce around a lot, and they
>> tend to
>> >>>> >> pick a lower number as a final result than I'd
>> >>>> >> like. If there's a way to capture a movie of it...
>> >>>> >> >
>> >>>> >> > And I like the "monitor only" mode idea.
>> >>>> >> > Perhaps we could create the HTB tree, but just not attach the
>> CAKE qdisc?
>> >>>> >> > Technically even HTB could reduce latency by itself on
>> overloaded APs , so it wouldn't be true passive monitoring, but it would
>> allow us to compare the before/after of CAKE while still having qdiscs we
>> can point cpumap-pping to.
>> >>>> >> >
>> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this
>> passive monitoring period?
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >>>> >> >>
>> >>>> >> >> Are you looking for a shaped result? That'll show "whatever
>> bandwidth
>> >>>> >> >> is left on the sector" without Libre, and around the
>> customer's paid-for
>> >>>> >> >> target with Libre (hopefully!).
>> >>>> >> >>
>> >>>> >> >> One thing Preseem does well is that their onboarding process
>> includes
>> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering data
>> before you
>> >>>> >> >> pull the switch (although most ISPs I've talked to just pull
>> the switch...)
>> >>>> >> >> It's quite enlightening to see a before/after, even with only
>> FQ_CODEL
>> >>>> >> >> as the shaper.
>> >>>> >> >>
>> >>>> >> >> We could probably do something similar with a "monitor only"
>> mode that
>> >>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy"
>> on
>> >>>> >> >> FreeBSD?).
>> >>>> >> >>
>> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >>>> >> >>>
>> >>>> >> >>> I am curious if anyone here has speedtest results with
>> libreqos on and
>> >>>> >> >>> off, and can send a screen shot?
>> >>>> >> >>>
>> >>>> >> >>> --
>> >>>> >> >>> This song goes out to all the folk that thought Stadia would
>> work:
>> >>>> >> >>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
>> >>>> >> >>> _______________________________________________
>> >>>> >> >>> LibreQoS mailing list
>> >>>> >> >>> LibreQoS@lists.bufferbloat.net
>> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >>>> >> >>
>> >>>> >> >> _______________________________________________
>> >>>> >> >> LibreQoS mailing list
>> >>>> >> >> LibreQoS@lists.bufferbloat.net
>> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >>>> >> >
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > --
>> >>>> >> > Robert Chacón
>> >>>> >> > CEO | JackRabbit Wireless LLC
>> >>>> >> > _______________________________________________
>> >>>> >> > LibreQoS mailing list
>> >>>> >> > LibreQoS@lists.bufferbloat.net
>> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> --
>> >>>> >> This song goes out to all the folk that thought Stadia would work:
>> >>>> >>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>>> >> Dave Täht CEO, TekLibre, LLC
>> >>>> >> _______________________________________________
>> >>>> >> LibreQoS mailing list
>> >>>> >> LibreQoS@lists.bufferbloat.net
>> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> This song goes out to all the folk that thought Stadia would work:
>> >>>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>>> Dave Täht CEO, TekLibre, LLC
>> >>>
>> >>> _______________________________________________
>> >>> LibreQoS mailing list
>> >>> LibreQoS@lists.bufferbloat.net
>> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >>
>> >> _______________________________________________
>> >> LibreQoS mailing list
>> >> LibreQoS@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >
>> > _______________________________________________
>> > LibreQoS mailing list
>> > LibreQoS@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/libreqos
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC
>> _______________________________________________
>> LibreQoS mailing list
>> LibreQoS@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/libreqos
>>
>
>
> --
> Robert Chacón
> CEO | JackRabbit Wireless LLC <http://jackrabbitwireless.com>
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>

[-- Attachment #2: Type: text/html, Size: 17508 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

On Mon, Oct 24, 2022 at 10:35 AM Robert Chacón
<robert.chacon@jackrabbitwireless.com> wrote:
>
> > Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
> which is a good idea, however perhaps drops, marks and drops+marks
> would be better.
>
> Yup, in InfluxDB for the stats by Tin (diffserv4) we display true drops as (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by sub quite yet.
> And ok, drops, marks and drops+marks can be shown instead - by subscriber or by Node (AP/Site).
> Dave - is it ok if we graph this data as a percentage of packets sent? So for drops, drop / sent_packets ? I just imagine that  would make comparing different subscribers' connections easier.

Drops and marks are proportional to the plan. You get a heck of a lot
more drops at 5mbits than 50, and the relationship
is not linear, closer to quadratic.

So comparing customers at their given tiers makes more sense than
across the board.

As for a percentage, you should see a clear difference in percentages
across tiers.

I know I'm not quite answering your question well, I'm a big believer
in just getting lots of data, graphing it every which way,
and seeing what's useful.

> Question for all - is having drops / marks/ drops+marks by subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we just show those stats by AP/Node?
> If preseem currently offers such stats at a subscriber level we can do the same, just wanna be sure I'm not overtaxing CPU.
>
> On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>
>> Some of the context of my request for libreqos on vs off is from here,
>> presentation in the wee hours, wed morning.
>>
>> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>>
>> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
>> <libreqos@lists.bufferbloat.net> wrote:
>> >
>> > we also have a bunch of 25/5.  It would be nice to get some more detailed numbers from ookla tests.
>>
>> +1! We've argued methods with many a provider, and ookla and samknows
>> are new to this game.
>>
>> As one example I don't know if they use decimal or binary mbits!
>>
>> ...
>>
>> ack-filtering in the libreqos case can and will drop acks in both
>> directions, due to the structure of how htb
>> works in this scenario, at any bandwidth ratio. I *think* it does very
>> little harm but I would use the ack-filter
>> not the aggressive ack filter out of caution.
>>
>> Secondly if you are gathering drop statistics from the higher level
>> htb categories, total drops will go WAY up,
>> and it is best to use the cake drop, ack_drop outputs statistics directly.
>>
>> ack_drops have nothing to do with congestion control and should be
>> ignored in most cases.
>>
>> Also I keep hoping more will pull out ecn marks separately. Public use
>> is way up by e2e transports, and I worry that a high ecn ratio is a
>> e2e network error, which might show up as a big backlog.
>>
>> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>> which is a good idea, however perhaps drops, marks and drops+marks
>> would be better.
>>
>> An inverse log scale is needed to plot marks, drops, and ack_drops.
>>
>> I'll always be of the opinion that it's best to run cake on egress at
>> the cpe, first, doing all this work before it hits the wire.
>> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>>
>> >We also have a lot of Eero units out there, so watching for the eero speed test would also be great.
>>
>> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
>> buggy fq_codel offload:
>>
>> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
>> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>
>> >> > sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>> >>
>> >> That's a good idea (I think there's a service somewhere for finding speedtest
>> >> servers?). We're discussing testing the ack-filter feature right now, and something
>> >> similar would work well for that. (We have a bunch of 25/5 and similarly asymmetric
>> >> connections, in theory it'll help...)
>> >>
>> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>>
>> >>> sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>> >>>
>> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >>>>
>> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>> >>>> >
>> >>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I don't think it's indicative over what's actually happening.
>> >>>>
>> >>>> A simultaneous packet capture and wireshark plot of the RTTs would
>> >>>> also be helpful.
>> >>>>
>> >>>> >
>> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>>> >>
>> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
>> >>>> >> >
>> >>>> >> > I can run an Ookla test in the middle of the night with LibreQoS off for a bit.
>> >>>> >>
>> >>>> >> Groovy.
>> >>>> >>
>> >>>> >> > From what I recall - without Libre we see download bloat of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each direction for most clients.
>> >>>> >>
>> >>>> >> The numbers report by speedtest bounce around a lot, and they tend to
>> >>>> >> pick a lower number as a final result than I'd
>> >>>> >> like. If there's a way to capture a movie of it...
>> >>>> >> >
>> >>>> >> > And I like the "monitor only" mode idea.
>> >>>> >> > Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
>> >>>> >> > Technically even HTB could reduce latency by itself on overloaded APs , so it wouldn't be true passive monitoring, but it would allow us to compare the before/after of CAKE while still having qdiscs we can point cpumap-pping to.
>> >>>> >> >
>> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this passive monitoring period?
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>>> >> >>
>> >>>> >> >> Are you looking for a shaped result? That'll show "whatever bandwidth
>> >>>> >> >> is left on the sector" without Libre, and around the customer's paid-for
>> >>>> >> >> target with Libre (hopefully!).
>> >>>> >> >>
>> >>>> >> >> One thing Preseem does well is that their onboarding process includes
>> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering data before you
>> >>>> >> >> pull the switch (although most ISPs I've talked to just pull the switch...)
>> >>>> >> >> It's quite enlightening to see a before/after, even with only FQ_CODEL
>> >>>> >> >> as the shaper.
>> >>>> >> >>
>> >>>> >> >> We could probably do something similar with a "monitor only" mode that
>> >>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
>> >>>> >> >> FreeBSD?).
>> >>>> >> >>
>> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>>> >> >>>
>> >>>> >> >>> I am curious if anyone here has speedtest results with libreqos on and
>> >>>> >> >>> off, and can send a screen shot?
>> >>>> >> >>>
>> >>>> >> >>> --
>> >>>> >> >>> This song goes out to all the folk that thought Stadia would work:
>> >>>> >> >>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
>> >>>> >> >>> _______________________________________________
>> >>>> >> >>> LibreQoS mailing list
>> >>>> >> >>> LibreQoS@lists.bufferbloat.net
>> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >>>> >> >>
>> >>>> >> >> _______________________________________________
>> >>>> >> >> LibreQoS mailing list
>> >>>> >> >> LibreQoS@lists.bufferbloat.net
>> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >>>> >> >
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > --
>> >>>> >> > Robert Chacón
>> >>>> >> > CEO | JackRabbit Wireless LLC
>> >>>> >> > _______________________________________________
>> >>>> >> > LibreQoS mailing list
>> >>>> >> > LibreQoS@lists.bufferbloat.net
>> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> --
>> >>>> >> This song goes out to all the folk that thought Stadia would work:
>> >>>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>>> >> Dave Täht CEO, TekLibre, LLC
>> >>>> >> _______________________________________________
>> >>>> >> LibreQoS mailing list
>> >>>> >> LibreQoS@lists.bufferbloat.net
>> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> This song goes out to all the folk that thought Stadia would work:
>> >>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >>>> Dave Täht CEO, TekLibre, LLC
>> >>>
>> >>> _______________________________________________
>> >>> LibreQoS mailing list
>> >>> LibreQoS@lists.bufferbloat.net
>> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >>
>> >> _______________________________________________
>> >> LibreQoS mailing list
>> >> LibreQoS@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >
>> > _______________________________________________
>> > LibreQoS mailing list
>> > LibreQoS@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/libreqos
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC
>> _______________________________________________
>> LibreQoS mailing list
>> LibreQoS@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/libreqos
>
>
>
> --
> Robert Chacón
> CEO | JackRabbit Wireless LLC



-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

Re: [LibreQoS] ookla speedtest results?

[dan]

[-- Attachment #1: Type: text/plain, Size: 11915 bytes --]

Some sort of % drop is better than count from an operator's perspective.
I'm actually happy with 'scaling issues' of % on small vs large plan
because those small plans are the one's we probably want to push for
upgrades if they're seeing a lot of drops.

That said, a 'score' method works really well from a high level.  And that
can be proportionally adjusted so a slightly higher % in a small plan gives
the same score as a larger plan would with fewer drops.


Also, on the SNMP statement above, we had the same issue with preseem being
hyper aggressive to pull stats.  Some mikrotik versions have some snmp
flaws and preseem can peg the CPU.  Hasn't happened in a while but that was
an issue.

On Mon, Oct 24, 2022 at 11:42 AM Dave Taht <dave.taht@gmail.com> wrote:

> On Mon, Oct 24, 2022 at 10:35 AM Robert Chacón
> <robert.chacon@jackrabbitwireless.com> wrote:
> >
> > > Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
> > which is a good idea, however perhaps drops, marks and drops+marks
> > would be better.
> >
> > Yup, in InfluxDB for the stats by Tin (diffserv4) we display true drops
> as (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by sub
> quite yet.
> > And ok, drops, marks and drops+marks can be shown instead - by
> subscriber or by Node (AP/Site).
> > Dave - is it ok if we graph this data as a percentage of packets sent?
> So for drops, drop / sent_packets ? I just imagine that  would make
> comparing different subscribers' connections easier.
>
> Drops and marks are proportional to the plan. You get a heck of a lot
> more drops at 5mbits than 50, and the relationship
> is not linear, closer to quadratic.
>
> So comparing customers at their given tiers makes more sense than
> across the board.
>
> As for a percentage, you should see a clear difference in percentages
> across tiers.
>
> I know I'm not quite answering your question well, I'm a big believer
> in just getting lots of data, graphing it every which way,
> and seeing what's useful.
>
> > Question for all - is having drops / marks/ drops+marks by
> subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we
> just show those stats by AP/Node?
> > If preseem currently offers such stats at a subscriber level we can do
> the same, just wanna be sure I'm not overtaxing CPU.
> >
> > On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >>
> >> Some of the context of my request for libreqos on vs off is from here,
> >> presentation in the wee hours, wed morning.
> >>
> >>
> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
> >>
> >> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
> >> <libreqos@lists.bufferbloat.net> wrote:
> >> >
> >> > we also have a bunch of 25/5.  It would be nice to get some more
> detailed numbers from ookla tests.
> >>
> >> +1! We've argued methods with many a provider, and ookla and samknows
> >> are new to this game.
> >>
> >> As one example I don't know if they use decimal or binary mbits!
> >>
> >> ...
> >>
> >> ack-filtering in the libreqos case can and will drop acks in both
> >> directions, due to the structure of how htb
> >> works in this scenario, at any bandwidth ratio. I *think* it does very
> >> little harm but I would use the ack-filter
> >> not the aggressive ack filter out of caution.
> >>
> >> Secondly if you are gathering drop statistics from the higher level
> >> htb categories, total drops will go WAY up,
> >> and it is best to use the cake drop, ack_drop outputs statistics
> directly.
> >>
> >> ack_drops have nothing to do with congestion control and should be
> >> ignored in most cases.
> >>
> >> Also I keep hoping more will pull out ecn marks separately. Public use
> >> is way up by e2e transports, and I worry that a high ecn ratio is a
> >> e2e network error, which might show up as a big backlog.
> >>
> >> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
> >> which is a good idea, however perhaps drops, marks and drops+marks
> >> would be better.
> >>
> >> An inverse log scale is needed to plot marks, drops, and ack_drops.
> >>
> >> I'll always be of the opinion that it's best to run cake on egress at
> >> the cpe, first, doing all this work before it hits the wire.
> >> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
> >>
> >> >We also have a lot of Eero units out there, so watching for the eero
> speed test would also be great.
> >>
> >> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
> >> buggy fq_codel offload:
> >>
> >>
> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
> >> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >>
> >> >> > sniff DNS for those speed test lookups and trigger a packet
> capture for x duration?
> >> >>
> >> >> That's a good idea (I think there's a service somewhere for finding
> speedtest
> >> >> servers?). We're discussing testing the ack-filter feature right
> now, and something
> >> >> similar would work well for that. (We have a bunch of 25/5 and
> similarly asymmetric
> >> >> connections, in theory it'll help...)
> >> >>
> >> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >>>
> >> >>> sniff DNS for those speed test lookups and trigger a packet capture
> for x duration?
> >> >>>
> >> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com>
> wrote:
> >> >>>>
> >> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
> >> >>>> >
> >> >>>> > Dave, I think that 'bouncing' is more CPU time in the browser.
> I don't think it's indicative over what's actually happening.
> >> >>>>
> >> >>>> A simultaneous packet capture and wireshark plot of the RTTs would
> >> >>>> also be helpful.
> >> >>>>
> >> >>>> >
> >> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >>>> >>
> >> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
> >> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
> >> >>>> >> >
> >> >>>> >> > I can run an Ookla test in the middle of the night with
> LibreQoS off for a bit.
> >> >>>> >>
> >> >>>> >> Groovy.
> >> >>>> >>
> >> >>>> >> > From what I recall - without Libre we see download bloat of
> 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each
> direction for most clients.
> >> >>>> >>
> >> >>>> >> The numbers report by speedtest bounce around a lot, and they
> tend to
> >> >>>> >> pick a lower number as a final result than I'd
> >> >>>> >> like. If there's a way to capture a movie of it...
> >> >>>> >> >
> >> >>>> >> > And I like the "monitor only" mode idea.
> >> >>>> >> > Perhaps we could create the HTB tree, but just not attach the
> CAKE qdisc?
> >> >>>> >> > Technically even HTB could reduce latency by itself on
> overloaded APs , so it wouldn't be true passive monitoring, but it would
> allow us to compare the before/after of CAKE while still having qdiscs we
> can point cpumap-pping to.
> >> >>>> >> >
> >> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this
> passive monitoring period?
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via
> LibreQoS <libreqos@lists.bufferbloat.net> wrote:
> >> >>>> >> >>
> >> >>>> >> >> Are you looking for a shaped result? That'll show "whatever
> bandwidth
> >> >>>> >> >> is left on the sector" without Libre, and around the
> customer's paid-for
> >> >>>> >> >> target with Libre (hopefully!).
> >> >>>> >> >>
> >> >>>> >> >> One thing Preseem does well is that their onboarding process
> includes
> >> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering
> data before you
> >> >>>> >> >> pull the switch (although most ISPs I've talked to just pull
> the switch...)
> >> >>>> >> >> It's quite enlightening to see a before/after, even with
> only FQ_CODEL
> >> >>>> >> >> as the shaper.
> >> >>>> >> >>
> >> >>>> >> >> We could probably do something similar with a "monitor only"
> mode that
> >> >>>> >> >> doesn't enable any queues (is there a TC equivalent to
> "dummy" on
> >> >>>> >> >> FreeBSD?).
> >> >>>> >> >>
> >> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >>>> >> >>>
> >> >>>> >> >>> I am curious if anyone here has speedtest results with
> libreqos on and
> >> >>>> >> >>> off, and can send a screen shot?
> >> >>>> >> >>>
> >> >>>> >> >>> --
> >> >>>> >> >>> This song goes out to all the folk that thought Stadia
> would work:
> >> >>>> >> >>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
> >> >>>> >> >>> _______________________________________________
> >> >>>> >> >>> LibreQoS mailing list
> >> >>>> >> >>> LibreQoS@lists.bufferbloat.net
> >> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >> >>>> >> >>
> >> >>>> >> >> _______________________________________________
> >> >>>> >> >> LibreQoS mailing list
> >> >>>> >> >> LibreQoS@lists.bufferbloat.net
> >> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> > --
> >> >>>> >> > Robert Chacón
> >> >>>> >> > CEO | JackRabbit Wireless LLC
> >> >>>> >> > _______________________________________________
> >> >>>> >> > LibreQoS mailing list
> >> >>>> >> > LibreQoS@lists.bufferbloat.net
> >> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
> >> >>>> >>
> >> >>>> >>
> >> >>>> >>
> >> >>>> >> --
> >> >>>> >> This song goes out to all the folk that thought Stadia would
> work:
> >> >>>> >>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> >>>> >> Dave Täht CEO, TekLibre, LLC
> >> >>>> >> _______________________________________________
> >> >>>> >> LibreQoS mailing list
> >> >>>> >> LibreQoS@lists.bufferbloat.net
> >> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> --
> >> >>>> This song goes out to all the folk that thought Stadia would work:
> >> >>>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> >>>> Dave Täht CEO, TekLibre, LLC
> >> >>>
> >> >>> _______________________________________________
> >> >>> LibreQoS mailing list
> >> >>> LibreQoS@lists.bufferbloat.net
> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >> >>
> >> >> _______________________________________________
> >> >> LibreQoS mailing list
> >> >> LibreQoS@lists.bufferbloat.net
> >> >> https://lists.bufferbloat.net/listinfo/libreqos
> >> >
> >> > _______________________________________________
> >> > LibreQoS mailing list
> >> > LibreQoS@lists.bufferbloat.net
> >> > https://lists.bufferbloat.net/listinfo/libreqos
> >>
> >>
> >>
> >> --
> >> This song goes out to all the folk that thought Stadia would work:
> >>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> Dave Täht CEO, TekLibre, LLC
> >> _______________________________________________
> >> LibreQoS mailing list
> >> LibreQoS@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/libreqos
> >
> >
> >
> > --
> > Robert Chacón
> > CEO | JackRabbit Wireless LLC
>
>
>
> --
> This song goes out to all the folk that thought Stadia would work:
>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
>

[-- Attachment #2: Type: text/html, Size: 18988 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

On Mon, Oct 24, 2022 at 10:46 AM dan <dandenson@gmail.com> wrote:
>
> Some sort of % drop is better than count from an operator's perspective.  I'm actually happy with 'scaling issues' of % on small vs large plan because those small plans are the one's we probably want to push for upgrades if they're seeing a lot of drops.
>
> That said, a 'score' method works really well from a high level.  And that can be proportionally adjusted so a slightly higher % in a small plan gives the same score as a larger plan would with fewer drops.
>
>
> Also, on the SNMP statement above, we had the same issue with preseem being hyper aggressive to pull stats.  Some mikrotik versions have some snmp flaws and preseem can peg the CPU.  Hasn't happened in a while but that was an issue.

I gave up on mikrotik long before they started working on routeros7,
and just reflashed everything to openwrt.

I have a nice little custom build for it assembled for the ubnt agws,
and the mikrotik hap ac lite. Anyone using those?

I would hope they don't have the snmp problem you describe, and if
they did I could get it fixed in days.

Vastly improved the wifi in particular, see before/after here:
https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/

(After about 40Mbits, the bloat starts moving to the wifi)

Most of my work in the past 9 months has been on improving the
mediatek mt76 derived products. I left off here:
https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/908

A lot of this stuff is shipping commercially in the openwifi project,
if you haven't been paying attention to that.

> On Mon, Oct 24, 2022 at 11:42 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> On Mon, Oct 24, 2022 at 10:35 AM Robert Chacón
>> <robert.chacon@jackrabbitwireless.com> wrote:
>> >
>> > > Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>> > which is a good idea, however perhaps drops, marks and drops+marks
>> > would be better.
>> >
>> > Yup, in InfluxDB for the stats by Tin (diffserv4) we display true drops as (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by sub quite yet.
>> > And ok, drops, marks and drops+marks can be shown instead - by subscriber or by Node (AP/Site).
>> > Dave - is it ok if we graph this data as a percentage of packets sent? So for drops, drop / sent_packets ? I just imagine that  would make comparing different subscribers' connections easier.
>>
>> Drops and marks are proportional to the plan. You get a heck of a lot
>> more drops at 5mbits than 50, and the relationship
>> is not linear, closer to quadratic.
>>
>> So comparing customers at their given tiers makes more sense than
>> across the board.
>>
>> As for a percentage, you should see a clear difference in percentages
>> across tiers.
>>
>> I know I'm not quite answering your question well, I'm a big believer
>> in just getting lots of data, graphing it every which way,
>> and seeing what's useful.
>>
>> > Question for all - is having drops / marks/ drops+marks by subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we just show those stats by AP/Node?
>> > If preseem currently offers such stats at a subscriber level we can do the same, just wanna be sure I'm not overtaxing CPU.
>> >
>> > On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >>
>> >> Some of the context of my request for libreqos on vs off is from here,
>> >> presentation in the wee hours, wed morning.
>> >>
>> >> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>> >>
>> >> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
>> >> <libreqos@lists.bufferbloat.net> wrote:
>> >> >
>> >> > we also have a bunch of 25/5.  It would be nice to get some more detailed numbers from ookla tests.
>> >>
>> >> +1! We've argued methods with many a provider, and ookla and samknows
>> >> are new to this game.
>> >>
>> >> As one example I don't know if they use decimal or binary mbits!
>> >>
>> >> ...
>> >>
>> >> ack-filtering in the libreqos case can and will drop acks in both
>> >> directions, due to the structure of how htb
>> >> works in this scenario, at any bandwidth ratio. I *think* it does very
>> >> little harm but I would use the ack-filter
>> >> not the aggressive ack filter out of caution.
>> >>
>> >> Secondly if you are gathering drop statistics from the higher level
>> >> htb categories, total drops will go WAY up,
>> >> and it is best to use the cake drop, ack_drop outputs statistics directly.
>> >>
>> >> ack_drops have nothing to do with congestion control and should be
>> >> ignored in most cases.
>> >>
>> >> Also I keep hoping more will pull out ecn marks separately. Public use
>> >> is way up by e2e transports, and I worry that a high ecn ratio is a
>> >> e2e network error, which might show up as a big backlog.
>> >>
>> >> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>> >> which is a good idea, however perhaps drops, marks and drops+marks
>> >> would be better.
>> >>
>> >> An inverse log scale is needed to plot marks, drops, and ack_drops.
>> >>
>> >> I'll always be of the opinion that it's best to run cake on egress at
>> >> the cpe, first, doing all this work before it hits the wire.
>> >> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>> >>
>> >> >We also have a lot of Eero units out there, so watching for the eero speed test would also be great.
>> >>
>> >> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
>> >> buggy fq_codel offload:
>> >>
>> >> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
>> >> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >>
>> >> >> > sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>> >> >>
>> >> >> That's a good idea (I think there's a service somewhere for finding speedtest
>> >> >> servers?). We're discussing testing the ack-filter feature right now, and something
>> >> >> similar would work well for that. (We have a bunch of 25/5 and similarly asymmetric
>> >> >> connections, in theory it'll help...)
>> >> >>
>> >> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >>>
>> >> >>> sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>> >> >>>
>> >> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >> >>>>
>> >> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>> >> >>>> >
>> >> >>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I don't think it's indicative over what's actually happening.
>> >> >>>>
>> >> >>>> A simultaneous packet capture and wireshark plot of the RTTs would
>> >> >>>> also be helpful.
>> >> >>>>
>> >> >>>> >
>> >> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >>>> >>
>> >> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>> >> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
>> >> >>>> >> >
>> >> >>>> >> > I can run an Ookla test in the middle of the night with LibreQoS off for a bit.
>> >> >>>> >>
>> >> >>>> >> Groovy.
>> >> >>>> >>
>> >> >>>> >> > From what I recall - without Libre we see download bloat of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each direction for most clients.
>> >> >>>> >>
>> >> >>>> >> The numbers report by speedtest bounce around a lot, and they tend to
>> >> >>>> >> pick a lower number as a final result than I'd
>> >> >>>> >> like. If there's a way to capture a movie of it...
>> >> >>>> >> >
>> >> >>>> >> > And I like the "monitor only" mode idea.
>> >> >>>> >> > Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
>> >> >>>> >> > Technically even HTB could reduce latency by itself on overloaded APs , so it wouldn't be true passive monitoring, but it would allow us to compare the before/after of CAKE while still having qdiscs we can point cpumap-pping to.
>> >> >>>> >> >
>> >> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this passive monitoring period?
>> >> >>>> >> >
>> >> >>>> >> >
>> >> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >>>> >> >>
>> >> >>>> >> >> Are you looking for a shaped result? That'll show "whatever bandwidth
>> >> >>>> >> >> is left on the sector" without Libre, and around the customer's paid-for
>> >> >>>> >> >> target with Libre (hopefully!).
>> >> >>>> >> >>
>> >> >>>> >> >> One thing Preseem does well is that their onboarding process includes
>> >> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering data before you
>> >> >>>> >> >> pull the switch (although most ISPs I've talked to just pull the switch...)
>> >> >>>> >> >> It's quite enlightening to see a before/after, even with only FQ_CODEL
>> >> >>>> >> >> as the shaper.
>> >> >>>> >> >>
>> >> >>>> >> >> We could probably do something similar with a "monitor only" mode that
>> >> >>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
>> >> >>>> >> >> FreeBSD?).
>> >> >>>> >> >>
>> >> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >>>> >> >>>
>> >> >>>> >> >>> I am curious if anyone here has speedtest results with libreqos on and
>> >> >>>> >> >>> off, and can send a screen shot?
>> >> >>>> >> >>>
>> >> >>>> >> >>> --
>> >> >>>> >> >>> This song goes out to all the folk that thought Stadia would work:
>> >> >>>> >> >>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
>> >> >>>> >> >>> _______________________________________________
>> >> >>>> >> >>> LibreQoS mailing list
>> >> >>>> >> >>> LibreQoS@lists.bufferbloat.net
>> >> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >>>> >> >>
>> >> >>>> >> >> _______________________________________________
>> >> >>>> >> >> LibreQoS mailing list
>> >> >>>> >> >> LibreQoS@lists.bufferbloat.net
>> >> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >>>> >> >
>> >> >>>> >> >
>> >> >>>> >> >
>> >> >>>> >> > --
>> >> >>>> >> > Robert Chacón
>> >> >>>> >> > CEO | JackRabbit Wireless LLC
>> >> >>>> >> > _______________________________________________
>> >> >>>> >> > LibreQoS mailing list
>> >> >>>> >> > LibreQoS@lists.bufferbloat.net
>> >> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >>
>> >> >>>> >> --
>> >> >>>> >> This song goes out to all the folk that thought Stadia would work:
>> >> >>>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >>>> >> Dave Täht CEO, TekLibre, LLC
>> >> >>>> >> _______________________________________________
>> >> >>>> >> LibreQoS mailing list
>> >> >>>> >> LibreQoS@lists.bufferbloat.net
>> >> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>> --
>> >> >>>> This song goes out to all the folk that thought Stadia would work:
>> >> >>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >>>> Dave Täht CEO, TekLibre, LLC
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> LibreQoS mailing list
>> >> >>> LibreQoS@lists.bufferbloat.net
>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >>
>> >> >> _______________________________________________
>> >> >> LibreQoS mailing list
>> >> >> LibreQoS@lists.bufferbloat.net
>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >
>> >> > _______________________________________________
>> >> > LibreQoS mailing list
>> >> > LibreQoS@lists.bufferbloat.net
>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >>
>> >>
>> >>
>> >> --
>> >> This song goes out to all the folk that thought Stadia would work:
>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> Dave Täht CEO, TekLibre, LLC
>> >> _______________________________________________
>> >> LibreQoS mailing list
>> >> LibreQoS@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >
>> >
>> >
>> > --
>> > Robert Chacón
>> > CEO | JackRabbit Wireless LLC
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC



-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

Re: [LibreQoS] ookla speedtest results?

[dan]

[-- Attachment #1: Type: text/plain, Size: 14515 bytes --]

We're pretty far past that hardware.  Minimum plans at most sites are now
100x20 and we're moving rapidly to 200x200-1Gx1G plans.  The implication
being that we need 4x4 AX wifi radios and that's real tough in an openwrt
compatible hardware right now, not a ton of options.

Flat out though, wisps in general simply do not have the option of
controlling every client side router or even sending out new/compatible
routers for each customer to pull this off.  It's a non-starter for us.
It may be better but it's just not an option that can be pursued.
ingress and egress edge shaping is what we need.

On Mon, Oct 24, 2022 at 11:57 AM Dave Taht <dave.taht@gmail.com> wrote:

> On Mon, Oct 24, 2022 at 10:46 AM dan <dandenson@gmail.com> wrote:
> >
> > Some sort of % drop is better than count from an operator's
> perspective.  I'm actually happy with 'scaling issues' of % on small vs
> large plan because those small plans are the one's we probably want to push
> for upgrades if they're seeing a lot of drops.
> >
> > That said, a 'score' method works really well from a high level.  And
> that can be proportionally adjusted so a slightly higher % in a small plan
> gives the same score as a larger plan would with fewer drops.
> >
> >
> > Also, on the SNMP statement above, we had the same issue with preseem
> being hyper aggressive to pull stats.  Some mikrotik versions have some
> snmp flaws and preseem can peg the CPU.  Hasn't happened in a while but
> that was an issue.
>
> I gave up on mikrotik long before they started working on routeros7,
> and just reflashed everything to openwrt.
>
> I have a nice little custom build for it assembled for the ubnt agws,
> and the mikrotik hap ac lite. Anyone using those?
>
> I would hope they don't have the snmp problem you describe, and if
> they did I could get it fixed in days.
>
> Vastly improved the wifi in particular, see before/after here:
> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/
>
> (After about 40Mbits, the bloat starts moving to the wifi)
>
> Most of my work in the past 9 months has been on improving the
> mediatek mt76 derived products. I left off here:
> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/908
>
> A lot of this stuff is shipping commercially in the openwifi project,
> if you haven't been paying attention to that.
>
> > On Mon, Oct 24, 2022 at 11:42 AM Dave Taht <dave.taht@gmail.com> wrote:
> >>
> >> On Mon, Oct 24, 2022 at 10:35 AM Robert Chacón
> >> <robert.chacon@jackrabbitwireless.com> wrote:
> >> >
> >> > > Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
> >> > which is a good idea, however perhaps drops, marks and drops+marks
> >> > would be better.
> >> >
> >> > Yup, in InfluxDB for the stats by Tin (diffserv4) we display true
> drops as (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by
> sub quite yet.
> >> > And ok, drops, marks and drops+marks can be shown instead - by
> subscriber or by Node (AP/Site).
> >> > Dave - is it ok if we graph this data as a percentage of packets
> sent? So for drops, drop / sent_packets ? I just imagine that  would make
> comparing different subscribers' connections easier.
> >>
> >> Drops and marks are proportional to the plan. You get a heck of a lot
> >> more drops at 5mbits than 50, and the relationship
> >> is not linear, closer to quadratic.
> >>
> >> So comparing customers at their given tiers makes more sense than
> >> across the board.
> >>
> >> As for a percentage, you should see a clear difference in percentages
> >> across tiers.
> >>
> >> I know I'm not quite answering your question well, I'm a big believer
> >> in just getting lots of data, graphing it every which way,
> >> and seeing what's useful.
> >>
> >> > Question for all - is having drops / marks/ drops+marks by
> subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we
> just show those stats by AP/Node?
> >> > If preseem currently offers such stats at a subscriber level we can
> do the same, just wanna be sure I'm not overtaxing CPU.
> >> >
> >> > On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >>
> >> >> Some of the context of my request for libreqos on vs off is from
> here,
> >> >> presentation in the wee hours, wed morning.
> >> >>
> >> >>
> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
> >> >>
> >> >> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
> >> >> <libreqos@lists.bufferbloat.net> wrote:
> >> >> >
> >> >> > we also have a bunch of 25/5.  It would be nice to get some more
> detailed numbers from ookla tests.
> >> >>
> >> >> +1! We've argued methods with many a provider, and ookla and samknows
> >> >> are new to this game.
> >> >>
> >> >> As one example I don't know if they use decimal or binary mbits!
> >> >>
> >> >> ...
> >> >>
> >> >> ack-filtering in the libreqos case can and will drop acks in both
> >> >> directions, due to the structure of how htb
> >> >> works in this scenario, at any bandwidth ratio. I *think* it does
> very
> >> >> little harm but I would use the ack-filter
> >> >> not the aggressive ack filter out of caution.
> >> >>
> >> >> Secondly if you are gathering drop statistics from the higher level
> >> >> htb categories, total drops will go WAY up,
> >> >> and it is best to use the cake drop, ack_drop outputs statistics
> directly.
> >> >>
> >> >> ack_drops have nothing to do with congestion control and should be
> >> >> ignored in most cases.
> >> >>
> >> >> Also I keep hoping more will pull out ecn marks separately. Public
> use
> >> >> is way up by e2e transports, and I worry that a high ecn ratio is a
> >> >> e2e network error, which might show up as a big backlog.
> >> >>
> >> >> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
> >> >> which is a good idea, however perhaps drops, marks and drops+marks
> >> >> would be better.
> >> >>
> >> >> An inverse log scale is needed to plot marks, drops, and ack_drops.
> >> >>
> >> >> I'll always be of the opinion that it's best to run cake on egress at
> >> >> the cpe, first, doing all this work before it hits the wire.
> >> >> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
> >> >>
> >> >> >We also have a lot of Eero units out there, so watching for the
> eero speed test would also be great.
> >> >>
> >> >> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
> >> >> buggy fq_codel offload:
> >> >>
> >> >>
> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
> >> >> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >> >>
> >> >> >> > sniff DNS for those speed test lookups and trigger a packet
> capture for x duration?
> >> >> >>
> >> >> >> That's a good idea (I think there's a service somewhere for
> finding speedtest
> >> >> >> servers?). We're discussing testing the ack-filter feature right
> now, and something
> >> >> >> similar would work well for that. (We have a bunch of 25/5 and
> similarly asymmetric
> >> >> >> connections, in theory it'll help...)
> >> >> >>
> >> >> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >> >>>
> >> >> >>> sniff DNS for those speed test lookups and trigger a packet
> capture for x duration?
> >> >> >>>
> >> >> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com>
> wrote:
> >> >> >>>>
> >> >> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com>
> wrote:
> >> >> >>>> >
> >> >> >>>> > Dave, I think that 'bouncing' is more CPU time in the
> browser.  I don't think it's indicative over what's actually happening.
> >> >> >>>>
> >> >> >>>> A simultaneous packet capture and wireshark plot of the RTTs
> would
> >> >> >>>> also be helpful.
> >> >> >>>>
> >> >> >>>> >
> >> >> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >> >>>> >>
> >> >> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
> >> >> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
> >> >> >>>> >> >
> >> >> >>>> >> > I can run an Ookla test in the middle of the night with
> LibreQoS off for a bit.
> >> >> >>>> >>
> >> >> >>>> >> Groovy.
> >> >> >>>> >>
> >> >> >>>> >> > From what I recall - without Libre we see download bloat
> of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat
> each direction for most clients.
> >> >> >>>> >>
> >> >> >>>> >> The numbers report by speedtest bounce around a lot, and
> they tend to
> >> >> >>>> >> pick a lower number as a final result than I'd
> >> >> >>>> >> like. If there's a way to capture a movie of it...
> >> >> >>>> >> >
> >> >> >>>> >> > And I like the "monitor only" mode idea.
> >> >> >>>> >> > Perhaps we could create the HTB tree, but just not attach
> the CAKE qdisc?
> >> >> >>>> >> > Technically even HTB could reduce latency by itself on
> overloaded APs , so it wouldn't be true passive monitoring, but it would
> allow us to compare the before/after of CAKE while still having qdiscs we
> can point cpumap-pping to.
> >> >> >>>> >> >
> >> >> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this
> passive monitoring period?
> >> >> >>>> >> >
> >> >> >>>> >> >
> >> >> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via
> LibreQoS <libreqos@lists.bufferbloat.net> wrote:
> >> >> >>>> >> >>
> >> >> >>>> >> >> Are you looking for a shaped result? That'll show
> "whatever bandwidth
> >> >> >>>> >> >> is left on the sector" without Libre, and around the
> customer's paid-for
> >> >> >>>> >> >> target with Libre (hopefully!).
> >> >> >>>> >> >>
> >> >> >>>> >> >> One thing Preseem does well is that their onboarding
> process includes
> >> >> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering
> data before you
> >> >> >>>> >> >> pull the switch (although most ISPs I've talked to just
> pull the switch...)
> >> >> >>>> >> >> It's quite enlightening to see a before/after, even with
> only FQ_CODEL
> >> >> >>>> >> >> as the shaper.
> >> >> >>>> >> >>
> >> >> >>>> >> >> We could probably do something similar with a "monitor
> only" mode that
> >> >> >>>> >> >> doesn't enable any queues (is there a TC equivalent to
> "dummy" on
> >> >> >>>> >> >> FreeBSD?).
> >> >> >>>> >> >>
> >> >> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
> libreqos@lists.bufferbloat.net> wrote:
> >> >> >>>> >> >>>
> >> >> >>>> >> >>> I am curious if anyone here has speedtest results with
> libreqos on and
> >> >> >>>> >> >>> off, and can send a screen shot?
> >> >> >>>> >> >>>
> >> >> >>>> >> >>> --
> >> >> >>>> >> >>> This song goes out to all the folk that thought Stadia
> would work:
> >> >> >>>> >> >>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> >> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
> >> >> >>>> >> >>> _______________________________________________
> >> >> >>>> >> >>> LibreQoS mailing list
> >> >> >>>> >> >>> LibreQoS@lists.bufferbloat.net
> >> >> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >> >> >>>> >> >>
> >> >> >>>> >> >> _______________________________________________
> >> >> >>>> >> >> LibreQoS mailing list
> >> >> >>>> >> >> LibreQoS@lists.bufferbloat.net
> >> >> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
> >> >> >>>> >> >
> >> >> >>>> >> >
> >> >> >>>> >> >
> >> >> >>>> >> > --
> >> >> >>>> >> > Robert Chacón
> >> >> >>>> >> > CEO | JackRabbit Wireless LLC
> >> >> >>>> >> > _______________________________________________
> >> >> >>>> >> > LibreQoS mailing list
> >> >> >>>> >> > LibreQoS@lists.bufferbloat.net
> >> >> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
> >> >> >>>> >>
> >> >> >>>> >>
> >> >> >>>> >>
> >> >> >>>> >> --
> >> >> >>>> >> This song goes out to all the folk that thought Stadia would
> work:
> >> >> >>>> >>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> >> >>>> >> Dave Täht CEO, TekLibre, LLC
> >> >> >>>> >> _______________________________________________
> >> >> >>>> >> LibreQoS mailing list
> >> >> >>>> >> LibreQoS@lists.bufferbloat.net
> >> >> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> >> >> >>>> --
> >> >> >>>> This song goes out to all the folk that thought Stadia would
> work:
> >> >> >>>>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> >> >>>> Dave Täht CEO, TekLibre, LLC
> >> >> >>>
> >> >> >>> _______________________________________________
> >> >> >>> LibreQoS mailing list
> >> >> >>> LibreQoS@lists.bufferbloat.net
> >> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
> >> >> >>
> >> >> >> _______________________________________________
> >> >> >> LibreQoS mailing list
> >> >> >> LibreQoS@lists.bufferbloat.net
> >> >> >> https://lists.bufferbloat.net/listinfo/libreqos
> >> >> >
> >> >> > _______________________________________________
> >> >> > LibreQoS mailing list
> >> >> > LibreQoS@lists.bufferbloat.net
> >> >> > https://lists.bufferbloat.net/listinfo/libreqos
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> This song goes out to all the folk that thought Stadia would work:
> >> >>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> >> Dave Täht CEO, TekLibre, LLC
> >> >> _______________________________________________
> >> >> LibreQoS mailing list
> >> >> LibreQoS@lists.bufferbloat.net
> >> >> https://lists.bufferbloat.net/listinfo/libreqos
> >> >
> >> >
> >> >
> >> > --
> >> > Robert Chacón
> >> > CEO | JackRabbit Wireless LLC
> >>
> >>
> >>
> >> --
> >> This song goes out to all the folk that thought Stadia would work:
> >>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> >> Dave Täht CEO, TekLibre, LLC
>
>
>
> --
> This song goes out to all the folk that thought Stadia would work:
>
> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
> Dave Täht CEO, TekLibre, LLC
>

[-- Attachment #2: Type: text/html, Size: 23530 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Herbert Wolverson]

[-- Attachment #1: Type: text/plain, Size: 15580 bytes --]

I think we have a handful of HAP lite's left? They aren't great, but they
are cheap - so
we have a small apartment complex using them as client-side devices. The
complex is a half-way house for paroled offenders, and they destroy a *lot*
of hardware. :-|

The biggest issue we've had (other than they aren't very resilient to
someone
who needs anger-management classes) is the antenna. There's just not a lot
of gain there.

On Mon, Oct 24, 2022 at 1:03 PM dan via LibreQoS <
libreqos@lists.bufferbloat.net> wrote:

> We're pretty far past that hardware.  Minimum plans at most sites are now
> 100x20 and we're moving rapidly to 200x200-1Gx1G plans.  The implication
> being that we need 4x4 AX wifi radios and that's real tough in an openwrt
> compatible hardware right now, not a ton of options.
>
> Flat out though, wisps in general simply do not have the option of
> controlling every client side router or even sending out new/compatible
> routers for each customer to pull this off.  It's a non-starter for us.
> It may be better but it's just not an option that can be pursued.
> ingress and egress edge shaping is what we need.
>
> On Mon, Oct 24, 2022 at 11:57 AM Dave Taht <dave.taht@gmail.com> wrote:
>
>> On Mon, Oct 24, 2022 at 10:46 AM dan <dandenson@gmail.com> wrote:
>> >
>> > Some sort of % drop is better than count from an operator's
>> perspective.  I'm actually happy with 'scaling issues' of % on small vs
>> large plan because those small plans are the one's we probably want to push
>> for upgrades if they're seeing a lot of drops.
>> >
>> > That said, a 'score' method works really well from a high level.  And
>> that can be proportionally adjusted so a slightly higher % in a small plan
>> gives the same score as a larger plan would with fewer drops.
>> >
>> >
>> > Also, on the SNMP statement above, we had the same issue with preseem
>> being hyper aggressive to pull stats.  Some mikrotik versions have some
>> snmp flaws and preseem can peg the CPU.  Hasn't happened in a while but
>> that was an issue.
>>
>> I gave up on mikrotik long before they started working on routeros7,
>> and just reflashed everything to openwrt.
>>
>> I have a nice little custom build for it assembled for the ubnt agws,
>> and the mikrotik hap ac lite. Anyone using those?
>>
>> I would hope they don't have the snmp problem you describe, and if
>> they did I could get it fixed in days.
>>
>> Vastly improved the wifi in particular, see before/after here:
>> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/
>>
>> (After about 40Mbits, the bloat starts moving to the wifi)
>>
>> Most of my work in the past 9 months has been on improving the
>> mediatek mt76 derived products. I left off here:
>> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/908
>>
>> A lot of this stuff is shipping commercially in the openwifi project,
>> if you haven't been paying attention to that.
>>
>> > On Mon, Oct 24, 2022 at 11:42 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >>
>> >> On Mon, Oct 24, 2022 at 10:35 AM Robert Chacón
>> >> <robert.chacon@jackrabbitwireless.com> wrote:
>> >> >
>> >> > > Presently (in v1.3.), robert (in grafina?) is summing drops +
>> marks.
>> >> > which is a good idea, however perhaps drops, marks and drops+marks
>> >> > would be better.
>> >> >
>> >> > Yup, in InfluxDB for the stats by Tin (diffserv4) we display true
>> drops as (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by
>> sub quite yet.
>> >> > And ok, drops, marks and drops+marks can be shown instead - by
>> subscriber or by Node (AP/Site).
>> >> > Dave - is it ok if we graph this data as a percentage of packets
>> sent? So for drops, drop / sent_packets ? I just imagine that  would make
>> comparing different subscribers' connections easier.
>> >>
>> >> Drops and marks are proportional to the plan. You get a heck of a lot
>> >> more drops at 5mbits than 50, and the relationship
>> >> is not linear, closer to quadratic.
>> >>
>> >> So comparing customers at their given tiers makes more sense than
>> >> across the board.
>> >>
>> >> As for a percentage, you should see a clear difference in percentages
>> >> across tiers.
>> >>
>> >> I know I'm not quite answering your question well, I'm a big believer
>> >> in just getting lots of data, graphing it every which way,
>> >> and seeing what's useful.
>> >>
>> >> > Question for all - is having drops / marks/ drops+marks by
>> subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we
>> just show those stats by AP/Node?
>> >> > If preseem currently offers such stats at a subscriber level we can
>> do the same, just wanna be sure I'm not overtaxing CPU.
>> >> >
>> >> > On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >> >>
>> >> >> Some of the context of my request for libreqos on vs off is from
>> here,
>> >> >> presentation in the wee hours, wed morning.
>> >> >>
>> >> >>
>> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>> >> >>
>> >> >> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
>> >> >> <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >
>> >> >> > we also have a bunch of 25/5.  It would be nice to get some more
>> detailed numbers from ookla tests.
>> >> >>
>> >> >> +1! We've argued methods with many a provider, and ookla and
>> samknows
>> >> >> are new to this game.
>> >> >>
>> >> >> As one example I don't know if they use decimal or binary mbits!
>> >> >>
>> >> >> ...
>> >> >>
>> >> >> ack-filtering in the libreqos case can and will drop acks in both
>> >> >> directions, due to the structure of how htb
>> >> >> works in this scenario, at any bandwidth ratio. I *think* it does
>> very
>> >> >> little harm but I would use the ack-filter
>> >> >> not the aggressive ack filter out of caution.
>> >> >>
>> >> >> Secondly if you are gathering drop statistics from the higher level
>> >> >> htb categories, total drops will go WAY up,
>> >> >> and it is best to use the cake drop, ack_drop outputs statistics
>> directly.
>> >> >>
>> >> >> ack_drops have nothing to do with congestion control and should be
>> >> >> ignored in most cases.
>> >> >>
>> >> >> Also I keep hoping more will pull out ecn marks separately. Public
>> use
>> >> >> is way up by e2e transports, and I worry that a high ecn ratio is a
>> >> >> e2e network error, which might show up as a big backlog.
>> >> >>
>> >> >> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>> >> >> which is a good idea, however perhaps drops, marks and drops+marks
>> >> >> would be better.
>> >> >>
>> >> >> An inverse log scale is needed to plot marks, drops, and ack_drops.
>> >> >>
>> >> >> I'll always be of the opinion that it's best to run cake on egress
>> at
>> >> >> the cpe, first, doing all this work before it hits the wire.
>> >> >> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>> >> >>
>> >> >> >We also have a lot of Eero units out there, so watching for the
>> eero speed test would also be great.
>> >> >>
>> >> >> eero 5s have cake (and fq_codel on the wifi), eero 6s have a
>> somewhat
>> >> >> buggy fq_codel offload:
>> >> >>
>> >> >>
>> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
>> >> >> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>
>> >> >> >> > sniff DNS for those speed test lookups and trigger a packet
>> capture for x duration?
>> >> >> >>
>> >> >> >> That's a good idea (I think there's a service somewhere for
>> finding speedtest
>> >> >> >> servers?). We're discussing testing the ack-filter feature right
>> now, and something
>> >> >> >> similar would work well for that. (We have a bunch of 25/5 and
>> similarly asymmetric
>> >> >> >> connections, in theory it'll help...)
>> >> >> >>
>> >> >> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>
>> >> >> >>> sniff DNS for those speed test lookups and trigger a packet
>> capture for x duration?
>> >> >> >>>
>> >> >> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com>
>> wrote:
>> >> >> >>>>
>> >> >> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com>
>> wrote:
>> >> >> >>>> >
>> >> >> >>>> > Dave, I think that 'bouncing' is more CPU time in the
>> browser.  I don't think it's indicative over what's actually happening.
>> >> >> >>>>
>> >> >> >>>> A simultaneous packet capture and wireshark plot of the RTTs
>> would
>> >> >> >>>> also be helpful.
>> >> >> >>>>
>> >> >> >>>> >
>> >> >> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>> >>
>> >> >> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>> >> >> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>> >> >
>> >> >> >>>> >> > I can run an Ookla test in the middle of the night with
>> LibreQoS off for a bit.
>> >> >> >>>> >>
>> >> >> >>>> >> Groovy.
>> >> >> >>>> >>
>> >> >> >>>> >> > From what I recall - without Libre we see download bloat
>> of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat
>> each direction for most clients.
>> >> >> >>>> >>
>> >> >> >>>> >> The numbers report by speedtest bounce around a lot, and
>> they tend to
>> >> >> >>>> >> pick a lower number as a final result than I'd
>> >> >> >>>> >> like. If there's a way to capture a movie of it...
>> >> >> >>>> >> >
>> >> >> >>>> >> > And I like the "monitor only" mode idea.
>> >> >> >>>> >> > Perhaps we could create the HTB tree, but just not attach
>> the CAKE qdisc?
>> >> >> >>>> >> > Technically even HTB could reduce latency by itself on
>> overloaded APs , so it wouldn't be true passive monitoring, but it would
>> allow us to compare the before/after of CAKE while still having qdiscs we
>> can point cpumap-pping to.
>> >> >> >>>> >> >
>> >> >> >>>> >> > Alternatively, maybe we could use eBPF PPing just for
>> this passive monitoring period?
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via
>> LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> Are you looking for a shaped result? That'll show
>> "whatever bandwidth
>> >> >> >>>> >> >> is left on the sector" without Libre, and around the
>> customer's paid-for
>> >> >> >>>> >> >> target with Libre (hopefully!).
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> One thing Preseem does well is that their onboarding
>> process includes
>> >> >> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering
>> data before you
>> >> >> >>>> >> >> pull the switch (although most ISPs I've talked to just
>> pull the switch...)
>> >> >> >>>> >> >> It's quite enlightening to see a before/after, even with
>> only FQ_CODEL
>> >> >> >>>> >> >> as the shaper.
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> We could probably do something similar with a "monitor
>> only" mode that
>> >> >> >>>> >> >> doesn't enable any queues (is there a TC equivalent to
>> "dummy" on
>> >> >> >>>> >> >> FreeBSD?).
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
>> libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> I am curious if anyone here has speedtest results with
>> libreqos on and
>> >> >> >>>> >> >>> off, and can send a screen shot?
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> --
>> >> >> >>>> >> >>> This song goes out to all the folk that thought Stadia
>> would work:
>> >> >> >>>> >> >>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
>> >> >> >>>> >> >>> _______________________________________________
>> >> >> >>>> >> >>> LibreQoS mailing list
>> >> >> >>>> >> >>> LibreQoS@lists.bufferbloat.net
>> >> >> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> _______________________________________________
>> >> >> >>>> >> >> LibreQoS mailing list
>> >> >> >>>> >> >> LibreQoS@lists.bufferbloat.net
>> >> >> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> > --
>> >> >> >>>> >> > Robert Chacón
>> >> >> >>>> >> > CEO | JackRabbit Wireless LLC
>> >> >> >>>> >> > _______________________________________________
>> >> >> >>>> >> > LibreQoS mailing list
>> >> >> >>>> >> > LibreQoS@lists.bufferbloat.net
>> >> >> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >> --
>> >> >> >>>> >> This song goes out to all the folk that thought Stadia
>> would work:
>> >> >> >>>> >>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> >>>> >> Dave Täht CEO, TekLibre, LLC
>> >> >> >>>> >> _______________________________________________
>> >> >> >>>> >> LibreQoS mailing list
>> >> >> >>>> >> LibreQoS@lists.bufferbloat.net
>> >> >> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> --
>> >> >> >>>> This song goes out to all the folk that thought Stadia would
>> work:
>> >> >> >>>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> >>>> Dave Täht CEO, TekLibre, LLC
>> >> >> >>>
>> >> >> >>> _______________________________________________
>> >> >> >>> LibreQoS mailing list
>> >> >> >>> LibreQoS@lists.bufferbloat.net
>> >> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>
>> >> >> >> _______________________________________________
>> >> >> >> LibreQoS mailing list
>> >> >> >> LibreQoS@lists.bufferbloat.net
>> >> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > LibreQoS mailing list
>> >> >> > LibreQoS@lists.bufferbloat.net
>> >> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> This song goes out to all the folk that thought Stadia would work:
>> >> >>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> Dave Täht CEO, TekLibre, LLC
>> >> >> _______________________________________________
>> >> >> LibreQoS mailing list
>> >> >> LibreQoS@lists.bufferbloat.net
>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Robert Chacón
>> >> > CEO | JackRabbit Wireless LLC
>> >>
>> >>
>> >>
>> >> --
>> >> This song goes out to all the folk that thought Stadia would work:
>> >>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> Dave Täht CEO, TekLibre, LLC
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC
>>
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>

[-- Attachment #2: Type: text/html, Size: 24792 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

On Mon, Oct 24, 2022 at 11:06 AM Herbert Wolverson via LibreQoS
<libreqos@lists.bufferbloat.net> wrote:
>
> I think we have a handful of HAP lite's left? They aren't great, but they are cheap - so
> we have a small apartment complex using them as client-side devices. The
> complex is a half-way house for paroled offenders, and they destroy a *lot*
> of hardware. :-|
>
> The biggest issue we've had (other than they aren't very resilient to someone
> who needs anger-management classes) is the antenna. There's just not a lot
> of gain there.

I agree on the antenna front.  There are plenty of other devices with
better antennas. Still, I'm frugal, I keep hoping the stuff we are
turning into green waste ends up instead, reflashed, operating at the
bandwidths available in the third world,
ipv6 capable, etc. I regularly ran routers down to SJDS, nicaragua
precovid, and gave them to coffee shops and airbnbs.

There are still a lot of people stuck (and unable to upgrade) at 25/3,
too, where a 20 dollar router
with cake and better wifi on it, would make a difference.

I am ever more aware that my hippie-dippy approach to life is not
shared by most! Here's a 3+ year old proposal that got no traction.

https://docs.google.com/document/d/1T21on7g1MqQZoK91epUdxLYFGdtyLRgBat0VXoC9e3I/edit

I was thinking with a tax subsidy, perhaps we could find uses for
these older routers, as meshy stuff, at least. To this day, tho, I
have dozens of routers, running modern openwrt, still in use, doing an
adaquate job. My favorite is a picostation above a pool I like, still
serving up 50+Mbit second, which was one of the very first routers I
reflashed to fq_codel back in 2012. I have every reason to believe it
will still be working 10 years from now.

>
> On Mon, Oct 24, 2022 at 1:03 PM dan via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>
>> We're pretty far past that hardware.  Minimum plans at most sites are now 100x20 and we're moving rapidly to 200x200-1Gx1G plans.  The implication being that we need 4x4 AX wifi radios and that's real tough in an openwrt compatible hardware right now, not a ton of options.
>>
>> Flat out though, wisps in general simply do not have the option of controlling every client side router or even sending out new/compatible routers for each customer to pull this off.  It's a non-starter for us.    It may be better but it's just not an option that can be pursued.  ingress and egress edge shaping is what we need.
>>
>> On Mon, Oct 24, 2022 at 11:57 AM Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>> On Mon, Oct 24, 2022 at 10:46 AM dan <dandenson@gmail.com> wrote:
>>> >
>>> > Some sort of % drop is better than count from an operator's perspective.  I'm actually happy with 'scaling issues' of % on small vs large plan because those small plans are the one's we probably want to push for upgrades if they're seeing a lot of drops.
>>> >
>>> > That said, a 'score' method works really well from a high level.  And that can be proportionally adjusted so a slightly higher % in a small plan gives the same score as a larger plan would with fewer drops.
>>> >
>>> >
>>> > Also, on the SNMP statement above, we had the same issue with preseem being hyper aggressive to pull stats.  Some mikrotik versions have some snmp flaws and preseem can peg the CPU.  Hasn't happened in a while but that was an issue.
>>>
>>> I gave up on mikrotik long before they started working on routeros7,
>>> and just reflashed everything to openwrt.
>>>
>>> I have a nice little custom build for it assembled for the ubnt agws,
>>> and the mikrotik hap ac lite. Anyone using those?
>>>
>>> I would hope they don't have the snmp problem you describe, and if
>>> they did I could get it fixed in days.
>>>
>>> Vastly improved the wifi in particular, see before/after here:
>>> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/
>>>
>>> (After about 40Mbits, the bloat starts moving to the wifi)
>>>
>>> Most of my work in the past 9 months has been on improving the
>>> mediatek mt76 derived products. I left off here:
>>> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/908
>>>
>>> A lot of this stuff is shipping commercially in the openwifi project,
>>> if you haven't been paying attention to that.
>>>
>>> > On Mon, Oct 24, 2022 at 11:42 AM Dave Taht <dave.taht@gmail.com> wrote:
>>> >>
>>> >> On Mon, Oct 24, 2022 at 10:35 AM Robert Chacón
>>> >> <robert.chacon@jackrabbitwireless.com> wrote:
>>> >> >
>>> >> > > Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>>> >> > which is a good idea, however perhaps drops, marks and drops+marks
>>> >> > would be better.
>>> >> >
>>> >> > Yup, in InfluxDB for the stats by Tin (diffserv4) we display true drops as (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by sub quite yet.
>>> >> > And ok, drops, marks and drops+marks can be shown instead - by subscriber or by Node (AP/Site).
>>> >> > Dave - is it ok if we graph this data as a percentage of packets sent? So for drops, drop / sent_packets ? I just imagine that  would make comparing different subscribers' connections easier.
>>> >>
>>> >> Drops and marks are proportional to the plan. You get a heck of a lot
>>> >> more drops at 5mbits than 50, and the relationship
>>> >> is not linear, closer to quadratic.
>>> >>
>>> >> So comparing customers at their given tiers makes more sense than
>>> >> across the board.
>>> >>
>>> >> As for a percentage, you should see a clear difference in percentages
>>> >> across tiers.
>>> >>
>>> >> I know I'm not quite answering your question well, I'm a big believer
>>> >> in just getting lots of data, graphing it every which way,
>>> >> and seeing what's useful.
>>> >>
>>> >> > Question for all - is having drops / marks/ drops+marks by subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we just show those stats by AP/Node?
>>> >> > If preseem currently offers such stats at a subscriber level we can do the same, just wanna be sure I'm not overtaxing CPU.
>>> >> >
>>> >> > On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>> >> >>
>>> >> >> Some of the context of my request for libreqos on vs off is from here,
>>> >> >> presentation in the wee hours, wed morning.
>>> >> >>
>>> >> >> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>>> >> >>
>>> >> >> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
>>> >> >> <libreqos@lists.bufferbloat.net> wrote:
>>> >> >> >
>>> >> >> > we also have a bunch of 25/5.  It would be nice to get some more detailed numbers from ookla tests.
>>> >> >>
>>> >> >> +1! We've argued methods with many a provider, and ookla and samknows
>>> >> >> are new to this game.
>>> >> >>
>>> >> >> As one example I don't know if they use decimal or binary mbits!
>>> >> >>
>>> >> >> ...
>>> >> >>
>>> >> >> ack-filtering in the libreqos case can and will drop acks in both
>>> >> >> directions, due to the structure of how htb
>>> >> >> works in this scenario, at any bandwidth ratio. I *think* it does very
>>> >> >> little harm but I would use the ack-filter
>>> >> >> not the aggressive ack filter out of caution.
>>> >> >>
>>> >> >> Secondly if you are gathering drop statistics from the higher level
>>> >> >> htb categories, total drops will go WAY up,
>>> >> >> and it is best to use the cake drop, ack_drop outputs statistics directly.
>>> >> >>
>>> >> >> ack_drops have nothing to do with congestion control and should be
>>> >> >> ignored in most cases.
>>> >> >>
>>> >> >> Also I keep hoping more will pull out ecn marks separately. Public use
>>> >> >> is way up by e2e transports, and I worry that a high ecn ratio is a
>>> >> >> e2e network error, which might show up as a big backlog.
>>> >> >>
>>> >> >> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>>> >> >> which is a good idea, however perhaps drops, marks and drops+marks
>>> >> >> would be better.
>>> >> >>
>>> >> >> An inverse log scale is needed to plot marks, drops, and ack_drops.
>>> >> >>
>>> >> >> I'll always be of the opinion that it's best to run cake on egress at
>>> >> >> the cpe, first, doing all this work before it hits the wire.
>>> >> >> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>>> >> >>
>>> >> >> >We also have a lot of Eero units out there, so watching for the eero speed test would also be great.
>>> >> >>
>>> >> >> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
>>> >> >> buggy fq_codel offload:
>>> >> >>
>>> >> >> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
>>> >> >> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>> >> >> >>
>>> >> >> >> > sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>>> >> >> >>
>>> >> >> >> That's a good idea (I think there's a service somewhere for finding speedtest
>>> >> >> >> servers?). We're discussing testing the ack-filter feature right now, and something
>>> >> >> >> similar would work well for that. (We have a bunch of 25/5 and similarly asymmetric
>>> >> >> >> connections, in theory it'll help...)
>>> >> >> >>
>>> >> >> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>> >> >> >>>
>>> >> >> >>> sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>>> >> >> >>>
>>> >> >> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>>> >> >> >>>>
>>> >> >> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>>> >> >> >>>> >
>>> >> >> >>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I don't think it's indicative over what's actually happening.
>>> >> >> >>>>
>>> >> >> >>>> A simultaneous packet capture and wireshark plot of the RTTs would
>>> >> >> >>>> also be helpful.
>>> >> >> >>>>
>>> >> >> >>>> >
>>> >> >> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>> >> >> >>>> >>
>>> >> >> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>>> >> >> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
>>> >> >> >>>> >> >
>>> >> >> >>>> >> > I can run an Ookla test in the middle of the night with LibreQoS off for a bit.
>>> >> >> >>>> >>
>>> >> >> >>>> >> Groovy.
>>> >> >> >>>> >>
>>> >> >> >>>> >> > From what I recall - without Libre we see download bloat of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each direction for most clients.
>>> >> >> >>>> >>
>>> >> >> >>>> >> The numbers report by speedtest bounce around a lot, and they tend to
>>> >> >> >>>> >> pick a lower number as a final result than I'd
>>> >> >> >>>> >> like. If there's a way to capture a movie of it...
>>> >> >> >>>> >> >
>>> >> >> >>>> >> > And I like the "monitor only" mode idea.
>>> >> >> >>>> >> > Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
>>> >> >> >>>> >> > Technically even HTB could reduce latency by itself on overloaded APs , so it wouldn't be true passive monitoring, but it would allow us to compare the before/after of CAKE while still having qdiscs we can point cpumap-pping to.
>>> >> >> >>>> >> >
>>> >> >> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this passive monitoring period?
>>> >> >> >>>> >> >
>>> >> >> >>>> >> >
>>> >> >> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>> >> >> >>>> >> >>
>>> >> >> >>>> >> >> Are you looking for a shaped result? That'll show "whatever bandwidth
>>> >> >> >>>> >> >> is left on the sector" without Libre, and around the customer's paid-for
>>> >> >> >>>> >> >> target with Libre (hopefully!).
>>> >> >> >>>> >> >>
>>> >> >> >>>> >> >> One thing Preseem does well is that their onboarding process includes
>>> >> >> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering data before you
>>> >> >> >>>> >> >> pull the switch (although most ISPs I've talked to just pull the switch...)
>>> >> >> >>>> >> >> It's quite enlightening to see a before/after, even with only FQ_CODEL
>>> >> >> >>>> >> >> as the shaper.
>>> >> >> >>>> >> >>
>>> >> >> >>>> >> >> We could probably do something similar with a "monitor only" mode that
>>> >> >> >>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
>>> >> >> >>>> >> >> FreeBSD?).
>>> >> >> >>>> >> >>
>>> >> >> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>> >> >> >>>> >> >>>
>>> >> >> >>>> >> >>> I am curious if anyone here has speedtest results with libreqos on and
>>> >> >> >>>> >> >>> off, and can send a screen shot?
>>> >> >> >>>> >> >>>
>>> >> >> >>>> >> >>> --
>>> >> >> >>>> >> >>> This song goes out to all the folk that thought Stadia would work:
>>> >> >> >>>> >> >>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> >> >> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
>>> >> >> >>>> >> >>> _______________________________________________
>>> >> >> >>>> >> >>> LibreQoS mailing list
>>> >> >> >>>> >> >>> LibreQoS@lists.bufferbloat.net
>>> >> >> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >> >>>> >> >>
>>> >> >> >>>> >> >> _______________________________________________
>>> >> >> >>>> >> >> LibreQoS mailing list
>>> >> >> >>>> >> >> LibreQoS@lists.bufferbloat.net
>>> >> >> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >> >>>> >> >
>>> >> >> >>>> >> >
>>> >> >> >>>> >> >
>>> >> >> >>>> >> > --
>>> >> >> >>>> >> > Robert Chacón
>>> >> >> >>>> >> > CEO | JackRabbit Wireless LLC
>>> >> >> >>>> >> > _______________________________________________
>>> >> >> >>>> >> > LibreQoS mailing list
>>> >> >> >>>> >> > LibreQoS@lists.bufferbloat.net
>>> >> >> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >> >>>> >>
>>> >> >> >>>> >>
>>> >> >> >>>> >>
>>> >> >> >>>> >> --
>>> >> >> >>>> >> This song goes out to all the folk that thought Stadia would work:
>>> >> >> >>>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> >> >> >>>> >> Dave Täht CEO, TekLibre, LLC
>>> >> >> >>>> >> _______________________________________________
>>> >> >> >>>> >> LibreQoS mailing list
>>> >> >> >>>> >> LibreQoS@lists.bufferbloat.net
>>> >> >> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >> >>>>
>>> >> >> >>>>
>>> >> >> >>>>
>>> >> >> >>>> --
>>> >> >> >>>> This song goes out to all the folk that thought Stadia would work:
>>> >> >> >>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> >> >> >>>> Dave Täht CEO, TekLibre, LLC
>>> >> >> >>>
>>> >> >> >>> _______________________________________________
>>> >> >> >>> LibreQoS mailing list
>>> >> >> >>> LibreQoS@lists.bufferbloat.net
>>> >> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >> >>
>>> >> >> >> _______________________________________________
>>> >> >> >> LibreQoS mailing list
>>> >> >> >> LibreQoS@lists.bufferbloat.net
>>> >> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >> >
>>> >> >> > _______________________________________________
>>> >> >> > LibreQoS mailing list
>>> >> >> > LibreQoS@lists.bufferbloat.net
>>> >> >> > https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> This song goes out to all the folk that thought Stadia would work:
>>> >> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> >> >> Dave Täht CEO, TekLibre, LLC
>>> >> >> _______________________________________________
>>> >> >> LibreQoS mailing list
>>> >> >> LibreQoS@lists.bufferbloat.net
>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Robert Chacón
>>> >> > CEO | JackRabbit Wireless LLC
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> This song goes out to all the folk that thought Stadia would work:
>>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> >> Dave Täht CEO, TekLibre, LLC
>>>
>>>
>>>
>>> --
>>> This song goes out to all the folk that thought Stadia would work:
>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>>> Dave Täht CEO, TekLibre, LLC
>>
>> _______________________________________________
>> LibreQoS mailing list
>> LibreQoS@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/libreqos
>
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos



-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

Although I'm enjoying a good wide-ranging debate... is it possible one
or more of you could
take a speedtest.net test, click on results so you see the
"responsiveness" data, and post a screen shot?

Perlease??

Re: [LibreQoS] ookla speedtest results?

[Robert Chacón]

[-- Attachment #1.1: Type: text/plain, Size: 956 bytes --]

Here's a customer on a 200/30 Mbps plan via Fixed Wireless
<https://www.speedtest.net/result/a/8780920652>.
LibreQoS using cake diffserv4.
LTU-LR is the client radio. Home WiFi is 802.11AC which is where most of
the bloat comes from.
[image: bloat_ookla.png]

When he tests by Ethernet bloat is 8ms instead of 40ms
<https://www.speedtest.net/result/13846818812>.


On Mon, Oct 24, 2022 at 12:28 PM Dave Taht via LibreQoS <
libreqos@lists.bufferbloat.net> wrote:

> Although I'm enjoying a good wide-ranging debate... is it possible one
> or more of you could
> take a speedtest.net test, click on results so you see the
> "responsiveness" data, and post a screen shot?
>
> Perlease??
> _______________________________________________
> LibreQoS mailing list
> LibreQoS@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>


-- 
Robert Chacón
CEO | JackRabbit Wireless LLC <http://jackrabbitwireless.com>

[-- Attachment #1.2: Type: text/html, Size: 1779 bytes --]

[-- Attachment #2: bloat_ookla.png --]
[-- Type: image/png, Size: 197455 bytes --]

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

On Mon, Oct 24, 2022 at 1:17 PM Robert Chacón
<robert.chacon@jackrabbitwireless.com> wrote:
>
> Here's a customer on a 200/30 Mbps plan via Fixed Wireless.
> LibreQoS using cake diffserv4.
> LTU-LR is the client radio. Home WiFi is 802.11AC which is where most of the bloat comes from.

Yep! I'd like to collect a list of APs that are decent here, out of
the box. 40ms does not seem "bad".

It would be pretty cool to be able to A/B test one of these:
https://openwrt.org/toh/tp-link/eap615-wall

with stock vs openwrt vs my current openwrt branch, on a test like
this. I don't know enough about the
structure of speedtest and wifi is naturally jittery, but I was down
to 8ms induced latency on a variety of tests
on the mt76 chipset(s) before I put it down. (more important for wifi,
is multi-station loading)

Anyone fiddling with openwisp.org?

> When he tests by Ethernet bloat is 8ms instead of 40ms.

thanks very much! That's gorgeous!

https://www.speedtest.net/result/13846818812

I keep hoping speedtest are internally dissecting data from certain AS
numbers and going "woah!" WTF are they doing?





>
> On Mon, Oct 24, 2022 at 12:28 PM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>>
>> Although I'm enjoying a good wide-ranging debate... is it possible one
>> or more of you could
>> take a speedtest.net test, click on results so you see the
>> "responsiveness" data, and post a screen shot?
>>
>> Perlease??
>> _______________________________________________
>> LibreQoS mailing list
>> LibreQoS@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/libreqos
>
>
>
> --
> Robert Chacón
> CEO | JackRabbit Wireless LLC



-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC

Re: [LibreQoS] ookla speedtest results?

[Mark Steckel]

I'm happy to send one of the TP-Link in-wall units to Robert if he wants to play with it.

Note - we are looking to deploy these in a 200 unit apartment building using rgnets.com to manage the networking, user accounts and billing via PPSK. (rgnets is pretty slick stuff!)

I would be even happier if OpenWRT supported PPSK but I'm fairly certain it doesn't (certainly not to the point to be useful for rgnets.com which is by need/use case).


 ---- On Mon, 24 Oct 2022 16:31:32 -0400  Dave Taht via LibreQoS  wrote --- 
 > On Mon, Oct 24, 2022 at 1:17 PM Robert Chacón
 > robert.chacon@jackrabbitwireless.com> wrote:
 > >
 > > Here's a customer on a 200/30 Mbps plan via Fixed Wireless.
 > > LibreQoS using cake diffserv4.
 > > LTU-LR is the client radio. Home WiFi is 802.11AC which is where most of the bloat comes from.
 > 
 > Yep! I'd like to collect a list of APs that are decent here, out of
 > the box. 40ms does not seem "bad".
 > 
 > It would be pretty cool to be able to A/B test one of these:
 > https://openwrt.org/toh/tp-link/eap615-wall
 > 
 > with stock vs openwrt vs my current openwrt branch, on a test like
 > this. I don't know enough about the
 > structure of speedtest and wifi is naturally jittery, but I was down
 > to 8ms induced latency on a variety of tests
 > on the mt76 chipset(s) before I put it down. (more important for wifi,
 > is multi-station loading)
 > 
 > Anyone fiddling with openwisp.org?
 > 
 > > When he tests by Ethernet bloat is 8ms instead of 40ms.
 > 
 > thanks very much! That's gorgeous!
 > 
 > https://www.speedtest.net/result/13846818812
 > 
 > I keep hoping speedtest are internally dissecting data from certain AS
 > numbers and going "woah!" WTF are they doing?
 > 
 > 
 > 
 > 
 > 
 > >
 > > On Mon, Oct 24, 2022 at 12:28 PM Dave Taht via LibreQoS libreqos@lists.bufferbloat.net> wrote:
 > >>
 > >> Although I'm enjoying a good wide-ranging debate... is it possible one
 > >> or more of you could
 > >> take a speedtest.net test, click on results so you see the
 > >> "responsiveness" data, and post a screen shot?
 > >>
 > >> Perlease??
 > >> _______________________________________________
 > >> LibreQoS mailing list
 > >> LibreQoS@lists.bufferbloat.net
 > >> https://lists.bufferbloat.net/listinfo/libreqos
 > >
 > >
 > >
 > > --
 > > Robert Chacón
 > > CEO | JackRabbit Wireless LLC
 > 
 > 
 > 
 > -- 
 > This song goes out to all the folk that thought Stadia would work:
 > https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
 > Dave Täht CEO, TekLibre, LLC
 > _______________________________________________
 > LibreQoS mailing list
 > LibreQoS@lists.bufferbloat.net
 > https://lists.bufferbloat.net/listinfo/libreqos
 > 

Re: [LibreQoS] ookla speedtest results?

[Dave Taht]

On Mon, Oct 24, 2022 at 11:03 AM dan <dandenson@gmail.com> wrote:
>
> We're pretty far past that hardware.  Minimum plans at most sites are now 100x20 and we're moving rapidly to 200x200-1Gx1G plans.  The implication being that we need 4x4 AX wifi radios and that's real tough in an openwrt compatible hardware right now, not a ton of options.

At a gbit, most router designs fall over in the first place, so I have
leaned away from the all-in-one approach to date, in favor of a base
unit + wifi satellites. Now things are getting better since this had
been written:

https://forum.openwrt.org/t/so-you-have-500mbps-1gbps-fiber-and-need-a-router-read-this-first/90305

But it's still quite hard to push a gbit, do firewalling, etc, etc.

>
> Flat out though, wisps in general simply do not have the option of controlling every client side router or even sending out new/compatible routers for each customer to pull this off.  It's a non-starter for us.    It may be better but it's just not an option that can be pursued.  ingress and egress edge shaping is what we need.

I agree that it is impossible to control every client side router, and
that will forevermore retain the demand for middlebox shaping.

I was trying to say, it would be better if as many routers as possible
- especially the wifi ones - also managed bufferbloat better,
especially at range. Most of the wifi routers I've tested hit a
performance cliff at ever decreasing ranges, it seems, mesh wifi (even
eeros) have really terrible responsiveness, and so on. Stuff in the 6
band is really flaky still.

One of the opportunities in herbert's new cpumap-qoe code is to
actually try to manage rtts better on the prem wifi, from the center.



> On Mon, Oct 24, 2022 at 11:57 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> On Mon, Oct 24, 2022 at 10:46 AM dan <dandenson@gmail.com> wrote:
>> >
>> > Some sort of % drop is better than count from an operator's perspective.  I'm actually happy with 'scaling issues' of % on small vs large plan because those small plans are the one's we probably want to push for upgrades if they're seeing a lot of drops.
>> >
>> > That said, a 'score' method works really well from a high level.  And that can be proportionally adjusted so a slightly higher % in a small plan gives the same score as a larger plan would with fewer drops.
>> >
>> >
>> > Also, on the SNMP statement above, we had the same issue with preseem being hyper aggressive to pull stats.  Some mikrotik versions have some snmp flaws and preseem can peg the CPU.  Hasn't happened in a while but that was an issue.
>>
>> I gave up on mikrotik long before they started working on routeros7,
>> and just reflashed everything to openwrt.
>>
>> I have a nice little custom build for it assembled for the ubnt agws,
>> and the mikrotik hap ac lite. Anyone using those?
>>
>> I would hope they don't have the snmp problem you describe, and if
>> they did I could get it fixed in days.
>>
>> Vastly improved the wifi in particular, see before/after here:
>> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/
>>
>> (After about 40Mbits, the bloat starts moving to the wifi)
>>
>> Most of my work in the past 9 months has been on improving the
>> mediatek mt76 derived products. I left off here:
>> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/908
>>
>> A lot of this stuff is shipping commercially in the openwifi project,
>> if you haven't been paying attention to that.
>>
>> > On Mon, Oct 24, 2022 at 11:42 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >>
>> >> On Mon, Oct 24, 2022 at 10:35 AM Robert Chacón
>> >> <robert.chacon@jackrabbitwireless.com> wrote:
>> >> >
>> >> > > Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>> >> > which is a good idea, however perhaps drops, marks and drops+marks
>> >> > would be better.
>> >> >
>> >> > Yup, in InfluxDB for the stats by Tin (diffserv4) we display true drops as (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by sub quite yet.
>> >> > And ok, drops, marks and drops+marks can be shown instead - by subscriber or by Node (AP/Site).
>> >> > Dave - is it ok if we graph this data as a percentage of packets sent? So for drops, drop / sent_packets ? I just imagine that  would make comparing different subscribers' connections easier.
>> >>
>> >> Drops and marks are proportional to the plan. You get a heck of a lot
>> >> more drops at 5mbits than 50, and the relationship
>> >> is not linear, closer to quadratic.
>> >>
>> >> So comparing customers at their given tiers makes more sense than
>> >> across the board.
>> >>
>> >> As for a percentage, you should see a clear difference in percentages
>> >> across tiers.
>> >>
>> >> I know I'm not quite answering your question well, I'm a big believer
>> >> in just getting lots of data, graphing it every which way,
>> >> and seeing what's useful.
>> >>
>> >> > Question for all - is having drops / marks/ drops+marks by subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we just show those stats by AP/Node?
>> >> > If preseem currently offers such stats at a subscriber level we can do the same, just wanna be sure I'm not overtaxing CPU.
>> >> >
>> >> > On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >>
>> >> >> Some of the context of my request for libreqos on vs off is from here,
>> >> >> presentation in the wee hours, wed morning.
>> >> >>
>> >> >> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>> >> >>
>> >> >> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
>> >> >> <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >
>> >> >> > we also have a bunch of 25/5.  It would be nice to get some more detailed numbers from ookla tests.
>> >> >>
>> >> >> +1! We've argued methods with many a provider, and ookla and samknows
>> >> >> are new to this game.
>> >> >>
>> >> >> As one example I don't know if they use decimal or binary mbits!
>> >> >>
>> >> >> ...
>> >> >>
>> >> >> ack-filtering in the libreqos case can and will drop acks in both
>> >> >> directions, due to the structure of how htb
>> >> >> works in this scenario, at any bandwidth ratio. I *think* it does very
>> >> >> little harm but I would use the ack-filter
>> >> >> not the aggressive ack filter out of caution.
>> >> >>
>> >> >> Secondly if you are gathering drop statistics from the higher level
>> >> >> htb categories, total drops will go WAY up,
>> >> >> and it is best to use the cake drop, ack_drop outputs statistics directly.
>> >> >>
>> >> >> ack_drops have nothing to do with congestion control and should be
>> >> >> ignored in most cases.
>> >> >>
>> >> >> Also I keep hoping more will pull out ecn marks separately. Public use
>> >> >> is way up by e2e transports, and I worry that a high ecn ratio is a
>> >> >> e2e network error, which might show up as a big backlog.
>> >> >>
>> >> >> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>> >> >> which is a good idea, however perhaps drops, marks and drops+marks
>> >> >> would be better.
>> >> >>
>> >> >> An inverse log scale is needed to plot marks, drops, and ack_drops.
>> >> >>
>> >> >> I'll always be of the opinion that it's best to run cake on egress at
>> >> >> the cpe, first, doing all this work before it hits the wire.
>> >> >> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>> >> >>
>> >> >> >We also have a lot of Eero units out there, so watching for the eero speed test would also be great.
>> >> >>
>> >> >> eero 5s have cake (and fq_codel on the wifi), eero 6s have a somewhat
>> >> >> buggy fq_codel offload:
>> >> >>
>> >> >> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
>> >> >> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>
>> >> >> >> > sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>> >> >> >>
>> >> >> >> That's a good idea (I think there's a service somewhere for finding speedtest
>> >> >> >> servers?). We're discussing testing the ack-filter feature right now, and something
>> >> >> >> similar would work well for that. (We have a bunch of 25/5 and similarly asymmetric
>> >> >> >> connections, in theory it'll help...)
>> >> >> >>
>> >> >> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>
>> >> >> >>> sniff DNS for those speed test lookups and trigger a packet capture for x duration?
>> >> >> >>>
>> >> >> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht@gmail.com> wrote:
>> >> >> >>>>
>> >> >> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson@gmail.com> wrote:
>> >> >> >>>> >
>> >> >> >>>> > Dave, I think that 'bouncing' is more CPU time in the browser.  I don't think it's indicative over what's actually happening.
>> >> >> >>>>
>> >> >> >>>> A simultaneous packet capture and wireshark plot of the RTTs would
>> >> >> >>>> also be helpful.
>> >> >> >>>>
>> >> >> >>>> >
>> >> >> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>> >>
>> >> >> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>> >> >> >>>> >> <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>> >> >
>> >> >> >>>> >> > I can run an Ookla test in the middle of the night with LibreQoS off for a bit.
>> >> >> >>>> >>
>> >> >> >>>> >> Groovy.
>> >> >> >>>> >>
>> >> >> >>>> >> > From what I recall - without Libre we see download bloat of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat each direction for most clients.
>> >> >> >>>> >>
>> >> >> >>>> >> The numbers report by speedtest bounce around a lot, and they tend to
>> >> >> >>>> >> pick a lower number as a final result than I'd
>> >> >> >>>> >> like. If there's a way to capture a movie of it...
>> >> >> >>>> >> >
>> >> >> >>>> >> > And I like the "monitor only" mode idea.
>> >> >> >>>> >> > Perhaps we could create the HTB tree, but just not attach the CAKE qdisc?
>> >> >> >>>> >> > Technically even HTB could reduce latency by itself on overloaded APs , so it wouldn't be true passive monitoring, but it would allow us to compare the before/after of CAKE while still having qdiscs we can point cpumap-pping to.
>> >> >> >>>> >> >
>> >> >> >>>> >> > Alternatively, maybe we could use eBPF PPing just for this passive monitoring period?
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> Are you looking for a shaped result? That'll show "whatever bandwidth
>> >> >> >>>> >> >> is left on the sector" without Libre, and around the customer's paid-for
>> >> >> >>>> >> >> target with Libre (hopefully!).
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> One thing Preseem does well is that their onboarding process includes
>> >> >> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering data before you
>> >> >> >>>> >> >> pull the switch (although most ISPs I've talked to just pull the switch...)
>> >> >> >>>> >> >> It's quite enlightening to see a before/after, even with only FQ_CODEL
>> >> >> >>>> >> >> as the shaper.
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> We could probably do something similar with a "monitor only" mode that
>> >> >> >>>> >> >> doesn't enable any queues (is there a TC equivalent to "dummy" on
>> >> >> >>>> >> >> FreeBSD?).
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> I am curious if anyone here has speedtest results with libreqos on and
>> >> >> >>>> >> >>> off, and can send a screen shot?
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> --
>> >> >> >>>> >> >>> This song goes out to all the folk that thought Stadia would work:
>> >> >> >>>> >> >>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
>> >> >> >>>> >> >>> _______________________________________________
>> >> >> >>>> >> >>> LibreQoS mailing list
>> >> >> >>>> >> >>> LibreQoS@lists.bufferbloat.net
>> >> >> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> _______________________________________________
>> >> >> >>>> >> >> LibreQoS mailing list
>> >> >> >>>> >> >> LibreQoS@lists.bufferbloat.net
>> >> >> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> > --
>> >> >> >>>> >> > Robert Chacón
>> >> >> >>>> >> > CEO | JackRabbit Wireless LLC
>> >> >> >>>> >> > _______________________________________________
>> >> >> >>>> >> > LibreQoS mailing list
>> >> >> >>>> >> > LibreQoS@lists.bufferbloat.net
>> >> >> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >> --
>> >> >> >>>> >> This song goes out to all the folk that thought Stadia would work:
>> >> >> >>>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> >>>> >> Dave Täht CEO, TekLibre, LLC
>> >> >> >>>> >> _______________________________________________
>> >> >> >>>> >> LibreQoS mailing list
>> >> >> >>>> >> LibreQoS@lists.bufferbloat.net
>> >> >> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> --
>> >> >> >>>> This song goes out to all the folk that thought Stadia would work:
>> >> >> >>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> >>>> Dave Täht CEO, TekLibre, LLC
>> >> >> >>>
>> >> >> >>> _______________________________________________
>> >> >> >>> LibreQoS mailing list
>> >> >> >>> LibreQoS@lists.bufferbloat.net
>> >> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>
>> >> >> >> _______________________________________________
>> >> >> >> LibreQoS mailing list
>> >> >> >> LibreQoS@lists.bufferbloat.net
>> >> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > LibreQoS mailing list
>> >> >> > LibreQoS@lists.bufferbloat.net
>> >> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> This song goes out to all the folk that thought Stadia would work:
>> >> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> Dave Täht CEO, TekLibre, LLC
>> >> >> _______________________________________________
>> >> >> LibreQoS mailing list
>> >> >> LibreQoS@lists.bufferbloat.net
>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Robert Chacón
>> >> > CEO | JackRabbit Wireless LLC
>> >>
>> >>
>> >>
>> >> --
>> >> This song goes out to all the folk that thought Stadia would work:
>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> Dave Täht CEO, TekLibre, LLC
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC



-- 
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC