[LibreQoS] ookla speedtest results?

[Herbert Wolverson]

I think we have a handful of HAP lite's left? They aren't great, but they
are cheap - so
we have a small apartment complex using them as client-side devices. The
complex is a half-way house for paroled offenders, and they destroy a *lot*
of hardware. :-|

The biggest issue we've had (other than they aren't very resilient to
someone
who needs anger-management classes) is the antenna. There's just not a lot
of gain there.

On Mon, Oct 24, 2022 at 1:03 PM dan via LibreQoS <
libreqos at lists.bufferbloat.net> wrote:

> We're pretty far past that hardware.  Minimum plans at most sites are now
> 100x20 and we're moving rapidly to 200x200-1Gx1G plans.  The implication
> being that we need 4x4 AX wifi radios and that's real tough in an openwrt
> compatible hardware right now, not a ton of options.
>
> Flat out though, wisps in general simply do not have the option of
> controlling every client side router or even sending out new/compatible
> routers for each customer to pull this off.  It's a non-starter for us.
> It may be better but it's just not an option that can be pursued.
> ingress and egress edge shaping is what we need.
>
> On Mon, Oct 24, 2022 at 11:57 AM Dave Taht <dave.taht at gmail.com> wrote:
>
>> On Mon, Oct 24, 2022 at 10:46 AM dan <dandenson at gmail.com> wrote:
>> >
>> > Some sort of % drop is better than count from an operator's
>> perspective.  I'm actually happy with 'scaling issues' of % on small vs
>> large plan because those small plans are the one's we probably want to push
>> for upgrades if they're seeing a lot of drops.
>> >
>> > That said, a 'score' method works really well from a high level.  And
>> that can be proportionally adjusted so a slightly higher % in a small plan
>> gives the same score as a larger plan would with fewer drops.
>> >
>> >
>> > Also, on the SNMP statement above, we had the same issue with preseem
>> being hyper aggressive to pull stats.  Some mikrotik versions have some
>> snmp flaws and preseem can peg the CPU.  Hasn't happened in a while but
>> that was an issue.
>>
>> I gave up on mikrotik long before they started working on routeros7,
>> and just reflashed everything to openwrt.
>>
>> I have a nice little custom build for it assembled for the ubnt agws,
>> and the mikrotik hap ac lite. Anyone using those?
>>
>> I would hope they don't have the snmp problem you describe, and if
>> they did I could get it fixed in days.
>>
>> Vastly improved the wifi in particular, see before/after here:
>> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/
>>
>> (After about 40Mbits, the bloat starts moving to the wifi)
>>
>> Most of my work in the past 9 months has been on improving the
>> mediatek mt76 derived products. I left off here:
>> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/908
>>
>> A lot of this stuff is shipping commercially in the openwifi project,
>> if you haven't been paying attention to that.
>>
>> > On Mon, Oct 24, 2022 at 11:42 AM Dave Taht <dave.taht at gmail.com> wrote:
>> >>
>> >> On Mon, Oct 24, 2022 at 10:35 AM Robert Chacón
>> >> <robert.chacon at jackrabbitwireless.com> wrote:
>> >> >
>> >> > > Presently (in v1.3.), robert (in grafina?) is summing drops +
>> marks.
>> >> > which is a good idea, however perhaps drops, marks and drops+marks
>> >> > would be better.
>> >> >
>> >> > Yup, in InfluxDB for the stats by Tin (diffserv4) we display true
>> drops as (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by
>> sub quite yet.
>> >> > And ok, drops, marks and drops+marks can be shown instead - by
>> subscriber or by Node (AP/Site).
>> >> > Dave - is it ok if we graph this data as a percentage of packets
>> sent? So for drops, drop / sent_packets ? I just imagine that  would make
>> comparing different subscribers' connections easier.
>> >>
>> >> Drops and marks are proportional to the plan. You get a heck of a lot
>> >> more drops at 5mbits than 50, and the relationship
>> >> is not linear, closer to quadratic.
>> >>
>> >> So comparing customers at their given tiers makes more sense than
>> >> across the board.
>> >>
>> >> As for a percentage, you should see a clear difference in percentages
>> >> across tiers.
>> >>
>> >> I know I'm not quite answering your question well, I'm a big believer
>> >> in just getting lots of data, graphing it every which way,
>> >> and seeing what's useful.
>> >>
>> >> > Question for all - is having drops / marks/ drops+marks by
>> subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we
>> just show those stats by AP/Node?
>> >> > If preseem currently offers such stats at a subscriber level we can
>> do the same, just wanna be sure I'm not overtaxing CPU.
>> >> >
>> >> > On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS <
>> libreqos at lists.bufferbloat.net> wrote:
>> >> >>
>> >> >> Some of the context of my request for libreqos on vs off is from
>> here,
>> >> >> presentation in the wee hours, wed morning.
>> >> >>
>> >> >>
>> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/
>> >> >>
>> >> >> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS
>> >> >> <libreqos at lists.bufferbloat.net> wrote:
>> >> >> >
>> >> >> > we also have a bunch of 25/5.  It would be nice to get some more
>> detailed numbers from ookla tests.
>> >> >>
>> >> >> +1! We've argued methods with many a provider, and ookla and
>> samknows
>> >> >> are new to this game.
>> >> >>
>> >> >> As one example I don't know if they use decimal or binary mbits!
>> >> >>
>> >> >> ...
>> >> >>
>> >> >> ack-filtering in the libreqos case can and will drop acks in both
>> >> >> directions, due to the structure of how htb
>> >> >> works in this scenario, at any bandwidth ratio. I *think* it does
>> very
>> >> >> little harm but I would use the ack-filter
>> >> >> not the aggressive ack filter out of caution.
>> >> >>
>> >> >> Secondly if you are gathering drop statistics from the higher level
>> >> >> htb categories, total drops will go WAY up,
>> >> >> and it is best to use the cake drop, ack_drop outputs statistics
>> directly.
>> >> >>
>> >> >> ack_drops have nothing to do with congestion control and should be
>> >> >> ignored in most cases.
>> >> >>
>> >> >> Also I keep hoping more will pull out ecn marks separately. Public
>> use
>> >> >> is way up by e2e transports, and I worry that a high ecn ratio is a
>> >> >> e2e network error, which might show up as a big backlog.
>> >> >>
>> >> >> Presently (in v1.3.), robert (in grafina?) is summing drops + marks.
>> >> >> which is a good idea, however perhaps drops, marks and drops+marks
>> >> >> would be better.
>> >> >>
>> >> >> An inverse log scale is needed to plot marks, drops, and ack_drops.
>> >> >>
>> >> >> I'll always be of the opinion that it's best to run cake on egress
>> at
>> >> >> the cpe, first, doing all this work before it hits the wire.
>> >> >> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2?
>> >> >>
>> >> >> >We also have a lot of Eero units out there, so watching for the
>> eero speed test would also be great.
>> >> >>
>> >> >> eero 5s have cake (and fq_codel on the wifi), eero 6s have a
>> somewhat
>> >> >> buggy fq_codel offload:
>> >> >>
>> >> >>
>> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/
>> >> >> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS <
>> libreqos at lists.bufferbloat.net> wrote:
>> >> >> >>
>> >> >> >> > sniff DNS for those speed test lookups and trigger a packet
>> capture for x duration?
>> >> >> >>
>> >> >> >> That's a good idea (I think there's a service somewhere for
>> finding speedtest
>> >> >> >> servers?). We're discussing testing the ack-filter feature right
>> now, and something
>> >> >> >> similar would work well for that. (We have a bunch of 25/5 and
>> similarly asymmetric
>> >> >> >> connections, in theory it'll help...)
>> >> >> >>
>> >> >> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS <
>> libreqos at lists.bufferbloat.net> wrote:
>> >> >> >>>
>> >> >> >>> sniff DNS for those speed test lookups and trigger a packet
>> capture for x duration?
>> >> >> >>>
>> >> >> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht <dave.taht at gmail.com>
>> wrote:
>> >> >> >>>>
>> >> >> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan <dandenson at gmail.com>
>> wrote:
>> >> >> >>>> >
>> >> >> >>>> > Dave, I think that 'bouncing' is more CPU time in the
>> browser.  I don't think it's indicative over what's actually happening.
>> >> >> >>>>
>> >> >> >>>> A simultaneous packet capture and wireshark plot of the RTTs
>> would
>> >> >> >>>> also be helpful.
>> >> >> >>>>
>> >> >> >>>> >
>> >> >> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS <
>> libreqos at lists.bufferbloat.net> wrote:
>> >> >> >>>> >>
>> >> >> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS
>> >> >> >>>> >> <libreqos at lists.bufferbloat.net> wrote:
>> >> >> >>>> >> >
>> >> >> >>>> >> > I can run an Ookla test in the middle of the night with
>> LibreQoS off for a bit.
>> >> >> >>>> >>
>> >> >> >>>> >> Groovy.
>> >> >> >>>> >>
>> >> >> >>>> >> > From what I recall - without Libre we see download bloat
>> of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat
>> each direction for most clients.
>> >> >> >>>> >>
>> >> >> >>>> >> The numbers report by speedtest bounce around a lot, and
>> they tend to
>> >> >> >>>> >> pick a lower number as a final result than I'd
>> >> >> >>>> >> like. If there's a way to capture a movie of it...
>> >> >> >>>> >> >
>> >> >> >>>> >> > And I like the "monitor only" mode idea.
>> >> >> >>>> >> > Perhaps we could create the HTB tree, but just not attach
>> the CAKE qdisc?
>> >> >> >>>> >> > Technically even HTB could reduce latency by itself on
>> overloaded APs , so it wouldn't be true passive monitoring, but it would
>> allow us to compare the before/after of CAKE while still having qdiscs we
>> can point cpumap-pping to.
>> >> >> >>>> >> >
>> >> >> >>>> >> > Alternatively, maybe we could use eBPF PPing just for
>> this passive monitoring period?
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via
>> LibreQoS <libreqos at lists.bufferbloat.net> wrote:
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> Are you looking for a shaped result? That'll show
>> "whatever bandwidth
>> >> >> >>>> >> >> is left on the sector" without Libre, and around the
>> customer's paid-for
>> >> >> >>>> >> >> target with Libre (hopefully!).
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> One thing Preseem does well is that their onboarding
>> process includes
>> >> >> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering
>> data before you
>> >> >> >>>> >> >> pull the switch (although most ISPs I've talked to just
>> pull the switch...)
>> >> >> >>>> >> >> It's quite enlightening to see a before/after, even with
>> only FQ_CODEL
>> >> >> >>>> >> >> as the shaper.
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> We could probably do something similar with a "monitor
>> only" mode that
>> >> >> >>>> >> >> doesn't enable any queues (is there a TC equivalent to
>> "dummy" on
>> >> >> >>>> >> >> FreeBSD?).
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS <
>> libreqos at lists.bufferbloat.net> wrote:
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> I am curious if anyone here has speedtest results with
>> libreqos on and
>> >> >> >>>> >> >>> off, and can send a screen shot?
>> >> >> >>>> >> >>>
>> >> >> >>>> >> >>> --
>> >> >> >>>> >> >>> This song goes out to all the folk that thought Stadia
>> would work:
>> >> >> >>>> >> >>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> >>>> >> >>> Dave Täht CEO, TekLibre, LLC
>> >> >> >>>> >> >>> _______________________________________________
>> >> >> >>>> >> >>> LibreQoS mailing list
>> >> >> >>>> >> >>> LibreQoS at lists.bufferbloat.net
>> >> >> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> _______________________________________________
>> >> >> >>>> >> >> LibreQoS mailing list
>> >> >> >>>> >> >> LibreQoS at lists.bufferbloat.net
>> >> >> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> > --
>> >> >> >>>> >> > Robert Chacón
>> >> >> >>>> >> > CEO | JackRabbit Wireless LLC
>> >> >> >>>> >> > _______________________________________________
>> >> >> >>>> >> > LibreQoS mailing list
>> >> >> >>>> >> > LibreQoS at lists.bufferbloat.net
>> >> >> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >> --
>> >> >> >>>> >> This song goes out to all the folk that thought Stadia
>> would work:
>> >> >> >>>> >>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> >>>> >> Dave Täht CEO, TekLibre, LLC
>> >> >> >>>> >> _______________________________________________
>> >> >> >>>> >> LibreQoS mailing list
>> >> >> >>>> >> LibreQoS at lists.bufferbloat.net
>> >> >> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> --
>> >> >> >>>> This song goes out to all the folk that thought Stadia would
>> work:
>> >> >> >>>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> >>>> Dave Täht CEO, TekLibre, LLC
>> >> >> >>>
>> >> >> >>> _______________________________________________
>> >> >> >>> LibreQoS mailing list
>> >> >> >>> LibreQoS at lists.bufferbloat.net
>> >> >> >>> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >>
>> >> >> >> _______________________________________________
>> >> >> >> LibreQoS mailing list
>> >> >> >> LibreQoS at lists.bufferbloat.net
>> >> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > LibreQoS mailing list
>> >> >> > LibreQoS at lists.bufferbloat.net
>> >> >> > https://lists.bufferbloat.net/listinfo/libreqos
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> This song goes out to all the folk that thought Stadia would work:
>> >> >>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> >> Dave Täht CEO, TekLibre, LLC
>> >> >> _______________________________________________
>> >> >> LibreQoS mailing list
>> >> >> LibreQoS at lists.bufferbloat.net
>> >> >> https://lists.bufferbloat.net/listinfo/libreqos
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Robert Chacón
>> >> > CEO | JackRabbit Wireless LLC
>> >>
>> >>
>> >>
>> >> --
>> >> This song goes out to all the folk that thought Stadia would work:
>> >>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> >> Dave Täht CEO, TekLibre, LLC
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>>
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC
>>
> _______________________________________________
> LibreQoS mailing list
> LibreQoS at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/libreqos
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/libreqos/attachments/20221024/91965bc3/attachment-0001.html>