I think we have a handful of HAP lite's left? They aren't great, but they are cheap - so we have a small apartment complex using them as client-side devices. The complex is a half-way house for paroled offenders, and they destroy a *lot* of hardware. :-| The biggest issue we've had (other than they aren't very resilient to someone who needs anger-management classes) is the antenna. There's just not a lot of gain there. On Mon, Oct 24, 2022 at 1:03 PM dan via LibreQoS < libreqos@lists.bufferbloat.net> wrote: > We're pretty far past that hardware. Minimum plans at most sites are now > 100x20 and we're moving rapidly to 200x200-1Gx1G plans. The implication > being that we need 4x4 AX wifi radios and that's real tough in an openwrt > compatible hardware right now, not a ton of options. > > Flat out though, wisps in general simply do not have the option of > controlling every client side router or even sending out new/compatible > routers for each customer to pull this off. It's a non-starter for us. > It may be better but it's just not an option that can be pursued. > ingress and egress edge shaping is what we need. > > On Mon, Oct 24, 2022 at 11:57 AM Dave Taht wrote: > >> On Mon, Oct 24, 2022 at 10:46 AM dan wrote: >> > >> > Some sort of % drop is better than count from an operator's >> perspective. I'm actually happy with 'scaling issues' of % on small vs >> large plan because those small plans are the one's we probably want to push >> for upgrades if they're seeing a lot of drops. >> > >> > That said, a 'score' method works really well from a high level. And >> that can be proportionally adjusted so a slightly higher % in a small plan >> gives the same score as a larger plan would with fewer drops. >> > >> > >> > Also, on the SNMP statement above, we had the same issue with preseem >> being hyper aggressive to pull stats. Some mikrotik versions have some >> snmp flaws and preseem can peg the CPU. Hasn't happened in a while but >> that was an issue. >> >> I gave up on mikrotik long before they started working on routeros7, >> and just reflashed everything to openwrt. >> >> I have a nice little custom build for it assembled for the ubnt agws, >> and the mikrotik hap ac lite. Anyone using those? >> >> I would hope they don't have the snmp problem you describe, and if >> they did I could get it fixed in days. >> >> Vastly improved the wifi in particular, see before/after here: >> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/ >> >> (After about 40Mbits, the bloat starts moving to the wifi) >> >> Most of my work in the past 9 months has been on improving the >> mediatek mt76 derived products. I left off here: >> https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/908 >> >> A lot of this stuff is shipping commercially in the openwifi project, >> if you haven't been paying attention to that. >> >> > On Mon, Oct 24, 2022 at 11:42 AM Dave Taht wrote: >> >> >> >> On Mon, Oct 24, 2022 at 10:35 AM Robert Chacón >> >> wrote: >> >> > >> >> > > Presently (in v1.3.), robert (in grafina?) is summing drops + >> marks. >> >> > which is a good idea, however perhaps drops, marks and drops+marks >> >> > would be better. >> >> > >> >> > Yup, in InfluxDB for the stats by Tin (diffserv4) we display true >> drops as (trueDrops = ecn_mark + drops - ack_drops). But we don't do it by >> sub quite yet. >> >> > And ok, drops, marks and drops+marks can be shown instead - by >> subscriber or by Node (AP/Site). >> >> > Dave - is it ok if we graph this data as a percentage of packets >> sent? So for drops, drop / sent_packets ? I just imagine that would make >> comparing different subscribers' connections easier. >> >> >> >> Drops and marks are proportional to the plan. You get a heck of a lot >> >> more drops at 5mbits than 50, and the relationship >> >> is not linear, closer to quadratic. >> >> >> >> So comparing customers at their given tiers makes more sense than >> >> across the board. >> >> >> >> As for a percentage, you should see a clear difference in percentages >> >> across tiers. >> >> >> >> I know I'm not quite answering your question well, I'm a big believer >> >> in just getting lots of data, graphing it every which way, >> >> and seeing what's useful. >> >> >> >> > Question for all - is having drops / marks/ drops+marks by >> subscriber/circuit worth the extra modest InfluxDB CPU use? Or should we >> just show those stats by AP/Node? >> >> > If preseem currently offers such stats at a subscriber level we can >> do the same, just wanna be sure I'm not overtaxing CPU. >> >> > >> >> > On Mon, Oct 24, 2022 at 11:14 AM Dave Taht via LibreQoS < >> libreqos@lists.bufferbloat.net> wrote: >> >> >> >> >> >> Some of the context of my request for libreqos on vs off is from >> here, >> >> >> presentation in the wee hours, wed morning. >> >> >> >> >> >> >> https://www.linkedin.com/feed/update/urn:li:activity:6989281945868283904/ >> >> >> >> >> >> On Mon, Oct 24, 2022 at 9:48 AM dan via LibreQoS >> >> >> wrote: >> >> >> > >> >> >> > we also have a bunch of 25/5. It would be nice to get some more >> detailed numbers from ookla tests. >> >> >> >> >> >> +1! We've argued methods with many a provider, and ookla and >> samknows >> >> >> are new to this game. >> >> >> >> >> >> As one example I don't know if they use decimal or binary mbits! >> >> >> >> >> >> ... >> >> >> >> >> >> ack-filtering in the libreqos case can and will drop acks in both >> >> >> directions, due to the structure of how htb >> >> >> works in this scenario, at any bandwidth ratio. I *think* it does >> very >> >> >> little harm but I would use the ack-filter >> >> >> not the aggressive ack filter out of caution. >> >> >> >> >> >> Secondly if you are gathering drop statistics from the higher level >> >> >> htb categories, total drops will go WAY up, >> >> >> and it is best to use the cake drop, ack_drop outputs statistics >> directly. >> >> >> >> >> >> ack_drops have nothing to do with congestion control and should be >> >> >> ignored in most cases. >> >> >> >> >> >> Also I keep hoping more will pull out ecn marks separately. Public >> use >> >> >> is way up by e2e transports, and I worry that a high ecn ratio is a >> >> >> e2e network error, which might show up as a big backlog. >> >> >> >> >> >> Presently (in v1.3.), robert (in grafina?) is summing drops + marks. >> >> >> which is a good idea, however perhaps drops, marks and drops+marks >> >> >> would be better. >> >> >> >> >> >> An inverse log scale is needed to plot marks, drops, and ack_drops. >> >> >> >> >> >> I'll always be of the opinion that it's best to run cake on egress >> at >> >> >> the cpe, first, doing all this work before it hits the wire. >> >> >> Are any of you doing that? Openwrt/dd-wrt/firewalla/mikrotik 7.2? >> >> >> >> >> >> >We also have a lot of Eero units out there, so watching for the >> eero speed test would also be great. >> >> >> >> >> >> eero 5s have cake (and fq_codel on the wifi), eero 6s have a >> somewhat >> >> >> buggy fq_codel offload: >> >> >> >> >> >> >> https://www.reddit.com/r/eero/comments/u7xm83/gen_2_sqm_vs_gen_3_sqm_stick_with_gen_2_if_you/ >> >> >> > On Mon, Oct 24, 2022 at 10:31 AM Herbert Wolverson via LibreQoS < >> libreqos@lists.bufferbloat.net> wrote: >> >> >> >> >> >> >> >> > sniff DNS for those speed test lookups and trigger a packet >> capture for x duration? >> >> >> >> >> >> >> >> That's a good idea (I think there's a service somewhere for >> finding speedtest >> >> >> >> servers?). We're discussing testing the ack-filter feature right >> now, and something >> >> >> >> similar would work well for that. (We have a bunch of 25/5 and >> similarly asymmetric >> >> >> >> connections, in theory it'll help...) >> >> >> >> >> >> >> >> On Mon, Oct 24, 2022 at 10:31 AM dan via LibreQoS < >> libreqos@lists.bufferbloat.net> wrote: >> >> >> >>> >> >> >> >>> sniff DNS for those speed test lookups and trigger a packet >> capture for x duration? >> >> >> >>> >> >> >> >>> On Mon, Oct 24, 2022 at 9:27 AM Dave Taht >> wrote: >> >> >> >>>> >> >> >> >>>> On Mon, Oct 24, 2022 at 8:20 AM dan >> wrote: >> >> >> >>>> > >> >> >> >>>> > Dave, I think that 'bouncing' is more CPU time in the >> browser. I don't think it's indicative over what's actually happening. >> >> >> >>>> >> >> >> >>>> A simultaneous packet capture and wireshark plot of the RTTs >> would >> >> >> >>>> also be helpful. >> >> >> >>>> >> >> >> >>>> > >> >> >> >>>> > On Mon, Oct 24, 2022 at 9:09 AM Dave Taht via LibreQoS < >> libreqos@lists.bufferbloat.net> wrote: >> >> >> >>>> >> >> >> >> >>>> >> On Mon, Oct 24, 2022 at 8:02 AM Robert Chacón via LibreQoS >> >> >> >>>> >> wrote: >> >> >> >>>> >> > >> >> >> >>>> >> > I can run an Ookla test in the middle of the night with >> LibreQoS off for a bit. >> >> >> >>>> >> >> >> >> >>>> >> Groovy. >> >> >> >>>> >> >> >> >> >>>> >> > From what I recall - without Libre we see download bloat >> of 300ms or so. With Libre, it's gone. On waveform we see 0ms added bloat >> each direction for most clients. >> >> >> >>>> >> >> >> >> >>>> >> The numbers report by speedtest bounce around a lot, and >> they tend to >> >> >> >>>> >> pick a lower number as a final result than I'd >> >> >> >>>> >> like. If there's a way to capture a movie of it... >> >> >> >>>> >> > >> >> >> >>>> >> > And I like the "monitor only" mode idea. >> >> >> >>>> >> > Perhaps we could create the HTB tree, but just not attach >> the CAKE qdisc? >> >> >> >>>> >> > Technically even HTB could reduce latency by itself on >> overloaded APs , so it wouldn't be true passive monitoring, but it would >> allow us to compare the before/after of CAKE while still having qdiscs we >> can point cpumap-pping to. >> >> >> >>>> >> > >> >> >> >>>> >> > Alternatively, maybe we could use eBPF PPing just for >> this passive monitoring period? >> >> >> >>>> >> > >> >> >> >>>> >> > >> >> >> >>>> >> > On Mon, Oct 24, 2022 at 8:02 AM Herbert Wolverson via >> LibreQoS wrote: >> >> >> >>>> >> >> >> >> >> >>>> >> >> Are you looking for a shaped result? That'll show >> "whatever bandwidth >> >> >> >>>> >> >> is left on the sector" without Libre, and around the >> customer's paid-for >> >> >> >>>> >> >> target with Libre (hopefully!). >> >> >> >>>> >> >> >> >> >> >>>> >> >> One thing Preseem does well is that their onboarding >> process includes >> >> >> >>>> >> >> "leave it in monitoring-only mode" for a week, gathering >> data before you >> >> >> >>>> >> >> pull the switch (although most ISPs I've talked to just >> pull the switch...) >> >> >> >>>> >> >> It's quite enlightening to see a before/after, even with >> only FQ_CODEL >> >> >> >>>> >> >> as the shaper. >> >> >> >>>> >> >> >> >> >> >>>> >> >> We could probably do something similar with a "monitor >> only" mode that >> >> >> >>>> >> >> doesn't enable any queues (is there a TC equivalent to >> "dummy" on >> >> >> >>>> >> >> FreeBSD?). >> >> >> >>>> >> >> >> >> >> >>>> >> >> On Mon, Oct 24, 2022 at 8:52 AM Dave Taht via LibreQoS < >> libreqos@lists.bufferbloat.net> wrote: >> >> >> >>>> >> >>> >> >> >> >>>> >> >>> I am curious if anyone here has speedtest results with >> libreqos on and >> >> >> >>>> >> >>> off, and can send a screen shot? >> >> >> >>>> >> >>> >> >> >> >>>> >> >>> -- >> >> >> >>>> >> >>> This song goes out to all the folk that thought Stadia >> would work: >> >> >> >>>> >> >>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz >> >> >> >>>> >> >>> Dave Täht CEO, TekLibre, LLC >> >> >> >>>> >> >>> _______________________________________________ >> >> >> >>>> >> >>> LibreQoS mailing list >> >> >> >>>> >> >>> LibreQoS@lists.bufferbloat.net >> >> >> >>>> >> >>> https://lists.bufferbloat.net/listinfo/libreqos >> >> >> >>>> >> >> >> >> >> >>>> >> >> _______________________________________________ >> >> >> >>>> >> >> LibreQoS mailing list >> >> >> >>>> >> >> LibreQoS@lists.bufferbloat.net >> >> >> >>>> >> >> https://lists.bufferbloat.net/listinfo/libreqos >> >> >> >>>> >> > >> >> >> >>>> >> > >> >> >> >>>> >> > >> >> >> >>>> >> > -- >> >> >> >>>> >> > Robert Chacón >> >> >> >>>> >> > CEO | JackRabbit Wireless LLC >> >> >> >>>> >> > _______________________________________________ >> >> >> >>>> >> > LibreQoS mailing list >> >> >> >>>> >> > LibreQoS@lists.bufferbloat.net >> >> >> >>>> >> > https://lists.bufferbloat.net/listinfo/libreqos >> >> >> >>>> >> >> >> >> >>>> >> >> >> >> >>>> >> >> >> >> >>>> >> -- >> >> >> >>>> >> This song goes out to all the folk that thought Stadia >> would work: >> >> >> >>>> >> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz >> >> >> >>>> >> Dave Täht CEO, TekLibre, LLC >> >> >> >>>> >> _______________________________________________ >> >> >> >>>> >> LibreQoS mailing list >> >> >> >>>> >> LibreQoS@lists.bufferbloat.net >> >> >> >>>> >> https://lists.bufferbloat.net/listinfo/libreqos >> >> >> >>>> >> >> >> >>>> >> >> >> >>>> >> >> >> >>>> -- >> >> >> >>>> This song goes out to all the folk that thought Stadia would >> work: >> >> >> >>>> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz >> >> >> >>>> Dave Täht CEO, TekLibre, LLC >> >> >> >>> >> >> >> >>> _______________________________________________ >> >> >> >>> LibreQoS mailing list >> >> >> >>> LibreQoS@lists.bufferbloat.net >> >> >> >>> https://lists.bufferbloat.net/listinfo/libreqos >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> >> LibreQoS mailing list >> >> >> >> LibreQoS@lists.bufferbloat.net >> >> >> >> https://lists.bufferbloat.net/listinfo/libreqos >> >> >> > >> >> >> > _______________________________________________ >> >> >> > LibreQoS mailing list >> >> >> > LibreQoS@lists.bufferbloat.net >> >> >> > https://lists.bufferbloat.net/listinfo/libreqos >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> This song goes out to all the folk that thought Stadia would work: >> >> >> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz >> >> >> Dave Täht CEO, TekLibre, LLC >> >> >> _______________________________________________ >> >> >> LibreQoS mailing list >> >> >> LibreQoS@lists.bufferbloat.net >> >> >> https://lists.bufferbloat.net/listinfo/libreqos >> >> > >> >> > >> >> > >> >> > -- >> >> > Robert Chacón >> >> > CEO | JackRabbit Wireless LLC >> >> >> >> >> >> >> >> -- >> >> This song goes out to all the folk that thought Stadia would work: >> >> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz >> >> Dave Täht CEO, TekLibre, LLC >> >> >> >> -- >> This song goes out to all the folk that thought Stadia would work: >> >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz >> Dave Täht CEO, TekLibre, LLC >> > _______________________________________________ > LibreQoS mailing list > LibreQoS@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/libreqos >