[LibreQoS] Fwd: A quick report from the WISPA conference

[Hayden Simon]

This list has been fantastic with respect to understanding some of the development choices made. The concerns being discussed in this thread are a non-issue in NZ (perhaps off the back of laws like yours; and privacy ones from the EU). In any event, if what the consumer wants to do doesn’t work – they simply vote with their wallet – I guess you could say we’re all kept fairly honest. In NZ, Gigabit (fibre) “unlimited data” to the home in any major town is ubiquitous and priced ~ $55/month USD. Rurally, unlimited FWA at (atleast) 50mbit down / 10ish up is available from all the major telco’s at a small premium. Yay for tiny populations?.....

HAYDEN​​ SIMON
UBER GROUP LIMITED
MANAGING DIRECTOR
 ‑
SUPREME OVERLORD
[http://uber.co.nz/wp-content/uploads/signatures/uber-facebook-icon.png]<https://www.facebook.com/UberGroup?_rdr=p>
[http://uber.co.nz/wp-content/uploads/signatures/uber-twitter-icon.png]<https://twitter.com/ubergroupltd>
E: h at uber.nz<mailto:h at uber.nz>
M: 021 0707 014<tel:021%200707%20014>
W: www.uber.nz<http://www.uber.nz/>
53 PORT ROAD | PO BOX 5083 | WHANGAREI | NEW ZEALAND





From: LibreQoS <libreqos-bounces at lists.bufferbloat.net> On Behalf Of dan via LibreQoS
Sent: Sunday, 13 November 2022 6:39 am
To: Herbert Wolverson <herberticus at gmail.com>
Cc: libreqos <libreqos at lists.bufferbloat.net>
Subject: Re: [LibreQoS] Fwd: A quick report from the WISPA conference

I have to argue with you on 'equally limited' streaming being ok.  That allows for 'internet-like' service but no/poor streaming, a service that clamps down really hard on streaming and then offers you an upgrade to make it work.  Neutrality doesn't mean 'kind of' neutral.  If it's called 'Internet' then no specific service or even type of service should be limited for the provider's direct benefit or whims.  I can see no positive case for a provider limiting any service/type below the customers quoted service.  The provider gets to save money, but since they offer an inferior product that is essentially false advertised and customers have no way to know this then capitalist tools like consumer choice can't work.  Consumers don't know they can get a better suited service, they don't show demand in a market, that prevents competition from having an opportunity, etc.   Most people have just 1 fast internet option.

Does a customer that can't figure out why their new Roku 4K on their new 4K TV refuses to stream 4K Netflix on their 100M internet service say "Netflix sucks!".  or Roku sucks or Sony sucks or whatever?  So this spills over into making other services take the blame for bad connections.



On Sat, Nov 12, 2022 at 10:19 AM Herbert Wolverson via LibreQoS <libreqos at lists.bufferbloat.net<mailto:libreqos at lists.bufferbloat.net>> wrote:
Net neutrality is complex (I'm generally in favor of the concept). There's nothing stopping an ISP from having rules that video streaming be limited - but ALL video streaming has to be equally limited. So ComCast could use Sandvine, but only without any vendor-specific limitations. Same for Cambium. Limit streams that look like video, and you're fine. With that said, it runs into something I say a lot: don't penalize your customers for using your product. If you're buying a 100mbit/s pipe, hooking it to your fancy 4k streaming system - it's entirely reasonable to expect sharp 4k video. In fact, your shaping emphasis should be on keeping their video nice and watchable even though someone just decided to download the newest Call of Duty, update their xbox, and update the firmware on their smart fridge. Likewise, if someone really wants the smallest 5 mbit connection you once offered - because they only ever use it to check email (we actually had a customer complain when we offered them a free upgrade from a 5mbit/s plan, years ago!) - you want to do your best to make it a usable tiny plan for them. Fair queuing (along with a bit of user education; we eventually convinced the customer that "no change in fees" actually meant not charging them more money) helps with all of that.

It can win customers, too. We have a couple of business customers who switched us from their backup to their primary because "our service felt snappier". We had a business go with us because their 24x7 lobby streaming video looked better on our (smaller) connection, and a couple of fraternities who likewise decided to use us because their overkill 4k video setup looked better (even with the 20+ Xboxes we could see on their network).

QUIC makes me chuckle, because it's exactly what we were doing in gamedev land in the late 90s to make deathmatches run smoothly. (Seriously, QuakeWorld and the original Unreal Tournament had the most amazing network code; they basically implemented what is now known as "reliable UDP" to incredible results).

With that said, I think we've got a remarkable amount of wiggle-room to make things better (and it's not at all shoddy right now!). XDP is amazing as-is, and is improving fast. (I'm currently reading https://github.com/ilejn/xdpbridge - an older project that doesn't seem to have gained much traction, but I'm wondering if it couldn't provide some seriously fast bridge offloading; I'll have to setup a better test environment to find out)

On Sat, Nov 12, 2022 at 10:09 AM Robert Chacón via LibreQoS <libreqos at lists.bufferbloat.net<mailto:libreqos at lists.bufferbloat.net>> wrote:
Thank you, Dan.
I completely agree with you 100% there. It's a risky way to save a tiny amount of money on bandwidth costs.
Comcast paid out $16 million in a settlement<https://arstechnica.com/tech-policy/2010/04/comcast-owes-p2p-users-16-yes-they-should-take-it/?comments=1&post=20329187> over using Sandvine's DPI in this way.
Since then, Comcast has switched from DPI to fair queueing with DOCSIS-pie<https://www.reddit.com/r/eero/comments/rc8e2j/getting_a_bufferbloat_grade_with_comcast_aqm/>.

California's SB-822<https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=201720180SB822&fbclid=IwAR2LmNyy5ZkbtzayIyNerJ-Q_OUwK4G0H4aYv4XZwVNz-QF41d5v0BiST-E> will be used as a template by many other states going forward on Net Neutrality.
SB-822 even clarified that "Reasonable Network Management" needs to be "as application-agnostic as possible".
These DPI approaches like Cambium's will not hold up in court the way Fair Queuing (Preseem, LibreQoS) based solutions will.
We can point to years of public research and RFCs when defending a Fair Queuing QoE approach.
They can't so easily defend using a product that advertised its big "scale down 4k to save money" knob or "accelerate their speeds only when they run speed tests" button.
It's concerning to me that these QoE vendors are not disclosing "hey, don't use this in NN states like California please".
The WISP industry will be hurt long-term by these products with consumers viewing us as an inferior technology that limits their usage in intrusive ways.
Like you said, let's let them use their plan however they want. It doesn't hurt us. But things like capping end-users at HD will hurt us long-term in lost sales.

Regarding TCP acceleration, I'm leaning toward the argument made by Dan from Preseem<https://youtu.be/3r4FWGKho3c?t=1879> that QUIC adoption is growing pretty quickly and we may not want to mess with TCP at the expense of UDP given the rise of QUIC.
Cloudlfare says HTTP/3 and QUIC are already at 28% and growing<https://radar.cloudflare.com/adoption-and-usage?range=28d>.
One point he brought up is that if we use TCP acceleration, and a middlebox has to be rebooted or something - all client TCP sessions will break when OSPF switches paths around the box.

On Sat, Nov 12, 2022 at 8:36 AM dan via LibreQoS <libreqos at lists.bufferbloat.net<mailto:libreqos at lists.bufferbloat.net>> wrote:
I think there's a thin grey line between what other vendors (paraqum, bequant/cambium) and libreQoS is trying to do.  The first two are REALLY pushing this selecting narrowing of traffic and they'll go straight after streaming for their demos.  Most people that drop in the cambium QoE appliance turn the knobs on Netflix down and then praise the 40% savings on their network.  Same with PQ.   I think this is fundamentally wrong.  This is the entire reason Net Neutrality keeps popping up.  Why does the ISP get to say "You only need 25Mbps for your netflix".  That's double wrong if they offer their own video services.  It's not about improving the customer's experience, it's about spending less on upstream bandwidth at the customer's expense.  As soon as they say 'pays for itself in bandwidth savings' you know what the product is primarily for.

This is why I like the preseem and libreqos model of just using a 'stock' cake and just finessing the data coming in to keep the pipe from clogging.  Who cares if someone pulls 90M bursts of apple TV+ on their 100M pipe.  Only dial that back so other requests they make get through cleanly.

preseem is going more towards wireless monitoring etc and kinda letting shaping be a secondary.... if they were doing more libreqos like stuff I'd just stick with preseem.

libreQoS may end up being the best and most net neutral product you can get.  The only current disadvantage (ignoring DPI) is lack of TCP accelleration that PQ and Bequant/cambium have.

I should add that 3 states already have laws on the books that straight up make PQ and Cambium's solution illegal because they target certain services for throttling.  My state (Montana) has the same rule except only when working with a government contract.  Libreqos is actually compliant (as is preseem).

On Sat, Nov 12, 2022 at 8:11 AM Dave Taht via LibreQoS <libreqos at lists.bufferbloat.net<mailto:libreqos at lists.bufferbloat.net>> wrote:
this report predates the libreqos list...

---------- Forwarded message ---------
From: Dave Taht <dave.taht at gmail.com<mailto:dave.taht at gmail.com>>
Date: Mon, Oct 17, 2022 at 8:15 PM
Subject: A quick report from the WISPA conference
To: Sina Khanifar <sina at waveform.com<mailto:sina at waveform.com>>
Cc: Cake List <cake at lists.bufferbloat.net<mailto:cake at lists.bufferbloat.net>>, Make-Wifi-fast
<make-wifi-fast at lists.bufferbloat.net<mailto:make-wifi-fast at lists.bufferbloat.net>>, Rpm
<rpm at lists.bufferbloat.net<mailto:rpm at lists.bufferbloat.net>>, Stuart Cheshire <cheshire at apple.com<mailto:cheshire at apple.com>>,
bloat <bloat at lists.bufferbloat.net<mailto:bloat at lists.bufferbloat.net>>


On Mon, Oct 17, 2022 at 7:51 PM Sina Khanifar <sina at waveform.com<mailto:sina at waveform.com>> wrote:
>
> Positive or negative, I can claim a bit of credit for this video :). We've been working with LTT on a few projects and we pitched them on doing something around bufferbloat. We've seen more traffic to our Waveforn test than ever before, which has been fun!

Thank you. Great job with that video! And waveform has become the goto
site for many now.

I can't help but wonder tho... are you collecting any statistics, over
time, as to how much better the problem is getting?

And any chance they could do something similar explaining wifi?

...

I was just at WISPA conference week before last. Preseem's booth
(fq_codel) was always packed. Vilo living had put cake in their wifi 6
product. A
keynote speaker had deployed it and talked about it with waveform
results on the big screen (2k people there). A large wireless vendor
demo'd privately to me their flent results before/after cake on their
next-gen radios... and people dissed tarana without me prompting for
their bad bufferbloat... and the best thing of all that happened to me
was... besides getting a hug from a young lady (megan) who'd salvaged
her schooling in alaska using sqm - I walked up to the paraqum booth
(another large QoE middlebox maker centered more in india) and asked.

"So... do y'all have fq_codel yet?"

And they smiled and said: "No, we have something better... we've got cake."

"Cake? What's that?" - I said, innocently.

They then stepped me through their 200Gbps (!!) product, which uses a
bunch of offloads, and can track rtt down to a ms with the intel
ethernet card they were using. They'd modifed cake to provide 16 (?)
levels of service, and were running under dpdk (I am not sure if cake
was). It was a great, convincing pitch...

... then I told 'em who I was. There's a video of the in-both concert after.

...

The downside to me (and the subject of my talk) was that in nearly
every person I talked to, fq_codel was viewed as a means to better
subscriber bandwidth plan enforcement (which is admittedly the market
that preseem pioneered) and it was not understood that I'd got
involved in this whole thing because I'd wanted an algorithm to deal
with "rain fade", running directly on the radios. People wanted to use
the statistics on the radios to drive the plan enforcement better
(which is an ok approach, I guess), and for 10+ I'd been whinging
about the... physics.

So I ranted about rfc7567 a lot and begged people now putting routerOS
7.2 and later out there (mikrotik is huge in this market), to kill
their fifos and sfqs at the native rates of the interfaces... and
watch their network improve that way also.

I think one more wispa conference will be a clean sweep of everyone in
the fixed wireless market to not only adopt these algorithms for plan
enforcement, but even more directly on the radios and more CPE.

I also picked up enough consulting business to keep me busy the rest
of this year, and possibly more than I can handle (anybody looking?)

I wonder what will happen at a fiber conference?

> On Mon, Oct 17, 2022 at 7:45 PM Dave Taht via Bloat <bloat at lists.bufferbloat.net<mailto:bloat at lists.bufferbloat.net>> wrote:
>>
>> On Mon, Oct 17, 2022 at 5:02 PM Stuart Cheshire <cheshire at apple.com<mailto:cheshire at apple.com>> wrote:
>> >
>> > On 9 Oct 2022, at 06:14, Dave Taht via Make-wifi-fast <make-wifi-fast at lists.bufferbloat.net<mailto:make-wifi-fast at lists.bufferbloat.net>> wrote:
>> >
>> > > This was so massively well done, I cried. Does anyone know how to get in touch with the ifxit folk?
>> > >
>> > > https://www.youtube.com/watch?v=UICh3ScfNWI
>> >
>> > I’m surprised that you liked this video. It seems to me that it repeats all the standard misinformation. The analogy they use is the standard terrible example of waiting in a long line at a grocery store, and the “solution” is letting certain traffic “jump the line, angering everyone behind them”.
>>
>> Accuracy be damned. The analogy to common experience resonates more.
>>
>> >
>> > Some quotes from the video:
>> >
>> > > it would be so much more efficient for them to let you skip the line and just check out, especially since you’re in a hurry, but they’re rudely refusing
>>
>> I think the person with the cheetos pulling out a gun and shooting
>> everyone in front of him (AQM) would not go down well.
>>
>> > > to go back to our grocery store analogy this would be like if a worker saw you standing at the back ... and either let you skip to the front of the line or opens up an express lane just for you
>>
>> Actually that analogy is fairly close to fair queuing. The multiple
>> checker analogy is one of the most common analogies in queue theory
>> itself.
>>
>> >
>> > The video describes the problem of bufferbloat, and then describes the same failed solution that hasn’t worked for the last three decades.
>>
>> Hmm? It establishes the scenario, explains the problem *quickly*,
>> disses gamer routers for not getting it right..  *points to an
>> accurate test*, and then to the ideas and products that *actually
>> work* with "smart queueing", with a screenshot of the most common
>> (eero's optimize for gaming and videoconferencing), and fq_codel and
>> cake *by name*, and points folk at the best known solution available,
>> openwrt.
>>
>> Bing, baddabang, boom. Also the comments were revealing. A goodly
>> percentage already knew the problem, more than a few were inspired to
>> take the test,
>> there was a whole bunch of "Aha!" success stories and 360k views,
>> which is more people than we've ever been able to reach in for
>> example, a nanog conference.
>>
>> I loved that folk taking the test actually had quite a few A results,
>> without having had to do anything. At least some ISPs are getting it
>> more right now!
>>
>> At this point I think gamers in particular know what "brands" we've
>> tried to establish - "Smart queues", "SQM", "OpenWrt", fq_codel and
>> now "cake" are "good" things to have, and are stimulating demand by
>> asking for them,   It's certainly working out better and better for
>> evenroute, firewalla, ubnt and others, and I saw an uptick in
>> questions about this on various user forums.
>>
>> I even like that there's a backlash now of people saying "fixing
>> bufferbloat doesn't solve everything" -
>>
>> >  Describing the obvious simple-minded (wrong) solution that any normal person would think of based on their personal human experience waiting in grocery stores and airports, is not describing the solution to bufferbloat. The solution to bufferbloat is not that if you are privileged then you get to “skip to the front of the line”. The solution to bufferbloat is that there is no line!
>>
>> I like the idea of a guru floating above a grocery cart with a better
>> string of explanations, explaining
>>
>>    - "no, grasshopper, the solution to bufferbloat is no line... at all".
>>
>> >
>> > With grocery stores and airports people’s arrivals are independent and not controlled. There is no way for a grocery store or airport to generate backpressure to tell people to wait at home when a queue begins to form. The key to solving bufferbloat is generating timely backpressure to prevent the queue forming in the first place, not accepting a huge queue and then deciding who deserves special treatment to get better service than all the other peons who still have to wait in a long queue, just like before.
>>
>> I am not huge on the word "backpressure" here. Needs to signal the
>> other side to slow down, is more accurate. So might say timely
>> signalling rather than timely backpressure?
>>
>> Other feedback I got  was that the video was too smarmy (I agree),
>> different audiences than gamers need different forms of outreach...
>>
>> but to me, winning the gamers has always been one of the most
>> important things, as they make a lot of buying decisions, and they
>> benefit the most for
>> fq and packet prioritization as we do today in gamer routers and in
>> cake + qosify.
>>
>> maybe that gets in the way of more serious markets. Certainly I would
>> like another video explaining what goes wrong with videoconferencing.
>>
>>
>>
>>
>>
>>
>> >
>> > Stuart Cheshire
>> >
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
>> Dave Täht CEO, TekLibre, LLC
>> _______________________________________________
>> Bloat mailing list
>> Bloat at lists.bufferbloat.net<mailto:Bloat at lists.bufferbloat.net>
>> https://lists.bufferbloat.net/listinfo/bloat



--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC


--
This song goes out to all the folk that thought Stadia would work:
https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXtz
Dave Täht CEO, TekLibre, LLC
_______________________________________________
LibreQoS mailing list
LibreQoS at lists.bufferbloat.net<mailto:LibreQoS at lists.bufferbloat.net>
https://lists.bufferbloat.net/listinfo/libreqos
_______________________________________________
LibreQoS mailing list
LibreQoS at lists.bufferbloat.net<mailto:LibreQoS at lists.bufferbloat.net>
https://lists.bufferbloat.net/listinfo/libreqos


--
Robert Chacón
CEO | JackRabbit Wireless LLC<http://jackrabbitwireless.com>
Dev | LibreQoS.io<http://LibreQoS.io>

_______________________________________________
LibreQoS mailing list
LibreQoS at lists.bufferbloat.net<mailto:LibreQoS at lists.bufferbloat.net>
https://lists.bufferbloat.net/listinfo/libreqos
_______________________________________________
LibreQoS mailing list
LibreQoS at lists.bufferbloat.net<mailto:LibreQoS at lists.bufferbloat.net>
https://lists.bufferbloat.net/listinfo/libreqos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/libreqos/attachments/20221113/6897d774/attachment-0001.html>