From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 37DC13B29E; Tue, 18 Oct 2022 15:04:13 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1666119850; bh=SmVt9kWpXJ2E21yOFydFQRS+D+fMwMPVbwuDoeuZE04=; h=X-UI-Sender-Class:Date:From:To:CC:Subject:In-Reply-To:References; b=Pqe61NYwXEKJClliKpibRm4OiQay8DwQfxc4bIgVxKXn4Noftdgs+JEvLucqN4vin yjrW6ndS1EPtF1rTeUf7j2R3JFU45Q9iwruERKJpz3WPmDawIdsFyB0mZ1nUi5trra ZmTeX3L9hr6APxzhB5nhJE4Pyna2Ktxq7AdIWKRY= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [127.0.0.1] ([79.234.22.28]) by mail.gmx.net (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MSKuA-1oZl0U2l32-00Sgvc; Tue, 18 Oct 2022 21:04:10 +0200 Date: Tue, 18 Oct 2022 21:04:08 +0200 From: Sebastian Moeller To: Sina Khanifar , Sina Khanifar via Bloat , Dave Taht CC: Cake List , Make-Wifi-fast , Rpm , bloat User-Agent: K-9 Mail for Android In-Reply-To: References: <938D9D45-DADA-4291-BD8A-84E4257CEE49@apple.com> Message-ID: <31A8072A-2482-4395-A521-64350035D8DD@gmx.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:bYn/H6YJChgdembqc26b8VUihyqNjAUtE/Jf2Q7VgbgP+qzNVW/ zap1oj46798yn/4KGx1yBcU8ifBvYxLyOWma01FrivOnVHyu2clWHkUKKBVZgH7ADx/B44q NiWfQiTJIBQqqpebtplbUCuD7MgYgl9y62e/BBCej+SNzRH7mZaJT92FnwMjEPI28S8lbNy gfn14M7gJvnHQQK7M+q5A== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:JpsPhtFZXug=:1NNSrr2cieR04rlneKY29H padoXVxh2yg/5dsYHOLa75T+NorJeCTGQJBztHdvMgPI1rhmM5T3pOy6xvDB54krvXLMGilyv Au4CnzmVP6yajndm+GYi9pvPjETTXibmoSW8X7GSRaEBn0UKDZgvT+OL4aDrEIzk2aPEEvhkG 67hdqEbaAWto1qxckffWX0SBA4HG0ROFh1clI9LEt71YpLlxwCWUebWRIWScvSIE66aNlHg5r kEznB3xm56TCIMGkgF6Jce1eOLMSflGO66E9GjsHXvL4J62hQChVcJbcg9ncy4TwIA915iahk JZ1hFXS50+yoFazFUxCdk7I7pn/gROcnYIkpM6EfFy8EyLj062QvCfmdlY2rICJyfqBjC2Y0A xz2O4PvYrxwnrnj/6qkYsTqXThQJinujnPBUAIUQeJ0d64GSwBVNdqdBi0kHCUKdlbe5tdy4D FIWXapmGmuMohRYHrGkaN3ibKW5KaKDbz20VY/r0pIhtFttvtt9Uq+W6Ep3M6mYanQ1i4PtFm jrg1ydFZ1tVdzxvv8i5awwreT++R3ZNmjEDVNcieCaJetjo07KeEsZMFSb+KFQQ8/+o6CPn10 Sp9eIu9bbFNE2le9fArmi8fxuHTHmDd3dTWmtCUBJwzwwsQFmpqpf64vUxPNfyDoW78jwvv7S yV5i++QgQ4yMBCi2nUcCp7VmJ0zCBMeJHjyoyRkSk96zAMjM0Yq2onIyipRSQDQ2iRUWThbUM Egq+Ez+bDUJGTM+wgBHOP0iAAxUzeYkjBW0nJUJy8L7uhiRHirWwrjwZL5OdMZRK48yt4k3lK fqSeJ91nZ1m0FWIheJNAYehKmlx7zSO8EXJJDIsl/utGbK4yyZV96elg857u3urREaNzstgkp m6VqOfnaMS5TpHCNhrvnRPJ4Ht6kLek4gataCryg7rPrffhdRb2LKENVqKF0ioOXKgywq33c9 6eb7qe5Dg3DNvyuIwvir64aQnDBjgLiLmhWPXYgp0/SLWxJQoC36K+UpI178lT5Rsq5DgE62B h/pvrrVTPkYAgNHsN029DMQicXNeBDRwq+8Tn4C679nzdvkm2ySlk/ZQP8/6wniCPzrdox7uM ga/rqecf1rT6jic6ds3jDwXuH9ohbr8yy2vZB1h0uZDgpm/FHakOLnqWlScws5WK/+xAW/5zR qZdrkvtAE0FIPLOzJvRS3Ki3E3x3wsXkBlCuDwFcazfFz9NOnQnKGlAgZXjhkaIQXmtyjTE0d gG5c3NvC1ziStvUyztjuOxdC4daa5g+/xOKzwuWNFY7kldI03GbvXaNcbUwryA2D3+UAujqh1 x8Lesgc6digcmTyV25wuG51gGzqdHwjJ2t3YXAzUgY71cmtYqxwCIj4oG/4kqSPpzOoBNddgD cISDoaNso2j3wI/Z6dZw5j/egCab1AGlU4CIfwxDZMMMxGXw355hXGHMiiO0YG2gLnM5qxOOi Bqf0x2IDfGt8BDPcFiYsl97Cud2OL0LdmFkpHm5PHvHxHsF+N1SB9nc30dttXsjMk7Wk8LPws PxphJENSnfNX8Tp333B7xKPwFkI05QvJZqnvOAB55RZ3H Subject: Re: [Rpm] [Bloat] A quick report from the WISPA conference X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2022 19:04:13 -0000 Hi Sina, On 18 October 2022 19:17:16 CEST, Sina Khanifar via Bloat wrote: >>=20 >>=20 >>=20 >> I can't help but wonder tho=2E=2E=2E are you collecting any statistics,= over >> time, as to how much better the problem is getting? >>=20 >>=20 >>=20 > >We are collecting anonymized=C2=A0data, but we haven't analyzed it yet=2E= If we get a bit of time we'll look at that hopefully=2E [SM] Just an observation, using Safari I see large maximal delays (like a = small group of samples far out to the right of the bulk) for both down- and= upload that essentially disappear when I switch to firefox=2E Now I tend to have a ton of tabs open in Safari while I only open firefox = for dedicated use-cases with a few tabs at most, so I do not intend to thro= w shade on Safari here; my point is more browsers can and do affect the rep= orted latency numbers, of you want to be able to test this, maybe ask users= to use the OS browser (safari, edge, konqueror ;) ) as well as firefox and= chrome so you can directly compare across browsers? > >>=20 >>=20 >>=20 >> And any chance they could do something similar explaining wifi? >>=20 >>=20 >>=20 > >I'm actually not exactly sure what mitigations exist for WiFi at the mome= nt - is there something I can read? > >On this note: when we were building our test one of the things we really = wished existed was a standardized way to test latency and throughput to rou= ters=2E=20 [SM] traceroute/mtr albeit not sure how well this approach works from insi= de the browser, can you e=2Eg=2E control TTL and do you receive error messa= ges via ICMP? It would be super helpful if there was a standard in consumer routers=C2= =A0that allowed users to both ping and fetch 0kB fils from their routers, a= nd also run download/upload tests=2E [SM] I think I see where you are coming from here=2E Over in the OpenWrt f= orum we often see that server performance with iperf2/3 or netperf on a rou= ter is not all that representative for its routing performance=2E What do you expect to deduce from upload/download to the router? (I might = misunderstand your point by a mile, if so please elaborate) Regards Sebastian > >>=20 >>=20 >>=20 >> I think one more wispa conference will be a clean sweep of everyone in = the >> fixed wireless market to not only adopt these algorithms for plan >> enforcement, but even more directly on the radios and more CPE=2E >>=20 >>=20 >>=20 > >T-Mobile has signed up 1m+ people to their new Home Internet over 5G, and= all of them have really meaningful bufferbloat issues=2E I've been pointin= g folks who reach out to this thread ( https://forum=2Eopenwrt=2Eorg/t/cake= -w-adaptive-bandwidth-historic/108848 ) about cake-autorate and sqm-autorat= e, but ideally it would be fixed at a network level, just not sure how to a= pply pressure (I'm in contact with the T-Mobile Home Internet team, but I t= hink this is above their heads)=2E > >On Mon, Oct 17, 2022 at 8:15 PM, Dave Taht < dave=2Etaht@gmail=2Ecom > wr= ote: > >>=20 >>=20 >>=20 >> On Mon, Oct 17, 2022 at 7:51 PM Sina Khanifar < sina@ waveform=2E com ( >> sina@waveform=2Ecom ) > wrote: >>=20 >>=20 >>=20 >>>=20 >>>=20 >>> Positive or negative, I can claim a bit of credit for this video :)=2E= We've >>> been working with LTT on a few projects and we pitched them on doing >>> something around bufferbloat=2E We've seen more traffic to our Wavefor= n test >>> than ever before, which has been fun! >>>=20 >>>=20 >>>=20 >>=20 >>=20 >>=20 >> Thank you=2E Great job with that video! And waveform has become the got= o >> site for many now=2E >>=20 >>=20 >>=20 >>=20 >> I can't help but wonder tho=2E=2E=2E are you collecting any statistics,= over >> time, as to how much better the problem is getting? >>=20 >>=20 >>=20 >>=20 >> And any chance they could do something similar explaining wifi? >>=20 >>=20 >>=20 >>=20 >> =2E=2E=2E >>=20 >>=20 >>=20 >>=20 >> I was just at WISPA conference week before last=2E Preseem's booth >> (fq_codel) was always packed=2E Vilo living had put cake in their wifi = 6 >> product=2E A >> keynote speaker had deployed it and talked about it with waveform resul= ts >> on the big screen (2k people there)=2E A large wireless vendor demo'd >> privately to me their flent results before/after cake on their next-gen >> radios=2E=2E=2E and people dissed tarana without me prompting for their= bad >> bufferbloat=2E=2E=2E and the best thing of all that happened to me was= =2E=2E=2E >> besides getting a hug from a young lady (megan) who'd salvaged her >> schooling in alaska using sqm - I walked up to the paraqum booth >> (another large QoE middlebox maker centered more in india) and asked=2E >>=20 >>=20 >>=20 >> "So=2E=2E=2E do y'all have fq_codel yet?" >>=20 >>=20 >>=20 >>=20 >> And they smiled and said: "No, we have something better=2E=2E=2E we've = got >> cake=2E" >>=20 >>=20 >>=20 >>=20 >> "Cake? What's that?" - I said, innocently=2E >>=20 >>=20 >>=20 >>=20 >> They then stepped me through their 200Gbps (!!) product, which uses a >> bunch of offloads, and can track rtt down to a ms with the intel ethern= et >> card they were using=2E They'd modifed cake to provide 16 (?) levels of >> service, and were running under dpdk (I am not sure if cake was)=2E It = was a >> great, convincing pitch=2E=2E=2E >>=20 >>=20 >>=20 >>=20 >> =2E=2E=2E then I told 'em who I was=2E There's a video of the in-both c= oncert >> after=2E >>=20 >>=20 >>=20 >>=20 >> =2E=2E=2E >>=20 >>=20 >>=20 >>=20 >> The downside to me (and the subject of my talk) was that in nearly ever= y >> person I talked to, fq_codel was viewed as a means to better subscriber >> bandwidth plan enforcement (which is admittedly the market that preseem >> pioneered) and it was not understood that I'd got involved in this whol= e >> thing because I'd wanted an algorithm to deal with "rain fade", running >> directly on the radios=2E People wanted to use the statistics on the ra= dios >> to drive the plan enforcement better >> (which is an ok approach, I guess), and for 10+ I'd been whinging about >> the=2E=2E=2E physics=2E >>=20 >>=20 >>=20 >> So I ranted about rfc7567 a lot and begged people now putting routerOS >> 7=2E2 and later out there (mikrotik is huge in this market), to kill th= eir >> fifos and sfqs at the native rates of the interfaces=2E=2E=2E and watch= their >> network improve that way also=2E >>=20 >>=20 >>=20 >> I think one more wispa conference will be a clean sweep of everyone in = the >> fixed wireless market to not only adopt these algorithms for plan >> enforcement, but even more directly on the radios and more CPE=2E >>=20 >>=20 >>=20 >>=20 >> I also picked up enough consulting business to keep me busy the rest of >> this year, and possibly more than I can handle (anybody looking?) >>=20 >>=20 >>=20 >>=20 >> I wonder what will happen at a fiber conference? >>=20 >>=20 >>=20 >>>=20 >>>=20 >>> On Mon, Oct 17, 2022 at 7:45 PM Dave Taht via Bloat < bloat@ lists=2E = bufferbloat=2E >>> net ( bloat@lists=2Ebufferbloat=2Enet ) > wrote: >>>=20 >>>=20 >>>=20 >>>>=20 >>>>=20 >>>> On Mon, Oct 17, 2022 at 5:02 PM Stuart Cheshire < cheshire@ apple=2E = com ( >>>> cheshire@apple=2Ecom ) > wrote: >>>>=20 >>>>=20 >>>>=20 >>>>>=20 >>>>>=20 >>>>> On 9 Oct 2022, at 06:14, Dave Taht via Make-wifi-fast < make-wifi-fa= st@ lists=2E >>>>> bufferbloat=2E net ( make-wifi-fast@lists=2Ebufferbloat=2Enet ) > wr= ote: >>>>>=20 >>>>>=20 >>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> This was so massively well done, I cried=2E Does anyone know how to= get in >>>>>> touch with the ifxit folk? >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> https:/ / www=2E youtube=2E com/ watch?v=3DUICh3ScfNWI ( >>>>>> https://www=2Eyoutube=2Ecom/watch?v=3DUICh3ScfNWI ) >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>=20 >>>>>=20 >>>>>=20 >>>>> I=E2=80=99m surprised that you liked this video=2E It seems to me th= at it repeats >>>>> all the standard misinformation=2E The analogy they use is the stand= ard >>>>> terrible example of waiting in a long line at a grocery store, and t= he >>>>> =E2=80=9Csolution=E2=80=9D is letting certain traffic =E2=80=9Cjump = the line, angering everyone >>>>> behind them=E2=80=9D=2E >>>>>=20 >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> Accuracy be damned=2E The analogy to common experience resonates more= =2E >>>>=20 >>>>=20 >>>>=20 >>>>>=20 >>>>>=20 >>>>> Some quotes from the video: >>>>>=20 >>>>>=20 >>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> it would be so much more efficient for them to let you skip the lin= e and >>>>>> just check out, especially since you=E2=80=99re in a hurry, but the= y=E2=80=99re rudely >>>>>> refusing >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> I think the person with the cheetos pulling out a gun and shooting >>>> everyone in front of him (AQM) would not go down well=2E >>>>=20 >>>>=20 >>>>=20 >>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> to go back to our grocery store analogy this would be like if a wor= ker saw >>>>>> you standing at the back =2E=2E=2E and either let you skip to the f= ront of the >>>>>> line or opens up an express lane just for you >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> Actually that analogy is fairly close to fair queuing=2E The multiple >>>> checker analogy is one of the most common analogies in queue theory >>>> itself=2E >>>>=20 >>>>=20 >>>>=20 >>>>>=20 >>>>>=20 >>>>> The video describes the problem of bufferbloat, and then describes t= he >>>>> same failed solution that hasn=E2=80=99t worked for the last three d= ecades=2E >>>>>=20 >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> Hmm? It establishes the scenario, explains the problem *quickly*, dis= ses >>>> gamer routers for not getting it right=2E=2E *points to an accurate t= est*, and >>>> then to the ideas and products that *actually work* with "smart queue= ing", >>>> with a screenshot of the most common >>>> (eero's optimize for gaming and videoconferencing), and fq_codel and = cake >>>> *by name*, and points folk at the best known solution available, open= wrt=2E >>>>=20 >>>>=20 >>>>=20 >>>> Bing, baddabang, boom=2E Also the comments were revealing=2E A goodly >>>> percentage already knew the problem, more than a few were inspired to= take >>>> the test, >>>> there was a whole bunch of "Aha!" success stories and 360k views, whi= ch is >>>> more people than we've ever been able to reach in for example, a nano= g >>>> conference=2E >>>>=20 >>>>=20 >>>>=20 >>>> I loved that folk taking the test actually had quite a few A results, >>>> without having had to do anything=2E At least some ISPs are getting i= t more >>>> right now! >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> At this point I think gamers in particular know what "brands" we've t= ried >>>> to establish - "Smart queues", "SQM", "OpenWrt", fq_codel and now "ca= ke" >>>> are "good" things to have, and are stimulating demand by asking for t= hem, >>>> It's certainly working out better and better for evenroute, firewalla= , >>>> ubnt and others, and I saw an uptick in questions about this on vario= us >>>> user forums=2E >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> I even like that there's a backlash now of people saying "fixing >>>> bufferbloat doesn't solve everything" - >>>>=20 >>>>=20 >>>>=20 >>>>>=20 >>>>>=20 >>>>> Describing the obvious simple-minded (wrong) solution that any norma= l >>>>> person would think of based on their personal human experience waiti= ng in >>>>> grocery stores and airports, is not describing the solution to >>>>> bufferbloat=2E The solution to bufferbloat is not that if you are pr= ivileged >>>>> then you get to =E2=80=9Cskip to the front of the line=E2=80=9D=2E T= he solution to >>>>> bufferbloat is that there is no line! >>>>>=20 >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> I like the idea of a guru floating above a grocery cart with a better >>>> string of explanations, explaining >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> - "no, grasshopper, the solution to bufferbloat is no line=2E=2E=2E a= t all"=2E >>>>=20 >>>>=20 >>>>=20 >>>>>=20 >>>>>=20 >>>>> With grocery stores and airports people=E2=80=99s arrivals are indep= endent and not >>>>> controlled=2E There is no way for a grocery store or airport to gene= rate >>>>> backpressure to tell people to wait at home when a queue begins to f= orm=2E >>>>> The key to solving bufferbloat is generating timely backpressure to >>>>> prevent the queue forming in the first place, not accepting a huge q= ueue >>>>> and then deciding who deserves special treatment to get better servi= ce >>>>> than all the other peons who still have to wait in a long queue, jus= t like >>>>> before=2E >>>>>=20 >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> I am not huge on the word "backpressure" here=2E Needs to signal the = other >>>> side to slow down, is more accurate=2E So might say timely signalling= rather >>>> than timely backpressure? >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> Other feedback I got was that the video was too smarmy (I agree), >>>> different audiences than gamers need different forms of outreach=2E= =2E=2E >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> but to me, winning the gamers has always been one of the most importa= nt >>>> things, as they make a lot of buying decisions, and they benefit the = most >>>> for >>>> fq and packet prioritization as we do today in gamer routers and in c= ake + >>>> qosify=2E >>>>=20 >>>>=20 >>>>=20 >>>> maybe that gets in the way of more serious markets=2E Certainly I wou= ld like >>>> another video explaining what goes wrong with videoconferencing=2E >>>>=20 >>>>=20 >>>>=20 >>>>>=20 >>>>>=20 >>>>> Stuart Cheshire >>>>>=20 >>>>>=20 >>>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> -- >>>> This song goes out to all the folk that thought Stadia would work: ht= tps:/ >>>> / www=2E linkedin=2E com/ posts/ dtaht_the-mushroom-song-activity-698= 1366665607352320-FXtz >>>> ( >>>> https://www=2Elinkedin=2Ecom/posts/dtaht_the-mushroom-song-activity-6= 981366665607352320-FXtz >>>> ) Dave T=C3=A4ht CEO, TekLibre, LLC >>>> _______________________________________________ >>>> Bloat mailing list >>>> Bloat@ lists=2E bufferbloat=2E net ( Bloat@lists=2Ebufferbloat=2Enet = ) >>>> https://lists=2Ebufferbloat=2Enet/listinfo/bloat >>>>=20 >>>>=20 >>>=20 >>>=20 >>=20 >>=20 >>=20 >> -- >> This song goes out to all the folk that thought Stadia would work: http= s:/ >> / www=2E linkedin=2E com/ posts/ dtaht_the-mushroom-song-activity-69813= 66665607352320-FXtz >> ( >> https://www=2Elinkedin=2Ecom/posts/dtaht_the-mushroom-song-activity-698= 1366665607352320-FXtz >> ) Dave T=C3=A4ht CEO, TekLibre, LLC >>=20 >>=20 >> --=20 Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E