From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id BCF893B29E; Mon, 13 Mar 2023 16:45:50 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1678740345; i=moeller0@gmx.de; bh=1wMPEv5oKEUqRZdDXYT/YifdC4eZGj1+cTaBVqbNjBU=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=m1ZpRWndwq1jp++te7ET2RN2KaT1lmKauIM3iy3YUHYo4RRpF3HRoCbMgvWrXmjtd J/26ccLMbsZQNvQN+w36w0ToNL6EU4ITvQ96XEe8PpgX9rcyDNw0xbtpiXfYByZBkc vvykhpTR0K5n0YNsVNvG7qEI9JSax6Mi9Y5kACU6rJJuy20tq57wf0o6detqlgqn8N CBUyBG3jwQBBWZWdm731Pjwnpws2S5hM1H5fSbOHRi92/1tTIzMCCUltcEm+UMeuiK p54iPmHP49g61w1zPZujlRLNey7atvWPWpb26PV7b/xct9V7ECyCct7zs8DbeAin3o 4dobVsk+cwbDg== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from smtpclient.apple ([95.112.228.62]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MXp9Y-1q3R6T29pE-00Y6Or; Mon, 13 Mar 2023 21:45:45 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) From: Sebastian Moeller In-Reply-To: Date: Mon, 13 Mar 2023 21:45:44 +0100 Cc: Jeremy Austin , Rpm , libreqos , Dave Taht via Starlink , rjmcmahon , bloat Content-Transfer-Encoding: quoted-printable Message-Id: <5B61716D-6DE9-4320-A06B-F8EE16FAA673@gmx.de> References: <1672786712.106922180@apps.rackspace.com> <77CCAD19-07E0-4F9E-88C1-D207CF7BF376@cable.comcast.com> <83ffc0dad19e3343e49271889369cefc@rjmcmahon.com> <3CD0B9E6-0B2A-4A70-8F53-ED0822DF77A6@gmx.de> <13DE6E53-665F-4C20-BBE2-70E685421E9D@gmx.de> <22C819FA-DDD7-4B9B-8C09-8008D4273287@gmx.de> To: dan X-Mailer: Apple Mail (2.3696.120.41.1.2) X-Provags-ID: V03:K1:v9/JKqEZHDfN9go2fQRYWC5x1un3V0wU28tOsiMJ9IQ64/c/JWR SaSb0oEh6jHkn0dd3rZratI6R9X3rvnQeB9fe7JKubv1d2ihIswR34MchebXw0eC0Uut8Sk a7baCl4hxgtzxYOpzNEIrVSvS2+8igv2QJmUwOkV6MAbYrO4CgoExD/S919wt/ZDFx26GTP ZWZdVydz3A3ybT99MsXwQ== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:SHrwbvJ5X+I=;bejzp+3w52eqnhO9h3T+BvOkmtd bjCQHJdoloQpaE6rTB4k11RnpjlTUtrKPqRvhM067d6OOAO9hiLcQTvmiMPNaYYmFoBg0RuAv X1ael8Yu/scVDTGkZXcgoNse+v4xpmgBPTiJqzSUFe6bG4cCZm2zCjfZCC/aY2RtkfpVaB8u0 i4LS/cUWslbqEWXBCq4CQxsWHxoeAfyR0YIN+HGjemD+e5ofWqa4csdG6F8xAyh3tiBzXBV8Z loMj+cj9essAbtA9xf4M9c7oO26g7ywcdnWvFIXUvHqvtSnmNGdD2c1y5vN5OCwh2X5kWw1sQ xA92GDEYEAJ0Tnrk0BBLOdQxjZDP0OOfI+yiWN+Znv1k0pLnHXfnxhlhEvhaHHN5nf3F/Ta/M ArDcAl3y7uNs8VBoSs2IIA6lcLBLuvn/Wchzx3qZF1lfuhCIjmZAVPo4VeMiXCtcFYq/Rrf6f 498dcZzwrLCl3A5lgn/TQWolMaLyTu33u4P7WVFVkrtY9M0fYVQ8FWsKAfcriJ9vo4NjW0ItO fdf5Qcz9IVMYjV24h0FatBh/2d72CMFVcQwEnuQyL4Mc5drjf5uYNro5Koh+KZAmJ/Yv+USe0 qubybd4DJAxkbd5zb19AsTHMJ5Ek40d8MFbbSkYMnSHrWq+a6zZFrxJ3kJy4GOdSdZXfMJ1Sj bZ47njxoUh2nZkwUCSJHh+KzpVCaMaKnZtNiYDNyJ/OdDz5sjFFIIDP9ugzKalC3Gau0DA/oi G5U1lr/oYvviQbUkDy9aiZlKML5ZdX9MfGRC9644/IWxIZvc+E62ZWIewjOEGEcNYVrhzidVZ 4wjednvPjxUp9dqm4RsSWa81MQLDBysOPbLPH/yCzoslMES7aVJhWUkTKH9ySOPnw9656tJUX naYrF7564uR16OnFRvxFKr9k6Ti76/X03V8usEkfe35ddKlrYn03Y0kqSj88yE8WwEFhbZHJp BJjAYCstNoSRdxhSBTVvnfFT9LI= Subject: Re: [Rpm] [Starlink] [LibreQoS] [EXTERNAL] Re: Researchers Seeking Probe Volunteers in USA X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Mar 2023 20:45:51 -0000 Hi Dan, > On Mar 13, 2023, at 20:33, dan wrote: >=20 >>=20 >> I respectfully disagree, if say, my modem had a 4 GB queue I = could theoretically burst ~4GB worth of data at line rate into that = buffer without learning anything about the the modem-link capacity. >=20 > so this is where we're getting into staw man arguments. Find me a > single device or shaper with a 4GB buffer and we'll talk. [SM] Sure, the moment you tell me how to measure true capacity = without load ;) but my point stays intial burtst from my router to the = modem will be absorbed by the modems queues and will not be indicative = of the ink capacity. >>=20 >>=20 >>> turn off the >>> shaper and run anything. run your speed test. don't look at the >>> speed test results, just use it to generate some traffic. you'll = find >>> your peak and where you hit the buffers on the DSL modem by = measuring >>> on the interface and measuring latency. >>=20 >> Peak of what? Exactly? The peak sending rate of my router is = well known, its 1 Gbps gross ethernet rate... >=20 > I don't know how I can say it any clearer. there is a port, any > speed. data flows across that port. The peak data flowing is the > measure. simultaneously measuring latency will give the 'best' rate. > so called 'goodput' which is a stupid name and I hate it but there it > is. [SM] Sorry the peak gross ate on my gigabit interface to the = modem in spite of my shaper is always going to be 1 Gbps what changes is = the duty cycle... so without averaging this method of yours looks only = partially useful. >=20 >>=20 >>=20 >>> That speed test isn't giving >>> you this data and more than Disney+, other than you get to pick when >>> it runs. >>=20 >> Hrm, no we sre back at actually saturating the link, >=20 > which we're doing all the time. it's the entire premise of QoE. > Links get saturated, manage them. [SM] Mmmh, my goal (and your promise) was to be able to estimate = the saturation capacity before/without actually hitting it. Which s the = whole reason why cake-autorate exists, if we knew that we would track (a = low passed version) of that value with our shaper and be done with... >=20 >>=20 >=20 >>=20 >> [SM] not really, given enough capacity, typical streaming = protocols will actually not hit the ceiling, at least the one's I look = at every now and then tend to stay well below actual capacity of the = link. >=20 > Not sure where you're getting this info, I'm looking right at > customers on everything from 25Mbps to 800Mbps plans. [SM] I guess I need to take a packet capture, I have a hunch = that I might see ECN in action and a lack of drops is not indicative of = slow-start not ending, Ho-hom something for my todo list.... > And again, I'm > not saying you can saturate the link intentionally, I'm saying that > the tool doing the saturation isn't actually giving you accurate > results. You have to look at the interface and the latency for the > results. The speed test is a traffic generator, not a measuring tool. > It's fundamentally cannot do the measuring, it's doesn't have the > ability to see other flows on the interface. [SM] Ah, now I get your point, but I also ignore that point = immediately as that is a) not the capacity resolution I typically need, = b) in cake autorate we actually extract interface counters exactly = because we want to see all traffic. But comparing cake-autorate traces = with speedtest curves (e.g. flent) looks pretty well correlated, so for = my use cases the typical speedtests gve actionable and useful (albeit = not perfect) capacity estimates, the longer running the better. This is = a strike against all of these 10-20 seconds tests, but e.g. fast.com can = be configured easily to measure each direction for a full minute, which = side steps our buffer filling versus filling the link capacity = discussion nicely, as my modem's buffers are not nearly large enough to = absorg a noticeable portion of this 60 second load. >=20 >>=20 >>=20 >> [SM] But my problem is that on variable rate links I want to = measure the instantaneous capacity such that I can do adaptive admission = control and avpid over filling my modem's DSL buffers (I wish they would = do something like BQL, but alas they don't). >=20 > Literally measure the interface on a schedule or constantly and you're > getting this measurement every time you use the link. and if you > measure the latency you're constantly finding the spot right below the > buffers. [SM] Well except I can not measure the relevant interface = veridically, the DSL SoC internal buffering for GINP retransmissions is = not exposed to the OS but handled inside the "modem". >=20 >=20 >>=20 >> [SM] With competent AQM (like cake on ingress and egress = configured for per-internal-IP isolation) I do not even notice whether a = speedtes runs or not, and from the reported capacity I can estimate the = concurrent load from other endhosts in my network. >=20 > exactly. EXACTLY. You might just be coming around. [SM] ONne way of looking at it, I would say I am already here = since a decade ad longer ;) > That speed test > was held back by the shaper for your benefit NOT the speed test's. > It's results are necessarily false. [SM] The question is not about false or wrong but about useful = or useless. And I maintain that even a speedtest from an end-user system = with all potential cross traffic is still useful. > YOU can estimate the capacity by > adding up the speedtest results and your other uses. [SM] But here is the rub, this being a VDSL2 link and me having = had a look in the standards I can calculate the maximum goodput over = that link and in routine speedtests I come close enough to it that I = consider most speedtests useful estimates of capacity. If I see a test = noticeably smaller than expected I tend to repeat that test with tighter = control... No i am typically not scraping numbers from kernel cpunters, = but I simply run iftop on the router which will quickly let me see = whether there are other noticeable data transfers on going. > Measuring the > outside interface gives you exactly that. the speed test does not. > it's just a traffic generator for when you aren't generating it on > your own. [SM] The perfect is the enemy of the good, I am depp in the = "gopd enough is good enough" camp. Even tough I am happy to obsess about = details of per-packet-overhead if I not remind myself of the "good = enough mantra" ;) >=20 >=20 >>=20 >>=20 >>> Speed test cannot return actual capacity >>=20 >> [SM] Conventional capcaity tests give a decent enough estimate = of current capacity to be useful, I could not care less that they are = potential not perfect, sorry. The question still is how to estimate = capacity without loading the link... >=20 > you have to load the link to know this. Again, the speed test is a > traffic generator, it's not a measuring tool. You have to measure at > the wan interface to know this, you can never get it from the speed > test. [SM] Agian I understand your point and where you are coming = from, even though I do not fully agree. Yes speedtests will not be 100% = accurate and precise, but no that is not a show stopper as I am less = concerned about individual Bps, and more whether I hit that capacity I = pay for +/- 10-15%. I am a stickler for details, but i am not going to = harass my ISP (which I am satisfied with) just because it slightly under = delivers the contracted capacity ;) > And no, the speed test isn't a decent enough estimate. =20 [SM] Welcome to the EU, over here speedtests are essentially the = officially blessed tool to check contract compliance by ISPs... and oin = the process of getting there the same arguments we currently exchange = where exchanged as well... The EU made a good point, ISP are free to put = those numbers into contracts as they see fit, so they need to deliver = these numbers in a user confirmable way. That means ISPs have to eat the = slack, e.g. My ISP gives me a 116 Mps sync for a nominal 100 Mbps = contract (the speedtest defaults tp IPv6 if possible) 116.7 * (64/65) * ((1500-8-40-20-12)/(1500+34)) =3D 106.36 Mbps which allows them to fulfill the contracted maximal rate of 100 easily = (to allow for some measuremnt slack it is sufficient if the end user = measures 90% of the contracted maximal rate). Tl; dr: the methods that you argue strongly to be useless are actually = mandated in the EU and in Germany these even have legal standing in = disputes between ISPs and their customers. If a strong point could be = made that these measurements would be so wrong to be useless, I guess on = of the sleazier ISPs would already have done so ;) > The > more important the data is to you the more likely the test is bad and > going to lie. Internet feeling slow? run a speed test and put more > pressure on the service and the speed test has less available to > return results on, all the other services getting their slice of the > pie. [SM] Bad excuse. All ISPs over subscribe, which I consider an = acceptable and economic way of operation, AS LONG as they expand = "capacity" (or cancel contracts) once a "segment" shows signs of = repeated and/or sustained overload. If you sell X Mbps to me, be = prepared to actually deliver these any time... Personally i wonder why ISPs do not simply offer something like "fair = share on an X Mbps aggregate link" shared with my neighbours, but as = long as they charge me for X Mpbs I expect that they acrually deliver = this. Again occasional overload is just fine, sustained and predictable = overload however is not... (which by the way is not my stance alone, it = is essentially encoded in the EU regulation 2015/2120) >=20 >>=20 >>=20 >>> Guess what the only way to get an actual measure of the capacity is? >>> my way. measure what's passing the interface and measure what = happens >>> to a reliable latency test during that time. >>=20 >> [SM] This is, respectfully, what we do in cake-autorate, but = that requires an actual load and only accidentally detects the capacity, = if a high enough load is sustained long enough to evoke a latency = increase. But I knew that already, what you initially wrote sounded to = me like you had a method to detect instantaneous capacity without = needing to generate load. (BTW, in cake-autorate we do not generate an = artificial load (only artificial/active latency probes) but use the = organic user generated traffic as load generator*). >>=20 >> *) If all endhosts are idle we do not care much about the capacity, = only if there is traffic, however the quicker we can estimate the = capacity the tigher our controller can operate. >>=20 >=20 > See, you're coming around. Cake is autorating (or very close, 'on > device') at the wan port. not the speed test device or software. [SM] We are purposefully not running speedtests to estimate = capacity as that would not be helpful, what we do is measure the = existing load and the indiced delay and if the induced delay is larger = than a threshold we reduce the shaper rate to reduce the load gently... > And > the accurate data is collected by cake, not the speed test tool. [SM] Nah, we are not using cake's counters here but the kernels = traffic counters for the interface that cake is instantiated on. and no = these counyers are not perfect either as the kernel does IIRC not update = them immediately but in a delayed fashion that is computationally = cheaper. But for our purpose that is plenty "good enough"... > That > tool is reporting false information because it must, it doesn't know > the other consumers on the network. It's 'truest' when the network is > quiet but the more talkers the more the tool lies. [SM] A tool does not lie, the interpretation of the reading = becomes trickier, but that is a problem of te user, not the tool. If i = use a hammer to hammer in a screwm blame me, not the hammer or the = screw). ;) >=20 > cake, the kernel, and the wan port all have real info, the speed test > tool does not. [SM] THis is not where we started... and not what cake-autorate = does, it also does not convince me that capacity tests are useless. I = happily concede that they are neither 100% accurate, precise or = reliable. There is a contimuum between 100% correct and useless ;) Regards Sebastian