[LibreQoS] [Starlink] [Rpm] [EXTERNAL] Re: Researchers Seeking Probe Volunteers in USA

[Sebastian Moeller]

Hi Dan,


> On Mar 13, 2023, at 20:33, dan <dandenson at gmail.com> wrote:
> 
>> 
>>        I respectfully disagree, if say, my modem had a 4 GB queue I could theoretically burst ~4GB worth of data at line rate into that buffer without learning anything about the the modem-link capacity.
> 
> so this is where we're getting into staw man arguments.  Find me a
> single device or shaper with a 4GB buffer and we'll talk.

	[SM] Sure, the moment you tell me how to measure true capacity without load ;) but my point stays intial burtst from my router to the modem will be absorbed by the modems queues and will not be indicative of the ink capacity.


>> 
>> 
>>> turn off the
>>> shaper and run anything.  run your speed test.  don't look at the
>>> speed test results, just use it to generate some traffic.  you'll find
>>> your peak and where you hit the buffers on the DSL modem by measuring
>>> on the interface and measuring latency.
>> 
>>        Peak of what? Exactly? The peak sending rate of my router is well known, its 1 Gbps gross ethernet rate...
> 
> I don't know how I can say it any clearer.  there is a port, any
> speed.  data flows across that port.  The peak data flowing is the
> measure.  simultaneously measuring latency will give the 'best' rate.
> so called 'goodput' which is a stupid name and I hate it but there it
> is.

	[SM] Sorry the peak gross ate on my gigabit interface to the modem in spite of my shaper is always going to be 1 Gbps what changes is the duty cycle... so without averaging this method of yours looks only partially useful.


> 
>> 
>> 
>>> That speed test isn't giving
>>> you this data and more than Disney+, other than you get to pick when
>>> it runs.
>> 
>>        Hrm, no we sre back at actually saturating the link,
> 
> which we're doing all the time.  it's the entire premise of QoE.
> Links get saturated, manage them.

	[SM] Mmmh, my goal (and your promise) was to be able to estimate the saturation capacity before/without actually hitting it. Which s the whole reason why cake-autorate exists, if we knew that we would track (a low passed version) of that value with our shaper and be done with...


> 
>> 
> 
>> 
>>        [SM] not really, given enough capacity, typical streaming protocols will actually not hit the ceiling, at least the one's I look at every now and then tend to stay well below actual capacity of the link.
> 
> Not sure where you're getting this info, I'm looking right at
> customers on everything from 25Mbps to 800Mbps plans.

	[SM] I guess I need to take a packet capture, I have a hunch that I might see ECN in action and a lack of drops is not indicative of slow-start not ending, Ho-hom something for my todo list....


>  And again, I'm
> not saying you can saturate the link intentionally, I'm saying that
> the tool doing the saturation isn't actually giving you accurate
> results.  You have to look at the interface and the latency for the
> results.  The speed test is a traffic generator, not a measuring tool.
> It's fundamentally cannot do the measuring, it's doesn't have the
> ability to see other flows on the interface.

	[SM] Ah, now I get your point, but I also ignore that point immediately as that is a) not the capacity resolution I typically need, b) in cake autorate we actually extract interface counters exactly  because we want to see all traffic. But comparing cake-autorate traces with speedtest curves (e.g. flent) looks pretty well correlated, so for my use cases the typical speedtests gve actionable and useful (albeit not perfect) capacity estimates, the longer running the better. This is a strike against all of these 10-20 seconds tests, but e.g. fast.com can be configured easily to measure each direction for a full minute, which side steps our buffer filling versus filling the link capacity discussion nicely, as my modem's buffers are not nearly large enough to absorg a noticeable portion of this 60 second load.


> 
>> 
>> 
>>        [SM] But my problem is that on variable rate links I want to measure the instantaneous capacity such that I can do adaptive admission control and avpid over filling my modem's DSL buffers (I wish they would do something like BQL, but alas they don't).
> 
> Literally measure the interface on a schedule or constantly and you're
> getting this measurement every time you use the link.  and if you
> measure the latency you're constantly finding the spot right below the
> buffers.

	[SM] Well except I can not measure the relevant interface veridically, the DSL SoC internal buffering for GINP retransmissions is not exposed to the OS but handled inside the "modem".


> 
> 
>> 
>>        [SM] With competent AQM (like cake on ingress and egress configured for per-internal-IP isolation) I do not even notice whether a speedtes runs or not, and from the reported capacity I can estimate the concurrent load from other endhosts in my network.
> 
> exactly.  EXACTLY.  You might just be coming around.

	[SM] ONne way of looking at it, I would say I am already here since a decade ad longer ;)

>  That speed test
> was held back by the shaper for your benefit NOT the speed test's.
> It's results are necessarily false.

	[SM] The question is not about false or wrong but about useful or useless. And I maintain that even a speedtest from an end-user system with all potential cross traffic is still useful.

>  YOU can estimate the capacity by
> adding up the speedtest results and your other uses.

	[SM] But here is the rub, this being a VDSL2 link and me having had a look in the standards I can calculate the maximum goodput over that link and in routine speedtests I come close enough to it that I consider most speedtests useful estimates of capacity. If I see a test noticeably smaller than expected I tend to repeat that test with tighter control... No i am typically not scraping numbers from kernel cpunters, but I simply run iftop on the router which will quickly let me see whether there are other noticeable data transfers on going.


>  Measuring the
> outside interface gives you exactly that.  the speed test does not.
> it's just a traffic generator for when you aren't generating it on
> your own.

	[SM] The perfect is the enemy of the good, I am depp in the "gopd enough is good enough" camp. Even tough I am happy to obsess about details of per-packet-overhead if I not remind myself of the "good enough mantra" ;)


> 
> 
>> 
>> 
>>> Speed test cannot return actual capacity
>> 
>>        [SM] Conventional capcaity tests give a decent enough estimate of current capacity to be useful, I could not care less that they are potential not perfect, sorry. The question still is how to estimate capacity without loading the link...
> 
> you have to load the link to know this.  Again, the speed test is a
> traffic generator, it's not a measuring tool.  You have to measure at
> the wan interface to know this, you can never get it from the speed
> test.

	[SM] Agian I understand your point and where you are coming from, even though I do not fully agree. Yes speedtests will not be 100% accurate and precise, but no that is not a show stopper as I am less concerned about individual Bps, and more whether I hit that capacity I pay for +/- 10-15%. I am a stickler for details, but i am not going to harass my ISP (which I am satisfied with) just because it slightly under delivers the contracted capacity ;)


>  And no, the speed test isn't a decent enough estimate.  

	[SM] Welcome to the EU, over here speedtests are essentially the officially blessed tool to check contract compliance by ISPs... and oin the process of getting there the same arguments we currently exchange where exchanged as well... The EU made a good point, ISP are free to put those numbers into contracts as they see fit, so they need to deliver these numbers in a user confirmable way. That means ISPs have to eat the slack, e.g. My ISP gives me a 116 Mps sync for a nominal 100 Mbps contract (the speedtest defaults tp IPv6 if possible)
116.7 * (64/65) * ((1500-8-40-20-12)/(1500+34)) = 106.36 Mbps
which allows them to fulfill the contracted maximal rate of 100 easily (to allow for some measuremnt slack it is sufficient if the end user measures 90% of the contracted maximal rate).
Tl; dr: the methods that you argue strongly to be useless are actually mandated in the EU and in Germany these even have legal standing in disputes between ISPs and their customers. If a strong point could be made that these measurements would be so wrong to be useless, I guess on of the sleazier ISPs would already have done so ;)


> The
> more important the data is to you the more likely the test is bad and
> going to lie.  Internet feeling slow? run a speed test and put more
> pressure on the service and the speed test has less available to
> return results on, all the other services getting their slice of the
> pie.

	[SM] Bad excuse. All ISPs over subscribe, which I consider an acceptable and economic way of operation, AS LONG as they expand "capacity" (or cancel contracts) once a "segment" shows signs of repeated and/or sustained overload. If you sell X Mbps to me, be prepared to actually deliver these any time...
Personally i wonder why ISPs do not simply offer something like "fair share on an X Mbps aggregate link" shared with my neighbours, but as long as they charge me for X Mpbs I expect that they acrually deliver this. Again occasional overload is just fine, sustained and predictable overload however is not... (which by the way is not my stance alone, it is essentially encoded in the EU regulation 2015/2120)


> 
>> 
>> 
>>> Guess what the only way to get an actual measure of the capacity is?
>>> my way.  measure what's passing the interface and measure what happens
>>> to a reliable latency test during that time.
>> 
>>        [SM] This is, respectfully, what we do in cake-autorate, but that requires an actual load and only accidentally detects the capacity, if a high enough load is sustained long enough to evoke a latency increase. But I knew that already, what you initially wrote sounded to me like you had a method to detect instantaneous capacity without needing to generate load. (BTW, in cake-autorate we do not generate an artificial load (only artificial/active latency probes) but use the organic user generated traffic as load generator*).
>> 
>> *) If all endhosts are idle we do not care much about the capacity, only if there is traffic, however the quicker we can estimate the capacity the tigher our controller can operate.
>> 
> 
> See, you're coming around.  Cake is autorating (or very close, 'on
> device') at the wan port.  not the speed test device or software.

	[SM] We are purposefully not running speedtests to estimate capacity as that would not be helpful, what we do is measure the existing load and the indiced delay and if the induced delay is larger than a threshold we reduce the shaper rate to reduce the load gently...


>  And
> the accurate data is collected by cake, not the speed test tool.

	[SM] Nah, we are not using cake's counters here but the kernels traffic counters for the interface that cake is instantiated on. and no these counyers are not perfect either as the kernel does IIRC not update them immediately but in a delayed fashion that is computationally cheaper. But for our purpose that is plenty "good enough"...


>  That
> tool is reporting false information because it must, it doesn't know
> the other consumers on the network.  It's 'truest' when the network is
> quiet but the more talkers the more the tool lies.

	[SM] A tool does not lie, the interpretation of the reading becomes trickier, but that is a problem of te user, not the tool. If i use a hammer to hammer in a screwm blame me, not the hammer or the screw). ;)

> 
> cake, the kernel, and the wan port all have real info, the speed test
> tool does not.

	[SM] THis is not where we started... and not what cake-autorate does, it also does not convince me that capacity tests are useless. I happily concede that they are neither 100% accurate, precise or reliable. There is a contimuum between 100% correct and useless ;)

Regards
	Sebastian