Re: [Starlink] Microstate Accounting and the Nyquist problem

[Dave Taht <davet@teklibre.net>]

[-- Attachment #1: Type: text/plain, Size: 6172 bytes --]



> On Jun 11, 2021, at 3:34 PM, Mike Puchol <mike@starlink.sx> wrote:
> 
> We know that Starlink recalculates topology every 15 seconds (this guy, who obviously has way too much spare time, came up with an indirect observation of this interval: https://blog.beerriot.com/2021/02/14/starlink-raster-scan/ <https://blog.beerriot.com/2021/02/14/starlink-raster-scan/> )
> 
> If we could align with this, we could at least know when potential changes in path delays happen, and try to observe other changes that happen at a similar cadence.
> 
> Other thoughts, try to plug more details out of the gRPC data, setup GPS-synced probes with a device at the exit PoP, measure differences between time-sync probes to an array of endpoints.
> 

It’s ironic that the device has to have gps in it, and thus should be  able to provide perfect time to clients directly behind it, isn’t.

I haven’t captured a dhcp or dhcpv6 transaction yet myself,
do they have a ntp option?

What gps software or driver might they have used? (esr’s gpsd is quite popular, but there are others) 

What’s the gps chip?


> Has nobody attacked the JTAG connector on a Dishy yet?
> 
> Best,
> 
> Mike
> On Jun 12, 2021, 00:14 +0200, David Collier-Brown <davecb.42@gmail.com>, wrote:
>> OK, Oh Smarter Colleagues, the challenge to you is to say if there is a "natural" place to capture state changes to get the data we want, and if so, is it common or similar enough between drivers to be worthy of attention?
>> 
>> --dave
>> 
>> On 2021-06-09 9:15 a.m., Dave Taht wrote:
>>> 
>>> 
>>>> Begin forwarded message:
>>>> 
>>>> From: David Collier-Brown <davecb.42@gmail.com <mailto:davecb.42@gmail.com>>
>>>> Subject: Microstate Accounting and the Nyquist problem
>>>> Date: June 9, 2021 at 4:44:14 AM PDT
>>>> To: Dave Taht <davet@teklibre.net <mailto:davet@teklibre.net>>
>>>> Cc: Dave Collier-Brown <dave.collier-brown@indexexchange.com <mailto:dave.collier-brown@indexexchange.com>>
>>>> Reply-To: davecb@spamcop.net <mailto:davecb@spamcop.net>
>>>> 
>>>> A million years ago (roughly around Solaris 9), Sun was suffering from the same problems in measuring their dispatcher as you are with "sloshing".
>>>> 
>>>> A CPU would be 100% busy in one microsecond, 10% busy in the next gazillion, and the average CPU utilization for our sample period would be maybe 10.1, if the sampler happened to sample right when the spike was happening.
>>>> 
>>>> This was utterly useless for things like the fair-share scheduler, so it got fixed in Solaris 10, by having the dispatcher record the time a process (well, kernel thread) had spent in a state when the state changed.
>>>> 
>>>> Initially "microstate accounting" could be toggled on and off, but the branch-around cost more time than always doing the calculation (as discovered by my mad friend Fred) and the kernel folks left it on. It's on to this day.
>>>> 
>>>> In Simon Sundberg's talk, the opportunity to measure occurs every 1,000 packets, when a suitable timestamp is provided. While the eBPF program can look at every packet and do after-the-fact book-keeping in a map, that's only good if the phenomenon you're measuring is persistent enough that it's around for ~2,000 packets.
>>>> 
>>>> I'm going to suggest that the right place to record the information you want is right where the event happens.  Preferably in c code, as performance is easy to mess up, but perhaps with an eBPF mechanism to export it.
>>>> 
>>>> In previous Solaris work, I reliably found that exporting kstats was a darn sight harder than collecting them, and in Eric's blog post[1] he notes that converting time is expensive and best done long after collecting, when someone wanted to read the data.
>>>> 
>>>> There was an effort to do kstats in Linux[2], but it had supposedly poor performance, and actual trouble when the clock frequency changed.
>>>> 
>>>> Is there, in your opinion, a "natural" place to capture state changes to get the data you want, and if so, is it common or similar enough between drivers to be worthy of attention?
>>>> 
>>>> --dave
>>>> 
>>>> 
>>>> 
>>>> References:
>>>> 
>>>> Solaris: http://dtrace.org/blogs/eschrock/2004/10/13/microstate-accounting-in-solaris-10/ <https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdtrace.org%2Fblogs%2Feschrock%2F2004%2F10%2F13%2Fmicrostate-accounting-in-solaris-10%2F&data=04%7C01%7C%7C7f7cd5aab2ca42e2e7e908d92d25e27f%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C637590463000477252%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=fdZDOtRCcBk%2BO1ksiTOSU%2FltR8IMwueHyj0kQG4UkHw%3D&reserved=0>
>>>> A failing Linux effort: https://lwn.net/Articles/127296/ <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flwn.net%2FArticles%2F127296%2F&data=04%7C01%7C%7C7f7cd5aab2ca42e2e7e908d92d25e27f%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C637590463000487248%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=uN0gq8vi0GJHMPpjKVYRjX6G5nQOc%2BugxUwUEk3%2BWJ8%3D&reserved=0>, https://sourceforge.net/projects/microstate/ <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsourceforge.net%2Fprojects%2Fmicrostate%2F&data=04%7C01%7C%7C7f7cd5aab2ca42e2e7e908d92d25e27f%7Cb07c069022b843668d8d7b845d088e18%7C1%7C0%7C637590463000497242%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=iMNi40Pl9hMmd1h7WrLFP5jmHQ60mJl7zehhO8miJv4%3D&reserved=0>
>>>> --  
>>>> David Collier-Brown,         | Always do right. This will gratify
>>>> System Programmer and Author | some people and astonish the rest
>>>> davecb@spamcop.net <mailto:davecb@spamcop.net>           |                      -- Mark Twain
>>> 
>> _______________________________________________
>> Starlink mailing list
>> Starlink@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/starlink
> _______________________________________________
> Starlink mailing list
> Starlink@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/starlink


[-- Attachment #2: Type: text/html, Size: 11756 bytes --]