<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">Begin forwarded message:</div><br class="Apple-interchange-newline"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">From: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">David Collier-Brown <<a href="mailto:davecb.42@gmail.com" class="">davecb.42@gmail.com</a>><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Subject: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class=""><b class="">Microstate Accounting and the Nyquist problem</b><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Date: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">June 9, 2021 at 4:44:14 AM PDT<br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">To: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">Dave Taht <<a href="mailto:davet@teklibre.net" class="">davet@teklibre.net</a>><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Cc: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">Dave Collier-Brown <<a href="mailto:dave.collier-brown@indexexchange.com" class="">dave.collier-brown@indexexchange.com</a>><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Reply-To: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class=""><a href="mailto:davecb@spamcop.net" class="">davecb@spamcop.net</a><br class=""></span></div><br class=""><div class="">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" class="">
<div class=""><p class="">A million years ago (roughly around Solaris 9), Sun was suffering
from the same problems in measuring their dispatcher as you are
with "sloshing".</p><p class="">A CPU would be 100% busy in one microsecond, 10% busy in the next
gazillion, and the average CPU utilization for our sample period
would be <i class="">maybe</i> 10.1, if the sampler happened to sample
right when the spike was happening.</p><p class="">This was utterly useless for things like the fair-share
scheduler, so it got fixed in Solaris 10, by having the dispatcher
record the time a process (well, kernel thread) had spent in a
state when the state changed.<br class="">
</p><p class="">Initially "microstate accounting" could be toggled on and off,
but the branch-around cost more time than always doing the
calculation (as discovered by my mad friend Fred) and the kernel
folks left it on. It's on to this day.</p><p class="">In Simon Sundberg's talk, the opportunity to measure occurs every
1,000 packets, when a suitable timestamp is provided. While the
eBPF program can look at every packet and do after-the-fact
book-keeping in a map, that's only good if the phenomenon you're
measuring is persistent enough that it's around for ~2,000
packets.</p><p class="">I'm going to suggest that the right place to record the
information you want is right where the event happens. Preferably
in c code, as performance is easy to mess up, but perhaps with an
eBPF mechanism to export it.</p><p class="">In previous Solaris work, I reliably found that exporting kstats
was a darn sight harder than collecting them, and in Eric's blog
post[1] he notes that converting time is expensive and best done
long after collecting, when someone wanted to read the data.</p><p class="">There was an effort to do kstats in Linux[2], but it had
supposedly poor performance, and actual trouble when the clock
frequency changed.<br class="">
</p><p class="">Is there, in your opinion, a "natural" place to capture state
changes to get the data you want, and if so, is it common or
similar enough between drivers to be worthy of attention?</p><p class="">--dave<br class="">
</p><p class=""><br class="">
</p><p class="">References:</p>
<ol class="">
<li class="">Solaris:
<a class="moz-txt-link-freetext" href="http://dtrace.org/blogs/eschrock/2004/10/13/microstate-accounting-in-solaris-10/">http://dtrace.org/blogs/eschrock/2004/10/13/microstate-accounting-in-solaris-10/</a>
<br class="">
</li>
<li class="">A failing Linux effort: <a class="moz-txt-link-freetext" href="https://lwn.net/Articles/127296/">https://lwn.net/Articles/127296/</a>,
<a class="moz-txt-link-freetext" href="https://sourceforge.net/projects/microstate/">https://sourceforge.net/projects/microstate/</a><br class="">
</li>
</ol>
<pre class="moz-signature" cols="72">--
David Collier-Brown, | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
<a class="moz-txt-link-abbreviated" href="mailto:davecb@spamcop.net">davecb@spamcop.net</a> | -- Mark Twain
</pre>
</div>
</div></blockquote></div><br class=""></body></html>