Re: [Rpm] apm metric - annoyance per minute

[Dave Taht <dave.taht@gmail.com>]

On Tue, Jan 11, 2022 at 11:44 AM Dave Taht <dave.taht@gmail.com> wrote:
>
> On Tue, Jan 11, 2022 at 9:51 AM Aaron Wood <woody77@gmail.com> wrote:
> >
> > I read it as the number of events per minute (not say the number of frames longer than 20ms, but the number of events that took at least 20ms longer than the base RTT, which I think is what Dave meant by "latency excursion").
>
> yes. thx for reading my tea leaves. The big thing to me was
> "annoyance" or "glitch" per minute somewhat in line of rpm's concept.
>
> Hey, this network does 2500RPM but with .5APM:
> https://blog.cerowrt.org/post/disabling_channel_scans/
>
> On my holiday trip, mostly staying in cheap hotels, not *one* hotel
> out of 6 could sustain a quality videoconference. 10-20GPM,
>  but web pages and netflix loaded fine.
>
> >
> > E.g. if you're doing DNS queries, and they usually return in 50ms, and 3 of them take 83, 94, and 106 ms respectively, in a given minute, than that would be an APM of 3?  ( or maybe an APM rate of 30% if you were doing 10/minute)
>
> Yes. You count the excursions from the (semi-smoothed) baseline, not
> the size of the excursion. A "glitch" happened.
>
> > I've found the tricky thing for metrics is sorting out the event-count vs. events-rate differences.  A lot of tests that are isochronous give a constant event-rate to base on (e.g. ping's default of once per second), but other tests, like the UDP and ICMP pings in flent (at least in the past), have a rate that's based on the RTT, so as RTT goes up, the rate of events goes down, which means that it oversamples the "fast" events, and undersamples "slow" events.
>
> The original rrul spec had an isochronous voip like flow, not ping rtt
> test here., which has the annoying flaws you describe above. Which we
> now have in the irtt tool, but most of our tests still use ping. At
> the time (when we were shooting for reductions of latency from seconds
> to 10s of ms) using "ping" wasn't as much of a problem as it is today.
> We need a rrul_v2 and a tcp_nup, tcp_ndown tests that just do
> isochronous flows at fixed (and ideally high frequency, irtt works
> well to about 3ms, opus codec can do 2.7ms)
>
> > Further, retries for failed events muddy the waters, as the events aren't independent measurements.  If a momentary drop in connectivity causes retries to happen, and each failed retry is counted, is that N failures?  Or just 1 failure?  I've split those out as separate metrics in some systems I've built, so that I can tease them apart.  I've also done things like the distribution (histogram) of "attempts before success" or "attempts before operation failed".  Usually those are dominated by "1 attempt before success", and "N attempts before operation failed" where N is the number of total attempts before just giving up.
>
> Histograms are great. I kind of wanted to separate the concepts that a
> "glitch" happened, and also measure the glitch duration (so X retries
> turns into a duration rather than a count), and (sigh) whether the
> glitch mattered or not. It doesn't matter to a web page if you have an
> 250ms RTO on one flow but it takes 3sec to load anyway.
>
> glitches matter more for videoconferencing and gaming. I don't know if
> there is any human factors research on this, but once I find my flow
 > in an application, a 20ms 'glitch' is roughly as annoying as a 3second
> long one.

Glitches matter even more for music and video playback. Anyone who's
ever experienced a pink floyd record as a whole rather than ripped
into tracks is an example, very few players get the transitions
between songs correct (all my floyd records I ripped as a single
song). Coincidentally I was trying to work out a rube goldberg machine
for a web page load and st ubmedl ed

https://www.youtube.com/watch?v=WyOSqjIABe0

...

stumbled across this wonderful ok-go video about what a 90% and 99%
reliability rate meant for the successive success of 130 events. And
while I was typing this, my lte connection glitched
and I ended up typing "st ubmedl ed" whilst it caught up. I don't know
how many people suffer from glitches as badly as I do - as one example
I turn spellchekcing off when working with a real editor and there's
no way to do that with gmail, I think - but finding and keeping my
flow doesn't last very long when my services "glitch" like this.

>
> >
> > On Tue, Jan 11, 2022 at 9:34 AM Christoph Paasch via Rpm <rpm@lists.bufferbloat.net> wrote:
> >>
> >> Hi Dave!
> >>
> >> > On Jan 9, 2022, at 6:57 PM, Dave Taht via Rpm <rpm@lists.bufferbloat.net> wrote:
> >> >
> >> > or gpm - glitch per minute
> >> >
> >> > defined as a latency excursion of more than 20ms.
> >>
> >> I kinda find that interesting :) Can you give an example? Would it count the number of times we miss a "20ms-deadline"? So, if the RTT is 100ms, GPM would be 5 ?
> >>
> >>
> >> Christoph
> >>
> >> >
> >> > ?
> >> >
> >> >
> >> > --
> >> > I tried to build a better future, a few times:
> >> > https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
> >> >
> >> > Dave Täht CEO, TekLibre, LLC
> >> > _______________________________________________
> >> > Rpm mailing list
> >> > Rpm@lists.bufferbloat.net
> >> > https://lists.bufferbloat.net/listinfo/rpm
> >>
> >> _______________________________________________
> >> Rpm mailing list
> >> Rpm@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/rpm
>
>
>
> --
> I tried to build a better future, a few times:
> https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org
>
> Dave Täht CEO, TekLibre, LLC



-- 
I tried to build a better future, a few times:
https://wayforward.archive.org/?site=https%3A%2F%2Fwww.icei.org

Dave Täht CEO, TekLibre, LLC