revolutions per minute - a new metric for measuring responsiveness
 help / color / mirror / Atom feed
From: Christoph Paasch <cpaasch@apple.com>
To: "Bjørn Mork" <bjorn@mork.no>
Cc: Paul Spooren <mail@aparcar.org>, Rpm <rpm@lists.bufferbloat.net>,
	openwrt-devel@lists.openwrt.org
Subject: Re: [Rpm] Seeking RPM Server package for OpenWrt
Date: Thu, 24 Mar 2022 13:15:20 -0700	[thread overview]
Message-ID: <C05623BD-4C86-427A-B726-5EF042FD2F0F@apple.com> (raw)
In-Reply-To: <87y2101ni8.fsf@miraculix.mork.no>



> On Mar 23, 2022, at 12:23 PM, Bjørn Mork <bjorn@mork.no> wrote:
> 
> Christoph Paasch <cpaasch@apple.com> writes:
> 
>> Hello Bjorn,
>> 
>> Thanks for taking a look at this! Please see inline:
>> 
>>> On Mar 23, 2022, at 5:34 AM, Bjørn Mork via Rpm <rpm@lists.bufferbloat.net> wrote:
>>> 
>>> Paul Spooren <mail@aparcar.org> writes:
>>> 
>>>> The spec wants a 8GB file which seems a bit much for common home
>>>> routers. We could look into reading from /dev/zero since the body
>>>> content isn’t relevant but still the device is likely slower at
>>>> offering the content than your laptop can chew. A dedicated device
>>>> could be required.
>>> 
>>> There is no need to read anything from a file or device. You can just
>>> serve the same memory buffer in a loop.
>> 
>> That's right! It does not really need to be a file. Some webserver
>> implementations have such a capability to generate random content in
>> memory. (e.g.,
>> https://docs.trafficserver.apache.org/en/9.0.x/admin-guide/plugins/generator.en.html
>> <https://docs.trafficserver.apache.org/en/9.0.x/admin-guide/plugins/generator.en.html>)
>> 
>>> I did a quick look at the document and it seems under-specified. Page
>>> after page with blah-blah, but
>>> - not defining Content-Type for any of the URLs
>> 
>> In what way is the content-type relevant for the responsiveness measurement ?
> 
> It becomes relevant once one of the client or server implementations
> start making assumptions about it. Worst case is that you have two
> implementations making different assumptions. So you specify strict
> requirments to avoid that.

We could add a line saying that the content-type does not matter and must be ignored.

> This is pretty basic for any API. Maybe use OpenAPI or similar to
> for clarity instead of the home-grown API spec? 

I feel like OpenAPI is quite a different beast compared to what we are doing here.

>>> - not defining ciphers or any other TLS options, despite the rather
>>> restrictive TLSv1.3 requirment
>> 
>> I'm not sure in what way the cipher-suites are relevant to the
>> responsiveness measurement itself. In terms of deployment, it is the
>> same as for any other webservice. It is something that is usually not
>> specified in an IETF-draft as cipher-suites come and go.
> 
> They're relevant the same way the Content-Type is. If you don't say
> anything then you might end up with all sorts of incompatible
> configurations.

This is a deployment problem, not a specification-problem for the particular purpose of measuring responsiveness.

For example, we don't specify whether IPv4 or IPv6 should be used.
Also, if we list cipher-suites and one of them gets deprecated our RFC would need to be updated as well.

>> The TLSv1.3 requirement comes from the fact that we want to measure
>> TLS handshake latency, and by requiring TLSv1.3 we know that the
>> handshake is exactly 1 round-trip. Probably something to clarify in
>> the draft! I filed
>> https://github.com/network-quality/draft-ietf-ippm-responsiveness/issues/37
>> <https://github.com/network-quality/draft-ietf-ippm-responsiveness/issues/37>.
> 
> Yes, that makes sense. Thanks for explaining. And I believe you should
> include the explanation in the draft as well.
> 
>>> - no config examples for common web servers
>> 
>> It is uncommon for an IETF-draft to provide such kind of
>> configurations, because IETF-drafts are aiming to be implementation
>> independent as implementations change, but standards don't. We have
>> several configurations (and two implementations - one in Go and one in
>> Swift) available at https://github.com/network-quality/server/
>> <https://github.com/network-quality/server/>.
> 
> I believe it's common to include a reference implementation if it's
> semi-trivial, like the server side of this spec is.
> 
> And it's not unheard of that this reference implementation is given as
> configuration examples, in cases where the documenent can be implemented
> by configuring existing software. For example:
> https://datatracker.ietf.org/doc/html/rfc8806

Yeah, it could make sense to put something in the appendix (https://github.com/network-quality/draft-ietf-ippm-responsiveness/issues/39)

> 
> Now I must admit that I haven't actually tried. But I assume it's
> possible to use most web servers for this purpose. Or a pretty simple
> python script, maybe. 
> 
>>> - no actual client algorithm
>> 
>> Section 4 of the draft tries to explain the client algorithm. With
>> specifically Section 4.1.4 formalizing the "working conditions"
>> generation. Can you explain a bit more what parts are unclear to you?
> 
> Re-reading this, I realize that I went out to harsh here. Sorry.
> 
> I think it can be improved by replacing things like
> 
> "It is left to the implementation what to do when saturation is not
> reached within that time-frame."
> 
> with a precise description of what to do.

There are two approaches here:

Either, the implementation aborts and errors out.
Or, the implementation nevertheless measures the responsiveness and either presents the result as a valid result or as a result with a low confidence score.

We can probably outline the options that an implementation has. (https://github.com/network-quality/draft-ietf-ippm-responsiveness/issues/40)


Thanks for your feedback,
Christoph



  parent reply	other threads:[~2022-03-24 20:15 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-23 11:28 Rich Brown
2022-03-23 12:05 ` Paul Spooren
2022-03-23 12:34   ` Bjørn Mork
2022-03-23 12:57     ` Rich Brown
2022-03-23 13:02       ` Paul Spooren
2022-03-23 18:40     ` Christoph Paasch
2022-03-23 19:23       ` Bjørn Mork
2022-03-24  3:58         ` Will Hawkins
2022-03-24 20:15         ` Christoph Paasch [this message]
2022-03-23 12:39   ` Rich Brown

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.bufferbloat.net/postorius/lists/rpm.lists.bufferbloat.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=C05623BD-4C86-427A-B726-5EF042FD2F0F@apple.com \
    --to=cpaasch@apple.com \
    --cc=bjorn@mork.no \
    --cc=mail@aparcar.org \
    --cc=openwrt-devel@lists.openwrt.org \
    --cc=rpm@lists.bufferbloat.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox