From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 3088F3B25E for ; Sat, 27 Aug 2016 15:18:21 -0400 (EDT) Received: by mail-wm0-x231.google.com with SMTP id o80so35898593wme.1 for ; Sat, 27 Aug 2016 12:18:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=DNZbziSMKsb3iiO30vTOM781JQZpBAmwyXFRpsFYFlY=; b=ZRmwqGDarkt7EauZZsODnrGrk4FZeb9jCUOHT2vdJkKICGNhXx2Jvf98TjskSa+rVC nD5ZG4iXvo4/n4rbvntKvAXdjxLV8Xe5G2SDcyH9P4GTFuCFcVkToruOtHzrTvgPMEQu pafvMnCLVWnNkEuso7q0Zmypg+GnKKq1T+JClT70ldsNcpqQi9szERL4CfQM3gXCzsw1 YpiTpHc6rwGMqCbJyD7ZYMEc7n9BuyiAz2P/Bt9OH26izXj6gUICuHef/KFIH522+G+l IaI2i5wGjAh1mFSAGIssx4A2x/swrCjIYoPrU+FfsKGtr6YitG5YRqYshoiJGjkggvyw LW1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=DNZbziSMKsb3iiO30vTOM781JQZpBAmwyXFRpsFYFlY=; b=j5ev5nMxjhlfcDdUyU3R6Itj5eTmE4dqaBEMQ+q8arm8oHNUGXgz0UrrqVe5OQf4NH Xv9epXPE9GCAN4HBqd1aA7kjdKrLbrW1rM5oTH9hkD5i1b4Gxy4ciiBZ2lSPIWH3zjXA Echw02g/8HJNpovuHVtxZ7PMKUTMWsJDRUPqsnYGN4PDaGXwKpvTIugeb4GFq/bBO5ZD vYjwZkk0jqKhRIDFOlr2FbwOIFdlPwhZLOw8JjGso8KnNhC2DJf5Gim5V/wIvZ/3yR/2 Cof4BeinZ1pyEFDILxdKJKDt8B/kFAb3BkzfvzMkIMJyeGW11k+dp3mBPFR8S8a+DlNA 3C1Q== X-Gm-Message-State: AE9vXwM/t/vt9QF7wRyH5G9KI843/WuRcN6qQ3O30Ki5KtTSAQQVuvBXx5VfU9EFyzpuzw== X-Received: by 10.195.12.77 with SMTP id eo13mr9169903wjd.142.1472325500185; Sat, 27 Aug 2016 12:18:20 -0700 (PDT) Received: from localhost.localdomain (host-89-243-172-136.as13285.net. [89.243.172.136]) by smtp.googlemail.com with ESMTPSA id u2sm5047845wmf.5.2016.08.27.12.18.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 27 Aug 2016 12:18:19 -0700 (PDT) To: Kathleen Nichols , bloat@lists.bufferbloat.net References: <05056A85-894D-477F-A10D-C912C7D52C2F@gmail.com> <448a25be-160f-701e-f56b-b3a33d49cff8@pollere.com> <63403b38-94fa-1670-908c-e14573f822a8@gmail.com> <8971f9f1-a79b-ce49-f7c4-660a355fdd43@pollere.com> From: Alan Jenkins Message-ID: <1091de30-017f-e6ee-a440-cd8425be0e1a@gmail.com> Date: Sat, 27 Aug 2016 20:18:16 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <8971f9f1-a79b-ce49-f7c4-660a355fdd43@pollere.com> Content-Type: multipart/alternative; boundary="------------D11355EB5895CA36919DD6CA" Subject: Re: [Bloat] Bufferbloat test measurements X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Aug 2016 19:18:21 -0000 This is a multi-part message in MIME format. --------------D11355EB5895CA36919DD6CA Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit On 27/08/16 18:37, Kathleen Nichols wrote: > In-line below. Only for geeks. Present. > On 8/27/16 9:03 AM, Alan Jenkins wrote: >> That's the simplest measure of bufferbloat though :). > Don't I know! :) Have spent a couple of years figuring out how to measure > experienced delay... >> Do you have a criticism in terms of dslreports.com? I think it's fairly >> transparent, showing idle v.s. download v.s. upload. The headline >> figures are an average, and you can look at all the data points. (You >> can increase the measurement frequency if you're specifically interested >> in that). > "Criticism" seems too harsh. The uplink and downlink speed stuff is great > and agrees with all other measures. The "bufferbloat" grade is, I think, > sort > of confusing. Also, it's not clear where the queue the test builds up is > located. > It could be in ISP or home. So, I ran the test while I was also streaming a > Netflix video. Under the column "RTT/jitter Avg" the test lists values that > range from 654 to 702 with +/- 5.2 to 20.8 ms (for the four servers). I > couldn't > figure out what that means. My assumption is the RTT is just read out from the TCP socket, i.e. it's one of the kernel statistics. http://stackoverflow.com/questions/16231600/fetching-the-tcp-rtt-in-linux/16232250#16232250 Looking in `ss.c` as suggested, the second figure shown by `ss` is `rttvar`. And that's the kernel's measure of RTT variation. If my assumption is right, that would tell us where the "jitter" figure comes from as well. > I can see the delay ramp up over the test > (the video > stream also follows that ramp though it's normal delay ranges from about > 1ms to > about 40ms). If I took the RT delay experienced by the packets to/from > those servers, > I got median values between 391 and 429 ms. The IQRs were about 240ms to > 536-616ms. The maximum values where all just over 700ms, agreeing with > the dslreports > number. But that number is listed as an average so I don't understand? > Also what is the > jitter. I looked around for info on the numbers but I didn't find it. > Probably I just totally > missed some obvious thing to click on. > > But the thing is, I've been doing a lot of monitoring of my link and I > don't normally see > those kinds of delays. In the note I put out a bit ago, there is > definitely this bursting > behavior that is particularly indulged in by netflix and google but when > our downstream > bandwidth went up, those bursts no longer caused such long transient > delays (okay, duh). > So, I'm not sure who should be tagged as the "responsible party" for the > grade that the > test gives. Nor am I convinced that users have to assume that means they > are going to see > those kinds of delays. But this is a general issue with active measurement. > > It's not a criticism of the test, but maybe of the presentation of the > result. > > Kathie Thanks. I wasn't clear what you meant the first time, particularly about the 40ms figure. Very comprehensive explanation of your point. It's easy to rant about ubiquitous dumb FIFO's in consumer equipment :). Whereas the queue on the ISP side of the link ("download"), can be more complex and varied. Mine gets good results on dslreports.com, but the latency isn't always so good during torrent downloads. I do have doubts about the highly "multi-threaded" test method. dslreports let you dial it down manually (Preferences). As you say, a better real-world test is to just use your connection normally & run smokeping... or an online "line monitor" like http://www.thinkbroadband.com/ping :). Alan --------------D11355EB5895CA36919DD6CA Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit On 27/08/16 18:37, Kathleen Nichols wrote:
In-line below. Only for geeks.
Present.

On 8/27/16 9:03 AM, Alan Jenkins wrote:

That's the simplest measure of bufferbloat though :).

Don't I know! :)  Have spent a couple of years figuring out how to measure
experienced delay...

Do you have a criticism in terms of dslreports.com?  I think it's fairly
transparent, showing idle v.s. download v.s. upload.  The headline
figures are an average, and you can look at all the data points.  (You
can increase the measurement frequency if you're specifically interested
in that).

"Criticism" seems too harsh. The uplink and downlink speed stuff is great
and agrees with all other measures. The "bufferbloat" grade is, I think,
sort
of confusing. Also, it's not clear where the queue the test builds up is
located.
It could be in ISP or home. So, I ran the test while I was also streaming a
Netflix video. Under the column "RTT/jitter Avg" the test lists values that
range from 654 to 702 with +/- 5.2 to 20.8 ms (for the four servers). I
couldn't
figure out what that means.

My assumption is the RTT is just read out from the TCP socket, i.e. it's one of the kernel statistics.

http://stackoverflow.com/questions/16231600/fetching-the-tcp-rtt-in-linux/16232250#16232250

Looking in `ss.c` as suggested, the second figure shown by `ss` is `rttvar`.  And that's the kernel's measure of RTT variation.  If my assumption is right, that would tell us where the "jitter" figure comes from as well.

 I can see the delay ramp up over the test
(the video
stream also follows that ramp though it's normal delay ranges from about
1ms to
about 40ms). If I took the RT delay experienced by the packets to/from
those servers,
I got median values between 391 and 429 ms. The IQRs were about 240ms to
536-616ms. The maximum values where all just over 700ms, agreeing with
the dslreports
number. But that number is listed as an average so I don't understand?
Also what is the
jitter. I looked around for info on the numbers but I didn't find it.
Probably I just totally
missed some obvious thing to click on.

But the thing is, I've been doing a lot of monitoring of my link and I
don't normally see
those kinds of delays. In the note I put out a bit ago, there is
definitely this bursting
behavior that is particularly indulged in by netflix and google but when
our downstream
bandwidth went up, those bursts no longer caused such long transient
delays (okay, duh).
So, I'm not sure who should be tagged as the "responsible party" for the
grade that the
test gives. Nor am I convinced that users have to assume that means they
are going to see
those kinds of delays. But this is a general issue with active measurement.

It's not a criticism of the test, but maybe of the presentation of the
result.

	Kathie

Thanks.  I wasn't clear what you meant the first time, particularly about the 40ms figure.  Very comprehensive explanation of your point.

It's easy to rant about ubiquitous dumb FIFO's in consumer equipment :).  Whereas the queue on the ISP side of the link ("download"), can be more complex and varied.  Mine gets good results on dslreports.com, but the latency isn't always so good during torrent downloads.

I do have doubts about the highly "multi-threaded" test method.  dslreports let you dial it down manually (Preferences).  As you say, a better real-world test is to just use your connection normally & run smokeping... or an online "line monitor" like http://www.thinkbroadband.com/ping :).

Alan
--------------D11355EB5895CA36919DD6CA--