[Cerowrt-devel] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Cerowrt-devel] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Dave Taht]

The theme of networks being "engineered for speedtest" has been a
common thread in nearly  every conversation I've had with ISPs and
vendors using every base technology out there, be it dsl, cable,
ethernet, or fiber, for the last 4 years. Perhaps, in pursuing better
code, and RFCs, and the like, we've been going about fixing
bufferbloat the wrong way.

If Verizon can petition the FCC to change the definition of
broadband... why can't we petition speedtest to *change their test*?
Switching to merely reporting the 98th percentile results for ping
during an upload or download, instead of the baseline ping, would be a
vast improvement on what happens today, and no doubt we could suggest
other improvements.

What if we could publish an open letter to the benchmark makers such
as speedtest, explaining how engineering for their test does *not*
make for a better internet? The press fallout from that letter, would
improve some user education, regardless if we could get the tests
changed or not.

Who here would sign?


On Wed, Sep 10, 2014 at 2:54 PM, Joel Wirāmu Pauling <joel@aenertia.net> wrote:
> I have been heavily involved with the UFB (Ultrafast Broadband) PON
> deployment here in New Zealand.
>
> I am not sure how the regulated environment is playing out in Canada
> (I am moving there in a month so I guess I will find out). But here
> the GPON architecture is METH based and Layer2 only. Providers (RSP's)
> are the ones responsible for asking for Handoffer buffer tweaks to the
> LFC(local fibre companies; the layer 0-2 outfits-) which have mandated
> targets for Latency (at most 4.5ms) accross their PON Access networks
> to the Handover port.
>
> Most of the time this has been to 'fix' Speedtest.net TCP based
> results to report whatever Marketed service (100/30 For example) is in
> everyones favourite site speedtest.net.
>
> This has meant at least for the Chorus LFC regions where they use
> Alcatel-Lucent 7450's as the handover/aggregation switches we have
> deliberately introduced buffer bloat to please the RSP's - who
> otherwise get whingy about customers whinging about speedtest not
> showing 100/30mbit. Of course user education is 'too hard' .

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Pedro Tumusok]

[-- Attachment #1: Type: text/plain, Size: 2854 bytes --]

Would it be possible to create this alternative test site, to show end
users that speed is not everything.

If you get a result with some comments, ala netalyzr, most people would
probably be happy with that?
Or just a netalyzr lite maybe?

Pedro

On Thu, Sep 11, 2014 at 6:03 PM, Dave Taht <dave.taht@gmail.com> wrote:

> The theme of networks being "engineered for speedtest" has been a
> common thread in nearly  every conversation I've had with ISPs and
> vendors using every base technology out there, be it dsl, cable,
> ethernet, or fiber, for the last 4 years. Perhaps, in pursuing better
> code, and RFCs, and the like, we've been going about fixing
> bufferbloat the wrong way.
>
> If Verizon can petition the FCC to change the definition of
> broadband... why can't we petition speedtest to *change their test*?
> Switching to merely reporting the 98th percentile results for ping
> during an upload or download, instead of the baseline ping, would be a
> vast improvement on what happens today, and no doubt we could suggest
> other improvements.
>
> What if we could publish an open letter to the benchmark makers such
> as speedtest, explaining how engineering for their test does *not*
> make for a better internet? The press fallout from that letter, would
> improve some user education, regardless if we could get the tests
> changed or not.
>
> Who here would sign?
>
>
> On Wed, Sep 10, 2014 at 2:54 PM, Joel Wirāmu Pauling <joel@aenertia.net>
> wrote:
> > I have been heavily involved with the UFB (Ultrafast Broadband) PON
> > deployment here in New Zealand.
> >
> > I am not sure how the regulated environment is playing out in Canada
> > (I am moving there in a month so I guess I will find out). But here
> > the GPON architecture is METH based and Layer2 only. Providers (RSP's)
> > are the ones responsible for asking for Handoffer buffer tweaks to the
> > LFC(local fibre companies; the layer 0-2 outfits-) which have mandated
> > targets for Latency (at most 4.5ms) accross their PON Access networks
> > to the Handover port.
> >
> > Most of the time this has been to 'fix' Speedtest.net TCP based
> > results to report whatever Marketed service (100/30 For example) is in
> > everyones favourite site speedtest.net.
> >
> > This has meant at least for the Chorus LFC regions where they use
> > Alcatel-Lucent 7450's as the handover/aggregation switches we have
> > deliberately introduced buffer bloat to please the RSP's - who
> > otherwise get whingy about customers whinging about speedtest not
> > showing 100/30mbit. Of course user education is 'too hard' .
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>



-- 
Best regards / Mvh
Jan Pedro Tumusok

[-- Attachment #2: Type: text/html, Size: 3664 bytes --]

Re: [Cerowrt-devel] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Maciej Soltysiak]

On Thu, Sep 11, 2014 at 6:03 PM, Dave Taht <dave.taht@gmail.com> wrote:
> If Verizon can petition the FCC to change the definition of
> broadband... why can't we petition speedtest to *change their test*?
> Switching to merely reporting the 98th percentile results for ping
> during an upload or download, instead of the baseline ping, would be a
> vast improvement on what happens today, and no doubt we could suggest
> other improvements.
Sounds like a very good idea. The latency would finally differ and
hopefully lead to more meaningful discussions. Or at least be grounds
to it.

> Who here would sign?
I would!

Best regards,
Maciej Soltysiak

Re: [Cerowrt-devel] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[David Personette]

[-- Attachment #1: Type: text/plain, Size: 1011 bytes --]

FWIW, I'll sign it.

-- 
Please excuse my brevity, this was sent from my Android.
On Sep 11, 2014 2:20 PM, "Maciej Soltysiak" <maciej@soltysiak.com> wrote:

> On Thu, Sep 11, 2014 at 6:03 PM, Dave Taht <dave.taht@gmail.com> wrote:
> > If Verizon can petition the FCC to change the definition of
> > broadband... why can't we petition speedtest to *change their test*?
> > Switching to merely reporting the 98th percentile results for ping
> > during an upload or download, instead of the baseline ping, would be a
> > vast improvement on what happens today, and no doubt we could suggest
> > other improvements.
> Sounds like a very good idea. The latency would finally differ and
> hopefully lead to more meaningful discussions. Or at least be grounds
> to it.
>
> > Who here would sign?
> I would!
>
> Best regards,
> Maciej Soltysiak
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

[-- Attachment #2: Type: text/html, Size: 1554 bytes --]

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Rich Brown]

[-- Attachment #1: Type: text/plain, Size: 2316 bytes --]

Dave,

I'll sign too. (And I like the 98th percentile measure for each direction to give a single number that represents what's happening. It could include ping loss rate, as well...)

But more importantly, an open letter would likely be more powerful than the results I got from sending a note to the purveyors of speed tests. I've appended the note I sent about six weeks ago. And here's a paraphrase of their responses. (None of them seem to have twigged to the value of measuring bloat.)

speedtest.net - We wanted to follow up and let you know that are developers were informed of your ideas. 

speedof.me - We may add such features in future versions

testmy.net - I think you're really going to like my next version... you'll be provided a much deeper insight. 

There's a flock of other vendors - I just googled "Internet speed test" and got a dozen or so more. Many are served by Ookla/Speedtest.net, but many seem to have independent implementations...

I'll pull together a list of people to send the open letter to (trade press, etc)

Rich

----- Summary of note sent to these speed test vendors in early August 2014 -----
Hi!

I would love to see real-time latency measurements, and a summary of min and max times in the final report. Your page currently displays a single "latency" value, but it appears that it's simply the measurement before you start the data transfers. It's really interesting to see that low value, but also the range/max of the latency. 

Why is latency interesting? I'm a member of the CeroWrt project that is working to reduce latency in home routers (and everywhere else). Our research has led to the development and testing of the fq_codel algorithm that virtually eliminates bufferbloat (high latency during heavy traffic). You can read about the project at: http://www.bufferbloat.net/projects/cerowrt

I'm asking you to consider implementing the web-equivalent of our "Quick Test for Bufferbloat"http://www.bufferbloat.net/projects/cerowrt/wiki/Quick_Test_for_Bufferbloat By showing what happens to latency during transfers, people can become aware of the problem, and that there's a fix for it. (I'm using the CeroWrt firmware at my house. Ping times to Google only go up by ~20-30 msec even when I max out the DSL link in both directions.)



[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

Re: [Cerowrt-devel] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 2709 bytes --]


I will sign.  It would be better if we had an actual demonstration of how to implement a speedtest improvement.
 


On Thursday, September 11, 2014 12:03pm, "Dave Taht" <dave.taht@gmail.com> said:



> The theme of networks being "engineered for speedtest" has been a
> common thread in nearly every conversation I've had with ISPs and
> vendors using every base technology out there, be it dsl, cable,
> ethernet, or fiber, for the last 4 years. Perhaps, in pursuing better
> code, and RFCs, and the like, we've been going about fixing
> bufferbloat the wrong way.
> 
> If Verizon can petition the FCC to change the definition of
> broadband... why can't we petition speedtest to *change their test*?
> Switching to merely reporting the 98th percentile results for ping
> during an upload or download, instead of the baseline ping, would be a
> vast improvement on what happens today, and no doubt we could suggest
> other improvements.
> 
> What if we could publish an open letter to the benchmark makers such
> as speedtest, explaining how engineering for their test does *not*
> make for a better internet? The press fallout from that letter, would
> improve some user education, regardless if we could get the tests
> changed or not.
> 
> Who here would sign?
> 
> 
> On Wed, Sep 10, 2014 at 2:54 PM, Joel Wirāmu Pauling
> <joel@aenertia.net> wrote:
> > I have been heavily involved with the UFB (Ultrafast Broadband) PON
> > deployment here in New Zealand.
> >
> > I am not sure how the regulated environment is playing out in Canada
> > (I am moving there in a month so I guess I will find out). But here
> > the GPON architecture is METH based and Layer2 only. Providers (RSP's)
> > are the ones responsible for asking for Handoffer buffer tweaks to the
> > LFC(local fibre companies; the layer 0-2 outfits-) which have mandated
> > targets for Latency (at most 4.5ms) accross their PON Access networks
> > to the Handover port.
> >
> > Most of the time this has been to 'fix' Speedtest.net TCP based
> > results to report whatever Marketed service (100/30 For example) is in
> > everyones favourite site speedtest.net.
> >
> > This has meant at least for the Chorus LFC regions where they use
> > Alcatel-Lucent 7450's as the handover/aggregation switches we have
> > deliberately introduced buffer bloat to please the RSP's - who
> > otherwise get whingy about customers whinging about speedtest not
> > showing 100/30mbit. Of course user education is 'too hard' .
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

[-- Attachment #2: Type: text/html, Size: 3667 bytes --]

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 2843 bytes --]


Among friends of mine, we can publicize this widely.  But those friends probably would like to see how the measurement would work.


On Thursday, September 11, 2014 8:13pm, "Rich Brown" <richb.hanover@gmail.com> said:



> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> Dave,
> 
> I'll sign too. (And I like the 98th percentile measure for each direction to give
> a single number that represents what's happening. It could include ping loss rate,
> as well...)
> 
> But more importantly, an open letter would likely be more powerful than the
> results I got from sending a note to the purveyors of speed tests. I've appended
> the note I sent about six weeks ago. And here's a paraphrase of their responses.
> (None of them seem to have twigged to the value of measuring bloat.)
> 
> speedtest.net - We wanted to follow up and let you know that are developers were
> informed of your ideas.
> 
> speedof.me - We may add such features in future versions
> 
> testmy.net - I think you're really going to like my next version... you'll be
> provided a much deeper insight.
> 
> There's a flock of other vendors - I just googled "Internet speed test" and got a
> dozen or so more. Many are served by Ookla/Speedtest.net, but many seem to have
> independent implementations...
> 
> I'll pull together a list of people to send the open letter to (trade press, etc)
> 
> Rich
> 
> ----- Summary of note sent to these speed test vendors in early August 2014 -----
> Hi!
> 
> I would love to see real-time latency measurements, and a summary of min and max
> times in the final report. Your page currently displays a single "latency" value,
> but it appears that it's simply the measurement before you start the data
> transfers. It's really interesting to see that low value, but also the range/max
> of the latency.
> 
> Why is latency interesting? I'm a member of the CeroWrt project that is working to
> reduce latency in home routers (and everywhere else). Our research has led to the
> development and testing of the fq_codel algorithm that virtually eliminates
> bufferbloat (high latency during heavy traffic). You can read about the project
> at: http://www.bufferbloat.net/projects/cerowrt
> 
> I'm asking you to consider implementing the web-equivalent of our "Quick Test for
> Bufferbloat"http://www.bufferbloat.net/projects/cerowrt/wiki/Quick_Test_for_Bufferbloat
> By showing what happens to latency during transfers, people can become aware of
> the problem, and that there's a fix for it. (I'm using the CeroWrt firmware at my
> house. Ping times to Google only go up by ~20-30 msec even when I max out the DSL
> link in both directions.)
> 
> 
> 

[-- Attachment #2: Type: text/html, Size: 3676 bytes --]

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Jonathan Morton]

On 12 Sep, 2014, at 3:35 am, dpreed@reed.com wrote:

> Among friends of mine, we can publicize this widely.  But those friends probably would like to see how the measurement would work.

Could we make use of the existing test servers (running netperf) for that demonstration?  How hard is the protocol to fake in Javascript?

Or would a netperf-wrapper demonstration suffice?  We've already got that, but we'd need to extract the single-figures-of-merit from the data.

I wonder if the speedof.me API can already be tricked into doing the right thing?

 - Jonathan Morton

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]


The speedof.me API probably can be used directly as the measurement of download and upload - you can create a competing download or upload in Javascript using a WebWorker talking to another server that supports the websocket API to force buffer overflow.  (sort of poor man's RRUL).
 
The speedof.me API would give you the measured performance, while the other path would just be aan easier to code test load to a source/sink.
 
Not sure that would help, but for a prototype it's not bad.


On Thursday, September 11, 2014 8:42pm, "Jonathan Morton" <chromatix99@gmail.com> said:



> 
> On 12 Sep, 2014, at 3:35 am, dpreed@reed.com wrote:
> 
> > Among friends of mine, we can publicize this widely. But those friends
> probably would like to see how the measurement would work.
> 
> Could we make use of the existing test servers (running netperf) for that
> demonstration? How hard is the protocol to fake in Javascript?
> 
> Or would a netperf-wrapper demonstration suffice? We've already got that, but
> we'd need to extract the single-figures-of-merit from the data.
> 
> I wonder if the speedof.me API can already be tricked into doing the right thing?
> 
> - Jonathan Morton
> 
> 

[-- Attachment #2: Type: text/html, Size: 2166 bytes --]

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Rich Brown]

Jonathan,

> Could we make use of the existing test servers (running netperf) for that demonstration?  How hard is the protocol to fake in Javascript?

Not having coded a stitch of this, I *think* it would require the following:

- Web page on netperf-xxx.bufferbloat.net that served out the javascript (required to get around cross-domain protections within the browser)

- Javascript function to connect back to that host on port 12865 and fake out the netserver with TCP_STREAM or TCP_MAERTS request

- Javascript that's efficient enough to source/swallow full-rate data stream

- Cloning the code from https://github.com/apenwarr/blip to make fake pings from TCP requests

Anyone know more than I do about this?

Rich

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Joel Wirāmu Pauling]

[-- Attachment #1: Type: text/plain, Size: 2161 bytes --]

Changing to better test suites isn't the answer, I know I've tried a lot to
educate and push for bespoke test sites for our customers .

This is all about happy illusions on the user side.

This is a branding and markets problem. In NZ, 'the' site for reference
that users go to is speedtest. Because of that RSPs deploy more speedtest
servers on their networks and you end up with a vicious cycle. Get enough
user driven complaints and you end up having a user driven push to make
your network 'look good' by doing otherwise silly things.

So if ookla implemented a udp based test, changed it's statical weighting
and data mining methods overnight. At least in NZ that might help.

For the RSPs it is always going to be easier to modify a buffer or queue
somewhere. And that is going to make their project managers and
stakeholders happier straight away.

-Joel
On 12 Sep 2014 13:24, <dpreed@reed.com> wrote:

> The speedof.me API probably can be used directly as the measurement of
> download and upload - you can create a competing download or upload in
> Javascript using a WebWorker talking to another server that supports the
> websocket API to force buffer overflow.  (sort of poor man's RRUL).
>
>
>
> The speedof.me API would give you the measured performance, while the
> other path would just be aan easier to code test load to a source/sink.
>
>
>
> Not sure that would help, but for a prototype it's not bad.
>
>
>
> On Thursday, September 11, 2014 8:42pm, "Jonathan Morton" <
> chromatix99@gmail.com> said:
>
>  >
> > On 12 Sep, 2014, at 3:35 am, dpreed@reed.com wrote:
> >
> > > Among friends of mine, we can publicize this widely. But those friends
> > probably would like to see how the measurement would work.
> >
> > Could we make use of the existing test servers (running netperf) for that
> > demonstration? How hard is the protocol to fake in Javascript?
> >
> > Or would a netperf-wrapper demonstration suffice? We've already got
> that, but
> > we'd need to extract the single-figures-of-merit from the data.
> >
> > I wonder if the speedof.me API can already be tricked into doing the
> right thing?
> >
> > - Jonathan Morton
> >
> >
>

[-- Attachment #2: Type: text/html, Size: 3513 bytes --]

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Jonathan Morton]

On 12 Sep, 2014, at 4:49 am, Joel Wirāmu Pauling wrote:

> So if ookla implemented a udp based test, changed it's statical weighting and data mining methods overnight. At least in NZ that might help. 

Isn't that the whole point of this discussion?

 - Jonathan Morton

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Joel Wirāmu Pauling]

The tendency has been (I am also guilty of this) to go down the route
of  'here I made this $bandwidth test' using some
$combination_of_scripts  - tell users to use this It's better and more
reflective of what users should be testing for"

That route is the one I am kinda trying to steer away from, as it
isn't solving the issue. Which is IMHO a user education one.

But yes you are right, you would have to solve this for each and every
measurement site out there, or at the least - the most popular one in
your particular market.

NZ has other challenges in that things like Speedofme are useless for
us because there are no test servers anywhere near our international
IX's and even on 1Gbit machines right on the Global gateway I get
nothing better than 20-30mbit out of it. (TCP BDP is a bitch in NZ).


-Joel

On 12 September 2014 14:04, Jonathan Morton <chromatix99@gmail.com> wrote:
>
> On 12 Sep, 2014, at 4:49 am, Joel Wirāmu Pauling wrote:
>
>> So if ookla implemented a udp based test, changed it's statical weighting and data mining methods overnight. At least in NZ that might help.
>
> Isn't that the whole point of this discussion?
>
>  - Jonathan Morton
>

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Jesper Dangaard Brouer]

[-- Attachment #1: Type: text/plain, Size: 1093 bytes --]

On Thu, 11 Sep 2014 20:13:19 -0400
Rich Brown <richb.hanover@gmail.com> wrote:

> I'll sign too. (And I like the 98th percentile measure for each
> direction to give a single number that represents what's happening. It
> could include ping loss rate, as well...)

I'll sign too.

The 98th percentile latency is of-cause nice, but we should remember to
keep our request simple.

"Dear speedtest.net/speedof.me please add a latency test, that measures
latency during the upload and download test-runs."

That will be the first baby step, don't make it too complicated for
them, afterwards we can start explaining if we want min/max or 98th
percentile.

They already have a ping facility, which the only run _before_ starting
the test run.  Thus, they should be-able to easily adapt their existing
"ping" code to measure during the test-runs... give then something they
can easily do.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [Cerowrt-devel] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]

Dave Taht <dave.taht@gmail.com> writes:

> What if we could publish an open letter to the benchmark makers such
> as speedtest, explaining how engineering for their test does *not*
> make for a better internet? The press fallout from that letter, would
> improve some user education, regardless if we could get the tests
> changed or not.
>
> Who here would sign?

I would! :)

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Rich Brown]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]


On Sep 12, 2014, at 3:17 AM, Jesper Dangaard Brouer <jbrouer@redhat.com> wrote:

> On Thu, 11 Sep 2014 20:13:19 -0400
> Rich Brown <richb.hanover@gmail.com> wrote:
> 
>> I'll sign too. (And I like the 98th percentile measure for each
>> direction to give a single number that represents what's happening. It
>> could include ping loss rate, as well...)
> 
> I'll sign too.
> 
> The 98th percentile latency is of-cause nice, but we should remember to
> keep our request simple.

I did try a simple request and got a lukewarm response. You can see the gist of my correspondence with the test companies in https://lists.bufferbloat.net/pipermail/cerowrt-devel/2014-September/003517.html

Rich
 
> "Dear speedtest.net/speedof.me please add a latency test, that measures
> latency during the upload and download test-runs."
> 
> That will be the first baby step, don't make it too complicated for
> them, afterwards we can start explaining if we want min/max or 98th
> percentile.
> 
> They already have a ping facility, which the only run _before_ starting
> the test run.  Thus, they should be-able to easily adapt their existing
> "ping" code to measure during the test-runs... give then something they
> can easily do.
> 
> -- 
> Best regards,
>  Jesper Dangaard Brouer
>  MSc.CS, Sr. Network Kernel Developer at Red Hat
>  Author of http://www.iptv-analyzer.org
>  LinkedIn: http://www.linkedin.com/in/brouer


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Jesper Dangaard Brouer]

[-- Attachment #1: Type: text/plain, Size: 2016 bytes --]

On Fri, 12 Sep 2014 08:16:46 -0400 Rich Brown <richb.hanover@gmail.com> wrote:

> On Sep 12, 2014, at 3:17 AM, Jesper Dangaard Brouer <jbrouer@redhat.com> wrote:
> 
> > On Thu, 11 Sep 2014 20:13:19 -0400 Rich Brown <richb.hanover@gmail.com> wrote:
> > 
[...]
> > 
> > The 98th percentile latency is of-cause nice, but we should remember to
> > keep our request simple.
> 
> I did try a simple request and got a lukewarm response. You can see the
> gist of my correspondence with the test companies in
> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2014-September/003517.html

Yes, I did notice your write-up.  And thank you for doing so!

When reading it, it strikes me, that you don't directly tell them what
to do; e.g. add a latency test during upload and download.  You say so
indirectly, and points to a website with an implemented solution.  My
point is, it requires work from their side, to lookup a solution and
understand it.

I want something even simpler like add test that: "measures latency
while measuring the speed" (your own formulation from betterspeedtest.sh)

The reason, this email and this requirement have to be passed on from
person reading the mail, to management, to the developers in the company.


 
> > "Dear speedtest.net/speedof.me please add a latency test, that measures
> > latency during the upload and download test-runs."
> > 
> > That will be the first baby step, don't make it too complicated for
> > them, afterwards we can start explaining if we want min/max or 98th
> > percentile.
> > 
> > They already have a ping facility, which the only run _before_ starting
> > the test run.  Thus, they should be-able to easily adapt their existing
> > "ping" code to measure during the test-runs... give then something they
> > can easily do.


-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[Rick Jones]

On 09/11/2014 06:48 PM, Rich Brown wrote:
> Jonathan,
>
>> Could we make use of the existing test servers (running netperf) for that demonstration?  How hard is the protocol to fake in Javascript?
>
> Not having coded a stitch of this, I *think* it would require the following:
>
> - Web page on netperf-xxx.bufferbloat.net that served out the javascript (required to get around cross-domain protections within the browser)
>
> - Javascript function to connect back to that host on port 12865 and fake out the netserver with TCP_STREAM or TCP_MAERTS request
>
> - Javascript that's efficient enough to source/swallow full-rate data stream
>
> - Cloning the code from https://github.com/apenwarr/blip to make fake pings from TCP requests
>
> Anyone know more than I do about this?

Not about the javascript stuff, but your high level description of the 
netperf side sounds plausible.  There are a few control messages netperf 
will exchange with netserver that if you want to leverage a remote 
netserver will need to be included.  You can run a netperf command with 
a higher debug level to see them.

rick jones

Re: [Cerowrt-devel] [Bloat] Fixing bufferbloat: How about an open letter to the web benchmarkers?

[David P. Reed]

[-- Attachment #1: Type: text/plain, Size: 1788 bytes --]

I have a working ping-over-http mobile browser app at  alt.reed.com. feel free to try it and look at the underlying packet stream with wireshark. I did a prototype of a RRUL test using Web sockets and a modified nginx websocket   module as a server that could be commanded to generate precise traffic and server end measurements ... it showed this can work up to a few 10 s of Mb/s.

It's slightly tricky and requires a good understanding of the Web sockets protocol stack.


On Sep 12, 2014, Rick Jones <rick.jones2@hp.com> wrote:
>On 09/11/2014 06:48 PM, Rich Brown wrote:
>> Jonathan,
>>
>>> Could we make use of the existing test servers (running netperf) for
>that demonstration?  How hard is the protocol to fake in Javascript?
>>
>> Not having coded a stitch of this, I *think* it would require the
>following:
>>
>> - Web page on netperf-xxx.bufferbloat.net that served out the
>javascript (required to get around cross-domain protections within the
>browser)
>>
>> - Javascript function to connect back to that host on port 12865 and
>fake out the netserver with TCP_STREAM or TCP_MAERTS request
>>
>> - Javascript that's efficient enough to source/swallow full-rate data
>stream
>>
>> - Cloning the code from https://github.com/apenwarr/blip to make fake
>pings from TCP requests
>>
>> Anyone know more than I do about this?
>
>Not about the javascript stuff, but your high level description of the
>netperf side sounds plausible.  There are a few control messages
>netperf
>will exchange with netserver that if you want to leverage a remote
>netserver will need to be included.  You can run a netperf command with
>
>a higher debug level to see them.
>
>rick jones

-- Sent from my Android device with K-@ Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 2934 bytes --]