[Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 2500 bytes --]

Hi everyone

This is to announce version 0.10.0 of Flent: The FLExible Network Tester
(formerly netperf-wrapper).

This is the first release under the new name: I'm hoping not too much
broke in the transition. A list of changes since v0.9.1 is included
below.

To install the release, use `pip install flent`. Arch Linux users can
install through the AUR. Pre-built packages for Debian and Ubuntu are
available from the Open Build Service at
https://software.opensuse.org/download.html?project=home:tohojo:flent&package=flent

Please report any bugs through Github at https://github.com/tohojo/flent/issues

Thanks,
-Toke


List of changes sine v0.9.1:

- Package installation should now work through `pip` on most platforms.
  In addition, two binaries are now installed: `flent` and `flent-gui`.
  The latter will launch the GUI directly (corresponding to passing
  --gui to the normal `flent` binary).

- The file format has changed. It now uses the .flnt extension and is
  bzip'ed instead of gzip'ed. In addition, raw data as parsed from the
  underlying test tools is now always included, but it has been moved
  outside the 'metadata' key and resides under its own top-level key
  ('raw_values') in the json object. Finally, the file format is now
  explicitly versioned, and a compatibility layer has been added so
  Flent will still load the old .json.gz files.

- Test runners can now depend on each other, so one test tool can be
  made to run when another finishes. In addition, a 'watchdog' feature
  has been introduced to the runners, so one runner exiting can trigger
  forceful kill of (an)other runner(s). This can be used along with the
  new 'timer' runner to enforce a maximum duration of tests, but can
  also be used with an external script to exit on arbitrary events; for
  instance when a bandwidth quota is used up.

- Flent should now better handle unicode output from test tools under
  different locales. Fixes bugs on OSX in particular, where output from
  the `ping` binary can be non-ASCII.

- Flent no longer crashes on broken pipe for text-based output
  formatters. Allows doing things like `flent -o csv | head` without
  errors.

- A couple of new tests has been added: `rrul_50_down`, `rrul_100_up`
  and `dslreports_8dn`.

- A --swap-up-down parameter has been added to swap the meaning of "up"
  and "down" when running tests.

- Various bug fixes.


This release announcement is also available on Github:
https://github.com/tohojo/flent/releases/tag/v0.10.0

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Dave Taht]

> - A couple of new tests has been added: `rrul_50_down`, `rrul_100_up`

These tests also include one saturating stream in the opposite direction,
and do not graph many flows very well, as yet.

https://github.com/tohojo/flent/issues/15

The results are "interesting". I have been using these to completely overwhelm
the sqm inbound shaper (pie,cake,fq_codel,and a policer all fail to manage
the concatinated 2sec comcast queue. Sigh.)

Also since we have tons more "flent" servers coming available, testing
more flows at more RTTs becomes feasible.

https://github.com/tohojo/flent/issues/36

>   and `dslreports_8dn`.

This last test is not really baked yet. My intent was to fully emulate
the dslreports tests (notably using an http pinger in addition to udp), and use
things like the new in-series runner to garner up and down results collected
sequentially, and calculate stuff including the "grade".  But the form of those
tests had not settled down upstream before this release....

Please do not expect binary compatability on the results of these
three tests going forward.

And: Ideas and Patches always wanted - especially now that the next
release window is open!

https://github.com/tohojo/flent/issues


-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Dave Taht]

um, er ah, can a modern web server or web browser decompress bzip?
gzip support is common, and it was always my hope to have a web
browser based parser for the output.

On Sun, May 24, 2015 at 9:41 AM, Dave Taht <dave.taht@gmail.com> wrote:
> also, nanog would be good to ping but this announcement is not generic
> enough for it. I can do nanog.
>
> Also you have hopefully collected a lot of addresses from your
> travels, like to caida, that are not on the bloat list.
>
> Best to do an announcement of that sort after you have a mailing list.
> I hope to migrate lists.bufferbloat.net to
> something new, leveraging your newfound expertise in that area. ;)
>
> Does stuff install to the mac using pip?
>
> On Sun, May 24, 2015 at 9:22 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>> - A couple of new tests has been added: `rrul_50_down`, `rrul_100_up`
>>
>> These tests also include one saturating stream in the opposite direction,
>> and do not graph many flows very well, as yet.
>>
>> https://github.com/tohojo/flent/issues/15
>>
>> The results are "interesting". I have been using these to completely overwhelm
>> the sqm inbound shaper (pie,cake,fq_codel,and a policer all fail to manage
>> the concatinated 2sec comcast queue. Sigh.)
>>
>> Also since we have tons more "flent" servers coming available, testing
>> more flows at more RTTs becomes feasible.
>>
>> https://github.com/tohojo/flent/issues/36
>>
>>>   and `dslreports_8dn`.
>>
>> This last test is not really baked yet. My intent was to fully emulate
>> the dslreports tests (notably using an http pinger in addition to udp), and use
>> things like the new in-series runner to garner up and down results collected
>> sequentially, and calculate stuff including the "grade".  But the form of those
>> tests had not settled down upstream before this release....
>>
>> Please do not expect binary compatability on the results of these
>> three tests going forward.
>>
>> And: Ideas and Patches always wanted - especially now that the next
>> release window is open!
>>
>> https://github.com/tohojo/flent/issues
>>
>>
>> --
>> Dave Täht
>> Open Networking needs **Open Source Hardware**
>>
>> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67
>
>
>
> --
> Dave Täht
> Open Networking needs **Open Source Hardware**
>
> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Dave Taht]

This contained the never-merged (some licencing issues) web based
parser for the data.

https://github.com/bipulkkuri/netperf-wrapper


On Sun, May 24, 2015 at 10:14 AM, Dave Taht <dave.taht@gmail.com> wrote:
> um, er ah, can a modern web server or web browser decompress bzip?
> gzip support is common, and it was always my hope to have a web
> browser based parser for the output.
>
> On Sun, May 24, 2015 at 9:41 AM, Dave Taht <dave.taht@gmail.com> wrote:
>> also, nanog would be good to ping but this announcement is not generic
>> enough for it. I can do nanog.
>>
>> Also you have hopefully collected a lot of addresses from your
>> travels, like to caida, that are not on the bloat list.
>>
>> Best to do an announcement of that sort after you have a mailing list.
>> I hope to migrate lists.bufferbloat.net to
>> something new, leveraging your newfound expertise in that area. ;)
>>
>> Does stuff install to the mac using pip?
>>
>> On Sun, May 24, 2015 at 9:22 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>>> - A couple of new tests has been added: `rrul_50_down`, `rrul_100_up`
>>>
>>> These tests also include one saturating stream in the opposite direction,
>>> and do not graph many flows very well, as yet.
>>>
>>> https://github.com/tohojo/flent/issues/15
>>>
>>> The results are "interesting". I have been using these to completely overwhelm
>>> the sqm inbound shaper (pie,cake,fq_codel,and a policer all fail to manage
>>> the concatinated 2sec comcast queue. Sigh.)
>>>
>>> Also since we have tons more "flent" servers coming available, testing
>>> more flows at more RTTs becomes feasible.
>>>
>>> https://github.com/tohojo/flent/issues/36
>>>
>>>>   and `dslreports_8dn`.
>>>
>>> This last test is not really baked yet. My intent was to fully emulate
>>> the dslreports tests (notably using an http pinger in addition to udp), and use
>>> things like the new in-series runner to garner up and down results collected
>>> sequentially, and calculate stuff including the "grade".  But the form of those
>>> tests had not settled down upstream before this release....
>>>
>>> Please do not expect binary compatability on the results of these
>>> three tests going forward.
>>>
>>> And: Ideas and Patches always wanted - especially now that the next
>>> release window is open!
>>>
>>> https://github.com/tohojo/flent/issues
>>>
>>>
>>> --
>>> Dave Täht
>>> Open Networking needs **Open Source Hardware**
>>>
>>> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67
>>
>>
>>
>> --
>> Dave Täht
>> Open Networking needs **Open Source Hardware**
>>
>> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67
>
>
>
> --
> Dave Täht
> Open Networking needs **Open Source Hardware**
>
> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Toke Høiland-Jørgensen]

Dave Taht <dave.taht@gmail.com> writes:

> This contained the never-merged (some licencing issues) web based
> parser for the data.
>
> https://github.com/bipulkkuri/netperf-wrapper

The main reason (apart from licensing issues) that it was never merged
was that I don't want to maintain two plotting implementations within
Flent itself. So the possible scenarios for a web-based viewer are, as I
see it, roughly:

1. Leverage the existing matplotlib plotting implementation to produce
   javascript plots. This is issue #27
   (https://github.com/tohojo/flent/issues/27) and depends on how well
   the mpl-to-d3 library works in practice.

2. Replace the whole plotting implementation with a javascript-based
   one, and serve that from within Flent itself.

3. Have an entirely separate web viewer implementation.


Which scenario that works best will probably to a large extent depend on
the actual use case this mystical beast is supposed to support. Which I
don't believe has been articulated properly...

-Toke

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Dave Taht]

well,1) we need a flent-devel mailing list... no need to have this
discussion so widely here..

On Sun, May 24, 2015 at 10:55 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Dave Taht <dave.taht@gmail.com> writes:
>
>> This contained the never-merged (some licencing issues) web based
>> parser for the data.
>>
>> https://github.com/bipulkkuri/netperf-wrapper
>
> The main reason (apart from licensing issues) that it was never merged
> was that I don't want to maintain two plotting implementations within
> Flent itself. So the possible scenarios for a web-based viewer are, as I
> see it, roughly:

I agree that *you* do not want to maintain two plotting
implementations inside flent. But:

I am hugely in favor of various implementations of stuff that can read
the file format and do various (new and interesting) things with it.
The number of python-matplotlib capable developers is small. The
number of javascript capable developers (and web based plotting tools)
is huge. Other means of dealing with the data (postgres json) are also
needed.

So the json.gz file format WAS readable and translatable by many web
servers and browsers, flent is not (so far as I know, even with adding
a mime type.)

I liked the original web based prototype, it needed a db back end, but
it seemed like a promising start.

> 1. Leverage the existing matplotlib plotting implementation to produce
>    javascript plots. This is issue #27
>    (https://github.com/tohojo/flent/issues/27) and depends on how well
>    the mpl-to-d3 library works in practice.

I don't have a lot of hope for the canvas approach, personally. Too
many abstractions.

>
> 2. Replace the whole plotting implementation with a javascript-based
>    one, and serve that from within Flent itself.

No, trying to optimize for local full throttle use make the tool most
useful for core developers.

>
> 3. Have an entirely separate web viewer implementation.
>
>
> Which scenario that works best will probably to a large extent depend on
> the actual use case this mystical beast is supposed to support. Which I
> don't believe has been articulated properly...

All three could be done.  (with sufficient developers with sufficient
skillsets with sufficient funding)

Personally what I most prefer is that the "expert" implementation be
blazing fast!

The third option makes it more possible to share and compare the
results over many browser types and OSes, including (ugh) windows.

> -Toke



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Toke Høiland-Jørgensen]

Dave Taht <dave.taht@gmail.com> writes:

> well,1) we need a flent-devel mailing list... no need to have this
> discussion so widely here..

Yup, will try to set one up soonish. ENOTIME so far.

> I agree that *you* do not want to maintain two plotting
> implementations inside flent. But:
>
> I am hugely in favor of various implementations of stuff that can read
> the file format and do various (new and interesting) things with it.
> The number of python-matplotlib capable developers is small. The
> number of javascript capable developers (and web based plotting tools)
> is huge. Other means of dealing with the data (postgres json) are also
> needed.

Oh, sure, quite happy with more implementations. That was option 3.
However, until someone actually starts working on such an implementation
this is all hypothetical. I'm happy to discuss interoperability issues
when such an implementation actually materialises :)

> So the json.gz file format WAS readable and translatable by many web
> servers and browsers, flent is not (so far as I know, even with adding
> a mime type.)

Yes, but .json.gz also makes it hard to distinguish from other random
gzip'ed json data. I'd rather have a separate file extension for offline
storage, and then solve the web problem as a separate problem, which as
far as I'm concerned is orthogonal to the encoding of the files on disk.
The important thing is the structure of the JSON object, not which
compression format and file extension it is stored in.

> I liked the original web based prototype, it needed a db back end, but
> it seemed like a promising start.

And odds are that db back end is going to store the JSON object in its
own encoding anyway, hence rendering this whole discussion moot... :)

> I don't have a lot of hope for the canvas approach, personally. Too
> many abstractions.

Yeah, I'm sceptical, but willing to give it the benefit of the doubt if
it means I have to write less code...

> Personally what I most prefer is that the "expert" implementation be
> blazing fast!

Yes, well, matplotlib is not exactly a cheetah...

-Toke

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Dave Taht]

On Sun, May 24, 2015 at 11:16 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Dave Taht <dave.taht@gmail.com> writes:
>
>> well,1) we need a flent-devel mailing list... no need to have this
>> discussion so widely here..
>
> Yup, will try to set one up soonish. ENOTIME so far.
>
>> I agree that *you* do not want to maintain two plotting
>> implementations inside flent. But:
>>
>> I am hugely in favor of various implementations of stuff that can read
>> the file format and do various (new and interesting) things with it.
>> The number of python-matplotlib capable developers is small. The
>> number of javascript capable developers (and web based plotting tools)
>> is huge. Other means of dealing with the data (postgres json) are also
>> needed.
>
> Oh, sure, quite happy with more implementations. That was option 3.
> However, until someone actually starts working on such an implementation
> this is all hypothetical. I'm happy to discuss interoperability issues
> when such an implementation actually materialises :)

Well, no. Usually we discuss interop issues in file and wire formats
in light of *possible* implementations... and a web implementation of
some plotting tools is a highly desirable implementation.


>> So the json.gz file format WAS readable and translatable by many web
>> servers and browsers, flent is not (so far as I know, even with adding
>> a mime type.)
>
> Yes, but .json.gz also makes it hard to distinguish from other random
> gzip'ed json data.

All in favor of .flent.gz. and .flent.bz. Not .flent the compressed
json encoded format.

> I'd rather have a separate file extension for offline
> storage, and then solve the web problem as a separate problem, which as
> far as I'm concerned is orthogonal to the encoding of the files on disk.
> The important thing is the structure of the JSON object, not which
> compression format and file extension it is stored in.
>
>> I liked the original web based prototype, it needed a db back end, but
>> it seemed like a promising start.
>
> And odds are that db back end is going to store the JSON object in its
> own encoding anyway, hence rendering this whole discussion moot... :)

And odds are that the db backend would only store the metadata and not
the raw data.

>> I don't have a lot of hope for the canvas approach, personally. Too
>> many abstractions.
>
> Yeah, I'm sceptical, but willing to give it the benefit of the doubt if
> it means I have to write less code...

Goal here is to make it as easy as possible for *others* to write code.

>> Personally what I most prefer is that the "expert" implementation be
>> blazing fast!
>
> Yes, well, matplotlib is not exactly a cheetah...

Yes, on my bad days, waiting for 100+ plots to render on my 16 core
box, I dream of rewriting the whole thing in go and parallizing the
hell out of it, for starters.

>
> -Toke



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Sebastian Moeller]

On May 24, 2015, at 20:16 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:
[...]
> 
>> So the json.gz file format WAS readable and translatable by many web
>> servers and browsers, flent is not (so far as I know, even with adding
>> a mime type.)
> 
> Yes, but .json.gz also makes it hard to distinguish from other random
> gzip'ed json data. I'd rather have a separate file extension for offline
> storage, and then solve the web problem as a separate problem, which as
> far as I'm concerned is orthogonal to the encoding of the files on disk.
> The important thing is the structure of the JSON object, not which
> compression format and file extension it is stored in.
[…]

*.flent.json.gz ?

Best Regards

	Sebastian

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[Dave Taht]

so far as I know browsers will autonegotiate gzip compression on the
wire or in the server. I do not believe (but am woefully out of date
on this) bzip can be handled in the browser.

http://redmine.lighttpd.net/projects/1/wiki/docs_modcompress
http://httpd.apache.org/docs/2.2/mod/mod_deflate.html

So my vote is that the native flent format be pure text json (with a
mime type mapping it to that) and the default output of flent be
flent.gz where it can be automagically decoded when presented with
whatever.flent and the .gz file exists instead.

And ya still have time to fix it before casting the bzip'd version in
stone. I am not big on fixing all the web browsers and servers in the
world for "yet another specialized mime type".

Even if javacript based decompressors exist for bzip, gzip was the
most common format back when i was paying attention (8 years ago).

Re: [Bloat] Announcing Flent (formerly netperf-wrapper) v0.10.0

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2893 bytes --]

xz is both faster and better compression than bzip, so unless there is browser 
support for bzip and not xz, bzip should be skipped.

David Lang

On Sun, 24 May 2015, Dave Taht wrote:

> um, er ah, can a modern web server or web browser decompress bzip?
> gzip support is common, and it was always my hope to have a web
> browser based parser for the output.
>
> On Sun, May 24, 2015 at 9:41 AM, Dave Taht <dave.taht@gmail.com> wrote:
>> also, nanog would be good to ping but this announcement is not generic
>> enough for it. I can do nanog.
>>
>> Also you have hopefully collected a lot of addresses from your
>> travels, like to caida, that are not on the bloat list.
>>
>> Best to do an announcement of that sort after you have a mailing list.
>> I hope to migrate lists.bufferbloat.net to
>> something new, leveraging your newfound expertise in that area. ;)
>>
>> Does stuff install to the mac using pip?
>>
>> On Sun, May 24, 2015 at 9:22 AM, Dave Taht <dave.taht@gmail.com> wrote:
>>>> - A couple of new tests has been added: `rrul_50_down`, `rrul_100_up`
>>>
>>> These tests also include one saturating stream in the opposite direction,
>>> and do not graph many flows very well, as yet.
>>>
>>> https://github.com/tohojo/flent/issues/15
>>>
>>> The results are "interesting". I have been using these to completely overwhelm
>>> the sqm inbound shaper (pie,cake,fq_codel,and a policer all fail to manage
>>> the concatinated 2sec comcast queue. Sigh.)
>>>
>>> Also since we have tons more "flent" servers coming available, testing
>>> more flows at more RTTs becomes feasible.
>>>
>>> https://github.com/tohojo/flent/issues/36
>>>
>>>>   and `dslreports_8dn`.
>>>
>>> This last test is not really baked yet. My intent was to fully emulate
>>> the dslreports tests (notably using an http pinger in addition to udp), and use
>>> things like the new in-series runner to garner up and down results collected
>>> sequentially, and calculate stuff including the "grade".  But the form of those
>>> tests had not settled down upstream before this release....
>>>
>>> Please do not expect binary compatability on the results of these
>>> three tests going forward.
>>>
>>> And: Ideas and Patches always wanted - especially now that the next
>>> release window is open!
>>>
>>> https://github.com/tohojo/flent/issues
>>>
>>>
>>> --
>>> Dave Täht
>>> Open Networking needs **Open Source Hardware**
>>>
>>> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67
>>
>>
>>
>> --
>> Dave Täht
>> Open Networking needs **Open Source Hardware**
>>
>> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67
>
>
>
> -- 
> Dave Täht
> Open Networking needs **Open Source Hardware**
>
> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat