[Make-wifi-fast] flent adds more text output

[Make-wifi-fast] flent adds more text output

[Dave Taht]

While I have always encouraged people to "go right to the graphs", I
have generally
wanted a bit more text output to make email comms easier, and toke has
just added
some basic output stats to flent in git head.

(there are also ways to get plenty of text data via other post-hoc
options, instead of graphs)

Example of the new output

d@dancer:~/uap$ flent -t uap-ath9-desk-dual-dancer-2v12 -H
172.22.148.9 --test-parameter=download_streams=2 tcp_ndown
Data file written to
./tcp_ndown-2016-11-19T095013.031996.uap_ath9_desk_dual_dancer_2v12.flent.gz.
Summary of tcp_ndown test run 'uap-ath9-desk-dual-dancer-2v12' (at
2016-11-19 17:50:13.031996):

                             avg /       median
 Ping (ms) ICMP   :        38.21 /        38.21 ms
 TCP download avg :         6.17 /         6.17 Mbits/s
 TCP download sum :        12.33 /        12.33 Mbits/s
 TCP download::1  :         6.32 /         6.32 Mbits/s
 TCP download::2  :         6.01 /         6.01 Mbits/s


-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Make-wifi-fast] flent adds more text output

[Sebastian Moeller]

Hi Dave,


> On Nov 19, 2016, at 19:10, Dave Taht <dave.taht@gmail.com> wrote:
> 
> While I have always encouraged people to "go right to the graphs", I
> have generally
> wanted a bit more text output to make email comms easier, and toke has
> just added
> some basic output stats to flent in git head.

	Is that pushed to github yet? I tried a git pull but got nothing new. Testing with:
date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 -l 60 -H netperf-eu.bufferbloat.net rrul_cs8 --remote-metadata=root@192.168.1.1 -p all_scaled -D . -t IPv4_SQM_LLA-simple-fq_codel-ETH_OH16_U095pct9545of10048K-D90pct46246of51385K_2_netperf-eu
did not give any stats. So is the output generic or restricted to specific tests?

Best Regards
	Sebastian


> 
> (there are also ways to get plenty of text data via other post-hoc
> options, instead of graphs)
> 
> Example of the new output
> 
> d@dancer:~/uap$ flent -t uap-ath9-desk-dual-dancer-2v12 -H
> 172.22.148.9 --test-parameter=download_streams=2 tcp_ndown
> Data file written to
> ./tcp_ndown-2016-11-19T095013.031996.uap_ath9_desk_dual_dancer_2v12.flent.gz.
> Summary of tcp_ndown test run 'uap-ath9-desk-dual-dancer-2v12' (at
> 2016-11-19 17:50:13.031996):
> 
>                             avg /       median
> Ping (ms) ICMP   :        38.21 /        38.21 ms
> TCP download avg :         6.17 /         6.17 Mbits/s
> TCP download sum :        12.33 /        12.33 Mbits/s
> TCP download::1  :         6.32 /         6.32 Mbits/s
> TCP download::2  :         6.01 /         6.01 Mbits/s
> 
> 
> -- 
> Dave Täht
> Let's go make home routers and wifi faster! With better software!
> http://blog.cerowrt.org
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Make-wifi-fast] flent adds more text output

[Toke Høiland-Jørgensen]

Sebastian Moeller <moeller0@gmx.de> writes:

> Hi Dave,
>
>> On Nov 19, 2016, at 19:10, Dave Taht <dave.taht@gmail.com> wrote:
>> 
>> While I have always encouraged people to "go right to the graphs", I
>> have generally
>> wanted a bit more text output to make email comms easier, and toke has
>> just added
>> some basic output stats to flent in git head.
>
> 	Is that pushed to github yet? I tried a git pull but got nothing new. Testing with:
> date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 -l 60 -H netperf-eu.bufferbloat.net rrul_cs8 --remote-metadata=root@192.168.1.1 -p all_scaled -D . -t IPv4_SQM_LLA-simple-fq_codel-ETH_OH16_U095pct9545of10048K-D90pct46246of51385K_2_netperf-eu
> did not give any stats. So is the output generic or restricted to
> specific tests?

No, it only happens if you don't select another output format; in this
case a plot. You can select this the summary formatter explicitly (and
thus turn off plots) by passing '-f summary'. Works when loading a
previous data file as well...

-Toke

Re: [Make-wifi-fast] flent adds more text output

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]

I am not convinced the text output is actually correct - see attached graph.

I am testing airtime fairness code on both an archer ap and ubuntu box
on the ath9k, and the udp flows show a peak of 80ms on the plot and a
median of 25ms on the ping, where...

Summary of rrul_be test run 'long-dense-airtime-9-ath9k-archer-c2' (at
2016-11-20 11:24:03.451990):

                              avg /       median
 Ping (ms) ICMP    :        32.75 /        32.75 ms
 Ping (ms) UDP BE1 :       158.23 /       158.23 ms
 Ping (ms) UDP BE2 :       125.31 /       125.31 ms
 Ping (ms) UDP BE3 :       121.95 /       121.95 ms
 Ping (ms) avg     :       135.16 /       135.16 ms
 TCP download BE   :         6.87 /         6.87 Mbits/s
 TCP download BE2  :         6.84 /         6.84 Mbits/s
 TCP download BE3  :         6.81 /         6.81 Mbits/s
 TCP download BE4  :         6.86 /         6.86 Mbits/s
 TCP download avg  :         6.84 /         6.84 Mbits/s
 TCP download sum  :        27.38 /        27.38 Mbits/s
 TCP totals        :        33.80 /        33.80 Mbits/s
 TCP upload BE     :         1.59 /         1.59 Mbits/s
 TCP upload BE2    :         1.62 /         1.62 Mbits/s
 TCP upload BE3    :         1.61 /         1.61 Mbits/s
 TCP upload BE4    :         1.60 /         1.60 Mbits/s
 TCP upload avg    :         1.60 /         1.60 Mbits/s
 TCP upload sum    :         6.42 /         6.42 Mbits/s



On Sun, Nov 20, 2016 at 3:17 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Sebastian Moeller <moeller0@gmx.de> writes:
>
>> Hi Dave,
>>
>>> On Nov 19, 2016, at 19:10, Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>> While I have always encouraged people to "go right to the graphs", I
>>> have generally
>>> wanted a bit more text output to make email comms easier, and toke has
>>> just added
>>> some basic output stats to flent in git head.
>>
>>       Is that pushed to github yet? I tried a git pull but got nothing new. Testing with:
>> date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 -l 60 -H netperf-eu.bufferbloat.net rrul_cs8 --remote-metadata=root@192.168.1.1 -p all_scaled -D . -t IPv4_SQM_LLA-simple-fq_codel-ETH_OH16_U095pct9545of10048K-D90pct46246of51385K_2_netperf-eu
>> did not give any stats. So is the output generic or restricted to
>> specific tests?
>
> No, it only happens if you don't select another output format; in this
> case a plot. You can select this the summary formatter explicitly (and
> thus turn off plots) by passing '-f summary'. Works when loading a
> previous data file as well...
>
> -Toke



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

[-- Attachment #2: text_output.png --]
[-- Type: image/png, Size: 63019 bytes --]

Re: [Make-wifi-fast] flent adds more text output

[Toke Høiland-Jørgensen]

Dave Taht <dave.taht@gmail.com> writes:

> I am not convinced the text output is actually correct - see attached graph.
>
> I am testing airtime fairness code on both an archer ap and ubuntu box
> on the ath9k, and the udp flows show a peak of 80ms on the plot and a
> median of 25ms on the ping, where...

The summary output formatter doesn't discard null data points before
computing the median. Guessing that's the reason for the discrepancy...

-Toke

Re: [Make-wifi-fast] flent adds more text output

[Toke Høiland-Jørgensen]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> Dave Taht <dave.taht@gmail.com> writes:
>
>> I am not convinced the text output is actually correct - see attached graph.
>>
>> I am testing airtime fairness code on both an archer ap and ubuntu box
>> on the ath9k, and the udp flows show a peak of 80ms on the plot and a
>> median of 25ms on the ping, where...
>
> The summary output formatter doesn't discard null data points before
> computing the median. Guessing that's the reason for the discrepancy...

Added filtering to data series before computing the median, and a new
column that shows the loss as a percent of data points filtered out.
Probably going to get in trouble for calling it loss, since it does not
correspond to packet loss (except for ICMP flows), just to lost data
points; not actually sure it's a good idea to have it in there...

-Toke

Re: [Make-wifi-fast] flent adds more text output

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 2854 bytes --]

nope.

root@dancer:~/uap# ./long.sh
Data file written to
./rrul_be-2016-11-20T053621.542683.long_bbr_dense_airtime_9_ath9k_archer_c2.flent.gz.
Summary of rrul_be test run 'long-bbr-dense-airtime-9-ath9k-archer-c2'
(at 2016-11-20 13:36:21.542683):

                              avg /       median                loss
 Ping (ms) ICMP    :        30.91 /        30.91 ms            -0.29 %
 Ping (ms) UDP BE1 :        86.81 /        86.81 ms             0.00 %
 Ping (ms) UDP BE2 :       186.92 /       186.92 ms             0.00 %
 Ping (ms) UDP BE3 :       123.76 /       123.76 ms             0.00 %
 Ping (ms) avg     :       132.49 /       132.49 ms             0.00 %
 TCP download BE   :         8.89 /         8.89 Mbits/s        0.00 %
 TCP download BE2  :         8.99 /         8.99 Mbits/s        0.00 %
 TCP download BE3  :         8.85 /         8.85 Mbits/s        0.00 %
 TCP download BE4  :         8.90 /         8.90 Mbits/s        0.00 %
 TCP download avg  :         8.91 /         8.91 Mbits/s        0.00 %
 TCP download sum  :        35.63 /        35.63 Mbits/s        0.00 %
 TCP totals        :        41.29 /        41.29 Mbits/s        0.00 %
 TCP upload BE     :         1.41 /         1.41 Mbits/s        0.00 %
 TCP upload BE2    :         1.42 /         1.42 Mbits/s        0.00 %
 TCP upload BE3    :         1.42 /         1.42 Mbits/s        0.00 %
 TCP upload BE4    :         1.41 /         1.41 Mbits/s        0.00 %
 TCP upload avg    :         1.42 /         1.42 Mbits/s        0.00 %
 TCP upload sum    :         5.66 /         5.66 Mbits/s        0.00 %
Data file written to
./tcp_ndown-2016-11-20T054204.988641.long_bbr_dense_airtime_9_ath9k_archer_c2_flows_1.flent.gz.



On Sun, Nov 20, 2016 at 5:08 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Toke Høiland-Jørgensen <toke@toke.dk> writes:
>
>> Dave Taht <dave.taht@gmail.com> writes:
>>
>>> I am not convinced the text output is actually correct - see attached graph.
>>>
>>> I am testing airtime fairness code on both an archer ap and ubuntu box
>>> on the ath9k, and the udp flows show a peak of 80ms on the plot and a
>>> median of 25ms on the ping, where...
>>
>> The summary output formatter doesn't discard null data points before
>> computing the median. Guessing that's the reason for the discrepancy...
>
> Added filtering to data series before computing the median, and a new
> column that shows the loss as a percent of data points filtered out.
> Probably going to get in trouble for calling it loss, since it does not
> correspond to packet loss (except for ICMP flows), just to lost data
> points; not actually sure it's a good idea to have it in there...
>
> -Toke



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

[-- Attachment #2: rrul_be-2016-11-20T053621.542683.long_bbr_dense_airtime_9_ath9k_archer_c2.flent.gz --]
[-- Type: application/x-gzip, Size: 1173150 bytes --]

Re: [Make-wifi-fast] flent adds more text output

[Toke Høiland-Jørgensen]

Dave Taht <dave.taht@gmail.com> writes:

> nope.

Ah. Turns out the bug was just that I was printing the mean twice
instead of the median. Fixed that, and turned 'loss' into a column
showing the number of total data points. That still makes it easy to
spot if a UDP flow is cut short, without sounding like it has anything
to do with packet loss.

-Toke

Re: [Make-wifi-fast] flent adds more text output

[Sebastian Moeller]

Hi Toke,

> On Nov 20, 2016, at 12:17, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Sebastian Moeller <moeller0@gmx.de> writes:
> 
>> Hi Dave,
>> 
>>> On Nov 19, 2016, at 19:10, Dave Taht <dave.taht@gmail.com> wrote:
>>> 
>>> While I have always encouraged people to "go right to the graphs", I
>>> have generally
>>> wanted a bit more text output to make email comms easier, and toke has
>>> just added
>>> some basic output stats to flent in git head.
>> 
>> 	Is that pushed to github yet? I tried a git pull but got nothing new. Testing with:
>> date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 -l 60 -H netperf-eu.bufferbloat.net rrul_cs8 --remote-metadata=root@192.168.1.1 -p all_scaled -D . -t IPv4_SQM_LLA-simple-fq_codel-ETH_OH16_U095pct9545of10048K-D90pct46246of51385K_2_netperf-eu
>> did not give any stats. So is the output generic or restricted to
>> specific tests?
> 
> No, it only happens if you don't select another output format; in this
> case a plot. You can select this the summary formatter explicitly (and
> thus turn off plots) by passing '-f summary'. Works when loading a
> previous data file as well…

Ah, now git pull acyually got me something new and without a -p directive I also see the stats output, great! Now I actually do wonder how hard it would be to add
a measure of variance, say the standard deviation or the standard error of the mean to the output (to qualify the average, which probably is the arithmetic mean, which might be worth noting in the header column). And finally for the latency probes, I believe maybe the 95th or 99th percentile would be closer to what we recommend people looking at? (I note that I typically start my flent runs with 10 un-contended ICMP echo requests from the command line to get a reasonable idea what the best case RTT actually is, a similar value could be extracted from the start of the flent run before the data streams have started?) I guess instead of writing emails I should actually look and write code… (maybe I find tie to look into that). Final questions why is this output mutually exclusive with plots?

Best Regards
	Sebastian

> 
> -Toke

Re: [Make-wifi-fast] flent adds more text output

[Toke Høiland-Jørgensen]

Sebastian Moeller <moeller0@gmx.de> writes:

> Hi Toke,
>
>> On Nov 20, 2016, at 12:17, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> Sebastian Moeller <moeller0@gmx.de> writes:
>> 
>>> Hi Dave,
>>> 
>>>> On Nov 19, 2016, at 19:10, Dave Taht <dave.taht@gmail.com> wrote:
>>>> 
>>>> While I have always encouraged people to "go right to the graphs", I
>>>> have generally
>>>> wanted a bit more text output to make email comms easier, and toke has
>>>> just added
>>>> some basic output stats to flent in git head.
>>> 
>>> 	Is that pushed to github yet? I tried a git pull but got nothing new. Testing with:
>>> date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 -l 60 -H netperf-eu.bufferbloat.net rrul_cs8 --remote-metadata=root@192.168.1.1 -p all_scaled -D . -t IPv4_SQM_LLA-simple-fq_codel-ETH_OH16_U095pct9545of10048K-D90pct46246of51385K_2_netperf-eu
>>> did not give any stats. So is the output generic or restricted to
>>> specific tests?
>> 
>> No, it only happens if you don't select another output format; in this
>> case a plot. You can select this the summary formatter explicitly (and
>> thus turn off plots) by passing '-f summary'. Works when loading a
>> previous data file as well…
>
> Ah, now git pull acyually got me something new and without a -p directive I also see the stats output, great! Now I actually do wonder how hard it would be to add
> a measure of variance, say the standard deviation or the standard error of the
> mean to the output (to qualify the average, which probably is the arithmetic
> mean, which might be worth noting in the header column). And finally for the
> latency probes, I believe maybe the 95th or 99th percentile would be closer to
> what we recommend people looking at? (I note that I typically start my flent
> runs with 10 un-contended ICMP echo requests from the command line to get a
> reasonable idea what the best case RTT actually is, a similar value could be
> extracted from the start of the flent run before the data streams have started?)
> I guess instead of writing emails I should actually look and write code… (maybe
> I find tie to look into that).

SummaryFormatter in formatters.py. Pretty straight forward :)

> Final questions why is this output mutually exclusive with plots?

Because internally Flent's notion of output is to format the data with a
Formatter class, only one of which can be active at a time. One of those
is the PlotFormatter that outputs plots, so if that is active no other
output is seen.

-Toke