[Bloat] [Codel] Network test tools for many parallel/concurrent connections?

[Rick Jones]

It will not match what one can get from tcptrace, or commercial 
solutions, but netperf can be asked to emit a number of potentially 
"intersting" things.  Using the "omni output selectors" one can request 
statistics for some interesting latencies:

raj at tardy:~$ netperf -- -O ? | grep LAT
RT_LATENCY
MIN_LATENCY
MAX_LATENCY
P50_LATENCY
P90_LATENCY
P99_LATENCY
MEAN_LATENCY
STDDEV_LATENCY

For a STREAM test those will be based on time in the send call.  For a 
MAERTS test those will be time in the receive call.  For an RR test 
those will be the round-trip times at the application layer.

You can also ./configure --enable-histogram and if the verbosity is set 
to 2 or more, a histogram of the distribution will be emitted which will 
resemble:

Histogram of time spent in send() call.
UNIT_USEC     :    0:    0:  434: 404912: 715323: 800663: 263305: 9336: 
2439: 1522
TEN_USEC      :    0: 2276:   41:   48:   97:   67:   79:   17:    5:    7
HUNDRED_USEC  :    0:   28:    2:    2:    0:    2:    0:    0:    1:    1
UNIT_MSEC     :    0:    3:    2:    0:    1:    0:    1:    0:    0:    0
TEN_MSEC      :    0:    0:    0:    0:    0:    0:    0:    0:    0:    0
HUNDRED_MSEC  :    0:    0:    0:    0:    0:    0:    0:    0:    0:    0
UNIT_SEC      :    0:    0:    0:    0:    0:    0:    0:    0:    0:    0
TEN_SEC       :    0:    0:    0:    0:    0:    0:    0:    0:    0:    0
 >100_SECS: 0
HIST_TOTAL:      2200614

when running under Linux, netperf also knows how to report the number of 
TCP retransmissions encountered over the life of the data connection:

raj at tardy:~$ netperf -- -O ? | grep -i retran
LOCAL_TRANSPORT_RETRANS
REMOTE_TRANSPORT_RETRANS

And if you want to have an idea of what each individual netperf was 
doing in terms of mbit/s or trans/s over discrete points in its 
lifetime, you can ./configure --enable-demo and it will emit interim 
results at roughly the requested interval which can then be 
post-processed.  An example of that being done can be found in 
doc/examples/runemomniaggdemo.sh script and doc/examples/post_proc.py

happy benchmarking,

rick jones