Hi All,
I've been doing a lot of work on iperf 2 to handle new testing features with a focus on TCP latency testing. Please do let me know your thoughts or suggest comments per the list below. No promises on time frame as this is not my primary day job ;)
The new features currently in development to include:
o) Reverse testing (invert the client/server roles after the connect) - requested by many for testing through NAT gateways
o) Latency tail analysis support (latency mostly needs to evaluated from a tail's perspective, provide those metrics w/o losing full distribution or, at least, tail information)
o) Support GPS (atomic clock) timestamps with all histogram metrics supporting post analysis of problem areas per distributed tooling (this assumes all tooling are synce'd to the GPS atomic clocks.)
o) TCP bidir testing support (bidirectional or full duplex traffic on the same socket)
o) TCP write-acks - the server will ack it's reads back (w/timestamp and byte no.) to the client per every client write(), useful for TCP streaming class of tests and doesn't require syncs to GPS clocks
o) TCP trip times, the write() to read() latency of every clients' writes - requires GPS clock sync - this is a one way metric
o) TCP connect only tests (run many parallel TCP connects and measure the connect times of each) - an issue for many mobile users is that TCP connect times are impacting user experience
o) -P support for more than 127 and now is only limited by the kernel/os (this already fixed)
o) CPU bound warning on exit - present a warning message when an iperf test seemed to be CPU bound vs network i/o bound
o) Mutex warn message on exit: present a warning on exit when the test spent time on mutex locks where that may be degrading network performance
o) Teporter thread consumption rate detector to add thread delay/suspends only when needed (currently, the code always add reporter thread delay.) This delay can and should reduce the cores cache invalidates per cache coherency and the use of shared memory (and its pointers) not under os mutex protection
o) Much better debugging support per ./configure --enable-thread-debug
o) Code clean up including using function pointers to replace per pacet, inline runtime tests (prepping to complete the C++ implementation vs today's hybrid of C and C++)
o) Support for raw sockets to test "L2 level" performance
o) Better code encapsulation (which includes freeing of memory)
Note: Here's an example debug that's now supported for those who want to delve into iperf internals.
Client:
[root@localhost iperf2-code]# src/iperf -c 192.168.1.1 -i 1 -t 4
THREAD(18576):[16:28:11.702942] Thread settings copy (malloc) from/to=0x67ac20/0x67b640
THREAD(18576):[16:28:11.703128] Thread_run_wrapper(0x67ac20 mode=2) thread counts tot/trfc=1/1
THREAD(18576):[16:28:11.703293] Thread_run_wrapper(0x67b640 mode=4) thread counts tot/trfc=2/1
THREAD(18577):[16:28:11.703305] Client thread started in constructor
THREAD(18578):[16:28:11.704364] Reporter thread started
THREAD(18577):[16:28:11.708777] Init settings report 0x7f2a38020fb0
THREAD(18577):[16:28:11.708839] Update connection report 0x7f2a38020fb0 winreq=0 actual=87040
THREAD(18577):[16:28:11.708853] Post report 0x7f2a38020fb0
------------------------------------------------------------
Client connecting to 192.168.1.1, TCP port 5001
THREAD(18577):[16:28:11.708895] Init 5000 element packet ring 0x7f2a38023ee0
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
THREAD(18577):[16:28:11.708961] Init data report 0x7f2a380239a0 size 1336 using packetring 0x7f2a38023ee0
THREAD(18578):[16:28:11.709000] Remove 0x7f2a38020fb0 from reporter job queue in rs
THREAD(18577):[16:28:11.709013] Init connection report 0x7f2a380239a0
THREAD(18578):[16:28:11.709037] Free 0x7f2a38020fb0 in rs
THREAD(18577):[16:28:11.709053] Update connection report 0x7f2a380239a0 winreq=0 actual=87040
THREAD(18577):[16:28:11.709080] Post report 0x7f2a380239a0
[ 3] local 192.168.1.4 port 36276 connected with 192.168.1.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 7.75 MBytes 65.0 Mbits/sec
[ 3] 1.0- 2.0 sec 7.75 MBytes 65.0 Mbits/sec
[ 3] 2.0- 3.0 sec 6.50 MBytes 54.5 Mbits/sec
[ 3] 3.0- 4.0 sec 7.12 MBytes 59.8 Mbits/sec
[ 3] 0.0- 4.0 sec 29.1 MBytes 60.5 Mbits/sec
THREAD(18578):[16:28:15.747044] Remove 0x7f2a380239a0 from reporter job queue in rs
THREAD(18577):[16:28:15.747045] Traffic thread thinks reporter is done with 0x7f2a380239a0
THREAD(18577):[16:28:15.747098] Client destructor close sock=3
THREAD(18577):[16:28:15.747126] Free packet ring 0x7f2a38023ee0 & condition variable await consumer 0x7f2a38023ef8
THREAD(18577):[16:28:15.747156] Free report hdr 0x7f2a380239a0 delay counter=994
THREAD(18577):[16:28:15.747168] Thread settings free=0x67ac20
THREAD(18578):[16:28:16.000164] Thread settings free=0x67b640
Server:
[root@localhost iperf2-code]# src/iperf -s -i 1
THREAD(19735):[16:27:43.620063] Thread settings copy (malloc) from/to=0x12dac20/0x12daf10
THREAD(19735):[16:27:43.620250] Thread_run_wrapper(0x12dac20 mode=8) thread counts tot/trfc=1/0
THREAD(19735):[16:27:43.620338] Thread_run_wrapper(0x12daf10 mode=3) thread counts tot/trfc=2/0
THREAD(19736):[16:27:43.620505] Init settings report 0x7f6c84020930
THREAD(19736):[16:27:43.620552] Update connection report 0x7f6c84020930 winreq=0 actual=87380
THREAD(19736):[16:27:43.620578] Post report 0x7f6c84020930
THREAD(19736):[16:27:43.620619] Thread settings copy (malloc) from/to=0x12dac20/0x7f6c84023410
THREAD(19736):[16:27:43.620636] Listener thread accepting for TCP (sock=3)
THREAD(19737):[16:27:43.620677] Reporter thread started
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
THREAD(19737):[16:27:43.620780] Remove 0x7f6c84020930 from reporter job queue in rs
THREAD(19737):[16:27:43.620799] Free 0x7f6c84020930 in rs
THREAD(19736):[16:28:11.711590] Thread_run_wrapper(0x7f6c84023410 mode=1) thread counts tot/trfc=3/1
THREAD(19736):[16:28:11.711658] Thread settings copy (malloc) from/to=0x12dac20/0x7f6c84020b90
THREAD(19739):[16:28:11.711715] Server thread started (sock=4)
THREAD(19736):[16:28:11.711844] Listener thread accepting for TCP (sock=3)
THREAD(19739):[16:28:11.712438] Init 5000 element packet ring 0x7f6c80020f80
THREAD(19739):[16:28:11.712470] Init data report 0x7f6c80020a40 size 1336 using packetring 0x7f6c80020f80
THREAD(19739):[16:28:11.712496] Init connection report 0x7f6c80020a40
THREAD(19739):[16:28:11.712519] Update connection report 0x7f6c80020a40 winreq=0 actual=374400
THREAD(19739):[16:28:11.712535] Post report 0x7f6c80020a40
[ 4] local 192.168.1.1 port 5001 connected with 192.168.1.4 port 36276
[ ID] Interval Transfer Bandwidth
[ 4] 0.0- 1.0 sec 6.70 MBytes 56.2 Mbits/sec
[ 4] 1.0- 2.0 sec 7.43 MBytes 62.3 Mbits/sec
[ 4] 2.0- 3.0 sec 6.91 MBytes 58.0 Mbits/sec
[ 4] 3.0- 4.0 sec 6.89 MBytes 57.8 Mbits/sec
[ 4] 0.0- 4.2 sec 29.1 MBytes 58.7 Mbits/sec
THREAD(19737):[16:28:15.878976] Remove 0x7f6c80020a40 from reporter job queue in rs
THREAD(19739):[16:28:16.000140] Traffic thread thinks reporter is done with 0x7f6c80020a40
THREAD(19739):[16:28:16.000208] Server destructor close sock=4 drop-sock=-1
THREAD(19739):[16:28:16.000387] Free packet ring 0x7f6c80020f80 & condition variable await consumer 0x7f6c80020f98
THREAD(19739):[16:28:16.000487] Free report hdr 0x7f6c80020a40 delay counter=1030
THREAD(19739):[16:28:16.000529] Thread settings free=0x7f6c84023410
THREAD(19737):[16:29:07.000175] Thread settings free=0x12daf10
Thanks,
Bob