<div dir="ltr">Just an FYI in case anybody has interest in traffic tooling.<div><br></div><div>Bob<br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">---------- Forwarded message ---------<br>From: <strong class="gmail_sendername" dir="auto">Bob McMahon</strong> <span dir="auto"><<a href="mailto:bob.mcmahon@broadcom.com">bob.mcmahon@broadcom.com</a>></span><br>Date: Wed, Jul 17, 2019 at 1:39 PM<br>Subject: iperf 2 "short term" road map<br>To: iPerf User Group <<a href="mailto:iperf-users@lists.sourceforge.net">iperf-users@lists.sourceforge.net</a>><br>Cc: Panos Vouzis <<a href="mailto:panos@netbeez.net">panos@netbeez.net</a>><br></div><br><br><div dir="ltr">Hi All,<div><br></div><div>I've been doing a lot of work on iperf 2 to handle new testing features with a focus on TCP latency testing. Please do let me know your thoughts or suggest comments per the list below. No promises on time frame as this is not my primary day job ;)<br><br></div><div>The new features currently in development to include:<br><br>o) Reverse testing (invert the client/server roles after the connect) - requested by many for testing through NAT gateways<br>o) Latency tail analysis support (latency mostly needs to evaluated from a tail's perspective, provide those metrics w/o losing full distribution or, at least, tail information)<br>o) Support GPS (atomic clock) timestamps with all histogram metrics supporting post analysis of problem areas per distributed tooling (this assumes all tooling are synce'd to the GPS atomic clocks.)<br>o) TCP bidir testing support (bidirectional or full duplex traffic on the same socket)<br>o) TCP write-acks - the server will ack it's reads back (w/timestamp and byte no.) to the client per every client write(), useful for TCP streaming class of tests and doesn't require syncs to GPS clocks<br>o) TCP trip times, the write() to read() latency of every clients' writes - requires GPS clock sync - this is a one way metric<br>o) TCP connect only tests (run many parallel TCP connects and measure the connect times of each) - an issue for many mobile users is that TCP connect times are impacting user experience<br>o) -P support for more than 127 and now is only limited by the kernel/os (this already fixed)<br>o) CPU bound warning on exit - present a warning message when an iperf test seemed to be CPU bound vs network i/o bound<br>o) Mutex warn message on exit: present a warning on exit when the test spent time on mutex locks where that may be degrading network performance<br>o) Teporter thread consumption rate detector to add thread delay/suspends only when needed (currently, the code always add reporter thread delay.) This delay can and should reduce the cores cache invalidates per cache coherency and the use of shared memory (and its pointers) not under os mutex protection<br>o) Much better debugging support per ./configure --enable-thread-debug<br>o) Code clean up including using function pointers to replace per pacet, inline runtime tests (prepping to complete the C++ implementation vs today's hybrid of C and C++)</div><div>o) Support for raw sockets to test "L2 level" performance</div><div>o) Better code encapsulation (which includes freeing of memory)<br><br>Note: Here's an example debug that's now supported for those who want to delve into iperf internals. <br><br>Client:<br>[root@localhost iperf2-code]# src/iperf -c 192.168.1.1 -i 1 -t 4<br>THREAD(18576):[16:28:11.702942] Thread settings copy (malloc) from/to=0x67ac20/0x67b640<br>THREAD(18576):[16:28:11.703128] Thread_run_wrapper(0x67ac20 mode=2) thread counts tot/trfc=1/1<br>THREAD(18576):[16:28:11.703293] Thread_run_wrapper(0x67b640 mode=4) thread counts tot/trfc=2/1<br>THREAD(18577):[16:28:11.703305] Client thread started in constructor<br>THREAD(18578):[16:28:11.704364] Reporter thread started<br>THREAD(18577):[16:28:11.708777] Init settings report 0x7f2a38020fb0<br>THREAD(18577):[16:28:11.708839] Update connection report 0x7f2a38020fb0 winreq=0 actual=87040<br>THREAD(18577):[16:28:11.708853] Post report 0x7f2a38020fb0<br>------------------------------------------------------------<br>Client connecting to 192.168.1.1, TCP port 5001<br>THREAD(18577):[16:28:11.708895] Init 5000 element packet ring 0x7f2a38023ee0<br>TCP window size: 85.0 KByte (default)<br>------------------------------------------------------------<br>THREAD(18577):[16:28:11.708961] Init data report 0x7f2a380239a0 size 1336 using packetring 0x7f2a38023ee0<br>THREAD(18578):[16:28:11.709000] Remove 0x7f2a38020fb0 from reporter job queue in rs<br>THREAD(18577):[16:28:11.709013] Init connection report 0x7f2a380239a0<br>THREAD(18578):[16:28:11.709037] Free 0x7f2a38020fb0 in rs<br>THREAD(18577):[16:28:11.709053] Update connection report 0x7f2a380239a0 winreq=0 actual=87040<br>THREAD(18577):[16:28:11.709080] Post report 0x7f2a380239a0<br>[ 3] local 192.168.1.4 port 36276 connected with 192.168.1.1 port 5001<br>[ ID] Interval Transfer Bandwidth<br>[ 3] 0.0- 1.0 sec 7.75 MBytes 65.0 Mbits/sec<br>[ 3] 1.0- 2.0 sec 7.75 MBytes 65.0 Mbits/sec<br>[ 3] 2.0- 3.0 sec 6.50 MBytes 54.5 Mbits/sec<br>[ 3] 3.0- 4.0 sec 7.12 MBytes 59.8 Mbits/sec<br>[ 3] 0.0- 4.0 sec 29.1 MBytes 60.5 Mbits/sec<br>THREAD(18578):[16:28:15.747044] Remove 0x7f2a380239a0 from reporter job queue in rs<br>THREAD(18577):[16:28:15.747045] Traffic thread thinks reporter is done with 0x7f2a380239a0<br>THREAD(18577):[16:28:15.747098] Client destructor close sock=3<br>THREAD(18577):[16:28:15.747126] Free packet ring 0x7f2a38023ee0 & condition variable await consumer 0x7f2a38023ef8<br>THREAD(18577):[16:28:15.747156] Free report hdr 0x7f2a380239a0 delay counter=994<br>THREAD(18577):[16:28:15.747168] Thread settings free=0x67ac20<br>THREAD(18578):[16:28:16.000164] Thread settings free=0x67b640<br><br>Server:<br><br>[root@localhost iperf2-code]# src/iperf -s -i 1 <br>THREAD(19735):[16:27:43.620063] Thread settings copy (malloc) from/to=0x12dac20/0x12daf10<br>THREAD(19735):[16:27:43.620250] Thread_run_wrapper(0x12dac20 mode=8) thread counts tot/trfc=1/0<br>THREAD(19735):[16:27:43.620338] Thread_run_wrapper(0x12daf10 mode=3) thread counts tot/trfc=2/0<br>THREAD(19736):[16:27:43.620505] Init settings report 0x7f6c84020930<br>THREAD(19736):[16:27:43.620552] Update connection report 0x7f6c84020930 winreq=0 actual=87380<br>THREAD(19736):[16:27:43.620578] Post report 0x7f6c84020930<br>THREAD(19736):[16:27:43.620619] Thread settings copy (malloc) from/to=0x12dac20/0x7f6c84023410<br>THREAD(19736):[16:27:43.620636] Listener thread accepting for TCP (sock=3)<br>THREAD(19737):[16:27:43.620677] Reporter thread started<br>------------------------------------------------------------<br>Server listening on TCP port 5001<br>TCP window size: 85.3 KByte (default)<br>------------------------------------------------------------<br>THREAD(19737):[16:27:43.620780] Remove 0x7f6c84020930 from reporter job queue in rs<br>THREAD(19737):[16:27:43.620799] Free 0x7f6c84020930 in rs<br>THREAD(19736):[16:28:11.711590] Thread_run_wrapper(0x7f6c84023410 mode=1) thread counts tot/trfc=3/1<br>THREAD(19736):[16:28:11.711658] Thread settings copy (malloc) from/to=0x12dac20/0x7f6c84020b90<br>THREAD(19739):[16:28:11.711715] Server thread started (sock=4)<br>THREAD(19736):[16:28:11.711844] Listener thread accepting for TCP (sock=3)<br>THREAD(19739):[16:28:11.712438] Init 5000 element packet ring 0x7f6c80020f80<br>THREAD(19739):[16:28:11.712470] Init data report 0x7f6c80020a40 size 1336 using packetring 0x7f6c80020f80<br>THREAD(19739):[16:28:11.712496] Init connection report 0x7f6c80020a40<br>THREAD(19739):[16:28:11.712519] Update connection report 0x7f6c80020a40 winreq=0 actual=374400<br>THREAD(19739):[16:28:11.712535] Post report 0x7f6c80020a40<br>[ 4] local 192.168.1.1 port 5001 connected with 192.168.1.4 port 36276<br>[ ID] Interval Transfer Bandwidth<br>[ 4] 0.0- 1.0 sec 6.70 MBytes 56.2 Mbits/sec<br>[ 4] 1.0- 2.0 sec 7.43 MBytes 62.3 Mbits/sec<br>[ 4] 2.0- 3.0 sec 6.91 MBytes 58.0 Mbits/sec<br>[ 4] 3.0- 4.0 sec 6.89 MBytes 57.8 Mbits/sec<br>[ 4] 0.0- 4.2 sec 29.1 MBytes 58.7 Mbits/sec<br>THREAD(19737):[16:28:15.878976] Remove 0x7f6c80020a40 from reporter job queue in rs<br>THREAD(19739):[16:28:16.000140] Traffic thread thinks reporter is done with 0x7f6c80020a40<br>THREAD(19739):[16:28:16.000208] Server destructor close sock=4 drop-sock=-1<br>THREAD(19739):[16:28:16.000387] Free packet ring 0x7f6c80020f80 & condition variable await consumer 0x7f6c80020f98<br>THREAD(19739):[16:28:16.000487] Free report hdr 0x7f6c80020a40 delay counter=1030<br>THREAD(19739):[16:28:16.000529] Thread settings free=0x7f6c84023410<br>THREAD(19737):[16:29:07.000175] Thread settings free=0x12daf10<br><br>Thanks,<br>Bob<br></div></div>
</div></div></div>