From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 803FF3CB36 for ; Wed, 17 Jul 2019 17:32:27 -0400 (EDT) Received: by mail-wr1-x432.google.com with SMTP id f9so26327378wre.12 for ; Wed, 17 Jul 2019 14:32:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=NI81JG3x53m+iohxS7cF5t1YaKNyLorfFnwjmawK+I4=; b=CBERNv3XTtvKQFEU7KgiNojlwURIdqbLS5LN0YPEQCngBbMcpWoZciSVBQZQAbT+TZ eT4CNzt089gUZ2onF9AZLXpQvyh0MmO5sqVV7MvKiyWsuHOJ5PNfixm5OdtaDgi8iwJU PBIPMcjnkYSviWmB+tcDhwAqyHl4XZuUP+zXs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=NI81JG3x53m+iohxS7cF5t1YaKNyLorfFnwjmawK+I4=; b=n+U8vjHjUPLe5u6TpoJ1WWe61FtOCHd/8eziliRUEnl7E1mPoc3nVlddiN7FtbKcq6 6ppRojn8pcVlFWH3HDsxaf1PBkAY0qVrzLQLzNZq4HB7Z2QscHQ0inbkASBfTihWFzuv u7q6KpWTmteYphWgVhns2NdC1dkrT4T4cl+2+PdnWksIwsmgHQkPp/AaAzEtmxxsd7Gi 40EZjkppLsFlNKLRusBY0KdoEQjcnnhYWeQDbEuTIxV9L5vhsD7llCWGlOkBwsFH8KPK rJA7oiYbryBwrRFkQNz6F5/AlJNK5hrj2QR8H1UREfEYoc8ec8aOmB8rK0rFKLJa7E1/ uIbw== X-Gm-Message-State: APjAAAVKIvyzGtkboJ7fMhZ60TvpFFjideAib9mq90IsjzInkWo/dNJ6 9vRuMMJn+4R9NuLmoaUNsbDCQuBEzdnUMsBn+NKN/rrx X-Google-Smtp-Source: APXvYqyeBdR8dgJjAEYg39wZyCQyJJdu5IKCA/zke9r+hoRmnQcP5DYj1m2HLWYtbxmNQozP8sleNdLUccwz2N8JC+I= X-Received: by 2002:adf:e941:: with SMTP id m1mr35759949wrn.279.1563399146066; Wed, 17 Jul 2019 14:32:26 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Bob McMahon Date: Wed, 17 Jul 2019 14:32:14 -0700 Message-ID: To: Make-Wifi-fast Content-Type: multipart/alternative; boundary="000000000000be8e2b058de73aa9" Subject: [Make-wifi-fast] Fwd: iperf 2 "short term" road map X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jul 2019 21:32:27 -0000 --000000000000be8e2b058de73aa9 Content-Type: text/plain; charset="UTF-8" Just an FYI in case anybody has interest in traffic tooling. Bob ---------- Forwarded message --------- From: Bob McMahon Date: Wed, Jul 17, 2019 at 1:39 PM Subject: iperf 2 "short term" road map To: iPerf User Group Cc: Panos Vouzis Hi All, I've been doing a lot of work on iperf 2 to handle new testing features with a focus on TCP latency testing. Please do let me know your thoughts or suggest comments per the list below. No promises on time frame as this is not my primary day job ;) The new features currently in development to include: o) Reverse testing (invert the client/server roles after the connect) - requested by many for testing through NAT gateways o) Latency tail analysis support (latency mostly needs to evaluated from a tail's perspective, provide those metrics w/o losing full distribution or, at least, tail information) o) Support GPS (atomic clock) timestamps with all histogram metrics supporting post analysis of problem areas per distributed tooling (this assumes all tooling are synce'd to the GPS atomic clocks.) o) TCP bidir testing support (bidirectional or full duplex traffic on the same socket) o) TCP write-acks - the server will ack it's reads back (w/timestamp and byte no.) to the client per every client write(), useful for TCP streaming class of tests and doesn't require syncs to GPS clocks o) TCP trip times, the write() to read() latency of every clients' writes - requires GPS clock sync - this is a one way metric o) TCP connect only tests (run many parallel TCP connects and measure the connect times of each) - an issue for many mobile users is that TCP connect times are impacting user experience o) -P support for more than 127 and now is only limited by the kernel/os (this already fixed) o) CPU bound warning on exit - present a warning message when an iperf test seemed to be CPU bound vs network i/o bound o) Mutex warn message on exit: present a warning on exit when the test spent time on mutex locks where that may be degrading network performance o) Teporter thread consumption rate detector to add thread delay/suspends only when needed (currently, the code always add reporter thread delay.) This delay can and should reduce the cores cache invalidates per cache coherency and the use of shared memory (and its pointers) not under os mutex protection o) Much better debugging support per ./configure --enable-thread-debug o) Code clean up including using function pointers to replace per pacet, inline runtime tests (prepping to complete the C++ implementation vs today's hybrid of C and C++) o) Support for raw sockets to test "L2 level" performance o) Better code encapsulation (which includes freeing of memory) Note: Here's an example debug that's now supported for those who want to delve into iperf internals. Client: [root@localhost iperf2-code]# src/iperf -c 192.168.1.1 -i 1 -t 4 THREAD(18576):[16:28:11.702942] Thread settings copy (malloc) from/to=0x67ac20/0x67b640 THREAD(18576):[16:28:11.703128] Thread_run_wrapper(0x67ac20 mode=2) thread counts tot/trfc=1/1 THREAD(18576):[16:28:11.703293] Thread_run_wrapper(0x67b640 mode=4) thread counts tot/trfc=2/1 THREAD(18577):[16:28:11.703305] Client thread started in constructor THREAD(18578):[16:28:11.704364] Reporter thread started THREAD(18577):[16:28:11.708777] Init settings report 0x7f2a38020fb0 THREAD(18577):[16:28:11.708839] Update connection report 0x7f2a38020fb0 winreq=0 actual=87040 THREAD(18577):[16:28:11.708853] Post report 0x7f2a38020fb0 ------------------------------------------------------------ Client connecting to 192.168.1.1, TCP port 5001 THREAD(18577):[16:28:11.708895] Init 5000 element packet ring 0x7f2a38023ee0 TCP window size: 85.0 KByte (default) ------------------------------------------------------------ THREAD(18577):[16:28:11.708961] Init data report 0x7f2a380239a0 size 1336 using packetring 0x7f2a38023ee0 THREAD(18578):[16:28:11.709000] Remove 0x7f2a38020fb0 from reporter job queue in rs THREAD(18577):[16:28:11.709013] Init connection report 0x7f2a380239a0 THREAD(18578):[16:28:11.709037] Free 0x7f2a38020fb0 in rs THREAD(18577):[16:28:11.709053] Update connection report 0x7f2a380239a0 winreq=0 actual=87040 THREAD(18577):[16:28:11.709080] Post report 0x7f2a380239a0 [ 3] local 192.168.1.4 port 36276 connected with 192.168.1.1 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 7.75 MBytes 65.0 Mbits/sec [ 3] 1.0- 2.0 sec 7.75 MBytes 65.0 Mbits/sec [ 3] 2.0- 3.0 sec 6.50 MBytes 54.5 Mbits/sec [ 3] 3.0- 4.0 sec 7.12 MBytes 59.8 Mbits/sec [ 3] 0.0- 4.0 sec 29.1 MBytes 60.5 Mbits/sec THREAD(18578):[16:28:15.747044] Remove 0x7f2a380239a0 from reporter job queue in rs THREAD(18577):[16:28:15.747045] Traffic thread thinks reporter is done with 0x7f2a380239a0 THREAD(18577):[16:28:15.747098] Client destructor close sock=3 THREAD(18577):[16:28:15.747126] Free packet ring 0x7f2a38023ee0 & condition variable await consumer 0x7f2a38023ef8 THREAD(18577):[16:28:15.747156] Free report hdr 0x7f2a380239a0 delay counter=994 THREAD(18577):[16:28:15.747168] Thread settings free=0x67ac20 THREAD(18578):[16:28:16.000164] Thread settings free=0x67b640 Server: [root@localhost iperf2-code]# src/iperf -s -i 1 THREAD(19735):[16:27:43.620063] Thread settings copy (malloc) from/to=0x12dac20/0x12daf10 THREAD(19735):[16:27:43.620250] Thread_run_wrapper(0x12dac20 mode=8) thread counts tot/trfc=1/0 THREAD(19735):[16:27:43.620338] Thread_run_wrapper(0x12daf10 mode=3) thread counts tot/trfc=2/0 THREAD(19736):[16:27:43.620505] Init settings report 0x7f6c84020930 THREAD(19736):[16:27:43.620552] Update connection report 0x7f6c84020930 winreq=0 actual=87380 THREAD(19736):[16:27:43.620578] Post report 0x7f6c84020930 THREAD(19736):[16:27:43.620619] Thread settings copy (malloc) from/to=0x12dac20/0x7f6c84023410 THREAD(19736):[16:27:43.620636] Listener thread accepting for TCP (sock=3) THREAD(19737):[16:27:43.620677] Reporter thread started ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 85.3 KByte (default) ------------------------------------------------------------ THREAD(19737):[16:27:43.620780] Remove 0x7f6c84020930 from reporter job queue in rs THREAD(19737):[16:27:43.620799] Free 0x7f6c84020930 in rs THREAD(19736):[16:28:11.711590] Thread_run_wrapper(0x7f6c84023410 mode=1) thread counts tot/trfc=3/1 THREAD(19736):[16:28:11.711658] Thread settings copy (malloc) from/to=0x12dac20/0x7f6c84020b90 THREAD(19739):[16:28:11.711715] Server thread started (sock=4) THREAD(19736):[16:28:11.711844] Listener thread accepting for TCP (sock=3) THREAD(19739):[16:28:11.712438] Init 5000 element packet ring 0x7f6c80020f80 THREAD(19739):[16:28:11.712470] Init data report 0x7f6c80020a40 size 1336 using packetring 0x7f6c80020f80 THREAD(19739):[16:28:11.712496] Init connection report 0x7f6c80020a40 THREAD(19739):[16:28:11.712519] Update connection report 0x7f6c80020a40 winreq=0 actual=374400 THREAD(19739):[16:28:11.712535] Post report 0x7f6c80020a40 [ 4] local 192.168.1.1 port 5001 connected with 192.168.1.4 port 36276 [ ID] Interval Transfer Bandwidth [ 4] 0.0- 1.0 sec 6.70 MBytes 56.2 Mbits/sec [ 4] 1.0- 2.0 sec 7.43 MBytes 62.3 Mbits/sec [ 4] 2.0- 3.0 sec 6.91 MBytes 58.0 Mbits/sec [ 4] 3.0- 4.0 sec 6.89 MBytes 57.8 Mbits/sec [ 4] 0.0- 4.2 sec 29.1 MBytes 58.7 Mbits/sec THREAD(19737):[16:28:15.878976] Remove 0x7f6c80020a40 from reporter job queue in rs THREAD(19739):[16:28:16.000140] Traffic thread thinks reporter is done with 0x7f6c80020a40 THREAD(19739):[16:28:16.000208] Server destructor close sock=4 drop-sock=-1 THREAD(19739):[16:28:16.000387] Free packet ring 0x7f6c80020f80 & condition variable await consumer 0x7f6c80020f98 THREAD(19739):[16:28:16.000487] Free report hdr 0x7f6c80020a40 delay counter=1030 THREAD(19739):[16:28:16.000529] Thread settings free=0x7f6c84023410 THREAD(19737):[16:29:07.000175] Thread settings free=0x12daf10 Thanks, Bob --000000000000be8e2b058de73aa9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Just an FYI in case anybody has interest in traffic toolin= g.

Bob

---------- Forwarded message ---------
From: Bob McMahon <bob.mcmahon@broad= com.com>
Date: Wed, Jul 17, 2019 at 1:39 PM
Subject: ip= erf 2 "short term" road map
To: iPerf User Group <iperf-users@lists.sourceforge= .net>
Cc: Panos Vouzis <p= anos@netbeez.net>


Hi All,

=
I've been doing a lot of work on iperf 2 to handle new testi= ng features with a focus on TCP latency testing.=C2=A0 Please do let me kno= w your thoughts or suggest comments per the list below.=C2=A0 No promises o= n time frame as this is not my primary day job ;)

The new= features currently in development to include:

o) Reverse testing (i= nvert the client/server roles after the connect) - requested by many for te= sting through NAT gateways
o) Latency tail analysis support (latency mos= tly needs to evaluated from a tail's perspective, provide those metrics= w/o losing full distribution or, at least, tail information)
o) Support= GPS (atomic clock) timestamps with all histogram metrics supporting post a= nalysis of problem areas per distributed tooling (this assumes all tooling = are synce'd to the GPS atomic clocks.)
o) TCP bidir testing support = (bidirectional or full duplex traffic on the same socket)
o) TCP write-a= cks - the server will ack it's reads back (w/timestamp and byte no.) to= the client per every client write(), useful for TCP streaming class of tes= ts and doesn't require syncs to GPS clocks
o) TCP trip times, the wr= ite() to read() latency of every clients' writes - requires GPS clock s= ync - this is a one way metric
o) TCP connect only tests (run many paral= lel TCP connects and measure the connect times of each) - an issue for many= mobile users is that TCP connect times are impacting user experience
o)= -P support for more than 127 and now is only limited by the kernel/os=C2= =A0 (this already fixed)
o) CPU bound warning on exit - present a warnin= g message when an iperf test seemed to be CPU bound vs network i/o boundo) Mutex warn message on exit: present a warning on exit when the test spe= nt time on mutex locks where that may be degrading network performance
o= ) Teporter thread consumption rate detector to add thread delay/suspends on= ly when needed (currently, the code always add reporter thread delay.)=C2= =A0 This delay can and should reduce the cores cache invalidates per cache = coherency and the use of shared memory (and its pointers) not under os mute= x protection
o) Much better debugging support per ./configure --enable-t= hread-debug
o) Code clean up including using function pointers to replac= e per pacet, inline runtime tests (prepping to complete the C++ implementat= ion vs today's hybrid of C and C++)
o) Support for raw socket= s to test "L2 level" performance
o) Better code encapsu= lation (which includes freeing of memory)

Note: =C2=A0Here's an = example debug that's now supported for those who want to delve into ipe= rf internals. =C2=A0

Client:
[root@localhost iperf2-code]# src/ip= erf -c 192.168.1.1 -i 1 -t 4
THREAD(18576):[16:28:11.702942] Thread sett= ings copy (malloc) from/to=3D0x67ac20/0x67b640
THREAD(18576):[16:28:11.7= 03128] Thread_run_wrapper(0x67ac20 mode=3D2) thread counts tot/trfc=3D1/1THREAD(18576):[16:28:11.703293] Thread_run_wrapper(0x67b640 mode=3D4) thr= ead counts tot/trfc=3D2/1
THREAD(18577):[16:28:11.703305] Client thread = started in constructor
THREAD(18578):[16:28:11.704364] Reporter thread s= tarted
THREAD(18577):[16:28:11.708777] Init settings report 0x7f2a38020f= b0
THREAD(18577):[16:28:11.708839] Update connection report 0x7f2a38020f= b0 winreq=3D0 actual=3D87040
THREAD(18577):[16:28:11.708853] Post report= 0x7f2a38020fb0
--------------------------------------------------------= ----
Client connecting to 192.168.1.1, TCP port 5001
THREAD(18577):[1= 6:28:11.708895] Init 5000 element packet ring 0x7f2a38023ee0
TCP window = size: 85.0 KByte (default)
---------------------------------------------= ---------------
THREAD(18577):[16:28:11.708961] Init data report 0x7f2a3= 80239a0 size 1336 using packetring 0x7f2a38023ee0
THREAD(18578):[16:28:1= 1.709000] Remove 0x7f2a38020fb0 from reporter job queue in rs
THREAD(185= 77):[16:28:11.709013] Init connection report 0x7f2a380239a0
THREAD(18578= ):[16:28:11.709037] Free 0x7f2a38020fb0 in rs
THREAD(18577):[16:28:11.70= 9053] Update connection report 0x7f2a380239a0 winreq=3D0 actual=3D87040
= THREAD(18577):[16:28:11.709080] Post report 0x7f2a380239a0
[ =C2=A03] lo= cal 192.168.1.4 port 36276 connected with 192.168.1.1 port 5001
[ ID] In= terval =C2=A0 =C2=A0 =C2=A0 Transfer =C2=A0 =C2=A0 Bandwidth
[ =C2=A03] = =C2=A00.0- 1.0 sec =C2=A07.75 MBytes =C2=A065.0 Mbits/sec
[ =C2=A03] =C2= =A01.0- 2.0 sec =C2=A07.75 MBytes =C2=A065.0 Mbits/sec
[ =C2=A03] =C2=A0= 2.0- 3.0 sec =C2=A06.50 MBytes =C2=A054.5 Mbits/sec
[ =C2=A03] =C2=A03.0= - 4.0 sec =C2=A07.12 MBytes =C2=A059.8 Mbits/sec
[ =C2=A03] =C2=A00.0- 4= .0 sec =C2=A029.1 MBytes =C2=A060.5 Mbits/sec
THREAD(18578):[16:28:15.74= 7044] Remove 0x7f2a380239a0 from reporter job queue in rs
THREAD(18577):= [16:28:15.747045] Traffic thread thinks reporter is done with 0x7f2a380239a= 0
THREAD(18577):[16:28:15.747098] Client destructor close sock=3D3
TH= READ(18577):[16:28:15.747126] Free packet ring 0x7f2a38023ee0 & conditi= on variable await consumer 0x7f2a38023ef8
THREAD(18577):[16:28:15.747156= ] Free report hdr 0x7f2a380239a0 delay counter=3D994
THREAD(18577):[16:2= 8:15.747168] Thread settings free=3D0x67ac20
THREAD(18578):[16:28:16.000= 164] Thread settings free=3D0x67b640

Server:

[root@localhost = iperf2-code]# src/iperf -s -i 1 =C2=A0
THREAD(19735):[16:27:43.620063] T= hread settings copy (malloc) from/to=3D0x12dac20/0x12daf10
THREAD(19735)= :[16:27:43.620250] Thread_run_wrapper(0x12dac20 mode=3D8) thread counts tot= /trfc=3D1/0
THREAD(19735):[16:27:43.620338] Thread_run_wrapper(0x12daf10= mode=3D3) thread counts tot/trfc=3D2/0
THREAD(19736):[16:27:43.620505] = Init settings report 0x7f6c84020930
THREAD(19736):[16:27:43.620552] Upda= te connection report 0x7f6c84020930 winreq=3D0 actual=3D87380
THREAD(197= 36):[16:27:43.620578] Post report 0x7f6c84020930
THREAD(19736):[16:27:43= .620619] Thread settings copy (malloc) from/to=3D0x12dac20/0x7f6c84023410THREAD(19736):[16:27:43.620636] Listener thread accepting for TCP (sock= =3D3)
THREAD(19737):[16:27:43.620677] Reporter thread started
-------= -----------------------------------------------------
Server listening o= n TCP port 5001
TCP window size: 85.3 KByte (default)
---------------= ---------------------------------------------
THREAD(19737):[16:27:43.62= 0780] Remove 0x7f6c84020930 from reporter job queue in rs
THREAD(19737):= [16:27:43.620799] Free 0x7f6c84020930 in rs
THREAD(19736):[16:28:11.7115= 90] Thread_run_wrapper(0x7f6c84023410 mode=3D1) thread counts tot/trfc=3D3/= 1
THREAD(19736):[16:28:11.711658] Thread settings copy (malloc) from/to= =3D0x12dac20/0x7f6c84020b90
THREAD(19739):[16:28:11.711715] Server threa= d started (sock=3D4)
THREAD(19736):[16:28:11.711844] Listener thread acc= epting for TCP (sock=3D3)
THREAD(19739):[16:28:11.712438] Init 5000 elem= ent packet ring 0x7f6c80020f80
THREAD(19739):[16:28:11.712470] Init data= report 0x7f6c80020a40 size 1336 using packetring 0x7f6c80020f80
THREAD(= 19739):[16:28:11.712496] Init connection report 0x7f6c80020a40
THREAD(19= 739):[16:28:11.712519] Update connection report 0x7f6c80020a40 winreq=3D0 a= ctual=3D374400
THREAD(19739):[16:28:11.712535] Post report 0x7f6c80020a4= 0
[ =C2=A04] local 192.168.1.1 port 5001 connected with 192.168.1.4 port= 36276
[ ID] Interval =C2=A0 =C2=A0 =C2=A0 Transfer =C2=A0 =C2=A0 Bandwi= dth
[ =C2=A04] =C2=A00.0- 1.0 sec =C2=A06.70 MBytes =C2=A056.2 Mbits/sec=
[ =C2=A04] =C2=A01.0- 2.0 sec =C2=A07.43 MBytes =C2=A062.3 Mbits/sec[ =C2=A04] =C2=A02.0- 3.0 sec =C2=A06.91 MBytes =C2=A058.0 Mbits/sec
[ = =C2=A04] =C2=A03.0- 4.0 sec =C2=A06.89 MBytes =C2=A057.8 Mbits/sec
[ =C2= =A04] =C2=A00.0- 4.2 sec =C2=A029.1 MBytes =C2=A058.7 Mbits/sec
THREAD(1= 9737):[16:28:15.878976] Remove 0x7f6c80020a40 from reporter job queue in rs=
THREAD(19739):[16:28:16.000140] Traffic thread thinks reporter is done = with 0x7f6c80020a40
THREAD(19739):[16:28:16.000208] Server destructor cl= ose sock=3D4 drop-sock=3D-1
THREAD(19739):[16:28:16.000387] Free packet = ring 0x7f6c80020f80 & condition variable await consumer 0x7f6c80020f98<= br>THREAD(19739):[16:28:16.000487] Free report hdr 0x7f6c80020a40 delay cou= nter=3D1030
THREAD(19739):[16:28:16.000529] Thread settings free=3D0x7f6= c84023410
THREAD(19737):[16:29:07.000175] Thread settings free=3D0x12daf= 10

Thanks,
Bob
--000000000000be8e2b058de73aa9--