From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 145413B2A4 for ; Fri, 13 Oct 2017 21:46:25 -0400 (EDT) Received: by mail-wm0-x22d.google.com with SMTP id b189so24151528wmd.4 for ; Fri, 13 Oct 2017 18:46:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=mlMaYP+Z+WZVmJsylz6v0xcgS5Z1r7Z8UXlvP9mBx0g=; b=fmpBS+Gy4K0QPTsSvxapJb9usMsP/rL0NchnDv7Z6fL4Ixl+4l3Jk/Vl5zu3kAyKSj xlx+7uN7ig1ybLspUwAhTslXmRf0iwCIJjMboz5HyOTMFVYCG7dowFO0LcsNPn4hbB52 x7XTyEmnbs6XpcaqVg5/7L1CsohHNpptp6Ezc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=mlMaYP+Z+WZVmJsylz6v0xcgS5Z1r7Z8UXlvP9mBx0g=; b=W9K3zakOqxH+EeCvi+iBwYEoG0buiUPtkfqeiLByGpTYvncbBAY1VSn8a8E4yDT0G4 hm1NhpaaUUMrvkLStku0SHew3PRmA7WKzSV5zOURugd2s1/uhD5WXbzyL+eu5F7S0101 EAQlYKQFb4rruI+po2u3Q20Ntoo4oMiJAzY8MFhcjT29s3TpW1Jp/0sWoSvIu6ZKkSsG TCh9a9MhV7xNQG9T0/+Ro5VFhlWQtUfIyMG8aUYuX+hM1QkzlMcFX61WaofNwZMHpON9 QrZ9cTfKtRj0b34mxFaGYsIiLT7W9nqQOYAQgCcxiRXAtjWqmiW7H2hkw4bBBFjl+eBC Sgcg== X-Gm-Message-State: AMCzsaXjbCxnFK3mu06n9pyUAM7n+qDf888Zy9/piv7MWC8Vy8EzQGgN WwnflBA8MyLXqRWr6dgnvBmBnYLrtMM5KdDnb5pr4w== X-Google-Smtp-Source: AOwi7QB2Gc86LgMJx862H2E3XT7B+5WE7tWsDCyGBlr+Gj3rkt3OQUP9xIBLMmN/uVEZuR/arRMiZZVH7yHhz8agHrM= X-Received: by 10.80.182.116 with SMTP id c49mr4158824ede.124.1507945584684; Fri, 13 Oct 2017 18:46:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.204.220 with HTTP; Fri, 13 Oct 2017 18:46:23 -0700 (PDT) In-Reply-To: References: <1507581711.45638427@apps.rackspace.com> <20171011233056.3634daea@redhat.com> <877ew0bwb3.fsf@toke.dk> <87mv4v9z2a.fsf@toke.dk> From: Bob McMahon Date: Fri, 13 Oct 2017 18:46:23 -0700 Message-ID: To: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= Cc: Jesper Dangaard Brouer , make-wifi-fast@lists.bufferbloat.net, Johannes Berg Content-Type: multipart/alternative; boundary="94eb2c0df8c4ea9b45055b77f0e8" Subject: Re: [Make-wifi-fast] less latency, more filling... for wifi X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 01:46:26 -0000 --94eb2c0df8c4ea9b45055b77f0e8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable FYI, here's an example output using two brix PCs over their wired ethernet connected by a Cisco SG300 switch. Note: the ptpd stats (not shown) displays the clock corrections and suggests them to be within 10 microseconds. [rjmcmahon@rjm-fedora etc]$ iperf -c 192.168.100.33 -u -e -t 2 -b 2kpps ------------------------------------------------------------ Client connecting to 192.168.100.33, UDP port 5001 with pid 29952 Sending 1470 byte datagrams, IPG target: 500.00 us (kalman adjust) UDP buffer size: 208 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.100.67 port 50062 connected with 192.168.100.33 port 5001 [ ID] Interval Transfer Bandwidth PPS [ 3] 0.00-2.00 sec 5.61 MBytes 23.5 Mbits/sec 1999 pps [ 3] Sent 4001 datagrams [ 3] Server Report: [ 3] 0.00-2.00 sec 5.61 MBytes 23.5 Mbits/sec 0.012 ms 0/ 4001 (0%) -/-/-/- ms 2000 pps [rjmcmahon@hera ~]$ iperf -s -u -e -i 0.1 ------------------------------------------------------------ Server listening on UDP port 5001 with pid 5178 Receiving 1470 byte datagrams UDP buffer size: 208 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.100.33 port 5001 connected with 192.168.100.67 port 57325 [ ID] Interval Transfer Bandwidth Jitter Lost/Total Latency avg/min/max/stdev PPS [ 3] 0.00-0.10 sec 289 KBytes 23.6 Mbits/sec 0.013 ms 0/ 201 (0%) 0.142/ 0.018/ 0.192/ 0.025 ms 2005 pps [ 3] 0.10-0.20 sec 287 KBytes 23.5 Mbits/sec 0.020 ms 0/ 200 (0%) 0.157/ 0.101/ 0.207/ 0.015 ms 1999 pps [ 3] 0.20-0.30 sec 287 KBytes 23.5 Mbits/sec 0.014 ms 0/ 200 (0%) 0.155/ 0.071/ 0.212/ 0.018 ms 2002 pps [ 3] 0.30-0.40 sec 287 KBytes 23.5 Mbits/sec 0.018 ms 0/ 200 (0%) 0.146/-0.007/ 0.187/ 0.018 ms 1999 pps [ 3] 0.40-0.50 sec 287 KBytes 23.5 Mbits/sec 0.021 ms 0/ 200 (0%) 0.151/ 0.021/ 0.208/ 0.018 ms 2000 pps [ 3] 0.50-0.60 sec 287 KBytes 23.5 Mbits/sec 0.016 ms 0/ 200 (0%) 0.148/ 0.043/ 0.192/ 0.018 ms 2000 pps [ 3] 0.60-0.70 sec 287 KBytes 23.5 Mbits/sec 0.019 ms 0/ 200 (0%) 0.152/ 0.041/ 0.199/ 0.018 ms 2001 pps [ 3] 0.70-0.80 sec 287 KBytes 23.5 Mbits/sec 0.016 ms 0/ 200 (0%) 0.144/ 0.071/ 0.206/ 0.017 ms 2001 pps [ 3] 0.80-0.90 sec 287 KBytes 23.5 Mbits/sec 0.015 ms 0/ 200 (0%) 0.140/ 0.111/ 0.186/ 0.014 ms 1999 pps [ 3] 0.90-1.00 sec 287 KBytes 23.5 Mbits/sec 0.022 ms 0/ 200 (0%) 0.154/ 0.111/ 0.222/ 0.019 ms 2000 pps [ 3] 1.00-1.10 sec 287 KBytes 23.5 Mbits/sec 0.014 ms 0/ 200 (0%) 0.152/ 0.036/ 0.197/ 0.017 ms 2000 pps [ 3] 1.10-1.20 sec 287 KBytes 23.5 Mbits/sec 0.015 ms 0/ 200 (0%) 0.153/-0.007/ 0.186/ 0.020 ms 2001 pps [ 3] 1.20-1.30 sec 287 KBytes 23.5 Mbits/sec 0.013 ms 0/ 200 (0%) 0.149/ 0.035/ 0.207/ 0.018 ms 2000 pps [ 3] 1.30-1.40 sec 287 KBytes 23.5 Mbits/sec 0.014 ms 0/ 200 (0%) 0.160/ 0.116/ 0.233/ 0.018 ms 2000 pps [ 3] 1.40-1.50 sec 287 KBytes 23.5 Mbits/sec 0.014 ms 0/ 200 (0%) 0.159/ 0.122/ 0.207/ 0.015 ms 2000 pps [ 3] 1.50-1.60 sec 287 KBytes 23.5 Mbits/sec 0.016 ms 0/ 200 (0%) 0.158/ 0.066/ 0.201/ 0.015 ms 1999 pps [ 3] 1.60-1.70 sec 287 KBytes 23.5 Mbits/sec 0.017 ms 0/ 200 (0%) 0.162/ 0.076/ 0.203/ 0.016 ms 2000 pps [ 3] 1.70-1.80 sec 287 KBytes 23.5 Mbits/sec 0.014 ms 0/ 200 (0%) 0.154/ 0.073/ 0.195/ 0.016 ms 2002 pps [ 3] 1.80-1.90 sec 287 KBytes 23.5 Mbits/sec 0.015 ms 0/ 200 (0%) 0.154/ 0.113/ 0.213/ 0.017 ms 1999 pps [ 3] 1.90-2.00 sec 287 KBytes 23.5 Mbits/sec 0.019 ms 0/ 200 (0%) 0.152/ 0.124/ 0.208/ 0.016 ms 1999 pps [ 3] 0.00-2.00 sec 5.61 MBytes 23.5 Mbits/sec 0.019 ms 0/ 4001 (0%) 0.151/-0.007/ 0.233/ 0.018 ms 2000 pps Here's a full line rate (1Gbs ethernet) run [rjmcmahon@rjm-fedora etc]$ iperf -c 192.168.100.33 -u -e -t 2 -b 200kpss -i 1 -w 2M ------------------------------------------------------------ Client connecting to 192.168.100.33, UDP port 5001 with pid 30626 Sending 1470 byte datagrams, IPG target: 5.00 us (kalman adjust) UDP buffer size: 416 KByte (WARNING: requested 2.00 MByte) ------------------------------------------------------------ [ 3] local 192.168.100.67 port 57349 connected with 192.168.100.33 port 5001 [ ID] Interval Transfer Bandwidth PPS [ 3] 0.00-1.00 sec 114 MBytes 958 Mbits/sec 81421 pps [ 3] 1.00-2.00 sec 114 MBytes 958 Mbits/sec 81380 pps [ 3] 0.00-2.00 sec 228 MBytes 957 Mbits/sec 81399 pps [ 3] Sent 162874 datagrams [ 3] Server Report: [ 3] 0.00-2.00 sec 228 MBytes 957 Mbits/sec 0.084 ms 0/162874 (0%) 1.641/ 0.253/ 2.466/ 0.114 ms 81383 pps [ 3] local 192.168.100.33 port 5001 connected with 192.168.100.67 port 57349 [ 3] 0.00-0.10 sec 11.4 MBytes 960 Mbits/sec 0.016 ms 0/ 8166 (0%) 1.615/ 0.253/ 2.309/ 0.355 ms 81606 pps [ 3] 0.10-0.20 sec 11.4 MBytes 956 Mbits/sec 0.016 ms 0/ 8133 (0%) 1.657/ 0.936/ 2.325/ 0.348 ms 81350 pps [ 3] 0.20-0.30 sec 11.4 MBytes 957 Mbits/sec 0.021 ms 0/ 8139 (0%) 1.657/ 0.950/ 2.400/ 0.348 ms 81383 pps [ 3] 0.30-0.40 sec 11.4 MBytes 958 Mbits/sec 0.016 ms 0/ 8144 (0%) 1.652/ 0.953/ 2.357/ 0.342 ms 81380 pps [ 3] 0.40-0.50 sec 11.4 MBytes 956 Mbits/sec 0.069 ms 0/ 8131 (0%) 1.644/ 0.947/ 2.368/ 0.341 ms 81384 pps [ 3] 0.50-0.60 sec 11.4 MBytes 957 Mbits/sec 0.072 ms 0/ 8138 (0%) 1.649/ 0.949/ 2.381/ 0.337 ms 81372 pps [ 3] 0.60-0.70 sec 11.4 MBytes 957 Mbits/sec 0.025 ms 0/ 8139 (0%) 1.639/ 0.952/ 2.357/ 0.342 ms 81383 pps [ 3] 0.70-0.80 sec 11.4 MBytes 957 Mbits/sec 0.014 ms 0/ 8135 (0%) 1.643/ 0.944/ 2.368/ 0.343 ms 81390 pps [ 3] 0.80-0.90 sec 11.4 MBytes 957 Mbits/sec 0.016 ms 0/ 8142 (0%) 1.639/ 0.946/ 2.361/ 0.341 ms 81366 pps [ 3] 0.90-1.00 sec 11.4 MBytes 957 Mbits/sec 0.015 ms 0/ 8142 (0%) 1.635/ 0.932/ 2.378/ 0.342 ms 81387 pps [ 3] 1.00-1.10 sec 11.4 MBytes 957 Mbits/sec 0.015 ms 0/ 8138 (0%) 1.633/ 0.934/ 2.359/ 0.341 ms 81373 pps [ 3] 1.10-1.20 sec 11.4 MBytes 957 Mbits/sec 0.010 ms 0/ 8135 (0%) 1.636/ 0.947/ 2.361/ 0.342 ms 81444 pps [ 3] 1.20-1.30 sec 11.4 MBytes 957 Mbits/sec 0.091 ms 0/ 8140 (0%) 1.624/ 0.908/ 2.363/ 0.354 ms 81400 pps [ 3] 1.30-1.40 sec 11.4 MBytes 956 Mbits/sec 0.016 ms 0/ 8133 (0%) 1.616/ 0.917/ 2.325/ 0.345 ms 81296 pps [ 3] 1.40-1.50 sec 11.4 MBytes 957 Mbits/sec 0.012 ms 0/ 8138 (0%) 1.626/ 0.918/ 2.361/ 0.346 ms 81414 pps [ 3] 1.50-1.60 sec 11.4 MBytes 957 Mbits/sec 0.015 ms 0/ 8136 (0%) 1.626/ 0.934/ 2.352/ 0.339 ms 81339 pps [ 3] 1.60-1.70 sec 11.4 MBytes 957 Mbits/sec 0.015 ms 0/ 8141 (0%) 1.633/ 0.930/ 2.351/ 0.341 ms 81376 pps [ 3] 1.70-1.80 sec 11.4 MBytes 956 Mbits/sec 0.017 ms 0/ 8133 (0%) 1.627/ 0.929/ 2.354/ 0.339 ms 81377 pps [ 3] 1.80-1.90 sec 11.4 MBytes 957 Mbits/sec 0.088 ms 0/ 8139 (0%) 1.659/ 0.895/ 2.396/ 0.343 ms 81330 pps [ 3] 1.90-2.00 sec 11.4 MBytes 956 Mbits/sec 0.013 ms 0/ 8128 (0%) 1.709/ 0.996/ 2.466/ 0.342 ms 81348 pps [ 3] 0.00-2.00 sec 228 MBytes 957 Mbits/sec 0.085 ms 0/162874 (0%) 1.641/ 0.253/ 2.466/ 0.344 ms 81383 pps I'll be testing with 802.11ax chips soon but probably can't share those numbers. Sorry about that. Bob On Fri, Oct 13, 2017 at 12:41 PM, Bob McMahon wrote: > PS. Thanks for writing flent and making it available. I'm a novice > w/flent but do plan to learn it. > > Bob > > On Fri, Oct 13, 2017 at 11:47 AM, Bob McMahon > wrote: > >> Hi Toke, >> >> The other thing that will cause the server thread(s) and listener thread >> to stop is -t when applied to the *server*, i.e. iperf -s -u -t 10 will >> cause a 10 second timeout for the server/listener thread(s) life. Some >> people don't want the Listener to stop so when -D (daemon) is applied, t= he >> -t will only terminate server trafffic threads. Many people asked for >> this because they wanted a way to time bound these threads, specifically >> over the life of many tests. >> >> Yeah, summing is a bit of a mess. I've some proto code I've been playin= g >> with but still not sure what is going to be released. >> >> For UDP, the source port must be unique per the quintuple (ip proto/src >> ip/ src port/ dst ip/ dst port). Since the UDP server is merely waiting >> for packets it doesn't have an knowledge about how to group. So it grou= ps >> based upon time, i.e. when a new traffic shows up it's put an existing >> active group for summing. >> >> I'm not sure a good way to fix this. I think the client would have to >> modify the payload, and per a -P tell the server the udp src ports that >> belong in the same group. Then the server could assign groups based upo= n a >> key in the payload. >> >> Thoughts and comments welcome, >> Bob >> >> On Fri, Oct 13, 2017 at 2:28 AM, Toke H=C3=B8iland-J=C3=B8rgensen >> wrote: >> >>> Bob McMahon writes: >>> >>> > Thanks Toke. Let me look into this. Is there packet loss during your >>> > tests? Can you share the output of the client and server per the erro= r >>> > scenario? >>> >>> Yeah, there's definitely packet loss. >>> >>> > With iperf 2 there is no TCP test exchange rather UDP test informatio= n >>> > is derived from packets in flight. The server determines a UDP test i= s >>> > finished by detecting a negative sequence number in the payload. In >>> > theory, this should separate UDP tests. The server detects a new UDP >>> > stream is by receiving a packet from a new source socket. If the >>> > packet carrying the negative sequence number is lost then summing >>> > across "tests" would be expected (even though not desired) per the >>> > current design and implementation. We intentionally left this as is a= s >>> > we didn't want to change the startup behavior nor require the network >>> > support TCP connections in order to run a UDP test. >>> >>> Ah, so basically, if the last packet from the client is dropped, the >>> server is not going to notice that the test ended and just keep >>> counting? That would definitely explain the behaviour I'm seeing. >>> >>> So if another test starts from a different source port, the server is >>> still going to count the same totals? That seems kinda odd :) >>> >>> > Since we know UDP is unreliable, we do control both client and server >>> over >>> > ssh pipes, and perform summing in flight per the interval reporting. >>> > Operating system signals are used to kill the server. The iperf >>> sum and >>> > final reports are ignored. Unfortunately, I can't publish this >>> package >>> > with iperf 2 for both technical and licensing reasons. There is som= e >>> skeleton >>> > code in Python 3.5 with asyncio >>> > >>> that >>> > may be of use. A next step here is to add support for pandas >>> > , and possibly some control char= t >>> > techniques (both single >>> and >>> > multivariate >>> > ) for >>> both >>> > regressions and outlier detection. >>> >>> No worries, I already have the setup scripts to handle restarting the >>> server, and I parse the output with Flent. Just wanted to point out thi= s >>> behaviour as it was giving me some very odd results before I started >>> systematically restarting the server... >>> >>> -Toke >>> >> >> > --94eb2c0df8c4ea9b45055b77f0e8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
FYI, here's an example output using two brix PCs over = their wired ethernet connected by a Cisco SG300 switch.=C2=A0 =C2=A0Note: t= he ptpd stats (not shown) displays the clock corrections and suggests them = to be within 10 microseconds.

[rjmcmahon@rjm-fedora = etc]$ iperf -c 192.168.100.33 -u=C2=A0 -e -t 2 -b 2kpps=C2=A0
---= ---------------------------------------------------------
Client = connecting to 192.168.100.33, UDP port 5001 with pid 29952
Sendin= g 1470 byte datagrams, IPG target: 500.00 us (kalman adjust)
UDP = buffer size:=C2=A0 208 KByte (default)
--------------------------= ----------------------------------
[=C2=A0 3] local 192.168.100.6= 7 port 50062 connected with 192.168.100.33 port 5001
[ ID] Interv= al=C2=A0 =C2=A0 =C2=A0 =C2=A0Transfer=C2=A0 =C2=A0 =C2=A0Bandwidth=C2=A0 = =C2=A0 =C2=A0 PPS
[=C2=A0 3] 0.00-2.00 sec=C2=A0 5.61 MBytes=C2= =A0 23.5 Mbits/sec 1999 pps
[=C2=A0 3] Sent 4001 datagrams
<= div>[=C2=A0 3] Server Report:
[=C2=A0 3] 0.00-2.00 sec=C2=A0 5.61= MBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.012 ms=C2=A0 =C2=A0 0/ 4001 (0%)= -/-/-/- ms 2000 pps

[rjmcmahon@hera ~]$ iperf -s = -u -e -i 0.1
----------------------------------------------------= --------
Server listening on UDP port 5001 with pid 5178
Receiving 1470 byte datagrams
UDP buffer size:=C2=A0 208 KByte = (default)
-------------------------------------------------------= -----
[=C2=A0 3] local 192.168.100.33 port 5001 connected with 19= 2.168.100.67 port 57325
[ ID] Interval=C2=A0 =C2=A0 =C2=A0 =C2=A0= Transfer=C2=A0 =C2=A0 =C2=A0Bandwidth=C2=A0 =C2=A0 =C2=A0 =C2=A0 Jitter=C2= =A0 =C2=A0Lost/Total=C2=A0 Latency avg/min/max/stdev PPS
[=C2=A0 = 3] 0.00-0.10 sec=C2=A0 =C2=A0289 KBytes=C2=A0 23.6 Mbits/sec=C2=A0 =C2=A00.= 013 ms=C2=A0 =C2=A0 0/=C2=A0 201 (0%)=C2=A0 0.142/ 0.018/ 0.192/ 0.025 ms 2= 005 pps
[=C2=A0 3] 0.10-0.20 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23.= 5 Mbits/sec=C2=A0 =C2=A00.020 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.157= / 0.101/ 0.207/ 0.015 ms 1999 pps
[=C2=A0 3] 0.20-0.30 sec=C2=A0 = =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.014 ms=C2=A0 =C2=A0 0/= =C2=A0 200 (0%)=C2=A0 0.155/ 0.071/ 0.212/ 0.018 ms 2002 pps
[=C2= =A0 3] 0.30-0.40 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2= =A00.018 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.146/-0.007/ 0.187/ 0.018= ms 1999 pps
[=C2=A0 3] 0.40-0.50 sec=C2=A0 =C2=A0287 KBytes=C2= =A0 23.5 Mbits/sec=C2=A0 =C2=A00.021 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2= =A0 0.151/ 0.021/ 0.208/ 0.018 ms 2000 pps
[=C2=A0 3] 0.50-0.60 s= ec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.016 ms=C2=A0 = =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.148/ 0.043/ 0.192/ 0.018 ms 2000 pps
=
[=C2=A0 3] 0.60-0.70 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec= =C2=A0 =C2=A00.019 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.152/ 0.041/ 0.= 199/ 0.018 ms 2001 pps
[=C2=A0 3] 0.70-0.80 sec=C2=A0 =C2=A0287 K= Bytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.016 ms=C2=A0 =C2=A0 0/=C2=A0 200 (= 0%)=C2=A0 0.144/ 0.071/ 0.206/ 0.017 ms 2001 pps
[=C2=A0 3] 0.80-= 0.90 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.015 ms= =C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.140/ 0.111/ 0.186/ 0.014 ms 1999 pp= s
[=C2=A0 3] 0.90-1.00 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 Mbit= s/sec=C2=A0 =C2=A00.022 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.154/ 0.11= 1/ 0.222/ 0.019 ms 2000 pps
[=C2=A0 3] 1.00-1.10 sec=C2=A0 =C2=A0= 287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.014 ms=C2=A0 =C2=A0 0/=C2=A0 = 200 (0%)=C2=A0 0.152/ 0.036/ 0.197/ 0.017 ms 2000 pps
[=C2=A0 3] = 1.10-1.20 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.015= ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.153/-0.007/ 0.186/ 0.020 ms 2001= pps
[=C2=A0 3] 1.20-1.30 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 M= bits/sec=C2=A0 =C2=A00.013 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.149/ 0= .035/ 0.207/ 0.018 ms 2000 pps
[=C2=A0 3] 1.30-1.40 sec=C2=A0 =C2= =A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.014 ms=C2=A0 =C2=A0 0/=C2= =A0 200 (0%)=C2=A0 0.160/ 0.116/ 0.233/ 0.018 ms 2000 pps
[=C2=A0= 3] 1.40-1.50 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00= .014 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.159/ 0.122/ 0.207/ 0.015 ms = 2000 pps
[=C2=A0 3] 1.50-1.60 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23= .5 Mbits/sec=C2=A0 =C2=A00.016 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.15= 8/ 0.066/ 0.201/ 0.015 ms 1999 pps
[=C2=A0 3] 1.60-1.70 sec=C2=A0= =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.017 ms=C2=A0 =C2=A0 0/= =C2=A0 200 (0%)=C2=A0 0.162/ 0.076/ 0.203/ 0.016 ms 2000 pps
[=C2= =A0 3] 1.70-1.80 sec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2= =A00.014 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.154/ 0.073/ 0.195/ 0.016= ms 2002 pps
[=C2=A0 3] 1.80-1.90 sec=C2=A0 =C2=A0287 KBytes=C2= =A0 23.5 Mbits/sec=C2=A0 =C2=A00.015 ms=C2=A0 =C2=A0 0/=C2=A0 200 (0%)=C2= =A0 0.154/ 0.113/ 0.213/ 0.017 ms 1999 pps
[=C2=A0 3] 1.90-2.00 s= ec=C2=A0 =C2=A0287 KBytes=C2=A0 23.5 Mbits/sec=C2=A0 =C2=A00.019 ms=C2=A0 = =C2=A0 0/=C2=A0 200 (0%)=C2=A0 0.152/ 0.124/ 0.208/ 0.016 ms 1999 pps
=
[=C2=A0 3] 0.00-2.00 sec=C2=A0 5.61 MBytes=C2=A0 23.5 Mbits/sec=C2=A0 = =C2=A00.019 ms=C2=A0 =C2=A0 0/ 4001 (0%)=C2=A0 0.151/-0.007/ 0.233/ 0.018 m= s 2000 pps


Here's a full line rate (1Gbs ethernet) ru= n

[rjmcmahon@rjm-fedora etc]$ iperf -c 192.168.100.33 -u=C2=A0 = -e -t 2 -b 200kpss -i 1 -w 2M
-----------------------------------= -------------------------
Client connecting to 192.168.100.33, UD= P port 5001 with pid 30626
Sending 1470 byte datagrams, IPG targe= t: 5.00 us (kalman adjust)
UDP buffer size:=C2=A0 416 KByte (WARN= ING: requested 2.00 MByte)
--------------------------------------= ----------------------
[=C2=A0 3] local 192.168.100.67 port 57349= connected with 192.168.100.33 port 5001
[ ID] Interval=C2=A0 =C2= =A0 =C2=A0 =C2=A0Transfer=C2=A0 =C2=A0 =C2=A0Bandwidth=C2=A0 =C2=A0 =C2=A0 = PPS
[=C2=A0 3] 0.00-1.00 sec=C2=A0 =C2=A0114 MBytes=C2=A0 =C2=A09= 58 Mbits/sec 81421 pps
[=C2=A0 3] 1.00-2.00 sec=C2=A0 =C2=A0114 M= Bytes=C2=A0 =C2=A0958 Mbits/sec 81380 pps
[=C2=A0 3] 0.00-2.00 se= c=C2=A0 =C2=A0228 MBytes=C2=A0 =C2=A0957 Mbits/sec 81399 pps
[=C2= =A0 3] Sent 162874 datagrams
[=C2=A0 3] Server Report:
= [=C2=A0 3] 0.00-2.00 sec=C2=A0 =C2=A0228 MBytes=C2=A0 =C2=A0957 Mbits/sec= =C2=A0 =C2=A00.084 ms=C2=A0 =C2=A0 0/162874 (0%)=C2=A0 1.641/ 0.253/ 2.466/= 0.114 ms 81383 pps


[=C2=A0 3] local 192.168.1= 00.33 port 5001 connected with 192.168.100.67 port 57349
[=C2=A0 = 3] 0.00-0.10 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0960 Mbits/sec=C2=A0 =C2=A00.= 016 ms=C2=A0 =C2=A0 0/ 8166 (0%)=C2=A0 1.615/ 0.253/ 2.309/ 0.355 ms 81606 = pps
[=C2=A0 3] 0.10-0.20 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0956 Mb= its/sec=C2=A0 =C2=A00.016 ms=C2=A0 =C2=A0 0/ 8133 (0%)=C2=A0 1.657/ 0.936/ = 2.325/ 0.348 ms 81350 pps
[=C2=A0 3] 0.20-0.30 sec=C2=A0 11.4 MBy= tes=C2=A0 =C2=A0957 Mbits/sec=C2=A0 =C2=A00.021 ms=C2=A0 =C2=A0 0/ 8139 (0%= )=C2=A0 1.657/ 0.950/ 2.400/ 0.348 ms 81383 pps
[=C2=A0 3] 0.30-0= .40 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0958 Mbits/sec=C2=A0 =C2=A00.016 ms=C2= =A0 =C2=A0 0/ 8144 (0%)=C2=A0 1.652/ 0.953/ 2.357/ 0.342 ms 81380 pps
=
[=C2=A0 3] 0.40-0.50 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0956 Mbits/sec= =C2=A0 =C2=A00.069 ms=C2=A0 =C2=A0 0/ 8131 (0%)=C2=A0 1.644/ 0.947/ 2.368/ = 0.341 ms 81384 pps
[=C2=A0 3] 0.50-0.60 sec=C2=A0 11.4 MBytes=C2= =A0 =C2=A0957 Mbits/sec=C2=A0 =C2=A00.072 ms=C2=A0 =C2=A0 0/ 8138 (0%)=C2= =A0 1.649/ 0.949/ 2.381/ 0.337 ms 81372 pps
[=C2=A0 3] 0.60-0.70 = sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0957 Mbits/sec=C2=A0 =C2=A00.025 ms=C2=A0 = =C2=A0 0/ 8139 (0%)=C2=A0 1.639/ 0.952/ 2.357/ 0.342 ms 81383 pps
[=C2=A0 3] 0.70-0.80 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0957 Mbits/sec=C2=A0= =C2=A00.014 ms=C2=A0 =C2=A0 0/ 8135 (0%)=C2=A0 1.643/ 0.944/ 2.368/ 0.343 = ms 81390 pps
[=C2=A0 3] 0.80-0.90 sec=C2=A0 11.4 MBytes=C2=A0 =C2= =A0957 Mbits/sec=C2=A0 =C2=A00.016 ms=C2=A0 =C2=A0 0/ 8142 (0%)=C2=A0 1.639= / 0.946/ 2.361/ 0.341 ms 81366 pps
[=C2=A0 3] 0.90-1.00 sec=C2=A0= 11.4 MBytes=C2=A0 =C2=A0957 Mbits/sec=C2=A0 =C2=A00.015 ms=C2=A0 =C2=A0 0/= 8142 (0%)=C2=A0 1.635/ 0.932/ 2.378/ 0.342 ms 81387 pps
[=C2=A0 = 3] 1.00-1.10 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0957 Mbits/sec=C2=A0 =C2=A00.= 015 ms=C2=A0 =C2=A0 0/ 8138 (0%)=C2=A0 1.633/ 0.934/ 2.359/ 0.341 ms 81373 = pps
[=C2=A0 3] 1.10-1.20 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0957 Mb= its/sec=C2=A0 =C2=A00.010 ms=C2=A0 =C2=A0 0/ 8135 (0%)=C2=A0 1.636/ 0.947/ = 2.361/ 0.342 ms 81444 pps
[=C2=A0 3] 1.20-1.30 sec=C2=A0 11.4 MBy= tes=C2=A0 =C2=A0957 Mbits/sec=C2=A0 =C2=A00.091 ms=C2=A0 =C2=A0 0/ 8140 (0%= )=C2=A0 1.624/ 0.908/ 2.363/ 0.354 ms 81400 pps
[=C2=A0 3] 1.30-1= .40 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0956 Mbits/sec=C2=A0 =C2=A00.016 ms=C2= =A0 =C2=A0 0/ 8133 (0%)=C2=A0 1.616/ 0.917/ 2.325/ 0.345 ms 81296 pps
=
[=C2=A0 3] 1.40-1.50 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0957 Mbits/sec= =C2=A0 =C2=A00.012 ms=C2=A0 =C2=A0 0/ 8138 (0%)=C2=A0 1.626/ 0.918/ 2.361/ = 0.346 ms 81414 pps
[=C2=A0 3] 1.50-1.60 sec=C2=A0 11.4 MBytes=C2= =A0 =C2=A0957 Mbits/sec=C2=A0 =C2=A00.015 ms=C2=A0 =C2=A0 0/ 8136 (0%)=C2= =A0 1.626/ 0.934/ 2.352/ 0.339 ms 81339 pps
[=C2=A0 3] 1.60-1.70 = sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0957 Mbits/sec=C2=A0 =C2=A00.015 ms=C2=A0 = =C2=A0 0/ 8141 (0%)=C2=A0 1.633/ 0.930/ 2.351/ 0.341 ms 81376 pps
[=C2=A0 3] 1.70-1.80 sec=C2=A0 11.4 MBytes=C2=A0 =C2=A0956 Mbits/sec=C2=A0= =C2=A00.017 ms=C2=A0 =C2=A0 0/ 8133 (0%)=C2=A0 1.627/ 0.929/ 2.354/ 0.339 = ms 81377 pps
[=C2=A0 3] 1.80-1.90 sec=C2=A0 11.4 MBytes=C2=A0 =C2= =A0957 Mbits/sec=C2=A0 =C2=A00.088 ms=C2=A0 =C2=A0 0/ 8139 (0%)=C2=A0 1.659= / 0.895/ 2.396/ 0.343 ms 81330 pps
[=C2=A0 3] 1.90-2.00 sec=C2=A0= 11.4 MBytes=C2=A0 =C2=A0956 Mbits/sec=C2=A0 =C2=A00.013 ms=C2=A0 =C2=A0 0/= 8128 (0%)=C2=A0 1.709/ 0.996/ 2.466/ 0.342 ms 81348 pps
[=C2=A0 = 3] 0.00-2.00 sec=C2=A0 =C2=A0228 MBytes=C2=A0 =C2=A0957 Mbits/sec=C2=A0 =C2= =A00.085 ms=C2=A0 =C2=A0 0/162874 (0%)=C2=A0 1.641/ 0.253/ 2.466/ 0.344 ms = 81383 pps

I'll be testing with 802.11ax chips soon but pr= obably can't share those numbers.=C2=A0 Sorry about that.=C2=A0=C2=A0
Bob



On Fri, Oct 13, 2017 at 12:41 PM, B= ob McMahon <bob.mcmahon@broadcom.com> wrote:
PS.=C2=A0 Thanks for writing fle= nt and making it available.=C2=A0 I'm a novice w/flent but do plan to l= earn it.

Bob

On Fri, Oct 13, 2017 at 11:= 47 AM, Bob McMahon <bob.mcmahon@broadcom.com> wrote:<= br>
Hi Toke,

The othe= r thing that will cause the server thread(s) and listener thread to stop is= -t when applied to the *server*, i.e. iperf -s -u -t 10 will cause a 10 se= cond timeout for the server/listener thread(s) life.=C2=A0 =C2=A0Some peopl= e don't want the Listener to stop so when -D (daemon) is applied, the -= t will only terminate server trafffic threads.=C2=A0 =C2=A0Many people aske= d for this because they wanted a way to time bound these threads, specifica= lly over the life of many tests.

Yeah, summing is a bit of a mess.= =C2=A0 I've some proto code I've been playing with but still not su= re what is going to be released.=C2=A0

For UDP, the source port mus= t be unique per the quintuple (ip proto/src ip/ src port/ dst ip/ dst port)= .=C2=A0 Since the UDP server is merely waiting for packets it doesn't h= ave an knowledge about how to group.=C2=A0 So it groups based upon time, i.= e. when a new traffic shows up it's put an existing active group for su= mming.=C2=A0

I'm not sure a good way to fix this.=C2=A0 I think= the client would have to modify the payload, and=C2=A0 per a -P tell the s= erver the udp src ports that belong in the same group.=C2=A0 Then the serve= r could assign groups based upon a key in the payload.

Thoughts and = comments welcome,
Bob

On Fri, Oct 13, 2017 at 2:28 AM, Toke H=C3=B8iland-J=C3= =B8rgensen <toke@toke.dk> wrote:
bob.mcmahon@broadcom.com> writes:

> Thanks Toke. Let me look into this. Is there packet loss during your > tests? Can you share the output of the client and server per the error=
> scenario?

Yeah, there's definitely packet loss.

> With iperf 2 there is no TCP test exchange rather UDP test information=
> is derived from packets in flight. The server determines a UDP test is=
> finished by detecting a negative sequence number in the payload. In > theory, this should separate UDP tests. The server detects a new UDP > stream is by receiving a packet from a new source socket. If the
> packet carrying the negative sequence number is lost then summing
> across "tests" would be expected (even though not desired) p= er the
> current design and implementation. We intentionally left this as is as=
> we didn't want to change the startup behavior nor require the netw= ork
> support TCP connections in order to run a UDP test.

Ah, so basically, if the last packet from the client is dropped, the=
server is not going to notice that the test ended and just keep
counting? That would definitely explain the behaviour I'm seeing.

So if another test starts from a different source port, the server is
still going to count the same totals? That seems kinda odd :)

> Since we know UDP is unreliable, we do control both client and server = over
> ssh pipes, and perform summing in flight per the interval reporting. >=C2=A0 Operating system signals are used to kill the server.=C2=A0 =C2= =A0 The iperf sum and
> final reports are ignored.=C2=A0 =C2=A0Unfortunately, I can't publ= ish this package
> with iperf 2 for both technical and licensing reasons.=C2=A0 =C2=A0The= re is some skeleton
> code in Python 3.5 with asyncio
> <https://sourcefor= ge.net/p/iperf2/code/ci/master/tree/flows/flows.py> that > may be of use.=C2=A0 =C2=A0A next step here is to add support fo= r pandas
> <http://pandas.pydata.org/index.html>,= and possibly some control chart
> <https://en.wikipedia.org/wiki/Control_chart= > techniques (both single and
> multivariate
> <http://www.itl.nist.gov/div89= 8/handbook/pmc/section3/pmc34.htm>) for both
> regressions and outlier detection.

No worries, I already have the setup scripts to handle restarting the
server, and I parse the output with Flent. Just wanted to point out this behaviour as it was giving me some very odd results before I started
systematically restarting the server...

-Toke



--94eb2c0df8c4ea9b45055b77f0e8--