From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yb0-x22b.google.com (mail-yb0-x22b.google.com [IPv6:2607:f8b0:4002:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 905233B25E for ; Tue, 20 Sep 2016 18:00:02 -0400 (EDT) Received: by mail-yb0-x22b.google.com with SMTP id u125so14654013ybg.3 for ; Tue, 20 Sep 2016 15:00:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=r29rZ1fFdwYYlGncbO/U+o05AcGtVk2hdf29ryWaLZQ=; b=WfDRS88te6LArHdDBUFE59UZuoHeNxfKE+88R2X+Fu8VJhzbmYn0MzeJ15eBdC1G4q 4f/6ELHJ0vcg2+MMfWBT1hvRKmzml8lmGrEqPDnTT5Z9uZU/iuv1lIesyFRBx8arQ3sw rVxdneXbcvGdVAHY4zIl1/mhyrXpBCjvsUtOZ8wS19E4W06OSx4vI8jthuXColIYzGSa tM83NCJgLQXaOYuRB5DTLQOiyAgFgfUP2eq7mYx7CbbL1Yyj/skTxurbkWVdROFWqwRA zFkZLqXzZqQPRjM/6rsjProv+4ThPVUwLSn0UyaMPqTha04Nj2i15/TkyliU+Hs8rJfg jvgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=r29rZ1fFdwYYlGncbO/U+o05AcGtVk2hdf29ryWaLZQ=; b=VjIk9wTIoPZMMStfsWt0gGDc+loJCjhf6R0kz0a4TTcJF9NKHfKqtKQIFY41MomY5h oMERFb9PdGCdwAroHxh89i+NgZ9jYioAovgWnnR4ShzB66rwSpDNdmZz8evh/lhY0JZS fh7oY43yn9eJS5R244VREpQo2vb4obDe0ab9HbitAV9mLKBgPaRj8wZ3cHTLtWZJiRio 8CXuOCfwszun36x+PBGJuo9kgAA/oyijvZAxRD80LURK/A5PN49svgw6xRTLDI9252Lz 7gQUKIBHpjSBnOQEeody04vS8AbNEzFw2WeJy09TU8b/yH8plytyMhBcIyksbGqh8zDS KQ6A== X-Gm-Message-State: AE9vXwNEoU4MJ/bXGZSDR7+fEGpMaZYzCn8IiIQiMe7XiHeELO/B8UQbZz6sMflS4RVkbKTUGzK8Pm52Z99D+w== X-Received: by 10.37.79.132 with SMTP id d126mr27787016ybb.175.1474408801778; Tue, 20 Sep 2016 15:00:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.117.214 with HTTP; Tue, 20 Sep 2016 15:00:01 -0700 (PDT) From: Aaron Wood Date: Tue, 20 Sep 2016 15:00:01 -0700 Message-ID: To: bloat Content-Type: multipart/alternative; boundary=001a113c4a44e2430a053cf78cf8 Subject: [Bloat] iperf3 and packet bursts X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Sep 2016 22:00:02 -0000 --001a113c4a44e2430a053cf78cf8 Content-Type: text/plain; charset=UTF-8 We were using iperf3 at work to test a network path (we wanted the fixed-rate throttling that it can do). And while TCP would fill the 50Mbps link from the ISP, UDP couldn't. UDP couldn't get over 8Mbps of goodput, no matter what rate we specified on the command line. We found a 100ms timer that's used to PWM the packet transmission to perform the throttling. Fine for TCP, fine where the end-to-end physical links are the same rate. But throw a 10:1 rate change through a switch into that, and suddenly you find out that the switch isn't that bloated. I modified iperf3 to use a 1ms timer, and was able to get things much smoother. I doubt it's as smooth as iperf3 gets on Linux when fq pacing is used, but it's a big improvement vs. the nice small buffers in switches. I put together a writeup with graphs on my blog: http://burntchrome.blogspot.com/2016/09/iperf3-and-microbursts.html I have a forked version of iperf3 on github: https://github.com/woody77/iperf This uses the 1ms timer, and has a few other fixes, such as it resets the throttling calculations at the start of each stats interval. That change stops iperf3 from transmitting at maximum rate after congestion has stopped it from achieving the target rate. There will be another writeup on that, but I need to get some good sample data together for graphing. -Aaron Wood --001a113c4a44e2430a053cf78cf8 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
We were using iperf3 at work to test a network path (we wa= nted the fixed-rate throttling that it can do).=C2=A0 And while TCP would f= ill the 50Mbps link from the ISP, UDP couldn't.=C2=A0 UDP couldn't = get over 8Mbps of goodput, no matter what rate we specified on the command = line.

We found a 100ms timer that's used to PWM the = packet transmission to perform the throttling.=C2=A0 Fine for TCP, fine whe= re the end-to-end physical links are the same rate.=C2=A0 But throw a 10:1 = rate change through a switch into that, and suddenly you find out that the = switch isn't that bloated.

I modified iperf3 t= o use a 1ms timer, and was able to get things much smoother.=C2=A0 I doubt = it's as smooth as iperf3 gets on Linux when fq pacing is used, but it&#= 39;s a big improvement vs. the nice small buffers in switches.
I put together a writeup with graphs on my blog:

I have a forked version of iperf3 on github:

This uses the 1ms timer, and has a = few other fixes, such as it resets the throttling calculations at the start= of each stats interval.=C2=A0 That change stops iperf3 from transmitting a= t maximum rate after congestion has stopped it from achieving the target ra= te.=C2=A0 There will be another writeup on that, but I need to get some goo= d sample data together for graphing.

-Aaron Wood
--001a113c4a44e2430a053cf78cf8--