From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 5A13E21F27E for ; Tue, 23 Jun 2015 10:25:36 -0700 (PDT) Received: from u-089-d065.biologie.uni-tuebingen.de ([134.2.89.65]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0MEXHd-1ZICX9020n-00FmId; Tue, 23 Jun 2015 19:25:34 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: Date: Tue, 23 Jun 2015 19:25:29 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <26463A88-821B-44B7-A728-64BCB0B7C7BB@gmx.de> <55847E32.9000405@gmail.com> <5584823E.4040207@gmail.com> <0129B5FB-9D1B-45FF-84CA-492A6A0B638B@gmx.de> To: Mikael Abrahamsson X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:xZNkEB95JsErk5aOkBCqKJAaUQ5qIceWLwGD4zfxLogbpDE926z yZGxnR4cSjVnaQjeeDPzvlV8IAoQlYQSFTCOO+ZxtqEJ336uhBRDEqUsnmBLJpl+bvbK79A poMxmMLoeK1+7u1QdJMrP7F6TRVI8W/wEfENpGgiGgbFtCc4NrWiX484Ii7TmS205Haur6N UtqCVZZ9vSySxR2IJCJOA== X-UI-Out-Filterresults: notjunk:1;V01:K0:uejfO1WDGdU=:AAAHZMEg+Aaygc0p0uWDoG 70GQwtvpzK7WxSBFGG9cCeDq8Yj3zEZgqMe1Q79ODxm3KdJBaIC7PI2HDX+IT3cvxYBDCjT2B cSgwLWjqNqJaYHvtuOKFvkq00/C2Q5zrkvwlXiNgF79CbiFEvHH4zwggzl1KeefmFPjuaTRUt cImpp9cux1cJGrGW0nomAwnJ1BJtTkYqYYayftr2SGxZ7VN4FHMER/SfIIcKqxa25hO4W351y rbzVpNU1pBRAWPtpBQhC8X+2nkw490JtLv/O9ACAssbD5snOTHQuKcy28aG/Cv92jYMbZ8Sbu DsDRGuYX9Mrk/HRGyeIUOBEnEEwLZJcbwZW16/PjY7z89Cpw8UVy+EIqfLVjpTLTOIUn96ZXa 7OVc+fE/ly5cZv4OQRcE+2wAokX7Vht958dRbcsGlmG4KSZiTu+gOkVuJNP2livtUkOdZJAy/ gFx9OGKGLE9E6lzGaAG1rR9VpjKB6+wecEc0I+p9i2PPmmi89Hou8NWd8WrUo2d3ovuXLnbbI WGyg9jkcDnolb/Y4H+bIP17EluHc8mUMWAgl/ZqaiVnc5p9HTPV0UM32Au3E27eiz9dGPwd6M QaZtWPXwsgOEtb97Yik/YHP3EwqRxygPPpH5YNBoFutk6qK4XZb9u9afYIXavt8EUzWykzah5 c83CbICNdH4EKnZcbYYDYQDKhn8eZvDa4Zv13uhAN8H5Gaf/lELL0TYL7gfLjBa+Kf/s= Cc: cerowrt-devel Subject: Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; "cake overhead 40" didn't) X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2015 17:26:06 -0000 Hi Mikael, On Jun 23, 2015, at 14:55 , Mikael Abrahamsson wrote: > On Tue, 23 Jun 2015, Sebastian Moeller wrote: >=20 >> Most likely not. Check http://wiki.openwrt.org/doc/howto/sqm . = Rich published a great set of instructions for setting up sqm-scripts = under openwrt proper. >=20 > I tried it on Linksys WRT1200AC with OpenWrt CC RC2. I configured sqm = to have 800 megabit/s each direction, and ran iperf3 over IPv4 with = NAT44 from Linux box behind WRT1200AC to an OSX macbook connected on a = switch on the same L2 subnet as the WAN port. >=20 > Linux <->WRT1200AC<->switch<->OSX Thanks a lot, interesting data! Was this test stressing both = directions at the same time? (My guess is if the test was UDP i don=92t = know, for a TCP test I am quite confident that it was uni-directional as = the @full MTU data does not show enough loss to accommodate the roughly = 2% reverse ACK traffic for the opposite direction). >=20 > I get 765 megabit/s of throughput using single session, at sirq load = of around 25%. If I lower the mss to 300 (to generate higher pps) I get = around 560 megabit/s of throughput at 50% sirq. With 10 parallel TCP = sessions, I get about the same. At MSS of 200 bytes, I get 400 megabit/s = at 70% sirq. I assume iperf3 uses TCP or UDP streams and reports the payload rate, = correct? Then we have a MSS of 1460 (with 20 bytes IPv4 header and 20 = bytes for TCP or UDP). @full MTU MSS: 1500 - 20 - 20 =3D 1460 byte Number of packets at 765 Mbps goodput: (765 * 1000^2) / ((1500 - 20 - 20) * 8) =3D 65496.5753425 =3D 65K On-the-wire packet size (OTWPS) assuming ethernet with FCS and no VLAN = tag)s): 1500 + 14 + 4 =3D 1518 bytes MSS to OTWPS ratio: (1500 - 20 - 20) / (1500 + 14 + 4) =3D 0.961791831357 raw bandwidth consumed by 765 Mbps good put: 765 / ((1500 - 20 - 20) / (1500 + 14 + 4)) =3D ((765 * 1000^2) / ((1500 = - 20 - 20) * 8)) * ((1500 + 14 + 4) * 8) =3D 795.390410959 Mbps So basically (1 - (795.390410959/800))*100 =3D 0.58 % unexplained loss, = not bad @MSS 300 MSS: 300 byte Number of packets at 560 Mbps goodput: (560 * 1000^2) / ((300) * 8) =3D 233333.333333 =3D 233K On-the-wire packet size (OTWPS) assuming ethernet with FCS: 300 + 20 + 20 + 14 + 4 =3D 358 bytes MSS to OTWPS ratio: (300) / (300 + 20 + 20 + 14 + 4) =3D 0.837988826816 raw bandwidth consumed by 560 Mbps good put: 560 / ((300) / (300 + 20 + 20 + 14 + 4 )) =3D ((560 * 1000^2) / ((300) * = 8)) * ((300 + 20 + 20 + 14 + 4 ) * 8) =3D 668.266666667 Mbps So basically (1 - (668.266666667/800))*100 =3D 16.4666666666 % = unexplained loss, not pretty but bearable I guess @MSS 200 MSS: 200 byte Number of packets at 400 Mbps goodput: (400 * 1000^2) / ((200) * 8) =3D 250000 =3D 250K On-the-wire packet size (OTWPS) assuming ethernet with FCS: 200 + 20 + 20 + 14 + 4 =3D 258 bytes MSS to OTWPS ratio: (200) / (200 + 20 + 20 + 14 + 4) =3D 0.77519379845 raw bandwidth consumed by 400 Mbps good put: 400 / ((200) / (200 + 20 + 20 + 14 + 4 )) =3D ((400 * 1000^2) / ((200) * = 8)) * ((200 + 20 + 20 + 14 + 4 ) * 8) =3D 516 Mbps So basically (1 - (516/800))*100 =3D 35.5 % unexplained loss, that is = sad. But the packet rate is still at 250K, I winder how this router does = with 64 byte ethernet frames >=20 > If I turn off SQM completely, I get 600 megabit/s at 200 byte MSS = single session at 80% sirq and 930 megabit/s at 26% sirq with default = MSS. Since no shaper was used, I think we need to include the = inter-frame-gap and preamble to calculate the maximal payload rates for = different packet sizes. @1Gbps MSS (1500 - 20 - 20) =3D 1460 byte Number of packets at 930 Mbps goodput: (930 * 1000^2) / ((1500 - 20 - 20) * 8) =3D 79623.2876712 =3D 80K To asses the maximum achievable at 1 GBE we need to take IFG and = preamble into account I think On-the-wire packet size (OTWPS) assuming ethernet with FCS plus inter = frame gap and preamble: 1500 + 14 + 4 + 12 + 8 =3D 1538 bytes MSS to OTWPS ratio: (1500 - 20 - 20) / (1500 + 14 + 4 + 12 + 8) =3D 0.949284785436 raw bandwidth consumed by 930 Mbps good put: 930 / ((1500 - 20 - 20) / (1500 + 14 + 4 + 12 + 8)) =3D ((930 * 1000^2) = / ((1500 - 20 - 20) * 8)) * ((1500 + 14 + 4 + 12 + 8) * 8) =3D = 979.684931507 Mbps So basically (1 - (979.684931507/1000))*100 =3D 2.0315068493 % = unexplained loss, not bad.=20 @1Gbps MSS 200 =3D 1460 byte Number of packets at 600 Mbps goodput: (600 * 1000^2) / ((200) * 8) =3D 375000 =3D 375K On-the-wire packet size (OTWPS) assuming ethernet with FCS plus inter = frame gap and preamble: 200 + 20 + 20 + 14 + 4 + 12 + 8 =3D 278 bytes MSS to OTWPS ratio: 200 / (200 + 20 + 20 + 14 + 4 + 12 + 8) =3D 0.719424460432 raw bandwidth consumed by 600 Mbps good put: 600 / ((200) / (200 + 20 + 20 + 14 + 4 + 12 + 8)) =3D ((600 * 1000^2) / = ((200) * 8)) * ((200 + 20 + 20 + 14 + 4 + 12 + 8) * 8) =3D 834 Mbps So basically (1 - (834/1000))*100 =3D 16.6 % unexplained loss, not bad.=20= As Dave said it would be nice see RRUL data from the same testbed. It = would be so nice if flint had a way to send different sized TCP packets=85= (I guess this might be faked with MSS clamping in the router and = relaying on path MTU discovery?) >=20 > So if you want high performing device that is OpenWRT compatible and = still does forwarding using CPU so you can test queuing algorithms, the = WRT1200AC and WRT1900ACv2 is the best I have been able to find currently = (unless you go for x86 platform). The 1200AC retailed for around 200EUR in Germany the 1900ACv2 = will be closer to 300EUR I guess, not too expensive but certainly above = my impulse buy limit ;) tack s=E5 mycket & Best Regards Sebastian >=20 > --=20 > Mikael Abrahamsson email: swmike@swm.pp.se > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel