Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; "cake overhead 40" didn't)

[Sebastian Moeller <moeller0@gmx.de>]

Hi Mikael,


On Jun 23, 2015, at 14:55 , Mikael Abrahamsson <swmike@swm.pp.se> wrote:

> On Tue, 23 Jun 2015, Sebastian Moeller wrote:
> 
>> 	Most likely not. Check http://wiki.openwrt.org/doc/howto/sqm . Rich published a great set of instructions for setting up sqm-scripts under openwrt proper.
> 
> I tried it on Linksys WRT1200AC with OpenWrt CC RC2. I configured sqm to have 800 megabit/s each direction, and ran iperf3 over IPv4 with NAT44 from Linux box behind WRT1200AC to an OSX macbook connected on a switch on the same L2 subnet as the WAN port.
> 
> Linux <->WRT1200AC<->switch<->OSX

	Thanks a lot, interesting data! Was this test stressing both directions at the same time? (My guess is if the test was UDP i don’t know, for a TCP test I am quite confident that it was uni-directional as the @full MTU data does not show enough loss to accommodate the roughly 2% reverse ACK traffic for the opposite direction).

> 
> I get 765 megabit/s of throughput using single session, at sirq load of around 25%. If I lower the mss to 300 (to generate higher pps) I get around 560 megabit/s of throughput at 50% sirq. With 10 parallel TCP sessions, I get about the same. At MSS of 200 bytes, I get 400 megabit/s at 70% sirq.

I assume iperf3 uses TCP or UDP streams and reports the payload rate, correct? Then we have a MSS of 1460 (with 20 bytes IPv4 header and 20 bytes for TCP or UDP).


@full MTU
MSS:
1500 - 20 - 20 = 1460 byte
Number of packets at 765 Mbps goodput:
(765 * 1000^2) / ((1500 - 20 - 20) * 8) = 65496.5753425 = 65K
On-the-wire packet size (OTWPS) assuming ethernet with FCS and no VLAN tag)s):
1500 + 14 + 4 = 1518 bytes
MSS to OTWPS ratio:
(1500 - 20 - 20) / (1500 + 14 + 4) = 0.961791831357
raw bandwidth consumed by 765 Mbps good put:
765 / ((1500 - 20 - 20) / (1500 + 14 + 4)) = ((765 * 1000^2) / ((1500 - 20 - 20) * 8)) * ((1500 + 14 + 4) * 8) = 795.390410959 Mbps
So basically (1 - (795.390410959/800))*100 = 0.58 % unexplained loss, not bad

@MSS 300
MSS:
300 byte
Number of packets at 560 Mbps goodput:
(560 * 1000^2) / ((300) * 8) = 233333.333333 = 233K
On-the-wire packet size (OTWPS) assuming ethernet with FCS:
300 + 20 + 20 + 14 + 4  = 358 bytes
MSS to OTWPS ratio:
(300) / (300 + 20 + 20 + 14 + 4) = 0.837988826816
raw bandwidth consumed by 560 Mbps good put:
560 / ((300) / (300 + 20 + 20 + 14 + 4 )) = ((560 * 1000^2) / ((300) * 8)) * ((300 + 20 + 20 + 14 + 4 ) * 8) = 668.266666667  Mbps
So basically (1 - (668.266666667/800))*100 = 16.4666666666 % unexplained loss, not pretty but bearable I guess

@MSS 200
MSS:
200 byte
Number of packets at 400 Mbps goodput:
(400 * 1000^2) / ((200) * 8) = 250000 = 250K
On-the-wire packet size (OTWPS) assuming ethernet with FCS:
200 + 20 + 20 + 14 + 4  = 258 bytes
MSS to OTWPS ratio:
(200) / (200 + 20 + 20 + 14 + 4) = 0.77519379845
raw bandwidth consumed by 400 Mbps good put:
400 / ((200) / (200 + 20 + 20 + 14 + 4 )) = ((400 * 1000^2) / ((200) * 8)) * ((200 + 20 + 20 + 14 + 4 ) * 8) = 516  Mbps
So basically (1 - (516/800))*100 = 35.5  % unexplained loss, that is sad. But the packet rate is still at 250K, I winder how this router does with 64 byte ethernet frames


> 
> If I turn off SQM completely, I get 600 megabit/s at 200 byte MSS single session at 80% sirq and 930 megabit/s at 26% sirq with default MSS.

	Since no shaper was used,  I think we need to include the inter-frame-gap and preamble to calculate the maximal payload rates for different packet sizes.

@1Gbps
MSS
(1500 - 20 - 20) = 1460 byte
Number of packets at 930 Mbps goodput:
(930 * 1000^2) / ((1500 - 20 - 20) * 8) = 79623.2876712 = 80K
To asses the maximum achievable at 1 GBE we need to take IFG and preamble into account I think
On-the-wire packet size (OTWPS) assuming ethernet with FCS plus inter frame gap and preamble:
1500 + 14 + 4 + 12 + 8  = 1538 bytes
MSS to OTWPS ratio:
(1500 - 20 - 20)  / (1500 + 14 + 4 + 12 + 8) = 0.949284785436
raw bandwidth consumed by 930 Mbps good put:
930 / ((1500 - 20 - 20) / (1500 + 14 + 4 + 12 + 8)) = ((930 * 1000^2) / ((1500 - 20 - 20) * 8)) * ((1500 + 14 + 4 + 12 + 8) * 8) = 979.684931507  Mbps
So basically (1 - (979.684931507/1000))*100 = 2.0315068493  % unexplained loss, not bad. 

@1Gbps
MSS
200 = 1460 byte
Number of packets at 600 Mbps goodput:
(600 * 1000^2) / ((200) * 8) = 375000 = 375K
On-the-wire packet size (OTWPS) assuming ethernet with FCS plus inter frame gap and preamble:
200 + 20 + 20 + 14 + 4 + 12 + 8  = 278 bytes
MSS to OTWPS ratio:
200  / (200 + 20 + 20 + 14 + 4 + 12 + 8) = 0.719424460432
raw bandwidth consumed by 600 Mbps good put:
600 / ((200) / (200 + 20 + 20 + 14 + 4 + 12 + 8)) = ((600 * 1000^2) / ((200) * 8)) * ((200 + 20 + 20 + 14 + 4 + 12 + 8) * 8) = 834  Mbps
So basically (1 - (834/1000))*100 = 16.6  % unexplained loss, not bad. 

As Dave said it would be nice see RRUL data from the same testbed. It would be so nice if flint had a way to send different sized TCP packets… (I guess this might be faked with MSS clamping in the router and relaying on path MTU discovery?)


> 
> So if you want high performing device that is OpenWRT compatible and still does forwarding using CPU so you can test queuing algorithms, the WRT1200AC and WRT1900ACv2 is the best I have been able to find currently (unless you go for x86 platform).

	The 1200AC retailed for around 200EUR in Germany the 1900ACv2 will be closer to 300EUR I guess, not too expensive but certainly above my impulse buy limit ;)


tack så mycket & Best Regards
	Sebastian

> 
> -- 
> Mikael Abrahamsson    email: swmike@swm.pp.se
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel