On Tue, 2 Sep 2014, Aaron Wood wrote:

>>
>> What this makes me realize is that I should go instrument the cpu stats
>> with each of the various operating modes:
>>
>> * no shaping, anywhere
>> * egress shaping
>> * egress and ingress shaping at various limited levels:
>>     * 10Mbps
>>     * 20Mbps
>>     * 50Mbps
>>     * 100Mbps
>>
>
> So I set this up tonight, and have a big pile of data to go through.  But
> the headline finding is that the WNDR3800 can't do more than 200Mbps
> ingress, with shaping turned off.  The GbE switch fabric and my setup were
> just fine (pushed some very nice numbers through those interfaces when on
> the switch), but going through the routing engine (NATing), and 200Mbps is
> about all it could do.

it's actually probably the connection tracking, not the routing engine or 
iptables. I've seen this a lot on high-traffic systems. I saw something earlier 
this week about how the connection tracking has a global lock, so it's 
effectivly single threaded, but there is work being done to fix this. Now, lock 
contention isn't an issue on a single-core box like the 3800, but the rest of 
the work is.

If you can find a place to set it up without NAT, (or with 1:1 NAT that doesn't 
need connection tracking), you will see much better performance from it.

For the Scale conference, I disable connection tracking and run them as bridges 
to a dedicated VLAN per SSID and do the firewalling and NAT upstream from the 
APs

David Lang

> I took tcp captures of it shaping past it's limit (configured for 150/12),
> with then rrul, tcp_download, tcp_upload tests.
>
> And I took a series of tests walking down from 100/12, 90/12, 80/12, ...
> down to 40/12, while capturing /proc/stats and /proc/softirqs once a second
> (roughly), so that can be processed to pull out where the load might be
> (initial peeking hints that it's all time spent in softirq).
>
> If anyone wants the raw data, let me know, I'll upload it somewhere.  The
> rrul pcap is large, the rest of it can be e-mailed easily.
>
> -Aaron
>