[Cerowrt-devel] [Bloat] Comcast upped service levels -> WNDR3800 can't cope...

Wed Sep 3 17:33:00 PDT 2014

On Wed, Sep 3, 2014 at 4:17 PM, Bill Ver Steeg (versteb)
<versteb at cisco.com> wrote:
> Speaking of IPv6 performance testing- In a recent FTTH field deployment, the network operator deployed an IPv6-only network and tunneled all subscriber IPv4 traffic over an IPv6 tunnel to the upstream network edge. It then unpacked the IPv4 traffic from the IPv6 tunnel and sent it on its merry way.

I tried to deploy ipv4 over ipv6 encapsulation when I was in Nicaragua
6 years back (the alternative was triple nat,
IPv4 addresses were really scarce on the ground), and got beat by the
encapsulation overhead, performance, multiple
bugs and bufferbloat, then. I figure that most of that has improved -
in particular I imagine that their encapsulated traffic
still has a 1500 MTU for the ipv4 traffic?

The original gear I'd had for the experiment could do a 2k MTU, which
was very helpful in making the ipv4 encapsulation
 1500 bytes as much of the internet expects, but a later version
couldn't quite get past 1540 bytes without problems.

> Long story short, the 4o6 tunneling code in the residential gateway was not nearly as performant as the IPv6 forwarding code. I actually got better IPv4 throughput running an IPv6 VPN on my end device, then sending my IPv4 traffic through that tunnel - thus avoiding the tunnel code on the gateway. If I recall correctly, the tunnel code capped out at about 20 Mbps and the IPv6 code went up to the 50Mbps SLA rate. I stumbled into this while running some IPTV video tests while running throughput benchmarks on my PC (with apparently pseudo-random results, until we figured out the various tunnels). Took me a while to figure it out. Delay also spiked when the gateway got bogged down......

I can believe it. I have seen many "bumps in the wire" do bad things
when run past their limits. Notable were several
PPPoe and PPPoA boxes. Older cablemodems, and last generation access
points are going to all have similar problems
when hooked up at these higher speeds. In the future, stuff that does
this sort of tunneling or encapsulation, or while
coverting from one media type to another, (say ethernet->cable,
ethernet->gpon, etc) may also run into it when the provider ups their
access speeds from one band to another, as both comcast and verizon
have.

This is of course, both a problem and an opportunity. A problem
because it will generate more support calls, and an opportunity to
sell better gear into the marketplace as ISP speeds are upgraded.

Some enterprising manufacturer could make a point of pitching their
product(s) as actually capable of modern transfer speeds on modern
ISPs, doing benchmarks, etc.

Given the mass delusional product naming in the home ap marketplace,
where nearly every product is named and pitched
over the base capability of the standards used, rather than the sordid
reality, I don't think anything short of a consumer reports, or legal
action, will result in sanity here.

Gigabit "routers", indeed, when only the switch is cable of that!
Nothing I've tried below 100 bucks can forward, well, at a gigabit,
with a number of real-world firewall rules. Even using x86 gear is
kind of problematic thus far.

http://snapon.lab.bufferbloat.net/~cero2/nuc-to-puck/results.html

> More capable gateways were deployed in the latter stages of the deployment, and they seemed to keep up with the 50 Mbps SLA rate.

What was the measured latency under load?

>
> Bill Ver Steeg
>
>
>
>
>
> -----Original Message-----
> From: bloat-bounces at lists.bufferbloat.net [mailto:bloat-bounces at lists.bufferbloat.net] On Behalf Of Dave Taht
> Sent: Wednesday, September 03, 2014 3:31 PM
> To: Sebastian Moeller
> Cc: cerowrt-devel at lists.bufferbloat.net; bloat
> Subject: Re: [Bloat] [Cerowrt-devel] Comcast upped service levels -> WNDR3800 can't cope...
>
> On Wed, Sep 3, 2014 at 12:22 PM, Sebastian Moeller <moeller0 at gmx.de> wrote:
>> Hi Aaron,
>>
>>
>> On Sep 3, 2014, at 17:12 , Aaron Wood <woody77 at gmail.com> wrote:
>>
>>> On Wed, Sep 3, 2014 at 4:08 AM, Jonathan Morton <chromatix99 at gmail.com> wrote:
>>> Given that the CPU load is confirmed as high, the pcap probably isn't as useful.  The rest would be interesting to look at.
>>>
>>> Are you able to test with smaller packet sizes?  That might help to isolate packet-throughput (ie. connection tracking) versus byte-throughput problems.
>>>
>>>  - Jonathan Morton
>>>
>>> Doing another test setup will take a few days (maybe not until the weekend).  But I can get the data uploaded, and do some preliminary crunching on it.
>>
>>         So the current SQM system allows to shape on multiple interfaces, so you could set up the shaper on se00 and test between sw10 and se00 (should work if you reliably get fast enough wifi connection, something like combined shaped bandwidth <= 70% of wifi rate should work). That would avoid the whole firewall and connection tracking logic.
>>         My home wifi environment is quite variable/noisy and not
>> well-suited for this test: with rrul_be I got stuck at around 70Mbps combined bandwidth, with different distributions of the up and down-leg for no-shaping, shaping to 50Mbps10Mbps, and shaping to 100Mbps50Mbps. SIRQ got pretty much pegged at 96-99% during all netperf-wrapper runs, so I assume this to be the bottleneck (the radio was in the > 200mbps range during the test with occasional drops to 150mbps). So my conclusion would: be it really is the shaping that is limited on my wndr3700v2 with cerowrt 3.10.50-1, again if I would be confident about the measurement which I am not (but EOUTOFTIME). That or my rf environment might only allow for roughly 70-80Mbps combined throughput. For what it is worth: test where performed between macbook running macosx 10.9.4 and hp proliant n54l running 64bit openSuse 13.1, kernel 3.11.10-17 (AMD turion with tg3 gbit ethernet adapter (BQL enabled), running fq_codel on eth0), with sha  ping on the se00 interface.
>
>
> A note on wifi throughput. CeroWrt routes, rather than bridges, between interfaces. So I would expect for simple benchmarks, openwrt (which bridges) might show much better wifi<-> ethernet behavior.
>
> We route, rather than bridge wifi, because of 1) it made it easier to debug it, and 2) the theory that multicast on busier networks messes up wifi far more than not-bridging slows it down. Have not accumulated a lot of proof of this, but this was kind of enlightening:
> http://tools.ietf.org/html/draft-desmouceaux-ipv6-mcast-wifi-power-usage-00
>
> I note that my regular benchmarking environment has mostly been 2 or more routers with nat and firewalling disabled.
>
> Given the trend towards looking at iptables and nat overhead on this thread, an ipv6 benchmark on this box might be revealing.
>
>> Best Regards
>>         Sebastian
>>
>>
>>>
>>> -Aaron
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel at lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>
>
> --
> Dave Täht
>
> https://www.bufferbloat.net/projects/make-wifi-fast
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

-- 
Dave Täht

https://www.bufferbloat.net/projects/make-wifi-fast