[Cake] FreeNet update

[Pete Heist <pete@heistp.net>]

We’ve got hfsc+fq_codel on one of our licensed spectrum full-duplex 100-Mbit backhaul links. I hope to switch to Cake, but I’m not yet sure the 3.16.7 kernel we’re using is stable enough to handle it.

I’d like to report how much better things are now even with fq_codel, but we keep finding (and fixing) problems elsewhere in the network that prevent us from actually filling the queue. The short list:

1) Switched TDD framing modes (Ubiquiti’s "flexible new" for now) as fixed framing was increasing RTT and killing download throughput (done)
2) Increased stability of CPE links (setting max tx rate, etc) to prevent loss / latency spikes that affect upstream ACKs and thus download throughput (done)
3) rx-vlan-offload must be broken in the r8169 driver in 3.16, was causing throughput problems for VLAN routing (fixed)
4) The current unsolved killer is a 0.03% downstream packet loss problem on the backhaul link itself, which per the Mathis equation does us no good

I can fill the queue on upload and show clear results with an rrul_up test, but not download yet, because of problem #4.

One thing that had to be handled right in the sqm setup script was VLANs, which I imagine most ISPs will have in the backhaul. The original qos script, using sfq, was adding qdiscs to both the VLAN devices (eth0.3300, etc) and the main devices on egress and ingress. This caused multiple problems:

- Routed packets would pass through three queues in each direction unnecessarily, and ingress shaping wasn’t needed in the backhaul anyway
- One-armed routers were effectively made half-duplex, because a qdisc was being added to eth0 without any filtering by VLAN, putting Internet facing and customer facing traffic on the same queue

Now I use tc filters to assign VLANs to queues and apply all qdiscs on eth0 under one hfsc/htb link sharing hierarchy.

So, one router almost done and 60 more to go, as we work to bloat up our queues so we can debloat them. :)

Sigh, I hope to get back to development soon. Enough for now, and thanks again to the list for the help…