[Bloat] Notes about hacking on AQMs

[Dave Taht]

So in addition to hacking on the switch, I've been poking into the behavior
of multiple AQM systems in the kernel, ranging from the wondershaper,
to the adsl-shaper, to the qos-scripts in openwrt.

I started off with something ambitious, which was to try and implement
a complete implementation of diffserv, using guidelines laid out by

not only across the outgoing to-the-internet interface, but across the
internal wired and wireless networks, something that would work with all
protocols.

I rapidly got bogged down. (or rather, I've been poking at it for months,
nay, years,
in part trying to find feedback loops that handled 'tiny monster' packets
like multicast on wireless)

Some notes:

There are as many philosophies to AQM as there are shapers and classifiers.

None of the Linux shaper scripts in the field handle ipv6 traffic.

HTB is the most commonly used qdisc, handles it's bandwidth limits by packet
drop
and doesn't do ECN. It's usually used in conjunction with other qdiscs, too.

An explanation of how diffserv (dsmark) and GRED are supposed to play ball
together
(starting here: http://www.opalsoft.net/qos/DS-27.htm) is so amazingly not
opaque.

SFB remains promising, but until I get a ported tc for it,
I can't play with it much.

SFQ is the second most commonly used qdisc, but doesn't balance in ways ESFQ
could.

ESFQ really looked like a winner and I'm sorry it never made the mainline
kernel.

HFSC is mind-bending as to what it tries to do.

Any form of fair queuing is useful for ethernet, but actually knowing the
link rate and port on the switch per dest macaddr would help in load
balancing streams.

Fair queueing is very bad on wireless when packet aggregation is used.

PFIFO_FAST is tied to TOS bits, not diffserv bits.

RED is, well, RED.

GRED is far less opaque than RED, as noted earlier.

MQ and MQPrio are horribly underdocumented. I still don't 'get' how to use
them
properly (I'm more focused on writing a good classifier at the moment)

802.11e does its prioritization at the vlan layer, not at the TOS or
diffserv bits. Getting from tos or diffserv to mq* seems painful but I
haven't looked into it too hard.

iptables seems to think ecn can only be looked at in TCP streams, where (for
example),
ecn bits can be copied to the outer header of a udp vpn stream, and marked
when needed.

ip6tables has no support for looking at ecn except through a u32 match.

You are in a maze of twisty little passages, all not quite going where you
want
to go. The intersection of all these 'solutions' is part of why wireless is
so messed up, as are home routers...
and I haven't even got to trying to figure out the multicast monster problem
yet!

Adding ECN capability to the other qdiscs looks like low hanging fruit...

Anyway, aside from the whining^H^H^H^H^^H descriptions above, here's a quick
and dirty bit of iptables useful for detecting ecn capability:

iptables -t mangle -X Wireless
iptables -t mangle -N Wireless
iptables -t mangle -F Wireless
iptables -t mangle -A Wireless -p tcp -m tcp --tcp-flags ALL SYN,ACK -m ecn
--ecn-tcp-ece -m recent --name ecn_enabled --set -m comment --comment 'ECN
enabled streams'
iptables -t mangle -A Wireless -p tcp -m tcp --tcp-flags ALL SYN,ACK -m ecn
! --ecn-tcp-ece -m recent --name ecn_disabled --set -m comment --comment
'ECN disab
led streams'

iptables -t mangle -F POSTROUTING
iptables -t mangle -A POSTROUTING -j Wireless

You can see what ips managed to do ECN or not via

cat /proc/net/xt_recent/ecn_*

But that's just a distraction from trying to converge on a
decent set of solutions for AQM. I AM happy to report that after getting
buffer sizes down (via ethtool, a switch patch, txqueuelen) I am finally
able to reliably see sub 10ms latencies on the wndr3700... but I wake up
these days, feeling doomed.

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://the-edge.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20110608/39098124/attachment-0002.html>