[Bloat] QoS for mission critical packets on wireless networks

[Dave Taht]

I think it would be more useful for me to describe the context behind the
fe80:: thread also
on this list.

In the case of wireless interfaces, very little QoS and packet
prioritization has been taking place
with the various qdiscs being used. (Linux uses "mq" by default, which is
severely underdocumented, and the features it has are rarely implemented by
packet classification before
it hits that qdisc, and it seems mostly intended to take advantage of
hardware QoS, (which at least on wireless n is something of a lose anyway -
aggregation is better)

Result: On a busy wireless network, mission critical, statistically rare
packets, are getting delayed or lost.

If it already wasn't apparent to me before I started on this trip a few
weeks ago,  it is glaringly obvious now.

Yesterday I was getting 3.5KB(!)/sec throughput on Georgia Tech's internal
wireless lan. I could "hear" over 32 access points.

And everywhere I've been, hotels, airports, etc, I've seen DHCP take
multiple attempts to get an address, and DNS timing out a lot. I'm sure
others have observed this sort of behavior!!!

On a busy wireless lan, and/or one with many access points and clients on
the same channel, DHCP queries go unanswered, ARP!! times out, DNS times
out, RA and ND fail, NTP gets delayed... While some of this is due to
interference (my god, "hearing" 32+APs!), I believe that a great deal of it
is due to large queue lengths and contention for the link the user is
connected to.

The default behavior for many of these protocols is to re-send the packets
after a short time - thus adding to the problem!

So...

Felix Fietkau reduced queue lengths significantly in the ath9k wireless
drivers used in the uberwrt/bismark/iscswrt projects and that has helped a
lot (still quite a ways to go to match my original patch but getting packet
aggregation to work well requires larger queues)

On the current build of bismark, I get 32ms avg delay on a latency under
load test, on an access point I'm connected to at 54Mbit/second. I'm going
to start doing connects at slower speeds today and expect the results to be
rather awful.

While this is not good (my old patches got around 4ms, debloat-testing hits
around 12ms), this is STILL better than what many are seeing in the field...

So...

I think some *more* of the problems can be fixed by packet prioritization on
the wireless side.

So I started working on a set of wireless QoS ideas to see if I could
improve the situation. (wilst working on red for external, wired interfaces)

I started off with a hard case - a broadcast multicast protocol that runs
over IPv6 called babel. It has the interesting property in that it sends a
broadcast every 20 seconds from an identifiable address (theoretically - see
the fe80 discussion also on this list) and thus I can easily (or so I
thought) observe, with enough nodes - delays and packet loss.

The idea I'm toying with now is to reserve a few percentage points of the
bandwidth to mission critical packets such as those listed so far, using
packet marking and a queuing discipline that does more of the "right thing"
(hsfc or htb+sfb)

(strict prioritization of things like ping would lead to trivial DDOSes on
any of these protocols, so a tiny bandwidth reserve strikes me as a better
approach)

Wireless multicast is additionally weird in that it falls back to a very low
rate in order to do its work.

Suggestions? comments? The current build of the "capetown" release of
bismark is very usable, has many cool features, including SFB, and supports
several off the shelf routers commonly available via best-buy and new-egg.

Anyone that's interested, please join in the effort. Certainly mere qdiscs
are something more folk can fiddle with at a scripting and tcpdump level...

Bismark documentation (supports Netgear wndr3700, wndr3700v2, nanostation
M5, buffalo, and I just build an x86 kvm virtual machine that needs love)

http://www.bufferbloat.net/projects/bismark/wiki

Capetown builds:
http://huchra.bufferbloat.net/~bismark/

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://the-edge.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20110511/bc664d19/attachment-0002.html>