[Bloat] debloat-testing rebased (3.0-rc3-db)

Fri Jun 17 11:12:13 PDT 2011

Well there is a lot of stuff that has been fed into 2.6.39 and later that
came
from the debloating work. In particular ECN and DSCP should 'just work'
now under all circumstances (or so I hope) on both ipv6 and ipv4.

So at the moment, debloat-testing is looking rather bare, but it's been
surving
a purpose.

My personal problem is that I've been unable to convince any custom kernel
to compile and run on ubuntu 10.10 on my laptop, which really hinders my own
efforts. (jg has been able to build debs that work, but for some reason I
can't.
It's embarrasing)

There have been so many ecn and dscp fixes pushed that the old
debloat-testing
was getting useless for that stuff, so I thank you for getting it done
today.

Moving forward...

I've been thinking about building kvm based openwrt versions of
debloat-testing... Someone could run multiple ones at the same time and
watch the traffic go by, much like how strongswan does it's testing, here:

http://www.strongswan.org/uml-testing.html

and we could start getting more  reproducable results from everyone.

People could fiddle with the gui. play with shapers. Stuff like that.

It also would be kind of helpful to be able to boot a kernel and small os
from a memory stick or ramdisk at this point.

(and I rather dislike mucking with my laptop anyway)

High on my want-to-do-but-mostly-wish-someone-else-would-do list are:

0) Add ecn support to HTB, and the other common shapers.
 1) debloating a string of common ethernet drivers, adding ethtool where
needed
2) working on vastly improving packet classification techniques with DSCP.
3) Making 802.11e 'just work' (see 2)
4) fiddling with netem to simulate delays
5) fidding with gred (why only 16 queues? why not 64??? Easy patch...)
    Does gred even work? nobody's touched the code since, like mid-05
6) writing a replacement for pfifo_fast
7) coming up with a sane way to measure the effect of massive multicast
packets'
     and feed that back into qdisc estimators
8) Moving the cerowrt/wndr work forward to 3.0 from the
heavily-now-backported
    stuff in 2.6.37.6...

Re: 0)

   Logic:

    mark_or_drop_packet(skb) - returns 0 if dropped, 1 if marked, 2 if
already marked, -1 on error
    (or whatever)

    Just that much would make me happy from a testing perspective. HTB drops
an extraordinary number of packets under load. A slight optimization would
be to try harder to shoot non-ecn packets, so trying a few more (4? 2?)
buckets upon detection of a (currently) rare already ecn marked packet would
be nice.

    a problem is actually pushing a common infrastructure statistics
infrastructure for marking/dropping into the rest of the stack which I don't
care about...

   While there is architecture in the kernel for seeing what packets are
being dropped
   and why, nobody is paying much attention to it.

   CONFIG_NET_DROP_MONITOR and CONFIG_SCHED_NETEM are
   looking more useful by the day.

Re: 2) - I finally got to where I was happy with the comprehensive Diffserv
classifier.

I like it. Being able to clearly see all the Mice packets and other packet
types against typical traffic is a real boon, and the code now works on
openwrt and more normal Linux boxes.

It works well mostly - With the plethora of iptables rules, it slows a
router capable of 150Mb/sec down to under 40. Fixing it is relatively easy
by adding some smarter
support for table lookups and priority reclassification in iptables...

(git repo: https://github.com/dtaht/Diffserv - I note that to work right,
this requires recently fixed bugs in netfilter (and/or debloat-testing or
3.0-rc2 or cerowrt))

Anyway... I really don't care about the performance issue right now, I
wanted, now that traffic was comprehensively classified, to start feeding it
into the appropriate shapers, and of course, ran smack in gred's 16 queue
limitation, just to start with. And then pfifo_fast started bugging me, in
that I wanted more fairness to all flows in the final qdisc...

and then I decided that if I merely got a usable build done of cerowrt-dbg
for
the wndr that someone else could use, I'd take the afternoon off.

It's up. It's working...

http://huchra.bufferbloat.net/~cerowrt/cerowrt-wndr3700-dbg/

I'm outta here. Happy downloading. The nanostation M5 build also mostly
works.

On Fri, Jun 17, 2011 at 11:14 AM, John W. Linville
<linville at tuxdriver.com>wrote:

> I finally got around to rebasing the debloat-testing tree -- I
> apologize for the delay!
>
> There isn't much in it aside from the 3.0-rc3 bits -- dtaht's ath9k
> buffer reduction and my (mostly abandoned) skb->destructor-based
> eBDP implementation for mac80211.  Feel free to nominate
> appropriate/interesting patches as you find them (including your own)!
>
> Thanks,
>
> John
> --
> John W. Linville                Someday the world will need a hero, and you
> linville at tuxdriver.com                  might be all we have.  Be ready.
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://the-edge.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20110617/ce1bc6a8/attachment.html>