From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-iy0-f171.google.com (mail-iy0-f171.google.com [209.85.210.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 58CBD20086A for ; Fri, 17 Jun 2011 10:47:26 -0700 (PDT) Received: by iyi12 with SMTP id 12so1897469iyi.16 for ; Fri, 17 Jun 2011 11:12:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=a9SX6BkqxPC0+i/fmhrYBMLbPdOvbaAk7PyG4dDuxXo=; b=xAZthpv0mjbz0/NB/iUucBy3SxxRTaY5pauVcNwNxTtCs0u18f05Z/iLdThzyxIhuh sJ6YeUAIwWQ1GwxOwXp5tnjrRABC03s4xuujY3WDX/gIx7Y2aBN5xBdzs+7f2vpBJl7d ZlI9K18iyfbVRwuJcDs1zRFPgL3lQj0T2uiAM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=GOmwvzxMAABS+bwHiT9ZsB4bXYVOv3ZEMzmFruuC0CXtXEIVYQiJrquH1RiuuCZ9Sv zxbFosFnwFZH2FIhkZQ3Lxm/xQCpp5Qg+iW76w/mPua2zqASc2gkP+QNwIJfZ9cRvaB8 Ze29iV8b9HPxPyjD9oOZ7JVnhAZ6EgV62/X94= MIME-Version: 1.0 Received: by 10.231.119.209 with SMTP id a17mr2073092ibr.88.1308334333750; Fri, 17 Jun 2011 11:12:13 -0700 (PDT) Received: by 10.231.13.76 with HTTP; Fri, 17 Jun 2011 11:12:13 -0700 (PDT) In-Reply-To: <20110617171452.GB6820@tuxdriver.com> References: <20110617171452.GB6820@tuxdriver.com> Date: Fri, 17 Jun 2011 12:12:13 -0600 Message-ID: From: Dave Taht To: "John W. Linville" Content-Type: multipart/alternative; boundary=0016369202fb35ef3c04a5ec5344 Cc: bloat@lists.bufferbloat.net Subject: Re: [Bloat] debloat-testing rebased (3.0-rc3-db) X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jun 2011 17:47:27 -0000 --0016369202fb35ef3c04a5ec5344 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Well there is a lot of stuff that has been fed into 2.6.39 and later that came from the debloating work. In particular ECN and DSCP should 'just work' now under all circumstances (or so I hope) on both ipv6 and ipv4. So at the moment, debloat-testing is looking rather bare, but it's been surving a purpose. My personal problem is that I've been unable to convince any custom kernel to compile and run on ubuntu 10.10 on my laptop, which really hinders my ow= n efforts. (jg has been able to build debs that work, but for some reason I can't. It's embarrasing) There have been so many ecn and dscp fixes pushed that the old debloat-testing was getting useless for that stuff, so I thank you for getting it done today. Moving forward... I've been thinking about building kvm based openwrt versions of debloat-testing... Someone could run multiple ones at the same time and watch the traffic go by, much like how strongswan does it's testing, here: http://www.strongswan.org/uml-testing.html and we could start getting more reproducable results from everyone. People could fiddle with the gui. play with shapers. Stuff like that. It also would be kind of helpful to be able to boot a kernel and small os from a memory stick or ramdisk at this point. (and I rather dislike mucking with my laptop anyway) High on my want-to-do-but-mostly-wish-someone-else-would-do list are: 0) Add ecn support to HTB, and the other common shapers. 1) debloating a string of common ethernet drivers, adding ethtool where needed 2) working on vastly improving packet classification techniques with DSCP. 3) Making 802.11e 'just work' (see 2) 4) fiddling with netem to simulate delays 5) fidding with gred (why only 16 queues? why not 64??? Easy patch...) Does gred even work? nobody's touched the code since, like mid-05 6) writing a replacement for pfifo_fast 7) coming up with a sane way to measure the effect of massive multicast packets' and feed that back into qdisc estimators 8) Moving the cerowrt/wndr work forward to 3.0 from the heavily-now-backported stuff in 2.6.37.6... Re: 0) Logic: mark_or_drop_packet(skb) - returns 0 if dropped, 1 if marked, 2 if already marked, -1 on error (or whatever) Just that much would make me happy from a testing perspective. HTB drop= s an extraordinary number of packets under load. A slight optimization would be to try harder to shoot non-ecn packets, so trying a few more (4? 2?) buckets upon detection of a (currently) rare already ecn marked packet woul= d be nice. a problem is actually pushing a common infrastructure statistics infrastructure for marking/dropping into the rest of the stack which I don'= t care about... While there is architecture in the kernel for seeing what packets are being dropped and why, nobody is paying much attention to it. CONFIG_NET_DROP_MONITOR and CONFIG_SCHED_NETEM are looking more useful by the day. Re: 2) - I finally got to where I was happy with the comprehensive Diffserv classifier. I like it. Being able to clearly see all the Mice packets and other packet types against typical traffic is a real boon, and the code now works on openwrt and more normal Linux boxes. It works well mostly - With the plethora of iptables rules, it slows a router capable of 150Mb/sec down to under 40. Fixing it is relatively easy by adding some smarter support for table lookups and priority reclassification in iptables... (git repo: https://github.com/dtaht/Diffserv - I note that to work right, this requires recently fixed bugs in netfilter (and/or debloat-testing or 3.0-rc2 or cerowrt)) Anyway... I really don't care about the performance issue right now, I wanted, now that traffic was comprehensively classified, to start feeding i= t into the appropriate shapers, and of course, ran smack in gred's 16 queue limitation, just to start with. And then pfifo_fast started bugging me, in that I wanted more fairness to all flows in the final qdisc... and then I decided that if I merely got a usable build done of cerowrt-dbg for the wndr that someone else could use, I'd take the afternoon off. It's up. It's working... http://huchra.bufferbloat.net/~cerowrt/cerowrt-wndr3700-dbg/ I'm outta here. Happy downloading. The nanostation M5 build also mostly works. On Fri, Jun 17, 2011 at 11:14 AM, John W. Linville wrote: > I finally got around to rebasing the debloat-testing tree -- I > apologize for the delay! > > There isn't much in it aside from the 3.0-rc3 bits -- dtaht's ath9k > buffer reduction and my (mostly abandoned) skb->destructor-based > eBDP implementation for mac80211. Feel free to nominate > appropriate/interesting patches as you find them (including your own)! > > Thanks, > > John > -- > John W. Linville Someday the world will need a hero, and y= ou > linville@tuxdriver.com might be all we have. Be ready. > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 http://the-edge.blogspot.com --0016369202fb35ef3c04a5ec5344 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Well there is a lot of stuff that has been fed into 2.6.39 and later that c= ame
from the debloating work. In particular ECN and DSCP should 'ju= st work'
now under all circumstances (or so I hope) on both ipv6 and= ipv4.

So at the moment, debloat-testing is looking rather bare, but it's = been surving
a purpose.

My personal problem is that I've bee= n unable to convince any custom kernel
to compile and run on ubuntu 10.1= 0 on my laptop, which really hinders my own efforts. (jg has been able to b= uild debs that work, but for some reason I can't.
It's embarrasing)

There have been so many ecn and dscp fixes pus= hed that the old debloat-testing
was getting useless for that stuff, so = I thank you for getting it done today.

Moving forward...

I've been thinking about building kvm based openwrt versions of debloat= -testing... Someone could run multiple ones at the same time and watch the = traffic go by, much like how strongswan does it's testing, here:

http://www.stron= gswan.org/uml-testing.html

and we could start getting more=A0 re= producable results from everyone.

People could fiddle with the gui.= play with shapers. Stuff like that.=A0

It also would be kind of helpful to be able to boot a kernel and small = os
from a memory stick or ramdisk at this point.

(and I rather di= slike mucking with my laptop anyway)

High on my want-to-do-but-mostl= y-wish-someone-else-would-do list are:

0) Add ecn support to HTB, and the other common shapers.
=A01) debl= oating a string of common ethernet drivers, adding ethtool where needed
= 2) working on vastly improving packet classification techniques with DSCP.<= br> 3) Making 802.11e 'just work' (see 2)
4) fiddling with netem to = simulate delays
5) fidding with gred (why only 16 queues? why not 64??? = Easy patch...)
=A0=A0=A0 Does gred even work? nobody's touched the c= ode since, like mid-05
6) writing a replacement for pfifo_fast
7) coming up with a sane way to = measure the effect of massive multicast packets'
=A0=A0=A0=A0 and f= eed that back into qdisc estimators
8) Moving the cerowrt/wndr work forw= ard to 3.0 from the heavily-now-backported
=A0=A0=A0 stuff in 2.6.37.6...

Re: 0)

=A0=A0 Logic:

=A0=A0=A0 mark_or_drop_packet(skb) - returns 0 if dropped, 1 if marked, 2 i= f already marked, -1 on error
=A0=A0=A0 (or whatever)

=A0=A0=A0 Just that much would make me happy from a testing perspective= . HTB drops an extraordinary number of packets under load. A slight optimiz= ation would be to try harder to shoot non-ecn packets, so trying a few more= (4? 2?) buckets upon detection of a (currently) rare already ecn marked pa= cket would be nice.

=A0=A0=A0 a problem is actually pushing a common infrastructure statistics = infrastructure for marking/dropping into the rest of the stack which I don&= #39;t care about...

=A0=A0 While there is architecture in the kernel= for seeing what packets are being dropped
=A0=A0 and why, nobody is paying much attention to it.

=A0=A0 CONFI= G_NET_DROP_MONITOR and CONFIG_SCHED_NETEM are
=A0=A0 looking more usefu= l by the day.

Re: 2) - I finally got to where I was happy with the c= omprehensive Diffserv classifier.

I like it. Being able to clearly see all the Mice packets and other pac= ket types against typical traffic is a real boon, and the code now works on= openwrt and more normal Linux boxes.

It works well mostly - With t= he plethora of iptables rules, it slows a router capable of 150Mb/sec down = to under 40. Fixing it is relatively easy by adding some smarter
support for table lookups and priority reclassification in iptables...
<= br>(git repo: https://github.= com/dtaht/Diffserv - I note that to work right, this requires recently = fixed bugs in netfilter (and/or debloat-testing or 3.0-rc2 or cerowrt))

Anyway... I really don't care about the performance issue right now= , I wanted, now that traffic was comprehensively classified, to start feedi= ng it into the appropriate shapers, and of course, ran smack in gred's = 16 queue limitation, just to start with. And then pfifo_fast started buggin= g me, in that I wanted more fairness to all flows in the final qdisc...

and then I decided that if I merely got a usable build done of cerowrt-= dbg for
the wndr that someone else could use, I'd take the afternoon= off.

It's up. It's working...

http://huchra.bufferbloa= t.net/~cerowrt/cerowrt-wndr3700-dbg/

I'm outta here. Happy downloading. The nanostation M5 build also mo= stly works.
=A0


On Fri, Jun 17, 20= 11 at 11:14 AM, John W. Linville <linville@tuxdriver.com> wrote:
I finally got aro= und to rebasing the debloat-testing tree -- I
apologize for the delay!

There isn't much in it aside from the 3.0-rc3 bits -- dtaht's ath9k=
buffer reduction and my (mostly abandoned) skb->destructor-based
eBDP implementation for mac80211. =A0Feel free to nominate
appropriate/interesting patches as you find them (including your own)!

Thanks,

John
--
John W. Linville =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Someday the world will need= a hero, and you
linville@tuxdriver.com =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0might be all we have. =A0Be ready.
_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net<= /a>
= https://lists.bufferbloat.net/listinfo/bloat



--
Dave T=E4ht
S= KYPE: davetaht
US Tel: 1-239-829-5608
http://the-edge.blogspot.com
--0016369202fb35ef3c04a5ec5344--