From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp81.iad3a.emailsrvr.com (smtp81.iad3a.emailsrvr.com [173.203.187.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id 3DB9C21F282 for ; Sun, 25 May 2014 13:18:40 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp11.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 0A427280094; Sun, 25 May 2014 16:18:39 -0400 (EDT) X-Virus-Scanned: OK Received: from app15.wa-webapps.iad3a (relay.iad3a.rsapps.net [172.27.255.110]) by smtp11.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id CDA08280090; Sun, 25 May 2014 16:18:38 -0400 (EDT) Received: from reed.com (localhost.localdomain [127.0.0.1]) by app15.wa-webapps.iad3a (Postfix) with ESMTP id B530480061; Sun, 25 May 2014 16:18:38 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@reed.com, from: dpreed@reed.com) with HTTP; Sun, 25 May 2014 16:18:38 -0400 (EDT) Date: Sun, 25 May 2014 16:18:38 -0400 (EDT) From: dpreed@reed.com To: "Dave Taht" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_20140525161838000000_97990" Importance: Normal X-Priority: 3 (Normal) X-Type: html In-Reply-To: References: Message-ID: <1401049118.740513082@apps.rackspace.com> X-Mailer: webmail7.0 Cc: "cerowrt-devel@lists.bufferbloat.net" Subject: Re: [Cerowrt-devel] =?utf-8?q?Fwd=3A_qos_in_open_commotion=3F?= X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 May 2014 20:18:40 -0000 ------=_20140525161838000000_97990 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =0ABesides my diversionary ramble in previous post, let me observe this.=0A= =0AUntil you realize that maintaining buffers inside the network never hel= ps with congestion in a resource limited network, you don't really understa= nd the problem.=0A =0AThe only reason to have buffers at all is to deal wit= h transient burst arrivals, and the whole goal is to stanch the sources qui= ckly.=0A =0ATherefore I suspect that commotion would probably work better b= y reducing buffering to fit into small machines' RAM constraints, and on la= rger machines, just letting the additional RAM be used for something else.= =0A =0AFor many "network experts" this idea is "heresy" because they've bee= n told that maximum throughput is gained only by big buffers in the network= . Buffers are thought of like processor cache memories - the more the bette= r for throughput.=0A =0AThat statement is generally not true. It is true in= certain kinds of throughput tests (single flows between the same source an= d destination, where end-to-end packet loss rates are high and not mitigate= d by link-level retrying once or twice). But in those tests there is no con= gestion, just an arbitrarily high queueing delay, which does not matter for= pure throughput tests.=0A =0ABuffering *is* congestion, when a link is sha= red among multiple flows.=0A =0A=0A=0AOn Sunday, May 25, 2014 3:56pm, "Dave= Taht" said:=0A=0A=0A=0A> meant to cc cerowrt-devel o= n this...=0A> =0A> =0A> ---------- Forwarded message ----------=0A> From: D= ave Taht =0A> Date: Sun, May 25, 2014 at 12:55 PM=0A> = Subject: qos in open commotion?=0A> To: andygunn@opentechinstitute.org, com= motion-dev@lists.chambana.net=0A> =0A> =0A> Dear Andy:=0A> =0A> In response= to your thread on qos in open commotion my list started a thread=0A> =0A> = https://lists.bufferbloat.net/pipermail/cerowrt-devel/2014-May/003044.html= =0A> =0A> summary:=0A> =0A> You can and should run packet scheduling/aqm/qo= s in routers with 32MB=0A> of memory or less. Some compromises are needed:= =0A> =0A> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2014-May/00= 3048.html=0A> =0A> FIRST:=0A> =0A> We strongly recomend that your edge gate= ways have aqm/packet=0A> scheduling/qos on all their connections to the int= ernet. See=0A> innumerable posting on bufferbloat and the fixes for it...= =0A> =0A> http://gettys.wordpress.com/=0A> =0A> Feel free to lift cerowrt's= SQM scripts and gui from the ceropackages=0A> repo for your own purposes. = Openwrt barrier breaker qos-scripts are=0A> pretty good too but don't work = with ipv6 at the moment...=0A> =0A> http://www.bufferbloat.net/projects/cer= owrt/wiki/Setting_up_SQM_for_CeroWrt_310=0A> =0A> For the kind of results w= e get on cable:=0A> =0A> http://snapon.lab.bufferbloat.net/~cero2/jimreiser= t/results.html=0A> =0A> Wifi has a built in QoS (802.11e) system but it doe= sn't work well in=0A> congested environments=0A> and optimizing wireless-n = aggregation works better.=0A> =0A> As for fixing wifi, well, we know what t= o do, but never found any=0A> funding for it. Ath9k is still horribly overb= uffered and while=0A> fq_codel takes some of the edge off of wifi (and rece= ntly we disabled=0A> 802.11e entirely in favor of fq_codel), and in cerowrt= we reduce=0A> aggregation to get better latency also - much more work rema= ins to=0A> truly make it scale down to levels of latency we consider reason= able=0A> while (In other words, wifi latencies suck horribly now no matter= =0A> what yet we think we know how to improve that. Feel free to do=0A> mea= surements of your mesh with tools like netperf-wrapper. There are=0A> also = a few papers out there now showing how bad wifi can get nowadays)=0A> =0A> = As for replacing pfifo_fast, openwrt barrier breaker replaced=0A> pfifo_fas= t with fq_codel in barrier breaker a year ago.=0A> =0A> fq_codel by default= is essentially zero cost (64k per interface*hw=0A> queues) and the default= in openwrt on all interfaces by default now...=0A> =0A> but the typical ro= uter cpus are so weak it is rare it kicks in except=0A> at 100mbit and belo= w. (where it can be wonderful) - and it's on a rate=0A> limited (eg dsl or = cable) system where it's most obviously useful.=0A> =0A> Presently.=0A> =0A= > Lastly, I've been running a deployed babel mesh network for 2 years=0A> w= ith fq_codel in it, 2 SSIDs per nanostation m5 and picostation, and=0A> it = runs pretty good. Recent tests on the ubnt edgerouter went well, as=0A> wel= l...=0A> =0A> Please give this stuff a shot. Your users will love it.=0A> = =0A> --=0A> Dave T=C3=A4ht=0A> =0A> NSFW:=0A> https://w2.eff.org/Censorship= /Internet_censorship_bills/russell_0296_indecent.article=0A> =0A> =0A> --= =0A> Dave T=C3=A4ht=0A> =0A> NSFW:=0A> https://w2.eff.org/Censorship/Intern= et_censorship_bills/russell_0296_indecent.article=0A> _____________________= __________________________=0A> Cerowrt-devel mailing list=0A> Cerowrt-devel= @lists.bufferbloat.net=0A> https://lists.bufferbloat.net/listinfo/cerowrt-d= evel=0A> ------=_20140525161838000000_97990 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Besides my= diversionary ramble in previous post, let me observe this.

=0A

 

=0A

Unti= l you realize that maintaining buffers inside the network never helps with = congestion in a resource limited network, you don't really understand the p= roblem.

=0A

 

=0A

The only reason to have buffers at all is to deal with tr= ansient burst arrivals, and the whole goal is to stanch the sources quickly= .

=0A

 

=0A

Therefore I suspect that commotion would probably work better b= y reducing buffering to fit into small machines' RAM constraints, and on la= rger machines, just letting the additional RAM be used for something else.<= /p>=0A

 

=0A

For many "network experts" this idea is "heresy" because they've = been told that maximum throughput is gained only by big buffers in the netw= ork. Buffers are thought of like processor cache memories - the more the be= tter for throughput.

=0A

 

=0AThat statement is generally not true. It is = true in certain kinds of throughput tests (single flows between the same so= urce and destination, where end-to-end packet loss rates are high and not m= itigated by link-level retrying once or twice). But in those tests there is= no congestion, just an arbitrarily high queueing delay, which does not mat= ter for pure throughput tests.

=0A

 = ;

=0A

Buffering *is* congestion, when a = link is shared among multiple flows.

=0A

 

=0A





On Sund= ay, May 25, 2014 3:56pm, "Dave Taht" <dave.taht@gmail.com> said:

=0A
=0A

> meant to cc cerowrt-devel on this...
>
>
> ---------- Forwarded message ----------
> From: Dave Taht &l= t;dave.taht@gmail.com>
> Date: Sun, May 25, 2014 at 12:55 PM
> Subject: qos in open commotion?
> To: andygunn@opentechinsti= tute.org, commotion-dev@lists.chambana.net
>
>
> = Dear Andy:
>
> In response to your thread on qos in open c= ommotion my list started a thread
>
> https://lists.buffer= bloat.net/pipermail/cerowrt-devel/2014-May/003044.html
>
>= summary:
>
> You can and should run packet scheduling/aqm= /qos in routers with 32MB
> of memory or less. Some compromises are= needed:
>
> https://lists.bufferbloat.net/pipermail/cerow= rt-devel/2014-May/003048.html
>
> FIRST:
>
&= gt; We strongly recomend that your edge gateways have aqm/packet
> = scheduling/qos on all their connections to the internet. See
> innu= merable posting on bufferbloat and the fixes for it...
>
>= http://gettys.wordpress.com/
>
> Feel free to lift cerowr= t's SQM scripts and gui from the ceropackages
> repo for your own p= urposes. Openwrt barrier breaker qos-scripts are
> pretty good too = but don't work with ipv6 at the moment...
>
> http://www.b= ufferbloat.net/projects/cerowrt/wiki/Setting_up_SQM_for_CeroWrt_310
&g= t;
> For the kind of results we get on cable:
>
>= http://snapon.lab.bufferbloat.net/~cero2/jimreisert/results.html
>=
> Wifi has a built in QoS (802.11e) system but it doesn't work we= ll in
> congested environments
> and optimizing wireless-n = aggregation works better.
>
> As for fixing wifi, well, we= know what to do, but never found any
> funding for it. Ath9k is st= ill horribly overbuffered and while
> fq_codel takes some of the ed= ge off of wifi (and recently we disabled
> 802.11e entirely in favo= r of fq_codel), and in cerowrt we reduce
> aggregation to get bette= r latency also - much more work remains to
> truly make it scale do= wn to levels of latency we consider reasonable
> while (In other w= ords, wifi latencies suck horribly now no matter
> what yet we thin= k we know how to improve that. Feel free to do
> measurements of yo= ur mesh with tools like netperf-wrapper. There are
> also a few pap= ers out there now showing how bad wifi can get nowadays)
>
&g= t; As for replacing pfifo_fast, openwrt barrier breaker replaced
> = pfifo_fast with fq_codel in barrier breaker a year ago.
>
>= ; fq_codel by default is essentially zero cost (64k per interface*hw
&= gt; queues) and the default in openwrt on all interfaces by default now...<= br />>
> but the typical router cpus are so weak it is rare it = kicks in except
> at 100mbit and below. (where it can be wonderful)= - and it's on a rate
> limited (eg dsl or cable) system where it's= most obviously useful.
>
> Presently.
>
>= ; Lastly, I've been running a deployed babel mesh network for 2 years
= > with fq_codel in it, 2 SSIDs per nanostation m5 and picostation, and> it runs pretty good. Recent tests on the ubnt edgerouter went well= , as
> well...
>
> Please give this stuff a shot. = Your users will love it.
>
> --
> Dave T=C3=A4ht>
> NSFW:
> https://w2.eff.org/Censorship/Internet_= censorship_bills/russell_0296_indecent.article
>
>
&= gt; --
> Dave T=C3=A4ht
>
> NSFW:
> https:= //w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.art= icle
> _______________________________________________
> Ce= rowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

=0A=
------=_20140525161838000000_97990--