From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-x22f.google.com (mail-lb0-x22f.google.com [IPv6:2a00:1450:4010:c04::22f]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id B59CF21F353 for ; Wed, 1 Oct 2014 20:46:13 -0700 (PDT) Received: by mail-lb0-f175.google.com with SMTP id u10so1542754lbd.34 for ; Wed, 01 Oct 2014 20:46:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=QRNe+eGeE/8VgElb/9XM4t2BbCqQinA54rSeilusLQQ=; b=myA0I6etJzmHhSHqwV4JDaq1gfdfjMAiM/n4J5+Y7f+mL/67hEGWmfltKAh07o0+1j OEFwD5TltWSZiPLYyxjOTQ0e6J4u+AJbydE+EtKZd+oRK8b+f2ArtHCTxUikz6//ocOc rUWOZLK94dgftSooiwoOIlMQls955BouZksQoOQks4CzMnS7dUFtsvcvj7psHehbz2IE 3RMvsDOcTV4LcmeulV2dLxtGKtvMvcQXV1b47h0JjvIOEH7j8UIhdbZY2RESsfHuzXL6 p+M9UsBoJOEhVJItHY5DCmE+sKqOW5F0isjXOYMpk+GortcWXsDGaxjc0glh7laA4bYp DHcA== MIME-Version: 1.0 X-Received: by 10.152.3.35 with SMTP id 3mr61144801laz.5.1412221570577; Wed, 01 Oct 2014 20:46:10 -0700 (PDT) Received: by 10.152.210.7 with HTTP; Wed, 1 Oct 2014 20:46:10 -0700 (PDT) Received: by 10.152.210.7 with HTTP; Wed, 1 Oct 2014 20:46:10 -0700 (PDT) In-Reply-To: <20141002014914.GA12656@lists.bufferbloat.net> References: <1396119385-16871-1-git-send-email-dave.taht@bufferbloat.net> <1396119385-16871-2-git-send-email-dave.taht@bufferbloat.net> <20140330222952.GA26806@lists.bufferbloat.net> <542C2786.6090704@gmail.com> <20141002014914.GA12656@lists.bufferbloat.net> Date: Thu, 2 Oct 2014 11:46:10 +0800 Message-ID: From: Alpha Sparc To: Dave Taht Content-Type: multipart/alternative; boundary=089e0141a0100f75e805046875d3 X-Mailman-Approved-At: Wed, 08 Oct 2014 19:34:32 -0700 Cc: Weedy , openwrt-devel , cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] [OpenWrt-Devel] [RFC PATCH] packages: Smart Queue Management for AQM Packet Scheduling and Qos from CeroWrt X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Oct 2014 03:46:42 -0000 --089e0141a0100f75e805046875d3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable How good is the throughput on CeroWrt compared to OpenWrt ? On Oct 2, 2014 9:55 AM, "Dave Taht" wrote: > On Wed, Oct 01, 2014 at 12:10:46PM -0400, Weedy wrote: > > On 30/03/14 06:29 PM, Dave Taht wrote: > > > On Sun, Mar 30, 2014 at 02:24:44PM -0400, Weedy wrote: > > >> On Sat, Mar 29, 2014 at 2:56 PM, Dave T=C3=A4ht >wrote: > > >> > > >>> From: Dave Taht > > >>> > > >>> This adds support for the bufferbloat project's "Smart Queue > Management" > > >>> (SQM) system, which improves over openwrt's qos-scripts in the > following > > >>> ways > > >>> > > >>> + Uses HTB with two models for managing traffic > > >>> a simplest one that merely uses fq_codel, and a three tier one > that does > > >>> some basic and tunable packet prioritization. > > >>> > > >>> + Works with ipv6 and ipv4 correctly (unlike qos-scripts) > > >>> + extensive support for fixing ADSL and PPOe framing problems > > >>> + Partial support for key diffserv markings > > >>> + highly tuned fq_codel implementation especially for low bandwidth= s > > >>> + Tested heavily on cable modems and on dsl devices > > >>> > > >>> It is a disimprovement in that: > > >>> > > >>> - There are no built-in tricks for doing l7 classification, > > >>> or other forms of packet inspection. > > >>> > > >>> - We haven't explored hfsc all that much, prefering to rely > > >>> on the predictable behavior of htb + fq_codel for everything > > >>> > > >>> - And there is support for a few qdiscs that are not in the linux > > >>> kernel mainline that remain experimental. > > >>> --- > > >>> net/sqm-scripts/Makefile | 48 +++ > > >>> net/sqm-scripts/files/etc/config/sqm | 11 + > > >>> net/sqm-scripts/files/etc/init.d/sqm | 23 ++ > > >>> net/sqm-scripts/files/usr/lib/sqm/functions.sh | 335 > > >>> ++++++++++++++++++++ > > >>> net/sqm-scripts/files/usr/lib/sqm/run.sh | 67 ++++ > > >>> net/sqm-scripts/files/usr/lib/sqm/simple.qos | 187 > +++++++++++ > > >>> net/sqm-scripts/files/usr/lib/sqm/simple.qos.help | 1 + > > >>> net/sqm-scripts/files/usr/lib/sqm/simplest.qos | 84 +++++ > > >>> .../files/usr/lib/sqm/simplest.qos.help | 1 + > > >>> net/sqm-scripts/files/usr/lib/sqm/stop.sh | 22 ++ > > >>> 10 files changed, 779 insertions(+) > > >>> create mode 100644 net/sqm-scripts/Makefile > > >>> create mode 100644 net/sqm-scripts/files/etc/config/sqm > > >>> create mode 100755 net/sqm-scripts/files/etc/init.d/sqm > > >>> create mode 100644 net/sqm-scripts/files/usr/lib/sqm/functions.sh > > >>> create mode 100755 net/sqm-scripts/files/usr/lib/sqm/run.sh > > >>> create mode 100755 net/sqm-scripts/files/usr/lib/sqm/simple.qos > > >>> create mode 100644 net/sqm-scripts/files/usr/lib/sqm/simple.qos.he= lp > > >>> create mode 100755 net/sqm-scripts/files/usr/lib/sqm/simplest.qos > > >>> create mode 100644 > net/sqm-scripts/files/usr/lib/sqm/simplest.qos.help > > >>> create mode 100755 net/sqm-scripts/files/usr/lib/sqm/stop.sh > > >>> > > >>> diff --git a/net/sqm-scripts/files/etc/config/sqm > > >>> b/net/sqm-scripts/files/etc/config/sqm > > >>> new file mode 100644 > > >>> index 0000000..547d321 > > >>> --- /dev/null > > >>> +++ b/net/sqm-scripts/files/etc/config/sqm > > >>> @@ -0,0 +1,11 @@ > > >>> + > > >>> +config queue 'ge00' > > >>> + option enabled '0' > > >>> + option interface 'ge00' > > >>> + option download '20000' > > >>> + option upload '4000' > > >>> + option qdisc 'fq_codel' > > >>> + option script 'simple.qos' > > >>> + option qdisc_advanced '0' > > >>> + option linklayer 'none' > > >>> + > > >>> > > >> > > >> How hard is this to config from the command line/vim? > > > > > > There are a few more options than this (for DSL compensation, ecn > > > and advanced configuration), the above would work if you changed > > > enabled to '1' and the device from ge00 to your wan device. (not > > > the "wan" firewall rule, presently. ) > > > > > > It does help to have a sane long term and realistic measurement of yo= ur > > > network using something like the rrul test rather than the oft-gamed > speedtest. > > > > > > > http://www.bufferbloat.net/projects/cerowrt/wiki/Setting_up_SQM_for_CeroW= rt_310 > > > > > > You are right, we should fully document all the variables in this fil= e. > > > Until recently they were kind of in flux. > > > > > >> I've never needed or really wanted luci on my box, I just use vim. > Going by > > >> this patch, there is either nothing to config or no examples. I woul= d > think > > >> shipping a roughly equivalent config to what ships in qos-scripts > would be > > >> a good start to get people testing. > > > > > > /etc/init.d/sqm start,stop etc work as expected. > > > > > >> IE: highest priority: small ARP, DNS, and SSH packets > > > > > > The SQM default is local DNS and ntp. fq_codel automagically optimize= s > > > for other sparse flows like arp, ssh, mosh, tcp syn, synack, etc, > > > no need to do that via classification. > > > > > >> normal: HTTP, HTTPS > > > > > > So you want to deprioritize vpn, smtp, rsync, dropbox, http on odd > ports, > > > caching servers, etc, all in favor of the web gods? > > > > > > we just toss all that into the normal (best effort bin) and let > fq_codel > > > sort it out. > > > > > >> bulk: everything else. > > > > > > We do respect the diffserv marking of CS1 (background) and toss it > > > into the background queue. > > > > > >> Side note: Will SQM do bandwidth slicing? Or at least handle "hostil= e" > > >> environments better? I say "hostile" as in roommates or maybe teenag= e > kids. > > >> Multiple people/devices thinking they are entitled to the entire WAN > > >> bandwidth at all times. > > > > > > fq_codel does fair (well, we call it "flow") queuing. > > > > > > https://tools.ietf.org/html/draft-hoeiland-joergensen-aqm-fq-codel-00 > > > > > > And manages the depth of flows via codel. > > > > > > http://tools.ietf.org/html/draft-nichols-tsvwg-codel-02 > > > > > > In other words, it will be fair to all fat flows generated by everyon= e, > > > and slice flows down to the defined quantum and turn them back into > > > packets. > > > > > > The "simplest.qos" model in SQM works remarkably well without trying = to > > > classify anything at all. I encourage people to try merely that and > have > > > their preconceptions altered. > > > > > > The three-tier model (simple.qos), is more like what people think the= y > want, > > > but the default is set to the bare minimum of what worked well in > testing. > > > > > > Example: a lot of flows are marked CS1 that shouldn't be, and starvin= g > > > that queue to like 5% rather than it's current 30% turned out badly. > > > > > > In terms of identifying and "punishing" abusers, well, the only thing > > > that stresses this code out even the slightest is dozens of torrent > flows. > > > > > > Give it a shot. :) > > > > > I feel like this died. > > It didn't die. > > *I* died. > > I'd been on a death march for the last 8 months trying to > get the last bugs out of openwrt/cerowrt, and when the last big one > got fixed (bug 442 in the cerowrt database, multiple other trackers) > > I put out a release of 3.10.50-1 pre BBrc1 and went to sleep. > > When I woke up, about a week ago, everybody had nearly 2 months > uptime, good throughput, and a bunch of minor nits here and there. > > Hooray! > > The prospect of resyncing with BBrcX intimidates me, and I have > had a ton of other things that slid to take care of, so I've > been catching up on those. Sebastian has been taking > care of SQM nits... > > https://github.com/dtaht/ceropackages-3.10/issues/8 > > And Jonathon morton has been pouring it all into > pure C - with an integral bandwidth shaper that we > hope will be faster and more efficient than htb. > > See an early result: > > http://pastebin.com/zz06WhJr > > It takes much of the heavy lifting out of the existing > sqm scripts. > > tc qdisc add dev eth1 root cake bandwidth 80mbit > > > So I don't know where to go. Certainly I'd like to > see the battle hardened sqm scripts (which are more > flexible than the C code above) get more widely used > and in BB. > > openwrt users can do that today by adding the ceropackages repo to their > build system. > or just installing the sqm-scripts and luci-app-sqm. > > or we can clean it up further for openwrt mainline. > > But I haven't seen one core openwrt dev say, yes, we want this mainlined, > here's what you need to fix, so I'm inclined to go back to my cave, get > more sleep, and work on the successor. > _______________________________________________ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel > --089e0141a0100f75e805046875d3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

How good is the throughput on CeroWrt compared to OpenWrt ?<= /p>

On Oct 2, 2014 9:55 AM, "Dave Taht" &l= t;dave.taht@bufferbloat.net> wrote:
On Wed,= Oct 01, 2014 at 12:10:46PM -0400, Weedy wrote:
> On 30/03/14 06:29 PM, Dave Taht wrote:
> > On Sun, Mar 30, 2014 at 02:24:44PM -0400, Weedy wrote:
> >> On Sat, Mar 29, 2014 at 2:56 PM, Dave T=C3=A4ht <dave.taht@bufferbloat.net>wrot= e:
> >>
> >>> From: Dave Taht <dave.taht@bufferbloat.net>
> >>>
> >>> This adds support for the bufferbloat project's "= ;Smart Queue Management"
> >>> (SQM) system, which improves over openwrt's qos-scrip= ts in the following
> >>> ways
> >>>
> >>> + Uses HTB with two models for managing traffic
> >>>=C2=A0 =C2=A0a simplest one that merely uses fq_codel, and= a three tier one that does
> >>>=C2=A0 =C2=A0some basic and tunable packet prioritization.=
> >>>
> >>> + Works with ipv6 and ipv4 correctly (unlike qos-scripts)=
> >>> + extensive support for fixing ADSL and PPOe framing prob= lems
> >>> + Partial support for key diffserv markings
> >>> + highly tuned fq_codel implementation especially for low= bandwidths
> >>> + Tested heavily on cable modems and on dsl devices
> >>>
> >>> It is a disimprovement in that:
> >>>
> >>> - There are no built-in tricks for doing l7 classificatio= n,
> >>> or other forms of packet inspection.
> >>>
> >>> - We haven't explored hfsc all that much, prefering t= o rely
> >>> on the predictable behavior of htb + fq_codel for everyth= ing
> >>>
> >>> - And there is support for a few qdiscs that are not in t= he linux
> >>> kernel mainline that remain experimental.
> >>> ---
> >>>=C2=A0 net/sqm-scripts/Makefile=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2= =A0 =C2=A048 +++
> >>>=C2=A0 net/sqm-scripts/files/etc/config/sqm=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A011 +
> >>>=C2=A0 net/sqm-scripts/files/etc/init.d/sqm=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A023 ++
> >>>=C2=A0 net/sqm-scripts/files/usr/lib/sqm/functions.sh=C2= =A0 =C2=A0 =C2=A0|=C2=A0 335
> >>> ++++++++++++++++++++
> >>>=C2=A0 net/sqm-scripts/files/usr/lib/sqm/run.sh=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2=A067 ++++
> >>>=C2=A0 net/sqm-scripts/files/usr/lib/sqm/simple.qos=C2=A0 = =C2=A0 =C2=A0 =C2=A0|=C2=A0 187 +++++++++++
> >>>=C2=A0 net/sqm-scripts/files/usr/lib/sqm/simple.qos.help= =C2=A0 |=C2=A0 =C2=A0 1 +
> >>>=C2=A0 net/sqm-scripts/files/usr/lib/sqm/simplest.qos=C2= =A0 =C2=A0 =C2=A0|=C2=A0 =C2=A084 +++++
> >>>=C2=A0 .../files/usr/lib/sqm/simplest.qos.help=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0 1 +
> >>>=C2=A0 net/sqm-scripts/files/usr/lib/sqm/stop.sh=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A022 ++
> >>>=C2=A0 10 files changed, 779 insertions(+)
> >>>=C2=A0 create mode 100644 net/sqm-scripts/Makefile
> >>>=C2=A0 create mode 100644 net/sqm-scripts/files/etc/config= /sqm
> >>>=C2=A0 create mode 100755 net/sqm-scripts/files/etc/init.d= /sqm
> >>>=C2=A0 create mode 100644 net/sqm-scripts/files/usr/lib/sq= m/functions.sh
> >>>=C2=A0 create mode 100755 net/sqm-scripts/files/usr/lib/sq= m/run.sh
> >>>=C2=A0 create mode 100755 net/sqm-scripts/files/usr/lib/sq= m/simple.qos
> >>>=C2=A0 create mode 100644 net/sqm-scripts/files/usr/lib/sq= m/simple.qos.help
> >>>=C2=A0 create mode 100755 net/sqm-scripts/files/usr/lib/sq= m/simplest.qos
> >>>=C2=A0 create mode 100644 net/sqm-scripts/files/usr/lib/sq= m/simplest.qos.help
> >>>=C2=A0 create mode 100755 net/sqm-scripts/files/usr/lib/sq= m/stop.sh
> >>>
> >>> diff --git a/net/sqm-scripts/files/etc/config/sqm
> >>> b/net/sqm-scripts/files/etc/config/sqm
> >>> new file mode 100644
> >>> index 0000000..547d321
> >>> --- /dev/null
> >>> +++ b/net/sqm-scripts/files/etc/config/sqm
> >>> @@ -0,0 +1,11 @@
> >>> +
> >>> +config queue 'ge00'
> >>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 option enabled '0' > >>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 option interface 'ge00&#= 39;
> >>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 option download '20000&#= 39;
> >>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 option upload '4000'=
> >>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 option qdisc 'fq_codel&#= 39;
> >>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 option script 'simple.qo= s'
> >>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 option qdisc_advanced '0= '
> >>> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 option linklayer 'none&#= 39;
> >>> +
> >>>
> >>
> >> How hard is this to config from the command line/vim?
> >
> > There are a few more options than this (for DSL compensation, ecn=
> > and advanced configuration), the above would work if you changed<= br> > > enabled to '1' and the device from ge00 to your wan devic= e. (not
> > the "wan" firewall rule, presently. )
> >
> > It does help to have a sane long term and realistic measurement o= f your
> > network using something like the rrul test rather than the oft-ga= med speedtest.
> >
> > http://www.bufferbloat.net/pro= jects/cerowrt/wiki/Setting_up_SQM_for_CeroWrt_310
> >
> > You are right, we should fully document all the variables in this= file.
> > Until recently they were kind of in flux.
> >
> >> I've never needed or really wanted luci on my box, I just= use vim. Going by
> >> this patch, there is either nothing to config or no examples.= I would think
> >> shipping a roughly equivalent config to what ships in qos-scr= ipts would be
> >> a good start to get people testing.
> >
> > /etc/init.d/sqm start,stop etc work as expected.
> >
> >> IE: highest priority: small ARP, DNS, and SSH packets
> >
> > The SQM default is local DNS and ntp. fq_codel automagically opti= mizes
> > for other sparse flows like arp, ssh, mosh, tcp syn, synack, etc,=
> > no need to do that via classification.
> >
> >> normal: HTTP, HTTPS
> >
> > So you want to deprioritize vpn, smtp, rsync, dropbox, http on od= d ports,
> > caching servers, etc, all in favor of the web gods?
> >
> > we just toss all that into the normal (best effort bin) and let f= q_codel
> > sort it out.
> >
> >> bulk: everything else.
> >
> > We do respect the diffserv marking of CS1 (background) and toss i= t
> > into the background queue.
> >
> >> Side note: Will SQM do bandwidth slicing? Or at least handle = "hostile"
> >> environments better? I say "hostile" as in roommate= s or maybe teenage kids.
> >> Multiple people/devices thinking they are entitled to the ent= ire WAN
> >> bandwidth at all times.
> >
> > fq_codel does fair (well, we call it "flow") queuing. > >
> > https://tools.ietf.org/html/draft-hoeila= nd-joergensen-aqm-fq-codel-00
> >
> > And manages the depth of flows via codel.
> >
> > http://tools.ietf.org/html/draft-nichols-tsvwg-codel-0= 2
> >
> > In other words, it will be fair to all fat flows generated by eve= ryone,
> > and slice flows down to the defined quantum and turn them back in= to
> > packets.
> >
> > The "simplest.qos" model in SQM works remarkably well w= ithout trying to
> > classify anything at all. I encourage people to try merely that a= nd have
> > their preconceptions altered.
> >
> > The three-tier model (simple.qos), is more like what people think= they want,
> > but the default is set to the bare minimum of what worked well in= testing.
> >
> > Example: a lot of flows are marked CS1 that shouldn't be, and= starving
> > that queue to like 5% rather than it's current 30% turned out= badly.
> >
> > In terms of identifying and "punishing" abusers, well, = the only thing
> > that stresses this code out even the slightest is dozens of torre= nt flows.
> >
> > Give it a shot. :)
> >
> I feel like this died.

It didn't die.

*I* died.

I'd been on a death march for the last 8 months trying to
get the last bugs out of openwrt/cerowrt, and when the last big one
got fixed (bug 442 in the cerowrt database, multiple other trackers)

I put out a release of 3.10.50-1 pre BBrc1 and went to sleep.

When I woke up, about a week ago, everybody had nearly 2 months
uptime, good throughput, and a bunch of minor nits here and there.

Hooray!

The prospect of resyncing with BBrcX intimidates me, and I have
had a ton of other things that slid to take care of, so I've
been catching up on those. Sebastian has been taking
care of SQM nits...

https://github.com/dtaht/ceropackages-3.10/issues/8

And Jonathon morton has been pouring it all into
pure C - with an integral bandwidth shaper that we
hope will be faster and more efficient than htb.

See an early result:

http://pastebin.= com/zz06WhJr

It takes much of the heavy lifting out of the existing
sqm scripts.

tc qdisc add dev eth1 root cake bandwidth 80mbit


So I don't know where to go. Certainly I'd like to
see the battle hardened sqm scripts (which are more
flexible than the C code above) get more widely used
and in BB.

openwrt users can do that today by adding the ceropackages repo to their bu= ild system.
or just installing the sqm-scripts and luci-app-sqm.

or we can clean it up further for openwrt mainline.

But I haven't seen one core openwrt dev say, yes, we want this mainline= d, here's what you need to fix, so I'm inclined to go back to my ca= ve, get more sleep, and work on the successor.
_______________________________________________
openwrt-devel mailing list
openwrt-devel@lists.open= wrt.org
https://lists.openwrt.org/cgi-bin/mailman/listinfo/open= wrt-devel
--089e0141a0100f75e805046875d3--