From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 4078721F180 for ; Mon, 6 Jan 2014 07:03:37 -0800 (PST) Received: by mail-wg0-f52.google.com with SMTP id x13so15629809wgg.31 for ; Mon, 06 Jan 2014 07:03:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=TT9gc4hGoUVZJjR5YIJCVx3LoZCrikii0N+MC8e6Rfo=; b=lqMoLgJxyq9SxeSCVKVClmxLMVI0FIguCaN2Rr0+GCB41SSzJkHGPnRDTB5G0B/et2 ZfRwBA9/gfv0BPLIguipDEmOuhetsmij3bYSzZUAvl4kQfw73f5c7NWJnI4xqmV80Idm Djc/cyrKlFwSmaJHK1XNvTMMEB5avaGmkf8WwcSqRSclPWEqTnpCYqNxp3nmw0LMc7hX L2vz59X09rcE7cBn8tm5Io+1RcxokDEklUqoSXUUVAWKTZmdrAwicwANb2LKCNqtzHCp XDlmPW8TjfK8czhzO1N6E40ptA59D/wfd20LRCIgb+vtuEDNp652W7kXJa7PE9cbTj+l h2dg== MIME-Version: 1.0 X-Received: by 10.194.90.144 with SMTP id bw16mr71165207wjb.1.1389020614894; Mon, 06 Jan 2014 07:03:34 -0800 (PST) Received: by 10.217.123.69 with HTTP; Mon, 6 Jan 2014 07:03:34 -0800 (PST) Received: by 10.217.123.69 with HTTP; Mon, 6 Jan 2014 07:03:34 -0800 (PST) In-Reply-To: <1A0169F3-2B57-4AFE-916E-66E32E782C50@gmx.de> References: <01558084-B7D8-448A-A4ED-CE36D18AAA97@gmail.com> <52C855B1.1040209@imap.cc> <1A0169F3-2B57-4AFE-916E-66E32E782C50@gmx.de> Date: Mon, 6 Jan 2014 07:03:34 -0800 Message-ID: From: Dave Taht To: Sebastian Moeller Content-Type: multipart/alternative; boundary=047d7bfcf35c5629b304ef4e90d7 Cc: cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] SQM Question #5: Link Layer Adaptation Overheads X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jan 2014 15:03:46 -0000 --047d7bfcf35c5629b304ef4e90d7 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On Jan 6, 2014 5:56 AM, "Sebastian Moeller" wrote: > > Hi Dave, hi List, > > On Jan 6, 2014, at 04:29 , Dave Taht wrote: > > > On Sat, Jan 4, 2014 at 10:40 AM, Fred Stratton wrote: > >> Link Names: > >> > >> For consistency, if ADSL is used as a portmanteau term, them VDSL should be > >> used as the equivalent for VDSL and VDSL2. > >> > >> CeroWRT has to decide whether it is an experimental build, or something that > >> will eventually be used in production, so these decisions can be made > >> consistently. > > > > Well, what I was aiming for was for us to get the sqm scripts and gui > > up to where they were better than the standard openwrt qos scripts and > > then push them up to openwrt to where they could be more widely > > deployed. > > > > Aside from being able to dynamically assign priorities in the gui, we > > are there. Except that nfq_codel is currently getting better results > > than fq_codel at low bandwidths, and I'm tempted to pour all of > > simple.qos into C. > > Since you wore nfq_codel, what is the secret sauce here? 1) It uses a 'tighter' version of Codel than what is currently in Linux. It doesn't work as well on longer rtts but holds down queue lengths at shorter rtts better and responds quicker than normal codel. This is a slightly more expensive version of codel too in that it uses two invsqrt via newtons method to get more accurate results. 2) it rotates the flow list more like how sfq does yielding better mixing which leads to higher survival rates for sparse flows and more balance across all flows. (This is a one line change to fq-codel) At higher bandwidths (say, 50mbit) being more drr like (fqcodel) actually tends to do better than sfqlike as bunching up some packet deliveries makes hosts respond quicker. 3) common to all the codels in this was elimination of the maxpacket check which mildly increases drop probability. Compared to the orders of magnitude we already get from fq codel the sum benefit of these fixes is in the very small percentage points. Without an extensive testing and simulation campaign I've been reluctant to attempt pushing them upstream. What I have mostly thought about instead was bundling up simple.QoS into c (call it cake or broadbandeq), Using these mods, adding in fixes for things that are hard now, like full diffserv support and something lighter than htb. But enotime, funding etc. Until 3 hit seeing benefit from nfqcodel was even harder to see, and I'd like to drop out 3 and revisit the data to see if the improvement is a chimera or not. > > > > As for cero's future - certainly since all the snowden revelations > > I've been going around saying that "friends don't let friends run > > factory firmware". I would like a stable build of sqm and cerowrt to > > emerge, and to then go off and work on improving wifi. Regrettably > > what seems to be happening is more backwards than forwards on the > > former, and ramping up on the ath9k and ath10k is taking more time > > than I'd like, and it seems likely I'll be working on those primarily > > on another platform and only eventually pushing the results out to > > cero, mainline kernel > > > > So it's still at the "keep plugging away" point for sqm, ipv6, cero in > > general, with the stable release always just out of sight. > > > > Tackling the ipv6 problem is next on my agenda on cero, and getting a > > test suite going is next on my day job. > > Any further hints on the nature of your day job possible :) > > > > >> I concur with your ADSL setup suggestion as default. I have been running the > >> Sebastian Moeller ping script overnight to calculate ADSL overhead for the > >> last several days. After several hours of curve fitting using Octave, an > >> overhead result is displayed. This novel approach works well. > > > > It would be nice to get to where we could autoconfigure a router using > > tools like these with no human intervention. This includes bandwidth > > estimation. > > I fully agree that it would be nice. Also it ail e hard unless we take control over the actual bottleneck link=85 With DSL connections, the D= SL modem knows a lot about the link properties, if the modem would be onboard we could programmatically as about bandwidth and encapsulation; for the more typical case with an independent modem or even modem router we have no clear path accessing that information. With cable I have even less hope, as will never get the modem into the router (then again DOCSIS 3.1's typically faster speeds and mandatory pie in the modem might make the situation less dire than on the DSL side). > So for ATM based links, I think we can estimate a number of relevant parameters about encapsulation and an aggregate up- and down-bandwidth measurement (which alas is not too helpful unless we know the degree of asymmetry or one of the bandwidth, in both cases we are likely to know everything a priori anyway). But the current prototype code I have is really slow (3 hours measurement time with otherwise quiet home network; ~20 processing) and memory demanding (the ascii ping traces take up ~260MB) so will not be able to run on the router (also the current implementation in matlab/octave does not lend itself well for an embedded platform=85) > > Best Regards > Sebastian > > > > > >> The overhead for the particular setup I use was 40 for PPPoE, and 10 for > >> PPPoA. > >> > >> The default you suggest is a suitable starting point, I suggest. > >> > >> > >> On 04/01/14 18:16, Rich Brown wrote: > >>> > >>> QUESTION #5: I still don=92t have any great answers for the Link Laye= r > >>> Adaptation overhead descriptions and recommendations. In an earlier message, > >>> (see > >>> https://lists.bufferbloat.net/pipermail/cerowrt-devel/2013-December/001914.= html > >>> and following messages), Fred Stratton described the overheads carried by > >>> various options, and Sebastian Moeller also gave some useful advice. > >>> > >>> After looking at the options, I despair of giving people a clear > >>> recommendation that would be optimal for their equipment. Consequently, I > >>> believe the best we can do is come up with =93good enough=94 recommendations > >>> that are not wrong, and still give decent performance. > >>> > >>> In this spirit, I have changed Draft #3 of the =93Setting up SQM=94 p= age to > >>> reflect this understanding. See > >>> http://www.bufferbloat.net/projects/cerowrt/wiki/Setting_up_AQM_for_CeroWrt= _310 > >>> > >>> ADSL/ATM link: Choose =93ADSL/ATM", and set Per Packet Overhea= d to > >>> 40 > >>> VDSL2 link: Choose =93VDSL=94, and set Per Packet Overhead to = 8 > >>> Other kind of link (e.g., Cable, Fiber, Ethernet, other not > >>> listed): Choose =93None (default)=94, and set Per Packet Overhead to = 0 > >>> > >>> NB: I have changed the first menu choice to =93ADSL/ATM=94 and the se= cond to > >>> =93VDSL=94 in the description. I would ask that we change to GUI to reflect > >>> those names as well. This makes it far easier/less confusing to talk about > >>> the options. > >>> > >>> As always, I welcome help in setting out clear recommendations that work > >>> well for the vast majority of people who try CeroWrt. Thanks. > >>> > >>> Rich > >>> _______________________________________________ > >>> Cerowrt-devel mailing list > >>> Cerowrt-devel@lists.bufferbloat.net > >>> https://lists.bufferbloat.net/listinfo/cerowrt-devel > >> > >> > >> _______________________________________________ > >> Cerowrt-devel mailing list > >> Cerowrt-devel@lists.bufferbloat.net > >> https://lists.bufferbloat.net/listinfo/cerowrt-devel > > > > > > > > -- > > Dave T=E4ht > > > > Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html > > _______________________________________________ > > Cerowrt-devel mailing list > > Cerowrt-devel@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/cerowrt-devel > --047d7bfcf35c5629b304ef4e90d7 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable


On Jan 6, 2014 5:56 AM, "Sebastian Moeller" <moeller0@gmx.de> wrote:
>
> Hi Dave, hi List,
>
> On Jan 6, 2014, at 04:29 , Dave Taht <dave.taht@gmail.com> wrote:
>
> > On Sat, Jan 4, 2014 at 10:40 AM, Fred Stratton <fredstratton@i= map.cc> wrote:
> >> Link Names:
> >>
> >> For consistency, if ADSL is used as a portmanteau term, them = VDSL should be
> >> used as the equivalent for VDSL and VDSL2.
> >>
> >> CeroWRT has to decide whether it is an experimental build, or= something that
> >> will eventually be used in production, so these decisions can= be made
> >> consistently.
> >
> > Well, what I was aiming for was for us to get the sqm scripts and= gui
> > up to where they were better than the standard openwrt qos script= s and
> > then push them up to openwrt to where they could be more widely > > deployed.
> >
> > Aside from being able to dynamically assign priorities in the gui= , we
> > are there. =A0Except that nfq_codel is currently getting better r= esults
> > than fq_codel at low bandwidths, and I'm tempted to pour all = of
> > simple.qos into C.
>
> =A0 =A0 =A0 =A0 Since you wore nfq_codel, what is the secret sauce her= e?

1) It uses a 'tighter' version of Codel than what is= currently in Linux. It doesn't work as well on longer rtts but holds d= own queue lengths at shorter rtts better and responds quicker than normal c= odel.

This is a slightly more expensive version of codel too in th= at it uses two invsqrt via newtons method to get more accurate results.

2) it rotates the flow list more like how sfq does yielding = better mixing which leads to higher survival rates for sparse flows and mor= e balance across all flows.

(This is a one line change to fq-codel)

At higher bandwidths (say, 50mbit) being more drr like (fqco= del) actually tends to do better than sfqlike as bunching up some packet de= liveries makes hosts respond quicker.

3) common to all the codels in this was elimination of the m= axpacket check which mildly increases drop probability.

Compared to the orders of magnitude we already get from fq c= odel the sum benefit of these fixes is in the very small percentage points.= Without an extensive testing and simulation campaign I've been relucta= nt to attempt pushing them upstream. What I have mostly thought about inste= ad was bundling up simple.QoS into c (call it cake or broadbandeq),
Using these mods, adding in fixes for things that are hard now, like full d= iffserv support and something lighter than htb.

But enotime, funding etc. Until 3 hit seeing benefit from nf= qcodel was even harder to see, and I'd like to drop out 3 and revisit t= he data to see if the improvement is a chimera or not.

> >
> > As for cero's future - certainly since all the snowden revela= tions
> > I've been going around saying that "friends don't le= t friends run
> > factory firmware". I would like a stable build of sqm and ce= rowrt to
> > emerge, and to then go off and work on improving wifi. Regrettabl= y
> > what seems to be happening is more backwards than forwards on the=
> > former, and ramping up on the ath9k and ath10k is taking more tim= e
> > than I'd like, and it seems likely I'll be working on tho= se primarily
> > on another platform and only eventually pushing the results out t= o
> > cero, mainline kernel
> >
> > So it's still at the "keep plugging away" point for= sqm, ipv6, cero in
> > general, with the stable release always just out of sight.
> >
> > Tackling the ipv6 problem is next on my agenda on cero, and getti= ng a
> > test suite going is next on my day job.
>
> =A0 =A0 =A0 =A0 Any further hints on the nature of your day job possib= le :)
>
> >
> >> I concur with your ADSL setup suggestion as default. I have b= een running the
> >> Sebastian Moeller ping script overnight to calculate ADSL ove= rhead for the
> >> last several days. After several hours of curve fitting using= Octave, an
> >> overhead result is displayed. This novel approach works well.=
> >
> > It would be nice to get to where we could autoconfigure a router = using
> > tools like these with no human intervention. This includes bandwi= dth
> > estimation.
>
> =A0 =A0 =A0 =A0 I fully agree that it would be nice. Also it ail e har= d unless we take control over the actual bottleneck link=85 With DSL connec= tions, the DSL modem knows a lot about the link properties, if the modem wo= uld be onboard we could programmatically as about bandwidth and encapsulati= on; for the more typical case with an independent modem or even modem route= r we have no clear path accessing that information. With cable I have even = less hope, as will never get the modem into the router (then again DOCSIS 3= .1's typically faster speeds and mandatory pie in the modem might make = the situation less dire than on the DSL side).
> =A0 =A0 =A0 =A0 So for ATM based links, I think we can estimate a numb= er of relevant parameters about encapsulation and an aggregate up- and down= -bandwidth measurement (which alas is not too helpful unless we know the de= gree of asymmetry or one of the bandwidth, in both cases we are likely to k= now everything a priori anyway). But the current prototype code I have is r= eally slow (3 hours measurement time with otherwise quiet home network; ~20= processing) and memory demanding (the ascii ping traces take up ~260MB) so= will not be able to run on the router (also the current implementation in = matlab/octave does not lend itself well for an embedded platform=85)
>
> Best Regards
> =A0 =A0 =A0 =A0 Sebastian
>
>
> >
> >> The overhead for the particular setup I use was 40 for PPPoE,= and 10 for
> >> PPPoA.
> >>
> >> The default you suggest is a suitable starting point, I sugge= st.
> >>
> >>
> >> On 04/01/14 18:16, Rich Brown wrote:
> >>>
> >>> QUESTION #5: I still don=92t have any great answers for t= he Link Layer
> >>> Adaptation overhead descriptions and recommendations. In = an earlier message,
> >>> (see
> >>> https://lists.bufferbloat.net/pipermail/= cerowrt-devel/2013-December/001914.html
> >>> and following messages), Fred Stratton described the over= heads carried by
> >>> various options, and Sebastian Moeller also gave some use= ful advice.
> >>>
> >>> After looking at the options, I despair of giving people = a clear
> >>> recommendation that would be optimal for their equipment.= Consequently, I
> >>> believe the best we can do is come up with =93good enough= =94 recommendations
> >>> that are not wrong, and still give decent performance. > >>>
> >>> In this spirit, I have changed Draft #3 of the =93Setting= up SQM=94 page to
> >>> reflect this understanding. See
> >>> http://www.bufferbloat.net/projects/cero= wrt/wiki/Setting_up_AQM_for_CeroWrt_310
> >>>
> >>> =A0 =A0 =A0 =A0ADSL/ATM link: Choose =93ADSL/ATM", a= nd set Per Packet Overhead to
> >>> 40
> >>> =A0 =A0 =A0 =A0VDSL2 link: Choose =93VDSL=94, and set Per= Packet Overhead to 8
> >>> =A0 =A0 =A0 =A0Other kind of link (e.g., Cable, Fiber, Et= hernet, other not
> >>> listed): Choose =93None (default)=94, and set Per Packet = Overhead to 0
> >>>
> >>> NB: I have changed the first menu choice to =93ADSL/ATM= =94 and the second to
> >>> =93VDSL=94 in the description. I would ask that we change= to GUI to reflect
> >>> those names as well. This makes it far easier/less confus= ing to talk about
> >>> the options.
> >>>
> >>> As always, I welcome help in setting out clear recommenda= tions that work
> >>> well for the vast majority of people who try CeroWrt. Tha= nks.
> >>>
> >>> Rich
> >>> _______________________________________________
> >>> Cerowrt-devel mailing list
> >>> Ce= rowrt-devel@lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> >>
> >>
> >> _______________________________________________
> >> Cerowrt-devel mailing list
> >> Cerowr= t-devel@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> >
> >
> >
> > --
> > Dave T=E4ht
> >
> > Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-de= vel@lists.bufferbloat.net
> > = https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

--047d7bfcf35c5629b304ef4e90d7--