From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 9A55421F182; Wed, 8 May 2013 15:25:53 -0700 (PDT) Received: by mail-ie0-f170.google.com with SMTP id aq17so4326945iec.1 for ; Wed, 08 May 2013 15:25:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=j4i5fL+sqfUeqyvBepdFPKJ0AAWJSNBhgxJty4AP0Dg=; b=xEp3tB4Rb2Z8qxPrNXGP9bXBNoy7CgA+UC8pZzYZyCkqB04rBgCK7ncliGQC+aovyU AUGNZo/sskWHO553rZqW3D+L+e/3vDzI+IzFsykUWQJghUB+HPRGyBrSdPUlHxo1476c 6uTfcib4M+G2y2wCabBLLMn+eB6sWGiP+Heq+SBIYU4P3x5+ULETqFPCbuOdvQvWG/FF xL8mtW+q/ZejsRHDZhCQIg7UvIwkVWYAFcexdfIxdQdfCM4m8SDoQuj7znZoLT1eVbSd 5sbpPP4oVrkFK2PvfW8HRScDugrHbm6nVrlprqKeIB7ciCgLgZJdqLLjOHcu5mrOq6cl BhUg== MIME-Version: 1.0 X-Received: by 10.50.236.100 with SMTP id ut4mr6466879igc.86.1368051952889; Wed, 08 May 2013 15:25:52 -0700 (PDT) Received: by 10.64.7.51 with HTTP; Wed, 8 May 2013 15:25:52 -0700 (PDT) In-Reply-To: <1367958622.13473.10.camel@edumazet-glaptop> References: <19404.1367933465@sandelman.ca> <1367958622.13473.10.camel@edumazet-glaptop> Date: Wed, 8 May 2013 15:25:52 -0700 Message-ID: From: Dave Taht To: Eric Dumazet Content-Type: multipart/alternative; boundary=14dae9399de3afc04204dc3c6a2c Cc: Wes Felter , codel@lists.bufferbloat.net, cerowrt-devel@lists.bufferbloat.net, bloat@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] [Bloat] Latest codel, fq_codel, and pie sim study from cablelabs now available X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 May 2013 22:25:54 -0000 --14dae9399de3afc04204dc3c6a2c Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Eric, you supplied the prio scheduler example as an example of how it shouldn't work, right? On Tue, May 7, 2013 at 1:30 PM, Eric Dumazet wrote= : > On Tue, 2013-05-07 at 14:56 -0500, Wes Felter wrote: > > > Is it time for prio_fq_codel or wfq_codel? That's what comes to mind > > when seeing the BitTorrent vs. VoIP results. > > 0) If evenly distributed, packet loss as high as reported here wouldn't actually affect VOIP much. I'd really like to have a better grip on the packet size, timing and other behavior of new codecs, notably VP8. Anyone? One thing that fq_codel enables is silence suppression actually can work, so there is no need to send VOIP as CBR, and most speech is silence. 1) All of the torrent clients I've looked at come, by default, with an upload rate limiter set to a very low value in the 50-150k range, so that the upload component of torrent problem has already been *solved via market demand*. I would certainly like those running windows and mac to let us know what the defaults are on a variety of torrent these days... If everybody here could just download one randomly http://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clients and try it on distributing some popular, but legal data, http://www.gutenberg.org/wiki/Gutenberg:The_CD_and_DVD_Project through their existing gateways and/or through cerowrt/openwrt? we'd be able to update here, and wikipedia with some info on that. A lot of people have been running scared of torrent for a long time, and my limited experience with current clients indicates strongly that it's not a huge problem anymore. I tend to see torrent packets marked CS1 too, which is either an artefact of all the DPI people ended up doing on it, or the clients co-operating.... Also LEDBAT and uTP seem to behave very differently, and I have some doubts about the quality of the TCP-ledbat model used in this study. 2) I have fiddled with various forms of wfq (qfq, some of my own stuff, prioritization, and currently a toy model with a very limited number of queues with a very well defined set of prioritizations) 3) It seems possible to make fq_codel's drop strategy a little more aggressive when there are high numbers of flows in multiple ways. The horizontal standing queue problem in fq_codel is bothersome and has been discussed on this list several times since day one. It comes from the original codel single queue "maxpacket" concept where dropping from a queue with a single packet in it is a bad idea - among other things it might stomp on packets in RTO or FIN, etc, etc. (note that maxpacket itself is an artifact of the ns2 code - it may not be necessary to keep a MTU's worth of packets around, merely a single packet might be fine) I long ago created a version of codel (codel3.h) that ripped out all single queue assumptions from the codel code... in preparation for finding something that worked better when there were multiple queues in play. I have a "special" version of fq_codel (efq_codel) in cerowrt where I have fiddled with various tweaks and tests and options. I've been pretty good about not documenting this until now, because everything I've tried to date was not much of an improvement, (nor did I have the robust set of test and edge cases I do now) My most recent idea for increasing fq_codel drop behavior under load involves relaxing the exit-the-drop-scheduler-at-maxpacket strategy when it would otherwise drop, and the measured delay in that fq_codel queue exceeds the current interval * X, maybe with an added qualification of packet size > 1000. It's hacky but I've repeatedly proven to myself that the most trivial of changes like this can have enormous side effects... and this is targetted specifically at the behavior of torrents, as best as I understand it. > Sure ! > > eth=3Deth0 > tc qdisc del dev $eth root 2>/dev/null > tc -batch << EOF > qdisc add dev $eth root handle 1: prio bands 3 > qdisc add dev $eth parent 1:1 handle 11: fq_codel > qdisc add dev $eth parent 1:2 handle 12: fq_codel > qdisc add dev $eth parent 1:3 handle 13: fq_codel > EOF > > Heh. I am hoping you are providing this as a negative proof!? as the strict prioritization of this particular linux scheduler means that a single full rate TCP flow in class 1:1 will completely starve classes 1:2 and 1:3. ..snip snip.. static struct sk_buff *prio_dequeue(struct Qdisc *sch) { struct prio_sched_data *q =3D qdisc_priv(sch); int prio; for (prio =3D 0; prio < q->bands; prio++) { struct Qdisc *qdisc =3D q->queues[prio]; struct sk_buff *skb =3D qdisc_dequeue_peeked(qdisc); if (skb) { qdisc_bstats_update(sch, skb); sch->q.qlen--; return skb; } } return NULL; } Some level of fairness between service classes is needed too. My most common setting for the "cake" shaper has been 20% minimum for the background traffic, for example. I am unsure if codel is really the right thing for the highest priority qdisc, as everything > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html --14dae9399de3afc04204dc3c6a2c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Eric, you supplied the prio scheduler example as an example of how it s= houldn't work, right?

On Tue, May 7,= 2013 at 1:30 PM, Eric Dumazet <eric.dumazet@gmail.com>= wrote:
On Tue, 2013-05-07 at 14:5= 6 -0500, Wes Felter wrote:

> Is it time for prio_fq_codel or wfq_codel? That's what comes to mi= nd
> when seeing the BitTorrent vs. VoIP results.


0) If evenly distributed, packet loss as hi= gh as reported here wouldn't actually affect VOIP much.

I'd= really like to have a better grip on the packet size, timing and other beh= avior of new codecs, notably VP8. Anyone?

One thing that fq_codel enables is silence suppression actually can wor= k, so there is no need to send VOIP as CBR, and most speech is silence.
=
1)=A0 All of the torrent clients I've looked at come, by default, w= ith an upload rate limiter set to a very low value in the 50-150k range, so= that the upload component of torrent problem has already been *solved via = market demand*.

I would certainly like those running windows and mac to let us know wha= t the defaults are on a variety of torrent these days... If everybody here = could just download one randomly

http://en.wikipedia.org/wiki/Compar= ison_of_BitTorrent_clients

and try it on distributing some popular, but legal data,

http://= www.gutenberg.org/wiki/Gutenberg:The_CD_and_DVD_Project

through= their existing gateways and/or through cerowrt/openwrt?

we'd be able to update here, and wikipedia with some info on that. =

A lot of people have been running scared of torrent for a long time= , and my limited experience with current clients indicates strongly that it= 's not a huge problem anymore. I tend to see torrent packets marked CS1= too, which is either an artefact of all the DPI people ended up doing on i= t, or the clients co-operating....

Also LEDBAT and uTP seem to behave very differently, and I have some do= ubts about the quality of the TCP-ledbat model used in this study.

2= ) I have fiddled with various forms of wfq (qfq, some of my own stuff, prio= ritization, and currently a toy model with a very limited number of queues = with a very well defined set of prioritizations)

3) It seems possible to make fq_codel's drop strategy a little more= aggressive when there are high numbers of flows in multiple ways.

= The horizontal standing queue problem in fq_codel is bothersome and has bee= n discussed on this list several times since day one. It comes from the ori= ginal codel single queue "maxpacket" concept where dropping from = a queue with a single packet in it is a bad idea - among other things it mi= ght stomp on packets in RTO or FIN, etc, etc. (note that maxpacket itself i= s an artifact of the ns2 code - it may not be necessary to keep a MTU's= worth of packets around, merely a single packet might be fine)

I long ago created a version of codel (codel3.h) that ripped out all si= ngle queue assumptions from the codel code... in preparation for finding so= mething that worked better when there were multiple queues in play.

I have a "special" version of fq_codel (efq_codel) in cerowrt= where I have fiddled with various tweaks and tests and options. I've b= een pretty good about not documenting this until now, because everything I&= #39;ve tried to date was not much of an improvement, (nor did I have the ro= bust set of test and edge cases I do now)

My most recent idea for increasing fq_codel drop behavior under load in= volves relaxing the exit-the-drop-scheduler-at-maxpacket strategy when it w= ould otherwise drop, and the measured delay in that fq_codel queue exceeds = the current interval * X, maybe with an added qualification of packet size = > 1000.

It's hacky but I've repeatedly proven to myself that the most t= rivial of changes like this can have enormous side effects... and this is t= argetted specifically at the behavior of torrents, as best as I understand = it.




=A0
Sure !

eth=3Deth0
tc qdisc del dev $eth root 2>/dev/null
tc -batch << EOF
qdisc add dev $eth root handle 1: prio bands 3
qdisc add dev $eth parent 1:1 handle 11: fq_codel
qdisc add dev $eth parent 1:2 handle 12: fq_codel
qdisc add dev $eth parent 1:3 handle 13: fq_codel
EOF

<= br>Heh. I am hoping you are providing this as a negative proof!? as the str= ict prioritization of this particular linux scheduler means that a single f= ull rate TCP flow in class 1:1 will completely starve classes 1:2 and 1:3.<= br>

..snip snip..

static struct sk_buff *prio_dequeue(struct Qdi= sc *sch)
{
=A0=A0=A0=A0=A0=A0=A0 struct prio_sched_data *q =3D qdisc_= priv(sch);
=A0=A0=A0=A0=A0=A0=A0 int prio;

=A0=A0=A0=A0=A0=A0=A0 = for (prio =3D 0; prio < q->bands; prio++) {
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 struct Qdisc *qdisc =3D q->= ;queues[prio];
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 struct sk_b= uff *skb =3D qdisc_dequeue_peeked(qdisc);
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 if (skb) {
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 qdisc_bstats_update(sch, skb);
=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sch->q.qlen--; =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 retur= n skb;
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 }
=A0=A0=A0=A0= =A0=A0=A0 }
=A0=A0=A0=A0=A0=A0=A0 return NULL;

}



S= ome level of fairness between service classes is needed too. My=20 most common setting for the "cake" shaper has been 20% minimum fo= r the=20 background traffic, for example. I am unsure if codel is really the=20 right thing for the highest priority qdisc, as everything

_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net<= /a>
= https://lists.bufferbloat.net/listinfo/bloat



--
Dave T=E4ht=

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/= subscribe.html=20 --14dae9399de3afc04204dc3c6a2c--