From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yk0-x232.google.com (mail-yk0-x232.google.com [IPv6:2607:f8b0:4002:c07::232]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 7DC4321F84D for ; Sun, 2 Aug 2015 12:04:42 -0700 (PDT) Received: by ykdu72 with SMTP id u72so95293933ykd.2 for ; Sun, 02 Aug 2015 12:04:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=TynsOCWlTt3G72AESvk4wj1GPBbnNuASGjGHpsWv6io=; b=kWEIU5BavUzs1CNx1g0hH4+1QEUcBWT/g6nYmKTerS0D0KGQn1cd7PHR66t7j4nFl6 0EZwNCeoIPcBsNRcRvhAkkqnH+BmGXfAcixUo/qAHYDlE0yBn7HhPcuVQTDM1H6AypR4 kp92djO9GvvicjqrSxxxZzWkn0WWPg061XALDKEy26yDl17yFR4CrteSOm+uU2bnOE6d dV1O6nq7xwbN3FFgLIF2fYcddwcNGQnSU4bnnEwvBoHSknZ7LX/sBxgNaVn/JnUyTvlI KQhhm4KNGeZpzx4hy05rk+gawbqzZwK81S+E+UgEkN+9mFg9Sb9qYZVc46GPd2rCs3aX UydQ== MIME-Version: 1.0 X-Received: by 10.170.135.74 with SMTP id b71mr15364577ykc.16.1438542281849; Sun, 02 Aug 2015 12:04:41 -0700 (PDT) Received: by 10.129.148.194 with HTTP; Sun, 2 Aug 2015 12:04:41 -0700 (PDT) In-Reply-To: <55B54BAE.5000002@gmail.com> References: <1437941360960.ed6ad09f@Nodemailer> <55B54BAE.5000002@gmail.com> Date: Sun, 2 Aug 2015 14:04:41 -0500 Message-ID: From: Benjamin Cronce To: Alan Jenkins Content-Type: multipart/alternative; boundary=001a11390b0cb4421b051c58b9c0 Cc: cake@lists.bufferbloat.net Subject: Re: [Cake] How to test Cake on TP-Link WDR3600 X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Aug 2015 19:05:11 -0000 --001a11390b0cb4421b051c58b9c0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Sun, Jul 26, 2015 at 4:05 PM, Alan Jenkins < alan.christopher.jenkins@gmail.com> wrote: > On 26/07/15 21:09, Alec Robertson wrote: > >> I=E2=80=99ve just updated to the newest trunk release of OpenWRT Chaos C= almer >> (fresh install) and the SQM QOS from OPKG interestingly does include Cak= e >> as a qdisc but neither layer_cake.qos nor piece_of_cake.qos are availabl= e >> as setup scripts. >> >> I=E2=80=99m still trying out Cake so I=E2=80=99ll be back soon with some= feedback. >> >> > You should find the cake option there does nothing? > > It'll only work if you have the "kmod-sched-cake" package providing > /lib/modules/*/sch_cake.ko. It's only in Dave's recent experimental buil= ds. > > fq_codel is the more supported option and serves the same functions. If > you can notice any difference yet, I think we'd love to hear about it. > Currently I believe the noticeable differences are > > 1. if your router has TCP offloads enabled, cake undoes ("peels") it some > to improve latency. (Getting this past review for mainline Linux sounds > increasingly "interesting"). > 2. for networks with many flows, cake works much harder to avoid "hash > collision" (entirely?), so every flow gets a fair share. fq_codel default= s > to 1000 hash buckets (but collision probability will increase well before > that point, see "birthday paradox"). > I was wondering about this. I'm going off of memory, but I read a paper a while back that said they tested link speeds from 500Mb/s to 2.5Gb/s and they saw these same characteristics when sending over 10,000 flows over the congested link. 1) Never more than 200 flows of data in the queue at any given time 2) Never more than 30 flows with more than one packet in the queue at a tim= e The birthday attack of all 200 flows into 1000 buckets is pretty bad, but most of those flows are not greedy at any given moment, it's mostly those 30 you need to worry about. The paper I was reading was talking about 10s of thousands of flows, so I assume there are many greedy TCP flows, but only 30 have more than one packet in the queue at a time. Assuming this is true, I wonder what implications this has and what Cake typically sees. Of course 500Mb is much faster than most consumer connections, but they stated they saw no difference in queuing even with a large difference in bandwidth. Because these were not consumer connections, I assume buffers were properly sized even if FIFO. Again, going off of memory, I could have gotten a few things out of context, but it seemed fairly strait forward. > > 1) seems a real concern for some new routers. If you are affected you > could add a boot script using ethtool. > > The idea is it's not optimal to disable offloads universally... maybe if > you're sharing a usb drive from the router as well or something. Having > cake handle it works as a great default configuration. (I just suspect > Linux devs would ask why the feature can't be enabled on other packet > schedulers, e.g. by using a stackable peeler qdisc). > > > Alan > --001a11390b0cb4421b051c58b9c0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Sun, Jul 26, 2015 at 4:05 PM, Alan Jenkins <alan.c= hristopher.jenkins@gmail.com> wrote:
On 26/07/15 21:09, Alec Robertson wrote:
I=E2=80=99ve just updated to the newest trunk release of OpenWRT Chaos Calm= er (fresh install) and the SQM QOS from OPKG interestingly does include Cak= e as a qdisc but neither layer_cake.qos nor piece_of_cake.qos are available= as setup scripts.

I=E2=80=99m still trying out Cake so I=E2=80=99ll be back soon with some fe= edback.


 You should find the cake option there does nothing?

It'll only work if you have the "kmod-sched-cake" package pro= viding /lib/modules/*/sch_cake.ko.=C2=A0 It's only in Dave's recent= experimental builds.

fq_codel is the more supported option and serves the same functions.=C2=A0 = If you can notice any difference yet, I think we'd love to hear about i= t.=C2=A0 Currently I believe the noticeable differences are

1. if your router has TCP offloads enabled, cake undoes ("peels")= it some to improve latency.=C2=A0 (Getting this past review for mainline L= inux sounds increasingly "interesting").
2. for networks with many flows, cake works much harder to avoid "hash= collision" (entirely?), so every flow gets a fair share. fq_codel def= aults to 1000 hash buckets (but collision probability will increase well be= fore that point, see "birthday paradox").
I was wondering about this. I'm going off of memory, but I= read a paper a while back that said they tested link speeds from 500Mb/s t= o 2.5Gb/s and they saw these same characteristics when sending over 10,000 = flows over the congested link.

1) Never more than = 200 flows of data in the queue at any given time
2) Never more th= an 30 flows with more than one packet in the queue at a time

=
The birthday attack of all 200 flows into 1000 buckets is pretty= bad, but most of those flows are not greedy at any given moment, it's = mostly those 30 you need to worry about. The paper I was reading was talkin= g about 10s of thousands of flows, so I assume there are many greedy TCP fl= ows, but only 30 have more than one packet in the queue at a time. Assuming= this is true, I wonder what implications this has and what Cake typically = sees. Of course 500Mb is much faster than most consumer connections, but th= ey stated they saw no difference in queuing even with a large difference in= bandwidth. Because these were not consumer connections, I assume buffers w= ere properly sized even if FIFO.

Again, going off = of memory, I could have gotten a few things out of context, but it seemed f= airly strait forward.
=C2=A0

1) seems a real concern for some new routers.=C2=A0 If you are affected you= could add a boot script using ethtool.

The idea is it's not optimal to disable offloads universally... maybe i= f you're sharing a usb drive from the router as well or something.=C2= =A0 Having cake handle it works as a great default configuration.=C2=A0 (I = just suspect Linux devs would ask why the feature can't be enabled on o= ther packet schedulers, e.g. by using a stackable peeler qdisc).


Alan

--001a11390b0cb4421b051c58b9c0--