From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id A1ABF21F245 for ; Sun, 25 May 2014 12:33:42 -0700 (PDT) Received: by mail-wi0-f177.google.com with SMTP id f8so3280507wiw.10 for ; Sun, 25 May 2014 12:33:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=yGLDZUoW9DJPIJ297oDvbNIEcaMVbVHsS6EMvPaMn3Q=; b=m1fKHvROgECAyQb2FCEp6cdfFE1hVnE3WdScn43l0n2Ez6wxKxeNpWzvc5KEvCD1rB SC9yXZdyPRW+flGenIw8vfPALMKCR1fNZvsbu1dmvGdbXHe+5ux1AgGs9Cyz1f8h939e HTmIlIG0zSjhJ5WXp2w2MvkqsEa21qaSQB5vgdLY/EA9uIEVFrVrb+f4tIOV98enXp1u BLRN6DEuv+UdBdMOfUbrWJefS493YcPEEG3SeHRtCoAvxhjAcFeWs27MA4XIekpxNM2H QxbForQOgVqMnuf+kixzbvWTYCwInEDAkNQ1fHvM6SGkvp1BGn0FqdmUuBu74XcRu5eZ dgXg== MIME-Version: 1.0 X-Received: by 10.194.6.106 with SMTP id z10mr22635903wjz.1.1401046420350; Sun, 25 May 2014 12:33:40 -0700 (PDT) Received: by 10.216.207.82 with HTTP; Sun, 25 May 2014 12:33:40 -0700 (PDT) In-Reply-To: <360688D7-8810-42F2-9FB1-BC0CD197993B@gmx.de> References: <360688D7-8810-42F2-9FB1-BC0CD197993B@gmx.de> Date: Sun, 25 May 2014 12:33:40 -0700 Message-ID: From: Dave Taht To: Sebastian Moeller Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "cerowrt-devel@lists.bufferbloat.net" Subject: Re: [Cerowrt-devel] Ubiquiti QOS X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 May 2014 19:33:43 -0000 On Sun, May 25, 2014 at 11:39 AM, Sebastian Moeller wrote= : > Hi Dane, > > > On May 25, 2014, at 08:17 , Dane Medic wrote: > >> Is it true that devices with less than 64 MB can't handle QOS? -> https:= //lists.chambana.net/pipermail/commotion-dev/2014-May/001816.html > > I think this means that the commotion developers think that 64MB = are required. A dev thinks so. It aint true. (I have a deployed mesh network of mostly nano and picostations and they never crash due to being out of memory. Sadly, they do crash for other reasons) As you get below 64MB some compromises are needed, and as you try to push more bits compromises are needed, and as you add more interfaces and queues compromises are needed. If you are using up all your memory running some application or another, instead of having it free for packets, you are generally in more trouble than you want to be. So the starting factor is how much free ram you have in normal operation with your applications running. Start with that as a baseline. (it isn't strictly true either, you can discard (not swap out) a great deal of program text and still have your applications run well) You typically want something like sqm on your gateway ethernet interface. If you have seriously limited free ram , just run a single fq_codel instance as in simplest.qos. As for the mesh backbone, well, there remains so much buffering underneath in the ath9k wifi drivers that fq_codel only takes the edge off a little. We just disabled 802.11e completely (getting rid of 3 out of 4 queues per interface), and my results have always been better for that, and I hope to catagorize them this summer - and it also saves on memory usage. > But it does not sound like they have first hand experience so this is eit= her hearsay, or commotions mesh networking is memory intensive. On the open= wrt side there seems no documentation of minim ram requirements. Doing a qu= ick back-of-the-envelop calculation here: > openWRT qos has 4 tiers which run fq_codel in both directions so we have = 8 fq_codel instances, with each fq_codel having a limit of 10240 packets, s= o worst case we expect: > > 4 * 2 * 10240 =3D 81920 packets > > at 1500bytes this equals > > 4 * 2 * 10240 * 1500 / (1024 * 1024) =3D 117.1875 MB Try to watch out for this sort of equivalence. Acks in the other direction are 66 bytes. Arguably we should have specified fq_codel's outside limit in bytes, not packets, and made it autotune to some ratio around free memory. And NONE of this memory is pre-allocated... more on that in a paragraph > this indeed is a bit heavy on a 32MB router, but honestly 64MB will not r= eally help you. Then again current openwrt has a limit off 800 instead of 1= 0240 so we end up at a worst case of: It's the extra SSIDs that hurt most these days, followed by the queues "needed" for 802.11e, but wait a paragraph... > 4 * 2 * 800 * 1500 / (1024 * 1024) =3D 9.1552734375 MB > > which should still be possible with 32MB. (Note that typically fq_codel d= oes not fill its queues up to limit, but it still would be bad if a router = can easily be DOSed into OOM and rebooting=E2=80=A6) The principal reason for the limit is! to avoid a DOS in memory limited routers. Otherwise I'd be perfectly happy if we could run it at the defaults. And the limit should go up some as we get closer to pushing gigE speeds (presently the router can only forward at about 330mbit). I am really hating seeing people cut/paste the limit into newer code without realizing that it was just there to keep a 32MB box from crashing under a dos... (and I note, that if you run out of memory to service packets, the odds are very good your box won't crash anyway - but this exercises code paths that are rarely touched.) > > (For current cerowrt with simple.qos the worst case is: > (1001 * 4 + 1000 * 13 + 800 * 12) * 1500 / (1024 * 1024) =3D 38.057327270= 5 MB > > yet this still works quite well on a 64MB device (only 4 of these queues = are connected to the WAN interface though) 3 up, 3 down, actually in simple.qos, 1 up, 1 down in simple. And each wifi SSID consumes 4 queues (although right now, 3 are unused) I want to make really clear: the FIXED overhead of fq_codel is something like 100+ 64 bytes*flows (usually 1024) - so each fq_codel instance eats 64K (not M!) of data to run. The limit just keeps packet data (don't quote me, something like 200 bytes + packet size) under control, and usually only builds up on the bottleneck device, not anywhere else. So you might have a bottleneck on your wifi, someone dos-ing you maybe, but the ethernet interface will be running at only a few packets outstanding at anytime. > One of the bigger issues with devices with small RAM is that often they h= ave relatively weak CPUs and I seem to recall that cerowrt tops out around = 60 to 70 Mbit/sec (total for ingress and egress) due to its shaping perform= ance. Yes. Doing considerably > > > So unless you want to run commotion you might want to ask on the openwrt = list=E2=80=A6 > > Best Regards > Sebastan > > > >> _______________________________________________ >> Cerowrt-devel mailing list >> Cerowrt-devel@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cerowrt-devel > > _______________________________________________ > Cerowrt-devel mailing list > Cerowrt-devel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cerowrt-devel --=20 Dave T=C3=A4ht NSFW: https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_= indecent.article