From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-gx0-f171.google.com (mail-gx0-f171.google.com [209.85.161.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id DAB93200588; Wed, 1 Feb 2012 11:49:02 -0800 (PST) Received: by ggnu1 with SMTP id u1so1476492ggn.16 for ; Wed, 01 Feb 2012 11:49:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=S2UNZ26jrHgFXJHMD6mFBzuzXb84jHjv2YdtMY+mlHg=; b=GdZrCvNlusNBdiWe1TdXA9h2GfwaYm134kWNJnZZaf8WtswULeS8zVItbh99qJq7nG ILgi/H67xK+AXocEJUlfnUZ3DtjCPyWqngpZUB66ukP192xCZk46PigNsosMoLqZNvC9 7kglKXxKqnkcm59eiednVr9HwQa58kRzh0/YA= MIME-Version: 1.0 Received: by 10.50.189.137 with SMTP id gi9mr14657952igc.29.1328125740598; Wed, 01 Feb 2012 11:49:00 -0800 (PST) Received: by 10.231.7.21 with HTTP; Wed, 1 Feb 2012 11:49:00 -0800 (PST) In-Reply-To: <4B1F748F-DD88-4498-8F9D-0C27E01721AE@gmx.de> References: <4B1F748F-DD88-4498-8F9D-0C27E01721AE@gmx.de> Date: Wed, 1 Feb 2012 19:49:00 +0000 Message-ID: From: Dave Taht To: Sebastian Moeller Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: cerowrt-devel@lists.bufferbloat.net, bloat Subject: Re: [Bloat] Testing Queue models X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2012 19:49:03 -0000 On Wed, Feb 1, 2012 at 7:17 PM, Sebastian Moeller wrote: > Hi Dave, > > > On Feb 1, 2012, at 10:17 AM, Dave Taht wrote: > >> can you reply to the lists for details like this? > > =A0 =A0 =A0 =A0Sorry, forgot to reply all, I hope I fixed it now. > >> >> The ADSL-shaper work that went into the stab option was pretty >> fundamental, and needed. >> >> A) Fix openwrt's shaper to be more right for their distro (which >> doesn't have SFQRED nor QFQ) > =A0 =A0 =A0 =A0Okay, I will try to coordinate with upstream if that can b= e somehow included in vanilla openwork (even though I hope that vanilla wil= l switch to your new scripts soon :) ) This might take a while, but It is o= n my todo list, But if vanilla sticks to its old shaper and shaper setup sc= ripts than sharing this with cerowrt is going to be only in idea not in imp= lementation... Well, there are several 'easy' fixes. 1) openwrt does not do any ipv6 classification at all presently. Easy to fix by merely calling ip6tables as well. 2) it's unclear to me how encapsulated packets (gre, 6in4) etc are handled 3) I'd like to campaign for the removal of esfq entirely from the build. SFQ long ago incorporated those features. 4) The queue depth for sfq is crazy. I'll put more details up on the related bug http://www.bufferbloat.net/issues/331 but first up to me is modeling the existing behavior, and finding scenarios where it misbehaves under common current in-home workloads. If I have any overall drive, it's to get to where it's possible to do a bakeoff of various models. I hope to enlist as many as possible (such as yourself!) in the field, as well as in (maybe a few) labs. http://www.bufferbloat.net/issues/301 Queuing behavior is not easy to understand in the first place, most papers use an avpkt of 1000, when about 80 is closer to normal for home use, the effects of packet aggregation are not well modeled, and while it has long been noted that head drop, and random drop, and ecn look like good ideas, nobody has any real experience with them in real situations. One of my biggest concerns is with ledbat (bittorrent) behavior, which has been only modeled with invalid models to date (IMHO). Another concern is with the observed behavior of things like netflix and youtube, which tend to send bursty streams.... That's a lot of new variables to play with! >> B) Fix it to be right on cerowrt > =A0 =A0 =A0 =A0This is I guess were I need your guidance (and where I act= ually need to switch to the most recent cerowrt first, I am still on ocean = city=85) Well, bql-30 turns out to have a kernel configuration error or two that I'm hopefully correcting now... pimv2 multicast isn't working for some reason, and I forgot to build oprofile and NO_HZ support. That said, it seems to work... > >> C) Clean up the web interfaces to make it obvious that DSL can be 'speci= al' > =A0 =A0 =A0 =A0That is beyond my capabilities, I am a scripting king of p= erson (I might give it a try but no ETA, ...) > I have spent most of my spare time in the last 2 months learning enough of lua to make a dent in the gui problem. I am emphatically not a UI person, however, and anything I design seems likely to be overcomplex and confusing to the end user. >> >> But first up, is models and analysis. Measure twice, cut once=85. > > =A0 =A0 =A0 =A0Yeah, I guess I will try the current cerowrt, hack stab in= side and see whether it still makes a difference (I assume it will), please= just ignore me until then=85 Please do. BUT, feel free to wait a week, the new stuff is shaping up quick= ly. > > >> >> I'm puzzled about your mtu setting, however=85 > > from tc-stab man page: > "mtu > maximum packet size we create size table for, assumed 2048 if not specifi= ed explicitly" > so this does not affect the real mtu. Opencoding this default was meant a= s a reminder for me to go back and try understanding and implement the foll= owing part of the man page: > "For ATM size tables, 16~bytes sized slots are perfectly enough. The defa= ult values of mtu and tsize create 4~bytes sized slots." > But I did not get around to that... > > Best > =A0 =A0 =A0 =A0Sebastian > > >> >> On Wed, Feb 1, 2012 at 6:13 PM, Sebastian Moeller wrot= e: >>> Hi Dave, >>> >>> cool work! >>> >>> One thing I noticed (ocean city on ADSL over ATM) is the I really neede= d to use tc's stab option in combination with the default openwork shaper t= o get latencies down to a useful level. >>> My crude hack was adding: >>> tc_stab_string=3D"stab overhead 18 mtu 2048 mpu 53 linklayer atm" >>> to generate.sh; and then modifying the addition of root disc like the f= ollowing >>> tc qdisc add dev $dev root handle 1: ${tc_stab_string} hfsc default ${c= lass_default}0 >>> Then I only needed to reduce the uplink and downlink speed marginally (= 5%) and got good and stable ping latencies even in the lieu of massive uplo= ads and opening 100 browser tabs at the same time. Without the stab option = I had to reduce nominal speeds to around 65%-70% of the line rate and still= got worse ping latencies than with the stab option. >>> =A0 =A0 =A0 =A0Obviously the exact invocation of the stab depends on a = number of factors, like the encapsulation used; since I do not know a gener= ic way of discovering those automatically (unless the del modem is part of = the router) I think this needs to be user editable... >>> >>> So it would be sweet if there would be an easy way (either per editor o= r GUI) to actually include stab options into the shaper to deal with this D= SL over ATM specific issue. >>> >>> >>> Thanks for deflating the buffers and rescuing decent latencies=85 >>> >>> Best >>> =A0 =A0 =A0 =A0Sebastian >>> >>> >>> On Feb 1, 2012, at 9:12 AM, Dave Taht wrote: >>> >>>> Now that I have BQL and the new SFQ, ARED, and SFQRED working in >>>> cerowrt, I have been working on modeling the behavior of existing >>>> shapers (wshaper, openwrt, sfqred, and something more torrent-proof >>>> that doesn't have a name yet) >>>> >>>> There are several things that will get in the way of doing it well at >>>> the present time, and I hope to evolve towards something that works >>>> well in nearly all cases. For purposes of this discussion I'll use a >>>> few abbreviations. >>>> >>>> "u" =3D user >>>> "d" =3D device >>>> "f" =3D flow >>>> >>>> Some general notes: >>>> >>>> Accurately determining uplink bandwidth is something that no tool does >>>> quite right today, and even then uplink bandwidth can be dynamic >>>> (notably on cable). Van and Kathie are working on something that can >>>> figure it out better, dynamically, but that code's not ready yet. >>>> Instead I'm trying to pull together a shaperprobe that detects modern >>>> conditions better, statically, at least. >>>> >>>> Uplink speed is the principal bottleneck problem that users deal with. >>>> >>>> I'm running SFQ on all interfaces. This helps break up streams into >>>> more manageable chunks. It can be really NICE, on ethernet. In fact, >>>> most of my network bottlenecks have moved to my desktops and laptops >>>> in the general case, and to get best results I have to run the debloat >>>> script and enable SFQ everywhere, in this wack-a-mole game. >>>> >>>> SFQ has positive effects on wireless as sparse streams move forward in >>>> the queue, and are thus shipped as an aggregate at the earliest >>>> opportunity. Regrettably there is so much rate-insensitive fixed >>>> buffering at present in the Linux wireless stack as to incur huge >>>> penalties at variable rates. >>>> >>>> It can have negative side effects on wireless, as it mucks with packet >>>> aggregation across devices. That said, in home use (rather than with >>>> dozens of wireless devices) it should help somewhat. I'm not worrying >>>> about wireless behavior all that much right now, I'm mostly trying to >>>> exercise the new code... as fixing the aggregation problems really >>>> needs to have a per-destination queue somehow. >>>> >>>> Developing correct configuration for RED and ARED is something of a da= rk art. >>>> >>>> The new SFQ prioritizes new streams very well. Perhaps overly well. >>>> Eric has a patch in progress >>>> to make it have more of an idea of 'how new' a stream is. It otherwise >>>> is very effective in reducing latency for short, sparse, streams. >>>> >>>> SFQRED's mental model is closer to what an ISP would use on it's >>>> downstream link. It is not (quite) right for what a site would use as >>>> an uplink. >>>> >>>> QFQ, which is what I was using to get 1/d level behavior in the >>>> prototype I did several months back, still is hanging under load on >>>> cerowrt. It's working on x86. =A0:( Might try working with classful SF= Q >>>> stuff instead, or I may give DRR a shot instead. >>>> >>>> Even getting 1/d is hard because the filters can only use IP and IPv6 >>>> addresses, so a device issuing traffic on both ipv4 and ipv6 can get >>>> twice what it deserves in terms of bandwidth, or more if it has >>>> multiple ipv6 addresses. >>>> >>>> My preferred solution to this (and part of the wireless problem) is to >>>> sort things out by source mac address. How to do that, remains a >>>> mystery. The way I'm leaning is to invent some sort of "src mac-to-id" >>>> filter that is global router-wide. >>>> >>>> I tend to view (in the home) as having 1/u network performance as the >>>> ideal. There are exceptions to this, notably video vs. say, >>>> bittorrent. A clever way of getting closer to 1/u might be to sense >>>> for more recent DNS queries and move that to a more interactive class. >>>> >>>> I got 1/d level performance, vs bittorrent, several months back, using >>>> a combination of HTB, and a >>>> two tier QFQ, then red model. I was happy with this, but QFQ was >>>> required. Perhaps I can make >>>> SFQRED do this with the right filters, or switch to a combination of >>>> DRR and SFQRED.... >>>> >>>> Anyway, at the present time, I'm trying a few approaches, the most >>>> promising one is a two or three tier model using SFQRED, with a very >>>> limited number of things prio 1 (dns), 2 (mostly everything), 3 >>>> classified and background traffic and torrent, where classifiable. >>>> Classification is a rathole however, and I'd rather be aiming for 1/d >>>> than 1/f. >>>> >>>> Regardless openwrt's default AQM does lousy queue depth management at >>>> present, and I expect to >>>> improve it tremendously over the coming week. >>>> >>>> -- >>>> Dave T=E4ht >>>> SKYPE: davetaht >>>> US Tel: 1-239-829-5608 >>>> FR Tel: 0638645374 >>>> http://www.bufferbloat.net >>>> _______________________________________________ >>>> Bloat mailing list >>>> Bloat@lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/bloat >>> >> >> >> >> -- >> Dave T=E4ht >> SKYPE: davetaht >> US Tel: 1-239-829-5608 >> FR Tel: 0638645374 >> http://www.bufferbloat.net > --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 FR Tel: 0638645374 http://www.bufferbloat.net