[Cerowrt-devel] [Bloat] Testing Queue models

Wed Feb 1 14:49:00 EST 2012

On Wed, Feb 1, 2012 at 7:17 PM, Sebastian Moeller <moeller0 at gmx.de> wrote:
> Hi Dave,
>
>
> On Feb 1, 2012, at 10:17 AM, Dave Taht wrote:
>
>> can you reply to the lists for details like this?
>
>        Sorry, forgot to reply all, I hope I fixed it now.
>
>>
>> The ADSL-shaper work that went into the stab option was pretty
>> fundamental, and needed.
>>
>> A) Fix openwrt's shaper to be more right for their distro (which
>> doesn't have SFQRED nor QFQ)
>        Okay, I will try to coordinate with upstream if that can be somehow included in vanilla openwork (even though I hope that vanilla will switch to your new scripts soon :) ) This might take a while, but It is on my todo list, But if vanilla sticks to its old shaper and shaper setup scripts than sharing this with cerowrt is going to be only in idea not in implementation...

Well, there are several 'easy' fixes.

1) openwrt does not do any ipv6 classification at all presently. Easy
to fix by merely calling
ip6tables as well.
2) it's unclear to me how encapsulated packets (gre, 6in4) etc are handled
3) I'd like to campaign for the removal of esfq entirely from the
build. SFQ long ago
incorporated those features.
4) The queue depth for sfq is crazy.

I'll put more details up on the related bug
http://www.bufferbloat.net/issues/331

but first up to me is modeling the existing behavior, and finding
scenarios where
it misbehaves under common current in-home workloads.

If I have any overall drive, it's to get to where it's possible to do
a bakeoff of various models.
I hope to enlist as many as possible (such as yourself!) in the field,
as well as in (maybe a few) labs.

http://www.bufferbloat.net/issues/301

Queuing behavior is not easy to understand in the first place, most
papers use an avpkt of 1000, when about 80 is closer to normal for
home use, the effects of packet aggregation are not well modeled, and
while it has long been noted that head drop, and random drop, and ecn
look like good ideas, nobody has any real experience with them in real
situations.

One of my biggest concerns is with ledbat (bittorrent) behavior, which
has been only modeled with invalid models to date (IMHO). Another
concern is with the observed behavior of things like netflix and
youtube, which tend to send bursty streams....

That's a lot of new variables to play with!

>> B) Fix it to be right on cerowrt
>        This is I guess were I need your guidance (and where I actually need to switch to the most recent cerowrt first, I am still on ocean city…)

Well, bql-30 turns out to have a kernel configuration error or two
that I'm hopefully correcting now... pimv2 multicast isn't working for
some reason, and I forgot to build oprofile and NO_HZ support. That
said, it seems to work...

>
>> C) Clean up the web interfaces to make it obvious that DSL can be 'special'
>        That is beyond my capabilities, I am a scripting king of person (I might give it a try but no ETA, ...)
>

I have spent most of my spare time in the last 2 months learning
enough of lua to make a dent in the gui problem. I am emphatically not
a UI person, however, and anything I design seems likely to be
overcomplex and confusing to the end user.

>>
>> But first up, is models and analysis. Measure twice, cut once….
>
>        Yeah, I guess I will try the current cerowrt, hack stab inside and see whether it still makes a difference (I assume it will), please just ignore me until then…

Please do. BUT, feel free to wait a week, the new stuff is shaping up quickly.

>
>
>>
>> I'm puzzled about your mtu setting, however…
>
> from tc-stab man page:
> "mtu
> maximum packet size we create size table for, assumed 2048 if not specified explicitly"
> so this does not affect the real mtu. Opencoding this default was meant as a reminder for me to go back and try understanding and implement the following part of the man page:
> "For ATM size tables, 16~bytes sized slots are perfectly enough. The default values of mtu and tsize create 4~bytes sized slots."
> But I did not get around to that...
>
> Best
>        Sebastian
>
>
>>
>> On Wed, Feb 1, 2012 at 6:13 PM, Sebastian Moeller <moeller0 at gmx.de> wrote:
>>> Hi Dave,
>>>
>>> cool work!
>>>
>>> One thing I noticed (ocean city on ADSL over ATM) is the I really needed to use tc's stab option in combination with the default openwork shaper to get latencies down to a useful level.
>>> My crude hack was adding:
>>> tc_stab_string="stab overhead 18 mtu 2048 mpu 53 linklayer atm"
>>> to generate.sh; and then modifying the addition of root disc like the following
>>> tc qdisc add dev $dev root handle 1: ${tc_stab_string} hfsc default ${class_default}0
>>> Then I only needed to reduce the uplink and downlink speed marginally (5%) and got good and stable ping latencies even in the lieu of massive uploads and opening 100 browser tabs at the same time. Without the stab option I had to reduce nominal speeds to around 65%-70% of the line rate and still got worse ping latencies than with the stab option.
>>>        Obviously the exact invocation of the stab depends on a number of factors, like the encapsulation used; since I do not know a generic way of discovering those automatically (unless the del modem is part of the router) I think this needs to be user editable...
>>>
>>> So it would be sweet if there would be an easy way (either per editor or GUI) to actually include stab options into the shaper to deal with this DSL over ATM specific issue.
>>>
>>>
>>> Thanks for deflating the buffers and rescuing decent latencies…
>>>
>>> Best
>>>        Sebastian
>>>
>>>
>>> On Feb 1, 2012, at 9:12 AM, Dave Taht wrote:
>>>
>>>> Now that I have BQL and the new SFQ, ARED, and SFQRED working in
>>>> cerowrt, I have been working on modeling the behavior of existing
>>>> shapers (wshaper, openwrt, sfqred, and something more torrent-proof
>>>> that doesn't have a name yet)
>>>>
>>>> There are several things that will get in the way of doing it well at
>>>> the present time, and I hope to evolve towards something that works
>>>> well in nearly all cases. For purposes of this discussion I'll use a
>>>> few abbreviations.
>>>>
>>>> "u" = user
>>>> "d" = device
>>>> "f" = flow
>>>>
>>>> Some general notes:
>>>>
>>>> Accurately determining uplink bandwidth is something that no tool does
>>>> quite right today, and even then uplink bandwidth can be dynamic
>>>> (notably on cable). Van and Kathie are working on something that can
>>>> figure it out better, dynamically, but that code's not ready yet.
>>>> Instead I'm trying to pull together a shaperprobe that detects modern
>>>> conditions better, statically, at least.
>>>>
>>>> Uplink speed is the principal bottleneck problem that users deal with.
>>>>
>>>> I'm running SFQ on all interfaces. This helps break up streams into
>>>> more manageable chunks. It can be really NICE, on ethernet. In fact,
>>>> most of my network bottlenecks have moved to my desktops and laptops
>>>> in the general case, and to get best results I have to run the debloat
>>>> script and enable SFQ everywhere, in this wack-a-mole game.
>>>>
>>>> SFQ has positive effects on wireless as sparse streams move forward in
>>>> the queue, and are thus shipped as an aggregate at the earliest
>>>> opportunity. Regrettably there is so much rate-insensitive fixed
>>>> buffering at present in the Linux wireless stack as to incur huge
>>>> penalties at variable rates.
>>>>
>>>> It can have negative side effects on wireless, as it mucks with packet
>>>> aggregation across devices. That said, in home use (rather than with
>>>> dozens of wireless devices) it should help somewhat. I'm not worrying
>>>> about wireless behavior all that much right now, I'm mostly trying to
>>>> exercise the new code... as fixing the aggregation problems really
>>>> needs to have a per-destination queue somehow.
>>>>
>>>> Developing correct configuration for RED and ARED is something of a dark art.
>>>>
>>>> The new SFQ prioritizes new streams very well. Perhaps overly well.
>>>> Eric has a patch in progress
>>>> to make it have more of an idea of 'how new' a stream is. It otherwise
>>>> is very effective in reducing latency for short, sparse, streams.
>>>>
>>>> SFQRED's mental model is closer to what an ISP would use on it's
>>>> downstream link. It is not (quite) right for what a site would use as
>>>> an uplink.
>>>>
>>>> QFQ, which is what I was using to get 1/d level behavior in the
>>>> prototype I did several months back, still is hanging under load on
>>>> cerowrt. It's working on x86.  :( Might try working with classful SFQ
>>>> stuff instead, or I may give DRR a shot instead.
>>>>
>>>> Even getting 1/d is hard because the filters can only use IP and IPv6
>>>> addresses, so a device issuing traffic on both ipv4 and ipv6 can get
>>>> twice what it deserves in terms of bandwidth, or more if it has
>>>> multiple ipv6 addresses.
>>>>
>>>> My preferred solution to this (and part of the wireless problem) is to
>>>> sort things out by source mac address. How to do that, remains a
>>>> mystery. The way I'm leaning is to invent some sort of "src mac-to-id"
>>>> filter that is global router-wide.
>>>>
>>>> I tend to view (in the home) as having 1/u network performance as the
>>>> ideal. There are exceptions to this, notably video vs. say,
>>>> bittorrent. A clever way of getting closer to 1/u might be to sense
>>>> for more recent DNS queries and move that to a more interactive class.
>>>>
>>>> I got 1/d level performance, vs bittorrent, several months back, using
>>>> a combination of HTB, and a
>>>> two tier QFQ, then red model. I was happy with this, but QFQ was
>>>> required. Perhaps I can make
>>>> SFQRED do this with the right filters, or switch to a combination of
>>>> DRR and SFQRED....
>>>>
>>>> Anyway, at the present time, I'm trying a few approaches, the most
>>>> promising one is a two or three tier model using SFQRED, with a very
>>>> limited number of things prio 1 (dns), 2 (mostly everything), 3
>>>> classified and background traffic and torrent, where classifiable.
>>>> Classification is a rathole however, and I'd rather be aiming for 1/d
>>>> than 1/f.
>>>>
>>>> Regardless openwrt's default AQM does lousy queue depth management at
>>>> present, and I expect to
>>>> improve it tremendously over the coming week.
>>>>
>>>> --
>>>> Dave Täht
>>>> SKYPE: davetaht
>>>> US Tel: 1-239-829-5608
>>>> FR Tel: 0638645374
>>>> http://www.bufferbloat.net
>>>> _______________________________________________
>>>> Bloat mailing list
>>>> Bloat at lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>
>>
>>
>>
>> --
>> Dave Täht
>> SKYPE: davetaht
>> US Tel: 1-239-829-5608
>> FR Tel: 0638645374
>> http://www.bufferbloat.net
>

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net