Re: [Bloat] Quick Fair Queue scheduler maturity and examples

Re: [Bloat] Quick Fair Queue scheduler maturity and examples

[David Täht]

[-- Attachment #1: Type: text/plain, Size: 3011 bytes --]

(Example elided, see thread on netdev)

On 11/02/2011 10:36 AM, Karel Rericha wrote:
> 2011/10/27 Eric Dumazet<eric.dumazet@gmail.com>:
> Thanks for example Eric. But it only added more confusion to me now 
> :-) I was under impression (and read somewhere) that QFQ is non work 
> conserving scheduler so I can use it more or less like HTB or HFSC to 
> set bandwidth constraints to flows. But from this example (and from 
> sources/patches/papers I try not to pretend I fully understand) it 
> looks to me like some multiqueue scheduler with arbitrary number of 
> queues and ability to arbitrary assign flows to this queues. So some 
> sort of fair division of available bandwidth to flows without 
> arbitrary bandwidth caps to these flows. 
This is what I want! It may not be what you want...
> I really dont see what is non work conserving here :-S Please save my 
> soul and enlighten me because I am at dead end now :-)

I initially had great hope for QFQ as I've been saying (mostly 
privately) that "PFIFO_FAST must die" for over a year now. What to 
replace it with is a rather large question, but I felt a start would be 
to adopt some FQ algorithm. Over the last couple weeks I read all the 
papers regarding DRR and QFQ and also poked into the source code and 
like you, am seriously un-enlightened.

I think eric's example is misleading as he divided up the queues by 
bandwidth, rather than flow, in the first tier of his tc hierarchy. 
useful as a test...

That said, buried in one of the papers on QFQ is the item that it isn't 
entirely fair queuing, as it has a maximum MSS of 5, rather than 1. 
Still that would be an improvement over diffserv based classification in 
the general case, and being somewhat bursty actually helps with 802.11n 
packet aggregation (at present).

Anyway, I've built QFQ into the latest cerowrt and am building it for 
another machine, and will try to come up with good ways to configure it 
this week.

My use cases are different than yours, however.

On a wireless STA (a laptop) - FQ all flows originating from that box. 
This would, for example, jump a DNS packet, ping, or TCP SYN attempt - 
to near the beginning of the transmit buffer while another elephant flow 
is taking place.

On the AP, FQ across the clients (so aggregation works better and 
bittorrent is suppressed), and FQ within each flow from/to that client 
(so as to reduce head of line blocking and have better performance for 
things like dns/ping/etc) - this would apply to the internal wireless 
and wired interfaces. As for the gateway interface, FQ across 
originating clients as well (but far more stuff than that needs to happen)

Haven't a clue how to do any of that right at all, at present. Clues 
wanted. I emailed one of the authors of QFQ for a clue, no reply as yet....

PS Also along the way whilst poking into that source code I found that 
there was already a fifo_drop_head tc queue type, which strikes me as 
almost useful for VO and VI wireless queues...

-- 
Dave Täht


[-- Attachment #2: dave_taht.vcf --]
[-- Type: text/x-vcard, Size: 204 bytes --]

begin:vcard
fn;quoted-printable:Dave T=C3=A4ht
n;quoted-printable:T=C3=A4ht;Dave
email;internet:dave.taht@gmail.com
tel;home:1-239-829-5608
tel;cell:0638645374
x-mozilla-html:FALSE
version:2.1
end:vcard

Re: [Bloat] Quick Fair Queue scheduler maturity and examples

[Eric Dumazet]

Le mercredi 02 novembre 2011 à 11:05 +0100, David Täht a écrit :
> (Example elided, see thread on netdev)
> 
> On 11/02/2011 10:36 AM, Karel Rericha wrote:
> > 2011/10/27 Eric Dumazet<eric.dumazet@gmail.com>:
> > Thanks for example Eric. But it only added more confusion to me now 
> > :-) I was under impression (and read somewhere) that QFQ is non work 
> > conserving scheduler so I can use it more or less like HTB or HFSC to 
> > set bandwidth constraints to flows. But from this example (and from 
> > sources/patches/papers I try not to pretend I fully understand) it 
> > looks to me like some multiqueue scheduler with arbitrary number of 
> > queues and ability to arbitrary assign flows to this queues. So some 
> > sort of fair division of available bandwidth to flows without 
> > arbitrary bandwidth caps to these flows. 
> This is what I want! It may not be what you want...
> > I really dont see what is non work conserving here :-S Please save my 
> > soul and enlighten me because I am at dead end now :-)
> 
> I initially had great hope for QFQ as I've been saying (mostly 
> privately) that "PFIFO_FAST must die" for over a year now. What to 
> replace it with is a rather large question, but I felt a start would be 
> to adopt some FQ algorithm. Over the last couple weeks I read all the 
> papers regarding DRR and QFQ and also poked into the source code and 
> like you, am seriously un-enlightened.
> 
> I think eric's example is misleading as he divided up the queues by 
> bandwidth, rather than flow, in the first tier of his tc hierarchy. 
> useful as a test...

It seems there is a bit of misunderstanding here.

QFQ is not a 'all is included' in one qdisc, like SFQ

You really need to setup qfq classes, and describe how packets are
mapped to qfq classes (this is done by an external flow classifier)

It also has no internal (default) flow classifier like SFQ did.

It has of course no bandwidth constraints. If you need to shape and use
QFQ, you'll have to use QFQ + a shaping qdisc. (This is why I used HTB
in my script because I wanted to shape)

If you dont need to shape, you still need to describe/setup qfq classes
and chose appropriate flow classifier.

Re: [Bloat] Quick Fair Queue scheduler maturity and examples

[David Täht]

[-- Attachment #1: Type: text/plain, Size: 2924 bytes --]

On 11/02/2011 11:31 AM, Eric Dumazet wrote:
> Le mercredi 02 novembre 2011 à 11:05 +0100, David Täht a écrit :
>> (Example elided, see thread on netdev)
>>
>> On 11/02/2011 10:36 AM, Karel Rericha wrote:
>>> 2011/10/27 Eric Dumazet<eric.dumazet@gmail.com>:
>>> Thanks for example Eric. But it only added more confusion to me now
>>> :-) I was under impression (and read somewhere) that QFQ is non work
>>> conserving scheduler so I can use it more or less like HTB or HFSC to
>>> set bandwidth constraints to flows. But from this example (and from
>>> sources/patches/papers I try not to pretend I fully understand) it
>>> looks to me like some multiqueue scheduler with arbitrary number of
>>> queues and ability to arbitrary assign flows to this queues. So some
>>> sort of fair division of available bandwidth to flows without
>>> arbitrary bandwidth caps to these flows.
>> This is what I want! It may not be what you want...
>>> I really dont see what is non work conserving here :-S Please save my
>>> soul and enlighten me because I am at dead end now :-)
>> I initially had great hope for QFQ as I've been saying (mostly
>> privately) that "PFIFO_FAST must die" for over a year now. What to
>> replace it with is a rather large question, but I felt a start would be
>> to adopt some FQ algorithm. Over the last couple weeks I read all the
>> papers regarding DRR and QFQ and also poked into the source code and
>> like you, am seriously un-enlightened.
>>
>> I think eric's example is misleading as he divided up the queues by
>> bandwidth, rather than flow, in the first tier of his tc hierarchy.
>> useful as a test...
> It seems there is a bit of misunderstanding here.
>
> QFQ is not a 'all is included' in one qdisc, like SFQ
I grok. (or rather, I did after some reading last week)
>
> You really need to setup qfq classes, and describe how packets are
> mapped to qfq classes (this is done by an external flow classifier)

It would be mildly better (in the case of wireless) to be able to do
flow classification based on the nexthop mac, which while introducing an
extra routing table lookup, would improve packet aggregation
probabilities in the multiple ip or multi-hop wireless case.

I don't know if a route lookup of dest mac could be correctly done at 
this layer.

> It also has no internal (default) flow classifier like SFQ did.
>
> It has of course no bandwidth constraints. If you need to shape and use
> QFQ, you'll have to use QFQ + a shaping qdisc. (This is why I used HTB
> in my script because I wanted to shape)

I just want FQ... for now.

> If you dont need to shape, you still need to describe/setup qfq classes
> and chose appropriate flow classifier.
>

Trying for two levels of flow classification here, which I still doubt I
can do here... hmm, perhaps with ifb...

don't want to shape, want to ultimately apply some level of a post-RED 
AQM to the overall flows
>


-- 
Dave Täht


[-- Attachment #2: dave_taht.vcf --]
[-- Type: text/x-vcard, Size: 214 bytes --]

begin:vcard
fn;quoted-printable:Dave T=C3=A4ht
n;quoted-printable:T=C3=A4ht;Dave
email;internet:dave.taht@gmail.com
tel;home:1-239-829-5608
tel;cell:0638645374
x-mozilla-html:FALSE
version:2.1
end:vcard