[Cake] Stranger target behaviour

[Cake] Stranger target behaviour

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 2891 bytes --]

Playing with latest JM authed commits on sch-cake & tc-adv.  Look at the
target values:

root@Router:~# tc -s qdisc change dev ifb4eth0 root cake internet
root@Router:~# tc -s qdisc show dev ifb4eth0
qdisc cake 8008: root refcnt 2 bandwidth 39300Kbit diffserv4 flows rtt
100.0ms noatm overhead 12
 Sent 285837 bytes 1240 pkt (dropped 0, overlimits 144 requeues 0)
 backlog 0b 0p requeues 0
capacity estimate: 10511Kbit
             Tin 0       Tin 1       Tin 2       Tin 3 
  thresh   39300Kbit   36843Kbit   29475Kbit    9825Kbit
  target       5.0ms       5.0ms       5.0ms       5.0ms
interval     100.0ms     100.0ms     100.0ms     100.0ms
Pk-delay         0us        22us         0us         0us
Av-delay         0us         6us         0us         0us
Sp-delay         0us         2us         0us         0us
  pkts             0        1240           0           0
  bytes            0      285837           0           0
way-inds           0           0           0           0
way-miss           0         125           0           0
way-cols           0           0           0           0
  drops            0           0           0           0
  marks            0           0           0           0
Sp-flows           0           0           0           0
Bk-flows           0           2           0           0
last-len           0          82           0           0
max-len            0        1514           0           0

root@Router:~# tc -s qdisc change dev ifb4eth0 root cake rtt 100ms
root@Router:~# tc -s qdisc show dev ifb4eth0
qdisc cake 8008: root refcnt 2 bandwidth 39300Kbit diffserv4 flows rtt
100.0ms noatm overhead 12
 Sent 757689 bytes 2374 pkt (dropped 0, overlimits 464 requeues 0)
 backlog 0b 0p requeues 0
capacity estimate: 5964Kbit
             Tin 0       Tin 1       Tin 2       Tin 3 
  thresh   39300Kbit   36843Kbit   29475Kbit    9825Kbit
  target       6.2ms       6.2ms       6.2ms       6.2ms
interval     100.0ms     100.0ms     100.0ms     100.0ms
Pk-delay         0us       286us         0us         0us
Av-delay         0us        70us         0us         0us
Sp-delay         0us         2us         0us         0us
  pkts             0        2374           0           0
  bytes            0      757689           0           0
way-inds           0           0           0           0
way-miss           0         157           0           0
way-cols           0           0           0           0
  drops            0           0           0           0
  marks            0           0           0           0
Sp-flows           0           0           0           0
Bk-flows           0           2           0           0
last-len           0          82           0           0
max-len            0        1514           0           0

Colour me confused :-)

Kevin


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4816 bytes --]

Re: [Cake] Stranger target behaviour

[Dave Taht]

The second one is from the interval >> 4, which equates to 6.2ms. I
note I picked that because some early data (2012) we had showed that
target slightly greater than cable media acquision (6ms) time was a
slight win, and it did not hurt to use a number more evenly divisible
in binary than the arbitrary 5%.

Also there was somewhere else in the original code where I'd stopped
using the target variable entirely and just used the shifted interval,
but given low rates where we have to account for the MTU, that
approach does not work. (otherwise anytime I can replace a memory
access with a mere shift, I'd do it)

As for the first 5ms sample below, I am really behind on keeping up
with the code. After next weekend I would hope to have time, but I do
hope someone can step through all the permutations upon which init and
change can be called to nail it.

Somewhere on these threads someone suggested moving the target
calculation into userspace, which does not work well, IMHO, when cake
is the default qdisc, and we somehow get a means to actually get the
actual line rate.
Dave Täht
I just invested five years of my life to making wifi better. And,
now... the FCC wants to make my work, illegal for people to install.
https://www.gofundme.com/savewifi


On Sat, Oct 31, 2015 at 3:35 PM, Kevin Darbyshire-Bryant
<kevin@darbyshire-bryant.me.uk> wrote:
> Playing with latest JM authed commits on sch-cake & tc-adv.  Look at the
> target values:
>
> root@Router:~# tc -s qdisc change dev ifb4eth0 root cake internet
> root@Router:~# tc -s qdisc show dev ifb4eth0
> qdisc cake 8008: root refcnt 2 bandwidth 39300Kbit diffserv4 flows rtt
> 100.0ms noatm overhead 12
>  Sent 285837 bytes 1240 pkt (dropped 0, overlimits 144 requeues 0)
>  backlog 0b 0p requeues 0
> capacity estimate: 10511Kbit
>              Tin 0       Tin 1       Tin 2       Tin 3
>   thresh   39300Kbit   36843Kbit   29475Kbit    9825Kbit
>   target       5.0ms       5.0ms       5.0ms       5.0ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk-delay         0us        22us         0us         0us
> Av-delay         0us         6us         0us         0us
> Sp-delay         0us         2us         0us         0us
>   pkts             0        1240           0           0
>   bytes            0      285837           0           0
> way-inds           0           0           0           0
> way-miss           0         125           0           0
> way-cols           0           0           0           0
>   drops            0           0           0           0
>   marks            0           0           0           0
> Sp-flows           0           0           0           0
> Bk-flows           0           2           0           0
> last-len           0          82           0           0
> max-len            0        1514           0           0
>
> root@Router:~# tc -s qdisc change dev ifb4eth0 root cake rtt 100ms
> root@Router:~# tc -s qdisc show dev ifb4eth0
> qdisc cake 8008: root refcnt 2 bandwidth 39300Kbit diffserv4 flows rtt
> 100.0ms noatm overhead 12
>  Sent 757689 bytes 2374 pkt (dropped 0, overlimits 464 requeues 0)
>  backlog 0b 0p requeues 0
> capacity estimate: 5964Kbit
>              Tin 0       Tin 1       Tin 2       Tin 3
>   thresh   39300Kbit   36843Kbit   29475Kbit    9825Kbit
>   target       6.2ms       6.2ms       6.2ms       6.2ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk-delay         0us       286us         0us         0us
> Av-delay         0us        70us         0us         0us
> Sp-delay         0us         2us         0us         0us
>   pkts             0        2374           0           0
>   bytes            0      757689           0           0
> way-inds           0           0           0           0
> way-miss           0         157           0           0
> way-cols           0           0           0           0
>   drops            0           0           0           0
>   marks            0           0           0           0
> Sp-flows           0           0           0           0
> Bk-flows           0           2           0           0
> last-len           0          82           0           0
> max-len            0        1514           0           0
>
> Colour me confused :-)
>
> Kevin
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>

Re: [Cake] Stranger target behaviour

[Sebastian Moeller]

Hi there,

On Oct 31, 2015, at 21:29 , Dave Taht <dave.taht@gmail.com> wrote:

> The second one is from the interval >> 4, which equates to 6.2ms. I
> note I picked that because some early data (2012) we had showed that
> target slightly greater than cable media acquision (6ms) time was a
> slight win, and it did not hurt to use a number more evenly divisible
> in binary than the arbitrary 5%.

	Well, but should it not be 6.2ms even if no rtt is explicitly requested? Currently cake has a lot of corer-cases that are not too well documented either.

> 
> Also there was somewhere else in the original code where I'd stopped
> using the target variable entirely and just used the shifted interval,
> but given low rates where we have to account for the MTU, that
> approach does not work. (otherwise anytime I can replace a memory
> access with a mere shift, I'd do it)

	Why? We could take the initial target, calculate the adjusted target, multiply that by 16 to get the adjusted interval/rtt; at that point target = interval >> 4 is true again ;)

> 
> As for the first 5ms sample below, I am really behind on keeping up
> with the code. After next weekend I would hope to have time, but I do
> hope someone can step through all the permutations upon which init and
> change can be called to nail it.

	I believe while we are at it we should reconsider the BUG_ONs; I believe it is going to be a hard sell for the kernel community to hose a whole system because a qdisc was a bit unhappy ;) I also want to repeat my conviction that the buffer limit needs to be exposed to the user and actually honored as a hard limit. I believe cake is aimed at non-expert users and having rtt and bandwidth settings have un-obviuos side effects on the worst case memory consumption does not seem suitable for non-experts...

> 
> Somewhere on these threads someone suggested moving the target
> calculation into userspace, which does not work well, IMHO, when cake
> is the default qdisc,

	I believe before that happens cake will need to acquire pfifo_fast priority bands as default ;); regarding the default qdisc issue, I hess the challenge is that nothing passes anything into sch_cake if it is the default, then obviously it should default to rtt 100ms target 6.2ms… So I should include cake as default qdisc in the cursory testing I do.

> and we somehow get a means to actually get the
> actual line rate.

	But we are far away from that, no?

Best Regards
	Sebastian

> Dave Täht
> I just invested five years of my life to making wifi better. And,
> now... the FCC wants to make my work, illegal for people to install.
> https://www.gofundme.com/savewifi
> 
> 
> On Sat, Oct 31, 2015 at 3:35 PM, Kevin Darbyshire-Bryant
> <kevin@darbyshire-bryant.me.uk> wrote:
>> Playing with latest JM authed commits on sch-cake & tc-adv.  Look at the
>> target values:
>> 
>> root@Router:~# tc -s qdisc change dev ifb4eth0 root cake internet
>> root@Router:~# tc -s qdisc show dev ifb4eth0
>> qdisc cake 8008: root refcnt 2 bandwidth 39300Kbit diffserv4 flows rtt
>> 100.0ms noatm overhead 12
>> Sent 285837 bytes 1240 pkt (dropped 0, overlimits 144 requeues 0)
>> backlog 0b 0p requeues 0
>> capacity estimate: 10511Kbit
>>             Tin 0       Tin 1       Tin 2       Tin 3
>>  thresh   39300Kbit   36843Kbit   29475Kbit    9825Kbit
>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk-delay         0us        22us         0us         0us
>> Av-delay         0us         6us         0us         0us
>> Sp-delay         0us         2us         0us         0us
>>  pkts             0        1240           0           0
>>  bytes            0      285837           0           0
>> way-inds           0           0           0           0
>> way-miss           0         125           0           0
>> way-cols           0           0           0           0
>>  drops            0           0           0           0
>>  marks            0           0           0           0
>> Sp-flows           0           0           0           0
>> Bk-flows           0           2           0           0
>> last-len           0          82           0           0
>> max-len            0        1514           0           0
>> 
>> root@Router:~# tc -s qdisc change dev ifb4eth0 root cake rtt 100ms
>> root@Router:~# tc -s qdisc show dev ifb4eth0
>> qdisc cake 8008: root refcnt 2 bandwidth 39300Kbit diffserv4 flows rtt
>> 100.0ms noatm overhead 12
>> Sent 757689 bytes 2374 pkt (dropped 0, overlimits 464 requeues 0)
>> backlog 0b 0p requeues 0
>> capacity estimate: 5964Kbit
>>             Tin 0       Tin 1       Tin 2       Tin 3
>>  thresh   39300Kbit   36843Kbit   29475Kbit    9825Kbit
>>  target       6.2ms       6.2ms       6.2ms       6.2ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk-delay         0us       286us         0us         0us
>> Av-delay         0us        70us         0us         0us
>> Sp-delay         0us         2us         0us         0us
>>  pkts             0        2374           0           0
>>  bytes            0      757689           0           0
>> way-inds           0           0           0           0
>> way-miss           0         157           0           0
>> way-cols           0           0           0           0
>>  drops            0           0           0           0
>>  marks            0           0           0           0
>> Sp-flows           0           0           0           0
>> Bk-flows           0           2           0           0
>> last-len           0          82           0           0
>> max-len            0        1514           0           0
>> 
>> Colour me confused :-)
>> 
>> Kevin
>> 
>> 
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
>> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] Stranger target behaviour

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 1125 bytes --]

On 01/11/15 09:47, Sebastian Moeller wrote:
> Hi there,
>
> On Oct 31, 2015, at 21:29 , Dave Taht <dave.taht@gmail.com> wrote:
>
>> The second one is from the interval >> 4, which equates to 6.2ms. I
>> note I picked that because some early data (2012) we had showed that
>> target slightly greater than cable media acquision (6ms) time was a
>> slight win, and it did not hurt to use a number more evenly divisible
>> in binary than the arbitrary 5%.
> 	Well, but should it not be 6.2ms even if no rtt is explicitly requested? Currently cake has a lot of corner-cases that are not too well documented either.
Sebastian as ever picks up on the exact source of my confusion/point.

If 'target=interval/16' is the new mantra then the defaults built in to
cake qdisc should be changed, otherwise we have the current situation
which is:  Default Interval=100ms, target=5ms.  If I specify rtt 100ms
via the joys of tc, then I have: Interval=100ms, target=6.2ms.  If I
specify a target keyword such as 'internet' which also has an interval=
100ms, then target become 5ms again.  Somewhat inconsistent.






[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4816 bytes --]

Re: [Cake] Stranger target behaviour

[Sebastian Moeller]

Dear friends of cake,

On Nov 1, 2015, at 14:21 , Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:

> On 01/11/15 09:47, Sebastian Moeller wrote:
>> Hi there,
>> 
>> On Oct 31, 2015, at 21:29 , Dave Taht <dave.taht@gmail.com> wrote:
>> 
>>> The second one is from the interval >> 4, which equates to 6.2ms. I
>>> note I picked that because some early data (2012) we had showed that
>>> target slightly greater than cable media acquision (6ms) time was a
>>> slight win, and it did not hurt to use a number more evenly divisible
>>> in binary than the arbitrary 5%.
>> 	Well, but should it not be 6.2ms even if no rtt is explicitly requested? Currently cake has a lot of corner-cases that are not too well documented either.
> Sebastian as ever picks up on the exact source of my confusion/point.
> 
> If 'target=interval/16' is the new mantra then the defaults built in to
> cake qdisc should be changed, otherwise we have the current situation
> which is:  Default Interval=100ms, target=5ms.  If I specify rtt 100ms
> via the joys of tc, then I have: Interval=100ms, target=6.2ms.  If I
> specify a target keyword such as 'internet' which also has an interval=
> 100ms, then target become 5ms again.  Somewhat inconsistent.

	I believe I just fixed this by initializing target in sch_cake to "interval >> 4”, I opted for the calculation instead of the simple number 6250 to sort of document the pattern and tried to put in a comment explaining why this is okay with codel's theory. Please note, I did not really test this, so for all I know it might break spectacularly...
	Slightly different question, I have a few cosmetic changes for tc, so which repository to commit this too, is tc-adv to be the gathering point for changes to be considered?

Best Regards
	Sebastian

Re: [Cake] Stranger target behaviour

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 2759 bytes --]

On 01/11/15 16:05, Sebastian Moeller wrote:
> Dear friends of cake,
>
> On Nov 1, 2015, at 14:21 , Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>
>> On 01/11/15 09:47, Sebastian Moeller wrote:
>>> Hi there,
>>>
>>> On Oct 31, 2015, at 21:29 , Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>>> The second one is from the interval >> 4, which equates to 6.2ms. I
>>>> note I picked that because some early data (2012) we had showed that
>>>> target slightly greater than cable media acquision (6ms) time was a
>>>> slight win, and it did not hurt to use a number more evenly divisible
>>>> in binary than the arbitrary 5%.
>>> 	Well, but should it not be 6.2ms even if no rtt is explicitly requested? Currently cake has a lot of corner-cases that are not too well documented either.
>> Sebastian as ever picks up on the exact source of my confusion/point.
>>
>> If 'target=interval/16' is the new mantra then the defaults built in to
>> cake qdisc should be changed, otherwise we have the current situation
>> which is:  Default Interval=100ms, target=5ms.  If I specify rtt 100ms
>> via the joys of tc, then I have: Interval=100ms, target=6.2ms.  If I
>> specify a target keyword such as 'internet' which also has an interval=
>> 100ms, then target become 5ms again.  Somewhat inconsistent.
> 	I believe I just fixed this by initializing target in sch_cake to "interval >> 4”, I opted for the calculation instead of the simple number 6250 to sort of document the pattern and tried to put in a comment explaining why this is okay with codel's theory. Please note, I did not really test this, so for all I know it might break spectacularly...
Looks ok to me, though the optimising compiler likely will turn that
calculation into a constant 6250 anyway ;-)  Only comment would be
'watch your line length' - no lines over 78 chars ideally in the
kernel.  On that note I've just committed some checkpatch
style/formatting related changes.
> 	Slightly different question, I have a few cosmetic changes for tc, so which repository to commit this too, is tc-adv to be the gathering point for changes to be considered?
I've no idea!  Recently I tried to merge Toke's repo version into my
copy of Dave's tc-adv without success.  Well the merge works, but
openwrt complains loudly at this recent version of iproute2 utilities
and so far it's beyond my wit to tease out how to fix.  *if* it worked
then we could be down to 1 repo, 1 version that worked for linux & openwrt.

Speaking purely personally, committing to tc-adv would mean openwrt type
people would test/play with it (me!)  Committing to kau.toke.de would
mean non-openwrters would play....of course you could cherrypick/commit
to both :-)

Kevin



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4816 bytes --]

Re: [Cake] Stranger target behaviour

[Sebastian Moeller]

Hi Kevin,

On Nov 1, 2015, at 18:18 , Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:

> On 01/11/15 16:05, Sebastian Moeller wrote:
>> Dear friends of cake,
>> 
>>> [...]
>> 	I believe I just fixed this by initializing target in sch_cake to "interval >> 4”, I opted for the calculation instead of the simple number 6250 to sort of document the pattern and tried to put in a comment explaining why this is okay with codel's theory. Please note, I did not really test this, so for all I know it might break spectacularly...
> Looks ok to me, though the optimising compiler likely will turn that
> calculation into a constant 6250 anyway ;-)  

	And that is fine it really is to document the fact for the source reader and to catch the hypothetical case that the default interval will be increased in the future...

> Only comment would be
> 'watch your line length' - no lines over 78 chars ideally in the
> kernel.

	Well, the powers that be need to decide, I believe the comment is helpful enough to having made it, but insignificant enough to not warrant its dedicated line or block of lines. First let’s see whether the comment survives in its verbosity and then care for style. That said, I am happy to keep monitoring this and potentially make the change.

>  On that note I've just committed some checkpatch
> style/formatting related changes.

	Oh, even better, I just see you fixed it for me; thanks a lot (sure beats having to clean up after myself ;) )

>> 	Slightly different question, I have a few cosmetic changes for tc, so which repository to commit this too, is tc-adv to be the gathering point for changes to be considered?
> I've no idea!  Recently I tried to merge Toke's repo version into my
> copy of Dave's tc-adv without success.  Well the merge works, but
> openwrt complains loudly at this recent version of iproute2 utilities
> and so far it's beyond my wit to tease out how to fix.  *if* it worked
> then we could be down to 1 repo, 1 version that worked for linux & openwrt.

	At least to me that sounds like a decent strategy, no?

> 
> Speaking purely personally, committing to tc-adv would mean openwrt type
> people would test/play with it (me!)  Committing to kau.toke.de would
> mean non-openwrters would play....of course you could cherrypick/commit
> to both :-)

	Since I only have commit permissions at tc-adv (I am not asking for more) I will just play with the tc-adv repository and hope Jonathan will cherry pick (or even just implement the ideas in his own way) everything he considers worth it…

Best Regards
	Sebastian

> 
> Kevin