[Cake] Bug or not Cake + hfsc or Cake + Drr

[Cake] Bug or not Cake + hfsc or Cake + Drr

[Martin Zaharinov]

[-- Attachment #1: Type: text/plain, Size: 1143 bytes --]

Hi i try to  make clear shaper with cake and drr
in this test i add drr for root and make class to set limit by ip
i use kernel 4.9.x and latest sch_cake from github

but in dmesg have error:

[  609.157445] drr_dequeue: cake qdisc 8004: is non-work-conserving?
[  609.212679] drr_dequeue: cake qdisc 8003: is non-work-conserving?


tc qdisc del dev eth1 root

tc qdisc add dev eth1 root handle 1:0000 drr

tc class add dev eth1 parent 1:0000 classid 1:0002 drr

tc class add dev eth1 parent 1:0002 classid 1:0001 drr

tc class add dev eth1 parent 1:0002 classid 1:0003 drr

tc class add dev eth1 parent 1:0003 classid 1:0004 drr

tc qdisc add dev eth1 parent 1:0004 cake besteffort bandwidth 5mbit
datacentre nat memlimit 32m


tc qdisc del dev eth0 root

tc qdisc add dev eth0 root handle 1:0000 drr

tc class add dev eth0 parent 1:0000 classid 1:0002 drr

tc class add dev eth0 parent 1:0002 classid 1:0001 drr

tc class add dev eth0 parent 1:0002 classid 1:0003 drr

tc class add dev eth0 parent 1:0003 classid 1:0004 drr

tc qdisc add dev eth0 parent 1:0004 cake besteffort bandwidth 7mbit
datacentre nat memlimit 32m



Best Regards,

m

[-- Attachment #2: Type: text/html, Size: 5962 bytes --]

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Toke Høiland-Jørgensen]

Martin Zaharinov <micron10@gmail.com> writes:

> Hi i try to  make clear shaper with cake and drr
> in this test i add drr for root and make class to set limit by ip
> i use kernel 4.9.x and latest sch_cake from github
>
> but in dmesg have error:
>
> [  609.157445] drr_dequeue: cake qdisc 8004: is non-work-conserving?
> [  609.212679] drr_dequeue: cake qdisc 8003: is non-work-conserving?

Yeah, you can't use cake as a DRR leaf qdisc. But why would you want to
do that? Cake contains its own DRR round-robin scheduler...

-Toke

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Martin Zaharinov]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

Hi Toke
i have little network with 100 users with ip 192.168.10.0/24

eth0 internet
eth1 lan + pppoe

on pppeo i have network 192.168.11.0/24

now i use hfsc to shape users by services
i have 3 services
1. 5mbit
2. 7mbit
3. 10mbit

with hfsc i use classid to select user and speed but in cake not have
classid support and this is a problem i want to use cake and set to every
users different speed


On Thu, Mar 7, 2019, 11:21 Toke Høiland-Jørgensen <toke@redhat.com> wrote:

> Martin Zaharinov <micron10@gmail.com> writes:
>
> > Hi i try to  make clear shaper with cake and drr
> > in this test i add drr for root and make class to set limit by ip
> > i use kernel 4.9.x and latest sch_cake from github
> >
> > but in dmesg have error:
> >
> > [  609.157445] drr_dequeue: cake qdisc 8004: is non-work-conserving?
> > [  609.212679] drr_dequeue: cake qdisc 8003: is non-work-conserving?
>
> Yeah, you can't use cake as a DRR leaf qdisc. But why would you want to
> do that? Cake contains its own DRR round-robin scheduler...
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1863 bytes --]

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Toke Høiland-Jørgensen]

Martin Zaharinov <micron10@gmail.com> writes:

> Hi Toke
> i have little network with 100 users with ip 192.168.10.0/24
>
> eth0 internet
> eth1 lan + pppoe
>
> on pppeo i have network 192.168.11.0/24
>
> now i use hfsc to shape users by services
> i have 3 services
> 1. 5mbit
> 2. 7mbit
> 3. 10mbit
>
> with hfsc i use classid to select user and speed but in cake not have
> classid support and this is a problem i want to use cake and set to
> every users different speed

Ah, right. Well, unfortunately you can't do that with CAKE, as you note.
You can keep your existing HFSC setup, though, and just use CAKE as a
leaf qdisc. But you'll have to turn off the shaper in CAKE and just use
the one in HFSC as before...

-Toke

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Martin Zaharinov]

[-- Attachment #1: Type: text/plain, Size: 944 bytes --]

please if you have time write my sample with leaf qdisc

On Thu, Mar 7, 2019, 12:30 Toke Høiland-Jørgensen <toke@redhat.com> wrote:

> Martin Zaharinov <micron10@gmail.com> writes:
>
> > Hi Toke
> > i have little network with 100 users with ip 192.168.10.0/24
> >
> > eth0 internet
> > eth1 lan + pppoe
> >
> > on pppeo i have network 192.168.11.0/24
> >
> > now i use hfsc to shape users by services
> > i have 3 services
> > 1. 5mbit
> > 2. 7mbit
> > 3. 10mbit
> >
> > with hfsc i use classid to select user and speed but in cake not have
> > classid support and this is a problem i want to use cake and set to
> > every users different speed
>
> Ah, right. Well, unfortunately you can't do that with CAKE, as you note.
> You can keep your existing HFSC setup, though, and just use CAKE as a
> leaf qdisc. But you'll have to turn off the shaper in CAKE and just use
> the one in HFSC as before...
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1564 bytes --]

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Martin Zaharinov]

[-- Attachment #1: Type: text/plain, Size: 1103 bytes --]

and do you have other solution for this case

On Thu, Mar 7, 2019, 12:41 Martin Zaharinov <micron10@gmail.com> wrote:

> please if you have time write my sample with leaf qdisc
>
> On Thu, Mar 7, 2019, 12:30 Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>
>> Martin Zaharinov <micron10@gmail.com> writes:
>>
>> > Hi Toke
>> > i have little network with 100 users with ip 192.168.10.0/24
>> >
>> > eth0 internet
>> > eth1 lan + pppoe
>> >
>> > on pppeo i have network 192.168.11.0/24
>> >
>> > now i use hfsc to shape users by services
>> > i have 3 services
>> > 1. 5mbit
>> > 2. 7mbit
>> > 3. 10mbit
>> >
>> > with hfsc i use classid to select user and speed but in cake not have
>> > classid support and this is a problem i want to use cake and set to
>> > every users different speed
>>
>> Ah, right. Well, unfortunately you can't do that with CAKE, as you note.
>> You can keep your existing HFSC setup, though, and just use CAKE as a
>> leaf qdisc. But you'll have to turn off the shaper in CAKE and just use
>> the one in HFSC as before...
>>
>> -Toke
>>
>

[-- Attachment #2: Type: text/html, Size: 2009 bytes --]

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Toke Høiland-Jørgensen]

Martin Zaharinov <micron10@gmail.com> writes:

> please if you have time write my sample with leaf qdisc

Basically, you can't use the "bandwidth" parameter to cake when it is
the leaf qdisc of HFSC/DRR/HTB. Presumably you have an existing script
you are using with HFSC? If you want to use cake in your setup, you can
just replace whatever you have as a leaf qdisc in there with cake...

-Toke

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Toke Høiland-Jørgensen]

Martin Zaharinov <micron10@gmail.com> writes:

> in original script i use sfq
>
> I what to change this line : tc qdisc add dev eth1 parent 1:000b sfq
> perturb 10
> to this
> tc qdisc add dev eth1 parent 1:000b cake bandwidth 10mbit
> and set on preview line ul rate to 15mbit or other.

Yeah, you can't do that. You will have to set the bandwidth in the HFSC
classes, and elide it from the cake invocation. So basically, just
replace 'sfq perturb 10' with 'cake' in your existing script. And modify
the bandwidth parameter in your 'tc class add' for HFSC.

-Toke

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Martin Zaharinov]

[-- Attachment #1: Type: text/plain, Size: 738 bytes --]

please write my one sample line with this

On Thu, Mar 7, 2019, 13:05 Toke Høiland-Jørgensen <toke@redhat.com> wrote:

> Martin Zaharinov <micron10@gmail.com> writes:
>
> > in original script i use sfq
> >
> > I what to change this line : tc qdisc add dev eth1 parent 1:000b sfq
> > perturb 10
> > to this
> > tc qdisc add dev eth1 parent 1:000b cake bandwidth 10mbit
> > and set on preview line ul rate to 15mbit or other.
>
> Yeah, you can't do that. You will have to set the bandwidth in the HFSC
> classes, and elide it from the cake invocation. So basically, just
> replace 'sfq perturb 10' with 'cake' in your existing script. And modify
> the bandwidth parameter in your 'tc class add' for HFSC.
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1153 bytes --]

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Toke Høiland-Jørgensen]

Martin Zaharinov <micron10@gmail.com> writes:

> please write my one sample line with this

From your script you'd need to use:

tc qdisc add dev eth1 parent 1:0004 cake

instead of

tc qdisc add dev eth1 parent 1:0004 sfq perturb 10

And adjust your HFSC parameter to shape to the rate you want. Can't help
you with that part, sorry, you'll need to go read up on shaping with
HFSC (or I'd suggest maybe using HTB instead as that is simpler to
configure). There are loads of tutorials for that online :)

-Toke

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Martin Zaharinov]

[-- Attachment #1: Type: text/plain, Size: 666 bytes --]

thanks i will try



On Thu, Mar 7, 2019, 13:25 Toke Høiland-Jørgensen <toke@redhat.com> wrote:

> Martin Zaharinov <micron10@gmail.com> writes:
>
> > please write my one sample line with this
>
> From your script you'd need to use:
>
> tc qdisc add dev eth1 parent 1:0004 cake
>
> instead of
>
> tc qdisc add dev eth1 parent 1:0004 sfq perturb 10
>
> And adjust your HFSC parameter to shape to the rate you want. Can't help
> you with that part, sorry, you'll need to go read up on shaping with
> HFSC (or I'd suggest maybe using HTB instead as that is simpler to
> configure). There are loads of tutorials for that online :)
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1105 bytes --]

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1447 bytes --]

If it’s just straight cake under hfsc with a rate here’s what I do (obviously replace 100mbit with your rate):

tc qdisc add dev eth1 root handle 1: hfsc default 1
tc class add dev eth1 parent 1: classid 1:1 hfsc ls rate 100mbit ul rate 100mbit
tc qdisc add dev eth1 parent 1:1 cake

Also, there were recent fixes to the Linux kernel and cake regarding GSO splitting. You may find with hfsc on older kernels that you have to use “cake no-split-gso” to avoid lockups.

> On Mar 7, 2019, at 12:32 PM, Martin Zaharinov <micron10@gmail.com> wrote:
> 
> thanks i will try
> 
> 
> 
> On Thu, Mar 7, 2019, 13:25 Toke Høiland-Jørgensen <toke@redhat.com <mailto:toke@redhat.com>> wrote:
> Martin Zaharinov <micron10@gmail.com <mailto:micron10@gmail.com>> writes:
> 
> > please write my one sample line with this
> 
> From your script you'd need to use:
> 
> tc qdisc add dev eth1 parent 1:0004 cake
> 
> instead of
> 
> tc qdisc add dev eth1 parent 1:0004 sfq perturb 10
> 
> And adjust your HFSC parameter to shape to the rate you want. Can't help
> you with that part, sorry, you'll need to go read up on shaping with
> HFSC (or I'd suggest maybe using HTB instead as that is simpler to
> configure). There are loads of tutorials for that online :)
> 
> -Toke
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake


[-- Attachment #2: Type: text/html, Size: 2720 bytes --]

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Martin Zaharinov]

[-- Attachment #1: Type: text/plain, Size: 2641 bytes --]

Hi Pete
yes i find this patch and add to cake source
and in my test with hfsc and cake
i use

tc qdisc add dev eth1 root handle 1: hfsc default 1
tc class add dev eth1 parent 1: classid 1:1 hfsc ls rate 7mbit ul rate 7mbit
tc qdisc add dev eth1 parent 1:1 cake besteffort bandwidth 5mbit datacentre
nat memlimit 32m

user speed is 5mbit and on hfsc i add extra 2 mbit to give free limit for
cake.
and work fine but in dmesg get :

[44835.530255] qdisc_peek_len: cake qdisc 8017: is non-work-conserving?
[44840.689403] HFSC: cake qdisc 8012: is non-work-conserving?


user receive 5mbit for upload and download and fine work.
mai be need to add no-split-gso to config

but after add have same error on dmesg :

[44955.641391] HFSC: cake qdisc 8021: is non-work-conserving?
[44955.958904] HFSC: cake qdisc 801C: is non-work-conserving?

but work fine and will remove warrning from kernel source to not list in
dmesg :)





i try same config with drr  to remove speed limit from hfsc and use only
cake bandwidth and work same fine but in time of work machine crash .


Best Regards,
Martin


On Thu, Mar 7, 2019 at 8:05 PM Pete Heist <pete@heistp.net> wrote:

> If it’s just straight cake under hfsc with a rate here’s what I do
> (obviously replace 100mbit with your rate):
>
> tc qdisc add dev eth1 root handle 1: hfsc default 1
> tc class add dev eth1 parent 1: classid 1:1 hfsc ls rate 100mbit ul rate
> 100mbit
> tc qdisc add dev eth1 parent 1:1 cake
>
> Also, there were recent fixes to the Linux kernel and cake regarding GSO
> splitting. You may find with hfsc on older kernels that you have to use
> “cake no-split-gso” to avoid lockups.
>
> On Mar 7, 2019, at 12:32 PM, Martin Zaharinov <micron10@gmail.com> wrote:
>
> thanks i will try
>
>
>
> On Thu, Mar 7, 2019, 13:25 Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>
>> Martin Zaharinov <micron10@gmail.com> writes:
>>
>> > please write my one sample line with this
>>
>> From your script you'd need to use:
>>
>> tc qdisc add dev eth1 parent 1:0004 cake
>>
>> instead of
>>
>> tc qdisc add dev eth1 parent 1:0004 sfq perturb 10
>>
>> And adjust your HFSC parameter to shape to the rate you want. Can't help
>> you with that part, sorry, you'll need to go read up on shaping with
>> HFSC (or I'd suggest maybe using HTB instead as that is simpler to
>> configure). There are loads of tutorials for that online :)
>>
>> -Toke
>>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>
>
>

[-- Attachment #2: Type: text/html, Size: 4338 bytes --]

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Toke Høiland-Jørgensen]

Martin Zaharinov <micron10@gmail.com> writes:

> Hi Pete
> yes i find this patch and add to cake source
> and in my test with hfsc and cake
> i use
>
> tc qdisc add dev eth1 root handle 1: hfsc default 1
> tc class add dev eth1 parent 1: classid 1:1 hfsc ls rate 7mbit ul rate 7mbit
> tc qdisc add dev eth1 parent 1:1 cake besteffort bandwidth 5mbit datacentre
> nat memlimit 32m
>
> user speed is 5mbit and on hfsc i add extra 2 mbit to give free limit for
> cake.
> and work fine but in dmesg get :
>
> [44835.530255] qdisc_peek_len: cake qdisc 8017: is non-work-conserving?
> [44840.689403] HFSC: cake qdisc 8012: is non-work-conserving?

Yes. As I said, you *cannot use the cake shaper* when running in under
HFSC.

> user receive 5mbit for upload and download and fine work.
> mai be need to add no-split-gso to config
>
> but after add have same error on dmesg :
>
> [44955.641391] HFSC: cake qdisc 8021: is non-work-conserving?
> [44955.958904] HFSC: cake qdisc 801C: is non-work-conserving?
>
> but work fine and will remove warrning from kernel source to not list in
> dmesg :)

I very much doubt that things will just continue to "work fine" if you
just remove the warning...

-Toke

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Martin Zaharinov]

[-- Attachment #1: Type: text/plain, Size: 1473 bytes --]

Yes Toke i know
i will make more test and will write

Best regards,
m.


On Thu, Mar 7, 2019 at 8:28 PM Toke Høiland-Jørgensen <toke@redhat.com>
wrote:

> Martin Zaharinov <micron10@gmail.com> writes:
>
> > Hi Pete
> > yes i find this patch and add to cake source
> > and in my test with hfsc and cake
> > i use
> >
> > tc qdisc add dev eth1 root handle 1: hfsc default 1
> > tc class add dev eth1 parent 1: classid 1:1 hfsc ls rate 7mbit ul rate
> 7mbit
> > tc qdisc add dev eth1 parent 1:1 cake besteffort bandwidth 5mbit
> datacentre
> > nat memlimit 32m
> >
> > user speed is 5mbit and on hfsc i add extra 2 mbit to give free limit for
> > cake.
> > and work fine but in dmesg get :
> >
> > [44835.530255] qdisc_peek_len: cake qdisc 8017: is non-work-conserving?
> > [44840.689403] HFSC: cake qdisc 8012: is non-work-conserving?
>
> Yes. As I said, you *cannot use the cake shaper* when running in under
> HFSC.
>
> > user receive 5mbit for upload and download and fine work.
> > mai be need to add no-split-gso to config
> >
> > but after add have same error on dmesg :
> >
> > [44955.641391] HFSC: cake qdisc 8021: is non-work-conserving?
> > [44955.958904] HFSC: cake qdisc 801C: is non-work-conserving?
> >
> > but work fine and will remove warrning from kernel source to not list in
> > dmesg :)
>
> I very much doubt that things will just continue to "work fine" if you
> just remove the warning...
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 2052 bytes --]

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Toke Høiland-Jørgensen]

Martin Zaharinov <micron10@gmail.com> writes:

> Yes Toke i know
> i will make more test and will write

Why do you want to run both shapers anyway?

-Toke

Re: [Cake] Bug or not Cake + hfsc or Cake + Drr

[Jonathan Morton]

> On 7 Mar, 2019, at 8:23 pm, Martin Zaharinov <micron10@gmail.com> wrote:
> 
> tc qdisc add dev eth1 root handle 1: hfsc default 1
> tc class add dev eth1 parent 1: classid 1:1 hfsc ls rate 7mbit ul rate 7mbit
> tc qdisc add dev eth1 parent 1:1 cake besteffort bandwidth 5mbit datacentre nat memlimit 32m
> 
> user speed is 5mbit and on hfsc i add extra 2 mbit to give free limit for cake.
> and work fine but in dmesg get :
> 
> [44835.530255] qdisc_peek_len: cake qdisc 8017: is non-work-conserving?
> [44840.689403] HFSC: cake qdisc 8012: is non-work-conserving?

Here's a setup that should work:

> tc qdisc add dev eth1 root handle 1: hfsc default 1
> tc class add dev eth1 parent 1: classid 1:1 hfsc ls rate 5mbit ul rate 5mbit
> tc qdisc add dev eth1 parent 1:1 cake unlimited besteffort internet nat memlimit 32m

Why don't you try that instead of editing the kernel?

(Also, don't use "datacentre" with bandwidths this low, it doesn't magically do what you're probably thinking of.)

 - Jonathan Morton