[Cake] Enforcing video quality question

[Cake] Enforcing video quality question

[Peter Lepeska]

[-- Attachment #1: Type: text/plain, Size: 283 bytes --]

Not sure this is the right forum but I am using Cake on my OpenWRT router
so I thought I would cross-post this question (
https://forum.openwrt.org/t/use-sqm-to-configure-video-quality/88984) here.

Please excuse if this is the wrong forum and thanks in advance for any help.

Peter

[-- Attachment #2: Type: text/html, Size: 446 bytes --]

Re: [Cake] Enforcing video quality question

[Toke Høiland-Jørgensen]

Peter Lepeska <bizzbyster@gmail.com> writes:

> Not sure this is the right forum but I am using Cake on my OpenWRT router
> so I thought I would cross-post this question (
> https://forum.openwrt.org/t/use-sqm-to-configure-video-quality/88984) here.

CAKE doesn't have any facilities to do per-device shaping like you're
asking, no. You could use fwmark to always put traffic into a lower
priority bin, but that still only limits in the aggregate, it's not a
per-device thing.

What you'd need is something like an HTB policy that puts each internal
IP into its own bin and caps the bandwidth of each. You could write a
.qos script and integrate it into sqm-scripts, but I don't think there's
anything already in there that would do what you want either...

-Toke

Re: [Cake] Enforcing video quality question

[Peter Lepeska]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

A user on the OpenWrt forum suggested hashlimit rules supported by
iptables. How does that idea sound to you?

Thanks!

Peter

On Thu, Feb 18, 2021 at 2:10 PM Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Peter Lepeska <bizzbyster@gmail.com> writes:
>
> > Not sure this is the right forum but I am using Cake on my OpenWRT router
> > so I thought I would cross-post this question (
> > https://forum.openwrt.org/t/use-sqm-to-configure-video-quality/88984)
> here.
>
> CAKE doesn't have any facilities to do per-device shaping like you're
> asking, no. You could use fwmark to always put traffic into a lower
> priority bin, but that still only limits in the aggregate, it's not a
> per-device thing.
>
> What you'd need is something like an HTB policy that puts each internal
> IP into its own bin and caps the bandwidth of each. You could write a
> .qos script and integrate it into sqm-scripts, but I don't think there's
> anything already in there that would do what you want either...
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1589 bytes --]

Re: [Cake] Enforcing video quality question

[Toke Høiland-Jørgensen]

Peter Lepeska <bizzbyster@gmail.com> writes:

> A user on the OpenWrt forum suggested hashlimit rules supported by
> iptables. How does that idea sound to you?

That will result in a cliff-edge policer (i.e., as soon as a device goes
over its limits it will see every packet get dropped). This doesn't
interact too well with the burstiness of TCP, so you'll likely get
erratic behaviour of the traffic if you do that. Doing the same thing
with HTB means the router will queue+shape each class (and with FQ-CoDel
on the leaves, you'll get a nice AQM behaviour as well), so that will be
smoother and less prone to bloat :)

-Toke

Re: [Cake] Enforcing video quality question

[Peter Lepeska]

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

Ok that makes sense. I'll do some reading. Thank you!

On Thu, Feb 18, 2021 at 2:28 PM Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Peter Lepeska <bizzbyster@gmail.com> writes:
>
> > A user on the OpenWrt forum suggested hashlimit rules supported by
> > iptables. How does that idea sound to you?
>
> That will result in a cliff-edge policer (i.e., as soon as a device goes
> over its limits it will see every packet get dropped). This doesn't
> interact too well with the burstiness of TCP, so you'll likely get
> erratic behaviour of the traffic if you do that. Doing the same thing
> with HTB means the router will queue+shape each class (and with FQ-CoDel
> on the leaves, you'll get a nice AQM behaviour as well), so that will be
> smoother and less prone to bloat :)
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1191 bytes --]

Re: [Cake] Enforcing video quality question

[N0man Tech]

[-- Attachment #1: Type: text/plain, Size: 1280 bytes --]

Try looking at fireqos, a component of firehol. It can do per-device limits
via a config file.

The developers provide an OpenWrt package and you can change the default
qdisc to Cake and modify Cake settings.

On Thu, Feb 18, 2021 at 2:43 PM Peter Lepeska <bizzbyster@gmail.com> wrote:

> Ok that makes sense. I'll do some reading. Thank you!
>
> On Thu, Feb 18, 2021 at 2:28 PM Toke Høiland-Jørgensen <toke@toke.dk>
> wrote:
>
>> Peter Lepeska <bizzbyster@gmail.com> writes:
>>
>> > A user on the OpenWrt forum suggested hashlimit rules supported by
>> > iptables. How does that idea sound to you?
>>
>> That will result in a cliff-edge policer (i.e., as soon as a device goes
>> over its limits it will see every packet get dropped). This doesn't
>> interact too well with the burstiness of TCP, so you'll likely get
>> erratic behaviour of the traffic if you do that. Doing the same thing
>> with HTB means the router will queue+shape each class (and with FQ-CoDel
>> on the leaves, you'll get a nice AQM behaviour as well), so that will be
>> smoother and less prone to bloat :)
>>
>> -Toke
>>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>

[-- Attachment #2: Type: text/html, Size: 2228 bytes --]

Re: [Cake] Enforcing video quality question

[John Yates]

On Thu, Feb 18, 2021 at 2:55 PM N0man Tech <n0manletter@gmail.com> wrote:
>
> Try looking at fireqos, a component of firehol. It can do per-device limits via a config file.
> The developers provide an OpenWrt package and you can change the default qdisc to Cake and modify Cake settings.

How live is this project?  Looking at github it seems moribund.

/john

Re: [Cake] Enforcing video quality question

[John Sager]

That's basically what I do. I set marks on outgoing traffic in the mangle 
table which are copied to connmark before egress. Then on ingress the 
connmark is restored to the packet and punted to ifb0 using 'action connmark 
action mirred egress redirect dev $IFB' as an ingress filter on the incoming 
interface (ppp0 in my case). Then I have HTB classes on ifb0 which set rate 
limits for different traffic classes indicated by the marks. I have only 6 
traffic classes (I bundle all video into one class), but as marks are 32 
bits wide there is lots of scope for classes for individual IP addresses.

John

On 18/02/2021 19:28, Toke Høiland-Jørgensen via Cake wrote:
> Peter Lepeska <bizzbyster@gmail.com> writes:
> 
>> A user on the OpenWrt forum suggested hashlimit rules supported by
>> iptables. How does that idea sound to you?
> 
> That will result in a cliff-edge policer (i.e., as soon as a device goes
> over its limits it will see every packet get dropped). This doesn't
> interact too well with the burstiness of TCP, so you'll likely get
> erratic behaviour of the traffic if you do that. Doing the same thing
> with HTB means the router will queue+shape each class (and with FQ-CoDel
> on the leaves, you'll get a nice AQM behaviour as well), so that will be
> smoother and less prone to bloat :)
> 
> -Toke
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
> 

Re: [Cake] Enforcing video quality question

[Peter Lepeska]

[-- Attachment #1: Type: text/plain, Size: 2003 bytes --]

Hi John

Does this result in the ability to set per internal host max ingress
bandwidth? If so, any chance you can share a snippet of a script? I will be
trying to reproduce your setup.

Thank you!

Peter

On Fri, Feb 19, 2021 at 7:16 AM John Sager <john@sager.me.uk> wrote:

> That's basically what I do. I set marks on outgoing traffic in the mangle
> table which are copied to connmark before egress. Then on ingress the
> connmark is restored to the packet and punted to ifb0 using 'action
> connmark
> action mirred egress redirect dev $IFB' as an ingress filter on the
> incoming
> interface (ppp0 in my case). Then I have HTB classes on ifb0 which set
> rate
> limits for different traffic classes indicated by the marks. I have only 6
> traffic classes (I bundle all video into one class), but as marks are 32
> bits wide there is lots of scope for classes for individual IP addresses.
>
> John
>
> On 18/02/2021 19:28, Toke Høiland-Jørgensen via Cake wrote:
> > Peter Lepeska <bizzbyster@gmail.com> writes:
> >
> >> A user on the OpenWrt forum suggested hashlimit rules supported by
> >> iptables. How does that idea sound to you?
> >
> > That will result in a cliff-edge policer (i.e., as soon as a device goes
> > over its limits it will see every packet get dropped). This doesn't
> > interact too well with the burstiness of TCP, so you'll likely get
> > erratic behaviour of the traffic if you do that. Doing the same thing
> > with HTB means the router will queue+shape each class (and with FQ-CoDel
> > on the leaves, you'll get a nice AQM behaviour as well), so that will be
> > smoother and less prone to bloat :)
> >
> > -Toke
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
> >
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>

[-- Attachment #2: Type: text/html, Size: 2872 bytes --]

Re: [Cake] Enforcing video quality question

[John Sager]

Yes. The marks are set on egress so you can select on inside IP address, 
port, protocol - in fact many characteristics that iptables rules can test 
for. I'll put together a toy iptables rules file and a toy script with the 
necessary tc commands. It'll take me a few days though as I'm busy with 
other stuff currently.

PS does the cake list allow attachments? It will be a small zip file.

John

On 19/02/2021 15:02, Peter Lepeska wrote:
> Hi John
> 
> Does this result in the ability to set per internal host max ingress 
> bandwidth? If so, any chance you can share a snippet of a script? I will be 
> trying to reproduce your setup.
> 
> Thank you!
> 
> Peter
> 
> On Fri, Feb 19, 2021 at 7:16 AM John Sager <john@sager.me.uk 
> <mailto:john@sager.me.uk>> wrote:
> 
>     That's basically what I do. I set marks on outgoing traffic in the mangle
>     table which are copied to connmark before egress. Then on ingress the
>     connmark is restored to the packet and punted to ifb0 using 'action
>     connmark
>     action mirred egress redirect dev $IFB' as an ingress filter on the
>     incoming
>     interface (ppp0 in my case). Then I have HTB classes on ifb0 which set rate
>     limits for different traffic classes indicated by the marks. I have only 6
>     traffic classes (I bundle all video into one class), but as marks are 32
>     bits wide there is lots of scope for classes for individual IP addresses.
> 
>     John
> 
>     On 18/02/2021 19:28, Toke Høiland-Jørgensen via Cake wrote:
>      > Peter Lepeska <bizzbyster@gmail.com <mailto:bizzbyster@gmail.com>>
>     writes:
>      >
>      >> A user on the OpenWrt forum suggested hashlimit rules supported by
>      >> iptables. How does that idea sound to you?
>      >
>      > That will result in a cliff-edge policer (i.e., as soon as a device goes
>      > over its limits it will see every packet get dropped). This doesn't
>      > interact too well with the burstiness of TCP, so you'll likely get
>      > erratic behaviour of the traffic if you do that. Doing the same thing
>      > with HTB means the router will queue+shape each class (and with FQ-CoDel
>      > on the leaves, you'll get a nice AQM behaviour as well), so that will be
>      > smoother and less prone to bloat :)
>      >
>      > -Toke
>      > _______________________________________________
>      > Cake mailing list
>      > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>      > https://lists.bufferbloat.net/listinfo/cake
>      >
>     _______________________________________________
>     Cake mailing list
>     Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>     https://lists.bufferbloat.net/listinfo/cake
> 

Re: [Cake] Enforcing video quality question

[Peter Lepeska]

[-- Attachment #1: Type: text/plain, Size: 3455 bytes --]

"I'll put together a toy iptables rules file and a toy script with the
necessary tc commands. "

Wow -- that would be really appreciated. Question: will you need to
explicitly call out the inside IP address of the host being shaped? Or, can
it be set to shape each inside host?

I'm not sure you can post to mailing list but maybe give it a shot (since
there may be general interest) and if it fails email me directly?

Thanks!!!

Peter

On Fri, Feb 19, 2021 at 2:04 PM John Sager <john@sager.me.uk> wrote:

> Yes. The marks are set on egress so you can select on inside IP address,
> port, protocol - in fact many characteristics that iptables rules can test
> for. I'll put together a toy iptables rules file and a toy script with the
> necessary tc commands. It'll take me a few days though as I'm busy with
> other stuff currently.
>
> PS does the cake list allow attachments? It will be a small zip file.
>
> John
>
> On 19/02/2021 15:02, Peter Lepeska wrote:
> > Hi John
> >
> > Does this result in the ability to set per internal host max ingress
> > bandwidth? If so, any chance you can share a snippet of a script? I will
> be
> > trying to reproduce your setup.
> >
> > Thank you!
> >
> > Peter
> >
> > On Fri, Feb 19, 2021 at 7:16 AM John Sager <john@sager.me.uk
> > <mailto:john@sager.me.uk>> wrote:
> >
> >     That's basically what I do. I set marks on outgoing traffic in the
> mangle
> >     table which are copied to connmark before egress. Then on ingress the
> >     connmark is restored to the packet and punted to ifb0 using 'action
> >     connmark
> >     action mirred egress redirect dev $IFB' as an ingress filter on the
> >     incoming
> >     interface (ppp0 in my case). Then I have HTB classes on ifb0 which
> set rate
> >     limits for different traffic classes indicated by the marks. I have
> only 6
> >     traffic classes (I bundle all video into one class), but as marks
> are 32
> >     bits wide there is lots of scope for classes for individual IP
> addresses.
> >
> >     John
> >
> >     On 18/02/2021 19:28, Toke Høiland-Jørgensen via Cake wrote:
> >      > Peter Lepeska <bizzbyster@gmail.com <mailto:bizzbyster@gmail.com
> >>
> >     writes:
> >      >
> >      >> A user on the OpenWrt forum suggested hashlimit rules supported
> by
> >      >> iptables. How does that idea sound to you?
> >      >
> >      > That will result in a cliff-edge policer (i.e., as soon as a
> device goes
> >      > over its limits it will see every packet get dropped). This
> doesn't
> >      > interact too well with the burstiness of TCP, so you'll likely get
> >      > erratic behaviour of the traffic if you do that. Doing the same
> thing
> >      > with HTB means the router will queue+shape each class (and with
> FQ-CoDel
> >      > on the leaves, you'll get a nice AQM behaviour as well), so that
> will be
> >      > smoother and less prone to bloat :)
> >      >
> >      > -Toke
> >      > _______________________________________________
> >      > Cake mailing list
> >      > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
> >      > https://lists.bufferbloat.net/listinfo/cake
> >      >
> >     _______________________________________________
> >     Cake mailing list
> >     Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
> >     https://lists.bufferbloat.net/listinfo/cake
> >
>

[-- Attachment #2: Type: text/html, Size: 4996 bytes --]

Re: [Cake] Enforcing video quality question

[John Sager]

You will need to specify the hosts explicitly, unless you can live with them 
all sharing one bandwidth class. In that case if you have more than one 
using bandwidth they would share the bandwidth in that class equally. I 
assume from your original post that you want each host to be limited in 
bandwidth to a specific value, but to do that you need a class for each host 
in the ingress HTB. What you probably need is a scheduler that has a limit 
per flow up to an overall ceiling beyond which it shares equally. I'm not 
aware that any of the schedulers do anything like that.

John

On 19/02/2021 20:33, Peter Lepeska wrote:
> "I'll put together a toy iptables rules file and a toy script with the
> necessary tc commands. "
> 
> Wow -- that would be really appreciated. Question: will you need to 
> explicitly call out the inside IP address of the host being shaped? Or, can 
> it be set to shape each inside host?
> 
> I'm not sure you can post to mailing list but maybe give it a shot (since 
> there may be general interest) and if it fails email me directly?
> 
> Thanks!!!
> 
> Peter
> 
> On Fri, Feb 19, 2021 at 2:04 PM John Sager <john@sager.me.uk 
> <mailto:john@sager.me.uk>> wrote:
> 
>     Yes. The marks are set on egress so you can select on inside IP address,
>     port, protocol - in fact many characteristics that iptables rules can test
>     for. I'll put together a toy iptables rules file and a toy script with the
>     necessary tc commands. It'll take me a few days though as I'm busy with
>     other stuff currently.
> 
>     PS does the cake list allow attachments? It will be a small zip file.
> 
>     John
> 
>     On 19/02/2021 15:02, Peter Lepeska wrote:
>      > Hi John
>      >
>      > Does this result in the ability to set per internal host max ingress
>      > bandwidth? If so, any chance you can share a snippet of a script? I
>     will be
>      > trying to reproduce your setup.
>      >
>      > Thank you!
>      >
>      > Peter
>      >
>      > On Fri, Feb 19, 2021 at 7:16 AM John Sager <john@sager.me.uk
>     <mailto:john@sager.me.uk>
>      > <mailto:john@sager.me.uk <mailto:john@sager.me.uk>>> wrote:
>      >
>      >     That's basically what I do. I set marks on outgoing traffic in
>     the mangle
>      >     table which are copied to connmark before egress. Then on ingress the
>      >     connmark is restored to the packet and punted to ifb0 using 'action
>      >     connmark
>      >     action mirred egress redirect dev $IFB' as an ingress filter on the
>      >     incoming
>      >     interface (ppp0 in my case). Then I have HTB classes on ifb0
>     which set rate
>      >     limits for different traffic classes indicated by the marks. I
>     have only 6
>      >     traffic classes (I bundle all video into one class), but as marks
>     are 32
>      >     bits wide there is lots of scope for classes for individual IP
>     addresses.
>      >
>      >     John
>      >
>      >     On 18/02/2021 19:28, Toke Høiland-Jørgensen via Cake wrote:
>      >      > Peter Lepeska <bizzbyster@gmail.com
>     <mailto:bizzbyster@gmail.com> <mailto:bizzbyster@gmail.com
>     <mailto:bizzbyster@gmail.com>>>
>      >     writes:
>      >      >
>      >      >> A user on the OpenWrt forum suggested hashlimit rules
>     supported by
>      >      >> iptables. How does that idea sound to you?
>      >      >
>      >      > That will result in a cliff-edge policer (i.e., as soon as a
>     device goes
>      >      > over its limits it will see every packet get dropped). This
>     doesn't
>      >      > interact too well with the burstiness of TCP, so you'll likely get
>      >      > erratic behaviour of the traffic if you do that. Doing the
>     same thing
>      >      > with HTB means the router will queue+shape each class (and
>     with FQ-CoDel
>      >      > on the leaves, you'll get a nice AQM behaviour as well), so
>     that will be
>      >      > smoother and less prone to bloat :)
>      >      >
>      >      > -Toke
>      >      > _______________________________________________
>      >      > Cake mailing list
>      >      > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>     <mailto:Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>>
>      >      > https://lists.bufferbloat.net/listinfo/cake
>      >      >
>      >     _______________________________________________
>      >     Cake mailing list
>      > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>     <mailto:Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>>
>      > https://lists.bufferbloat.net/listinfo/cake
>      >
> 

Re: [Cake] Enforcing video quality question

[Jeremy Marks]

Learning how to have this level of control on incoming traffic is really interesting. I would be very interested in a looking at an example script, if it is possible to attach a text file or a tiny instructive post on the CAKE site. 



On Feb 19, 2021, at 5:06 PM, John Sager <john@sager.me.uk> wrote:

You will need to specify the hosts explicitly, unless you can live with them all sharing one bandwidth class. In that case if you have more than one using bandwidth they would share the bandwidth in that class equally. I assume from your original post that you want each host to be limited in bandwidth to a specific value, but to do that you need a class for each host in the ingress HTB. What you probably need is a scheduler that has a limit per flow up to an overall ceiling beyond which it shares equally. I'm not aware that any of the schedulers do anything like that.

John

On 19/02/2021 20:33, Peter Lepeska wrote:
> "I'll put together a toy iptables rules file and a toy script with the
> necessary tc commands. "
> Wow -- that would be really appreciated. Question: will you need to explicitly call out the inside IP address of the host being shaped? Or, can it be set to shape each inside host?
> I'm not sure you can post to mailing list but maybe give it a shot (since there may be general interest) and if it fails email me directly?
> Thanks!!!
> Peter
> On Fri, Feb 19, 2021 at 2:04 PM John Sager <john@sager.me.uk <mailto:john@sager.me.uk>> wrote:
>    Yes. The marks are set on egress so you can select on inside IP address,
>    port, protocol - in fact many characteristics that iptables rules can test
>    for. I'll put together a toy iptables rules file and a toy script with the
>    necessary tc commands. It'll take me a few days though as I'm busy with
>    other stuff currently.
>    PS does the cake list allow attachments? It will be a small zip file.
>    John
>    On 19/02/2021 15:02, Peter Lepeska wrote:
>     > Hi John
>     >
>     > Does this result in the ability to set per internal host max ingress
>     > bandwidth? If so, any chance you can share a snippet of a script? I
>    will be
>     > trying to reproduce your setup.
>     >
>     > Thank you!
>     >
>     > Peter
>     >
>     > On Fri, Feb 19, 2021 at 7:16 AM John Sager <john@sager.me.uk
>    <mailto:john@sager.me.uk>
>     > <mailto:john@sager.me.uk <mailto:john@sager.me.uk>>> wrote:
>     >
>     >     That's basically what I do. I set marks on outgoing traffic in
>    the mangle
>     >     table which are copied to connmark before egress. Then on ingress the
>     >     connmark is restored to the packet and punted to ifb0 using 'action
>     >     connmark
>     >     action mirred egress redirect dev $IFB' as an ingress filter on the
>     >     incoming
>     >     interface (ppp0 in my case). Then I have HTB classes on ifb0
>    which set rate
>     >     limits for different traffic classes indicated by the marks. I
>    have only 6
>     >     traffic classes (I bundle all video into one class), but as marks
>    are 32
>     >     bits wide there is lots of scope for classes for individual IP
>    addresses.
>     >
>     >     John
>     >
>     >     On 18/02/2021 19:28, Toke Høiland-Jørgensen via Cake wrote:
>     >      > Peter Lepeska <bizzbyster@gmail.com
>    <mailto:bizzbyster@gmail.com> <mailto:bizzbyster@gmail.com
>    <mailto:bizzbyster@gmail.com>>>
>     >     writes:
>     >      >
>     >      >> A user on the OpenWrt forum suggested hashlimit rules
>    supported by
>     >      >> iptables. How does that idea sound to you?
>     >      >
>     >      > That will result in a cliff-edge policer (i.e., as soon as a
>    device goes
>     >      > over its limits it will see every packet get dropped). This
>    doesn't
>     >      > interact too well with the burstiness of TCP, so you'll likely get
>     >      > erratic behaviour of the traffic if you do that. Doing the
>    same thing
>     >      > with HTB means the router will queue+shape each class (and
>    with FQ-CoDel
>     >      > on the leaves, you'll get a nice AQM behaviour as well), so
>    that will be
>     >      > smoother and less prone to bloat :)
>     >      >
>     >      > -Toke
>     >      > _______________________________________________
>     >      > Cake mailing list
>     >      > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>    <mailto:Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>>
>     >      > https://lists.bufferbloat.net/listinfo/cake
>     >      >
>     >     _______________________________________________
>     >     Cake mailing list
>     > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>    <mailto:Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>>
>     > https://lists.bufferbloat.net/listinfo/cake
>     >
_______________________________________________
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] Enforcing video quality question

[Toke Høiland-Jørgensen]

John Sager <john@sager.me.uk> writes:

> You will need to specify the hosts explicitly, unless you can live with them 
> all sharing one bandwidth class. In that case if you have more than one 
> using bandwidth they would share the bandwidth in that class equally. I 
> assume from your original post that you want each host to be limited in 
> bandwidth to a specific value, but to do that you need a class for each host 
> in the ingress HTB.

Just do enough classes that you can cover the whole IP space? At least
for IPv4 that's trivial; for IPv6 you'll probably need to hash and hope
that there are not too many collisions...

> What you probably need is a scheduler that has a limit per flow up to
> an overall ceiling beyond which it shares equally. I'm not aware that
> any of the schedulers do anything like that.

If you use FQ-CoDel as the leaf qdisc in HTB you'll get flow scheduling
to each host. There won't be a per-flow *limit*, but you'll get nice
scheduling of all flows going towards each host.

-Toke

Re: [Cake] Enforcing video quality question

[John Sager]

On 20/02/2021 11:53, Toke Høiland-Jørgensen wrote:
> John Sager <john@sager.me.uk> writes:
> 
>> You will need to specify the hosts explicitly, unless you can live with them
>> all sharing one bandwidth class. In that case if you have more than one
>> using bandwidth they would share the bandwidth in that class equally. I
>> assume from your original post that you want each host to be limited in
>> bandwidth to a specific value, but to do that you need a class for each host
>> in the ingress HTB.
> 
> Just do enough classes that you can cover the whole IP space? At least
> for IPv4 that's trivial; for IPv6 you'll probably need to hash and hope
> that there are not too many collisions...

Thinking about that, one could set up, say 16 classes for 16 marks and 
generate the marks using the HMARK target. That could hash on src,dst and 
include the ports if necessary. Then the connections would distribute across 
the HTB classes. However one video connection would generate multiple flows 
(DNS, metadata, etc before & perhaps during the video flow) so simultaneous 
video sessions from several users would likely interfere with each other.

My current solution marks on source IP address or MAC address so all the 
traffic for one host goes into one class.

John

> 
>> What you probably need is a scheduler that has a limit per flow up to
>> an overall ceiling beyond which it shares equally. I'm not aware that
>> any of the schedulers do anything like that.
> 
> If you use FQ-CoDel as the leaf qdisc in HTB you'll get flow scheduling
> to each host. There won't be a per-flow *limit*, but you'll get nice
> scheduling of all flows going towards each host.
> 
> -Toke
> 

Re: [Cake] Enforcing video quality question

[Toke Høiland-Jørgensen]

John Sager <john@sager.me.uk> writes:

> PS does the cake list allow attachments? It will be a small zip file.

It'll likely end up in the moderation queue (don't recall the size
limit), but I can manually approve it from there...

-Toke

Re: [Cake] Enforcing video quality question

[John Sager]

[-- Attachment #1: Type: text/plain, Size: 3464 bytes --]

Here is the toy QoS solution for linux, which is a simplified version of the 
one I uses successfully on my home network. It uses quite a few scheduler 
modules - sch_htb, act_connmark, em_meta etc - that may not load 
automatically, so they may need to be listed in /etc/modules.

Toke, thanks for agreeing to let the attachment through.

John

On 19/02/2021 19:04, John Sager wrote:
> Yes. The marks are set on egress so you can select on inside IP address, 
> port, protocol - in fact many characteristics that iptables rules can test 
> for. I'll put together a toy iptables rules file and a toy script with the 
> necessary tc commands. It'll take me a few days though as I'm busy with 
> other stuff currently.
> 
> PS does the cake list allow attachments? It will be a small zip file.
> 
> John
> 
> On 19/02/2021 15:02, Peter Lepeska wrote:
>> Hi John
>>
>> Does this result in the ability to set per internal host max ingress 
>> bandwidth? If so, any chance you can share a snippet of a script? I will 
>> be trying to reproduce your setup.
>>
>> Thank you!
>>
>> Peter
>>
>> On Fri, Feb 19, 2021 at 7:16 AM John Sager <john@sager.me.uk 
>> <mailto:john@sager.me.uk>> wrote:
>>
>>     That's basically what I do. I set marks on outgoing traffic in the mangle
>>     table which are copied to connmark before egress. Then on ingress the
>>     connmark is restored to the packet and punted to ifb0 using 'action
>>     connmark
>>     action mirred egress redirect dev $IFB' as an ingress filter on the
>>     incoming
>>     interface (ppp0 in my case). Then I have HTB classes on ifb0 which set 
>> rate
>>     limits for different traffic classes indicated by the marks. I have 
>> only 6
>>     traffic classes (I bundle all video into one class), but as marks are 32
>>     bits wide there is lots of scope for classes for individual IP addresses.
>>
>>     John
>>
>>     On 18/02/2021 19:28, Toke Høiland-Jørgensen via Cake wrote:
>>      > Peter Lepeska <bizzbyster@gmail.com <mailto:bizzbyster@gmail.com>>
>>     writes:
>>      >
>>      >> A user on the OpenWrt forum suggested hashlimit rules supported by
>>      >> iptables. How does that idea sound to you?
>>      >
>>      > That will result in a cliff-edge policer (i.e., as soon as a device 
>> goes
>>      > over its limits it will see every packet get dropped). This doesn't
>>      > interact too well with the burstiness of TCP, so you'll likely get
>>      > erratic behaviour of the traffic if you do that. Doing the same thing
>>      > with HTB means the router will queue+shape each class (and with 
>> FQ-CoDel
>>      > on the leaves, you'll get a nice AQM behaviour as well), so that 
>> will be
>>      > smoother and less prone to bloat :)
>>      >
>>      > -Toke
>>      > _______________________________________________
>>      > Cake mailing list
>>      > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>>      > https://lists.bufferbloat.net/listinfo/cake
>>      >
>>     _______________________________________________
>>     Cake mailing list
>>     Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>>     https://lists.bufferbloat.net/listinfo/cake
>>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
> 

[-- Attachment #2: linux_toy_qos.zip --]
[-- Type: application/zip, Size: 4508 bytes --]

Re: [Cake] Enforcing video quality question

[Peter Lepeska]

[-- Attachment #1: Type: text/plain, Size: 3877 bytes --]

Hi John,

Thanks for sharing!

Peter

On Tue, Feb 23, 2021 at 6:15 AM John Sager <john@sager.me.uk> wrote:

> Here is the toy QoS solution for linux, which is a simplified version of
> the
> one I uses successfully on my home network. It uses quite a few scheduler
> modules - sch_htb, act_connmark, em_meta etc - that may not load
> automatically, so they may need to be listed in /etc/modules.
>
> Toke, thanks for agreeing to let the attachment through.
>
> John
>
> On 19/02/2021 19:04, John Sager wrote:
> > Yes. The marks are set on egress so you can select on inside IP address,
> > port, protocol - in fact many characteristics that iptables rules can
> test
> > for. I'll put together a toy iptables rules file and a toy script with
> the
> > necessary tc commands. It'll take me a few days though as I'm busy with
> > other stuff currently.
> >
> > PS does the cake list allow attachments? It will be a small zip file.
> >
> > John
> >
> > On 19/02/2021 15:02, Peter Lepeska wrote:
> >> Hi John
> >>
> >> Does this result in the ability to set per internal host max ingress
> >> bandwidth? If so, any chance you can share a snippet of a script? I
> will
> >> be trying to reproduce your setup.
> >>
> >> Thank you!
> >>
> >> Peter
> >>
> >> On Fri, Feb 19, 2021 at 7:16 AM John Sager <john@sager.me.uk
> >> <mailto:john@sager.me.uk>> wrote:
> >>
> >>     That's basically what I do. I set marks on outgoing traffic in the
> mangle
> >>     table which are copied to connmark before egress. Then on ingress
> the
> >>     connmark is restored to the packet and punted to ifb0 using 'action
> >>     connmark
> >>     action mirred egress redirect dev $IFB' as an ingress filter on the
> >>     incoming
> >>     interface (ppp0 in my case). Then I have HTB classes on ifb0 which
> set
> >> rate
> >>     limits for different traffic classes indicated by the marks. I have
> >> only 6
> >>     traffic classes (I bundle all video into one class), but as marks
> are 32
> >>     bits wide there is lots of scope for classes for individual IP
> addresses.
> >>
> >>     John
> >>
> >>     On 18/02/2021 19:28, Toke Høiland-Jørgensen via Cake wrote:
> >>      > Peter Lepeska <bizzbyster@gmail.com <mailto:bizzbyster@gmail.com
> >>
> >>     writes:
> >>      >
> >>      >> A user on the OpenWrt forum suggested hashlimit rules supported
> by
> >>      >> iptables. How does that idea sound to you?
> >>      >
> >>      > That will result in a cliff-edge policer (i.e., as soon as a
> device
> >> goes
> >>      > over its limits it will see every packet get dropped). This
> doesn't
> >>      > interact too well with the burstiness of TCP, so you'll likely
> get
> >>      > erratic behaviour of the traffic if you do that. Doing the same
> thing
> >>      > with HTB means the router will queue+shape each class (and with
> >> FQ-CoDel
> >>      > on the leaves, you'll get a nice AQM behaviour as well), so that
> >> will be
> >>      > smoother and less prone to bloat :)
> >>      >
> >>      > -Toke
> >>      > _______________________________________________
> >>      > Cake mailing list
> >>      > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
> >>      > https://lists.bufferbloat.net/listinfo/cake
> >>      >
> >>     _______________________________________________
> >>     Cake mailing list
> >>     Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
> >>     https://lists.bufferbloat.net/listinfo/cake
> >>
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
> >
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>

[-- Attachment #2: Type: text/html, Size: 5967 bytes --]

Re: [Cake] Enforcing video quality question

[Jeremy Marks]

Thank you very much!

---Original Message-----
From: Cake <cake-bounces@lists.bufferbloat.net> On Behalf Of John Sager
Sent: Tuesday, February 23, 2021 6:15 AM
To: cake@lists.bufferbloat.net
Subject: Re: [Cake] Enforcing video quality question

Here is the toy QoS solution for linux, which is a simplified version of the one I uses successfully on my home network. It uses quite a few scheduler modules - sch_htb, act_connmark, em_meta etc - that may not load automatically, so they may need to be listed in /etc/modules.

Toke, thanks for agreeing to let the attachment through.

John

On 19/02/2021 19:04, John Sager wrote:
> Yes. The marks are set on egress so you can select on inside IP 
> address, port, protocol - in fact many characteristics that iptables 
> rules can test for. I'll put together a toy iptables rules file and a 
> toy script with the necessary tc commands. It'll take me a few days 
> though as I'm busy with other stuff currently.
> 
> PS does the cake list allow attachments? It will be a small zip file.
> 
> John
> 
> On 19/02/2021 15:02, Peter Lepeska wrote:
>> Hi John
>>
>> Does this result in the ability to set per internal host max ingress 
>> bandwidth? If so, any chance you can share a snippet of a script? I 
>> will be trying to reproduce your setup.
>>
>> Thank you!
>>
>> Peter
>>
>> On Fri, Feb 19, 2021 at 7:16 AM John Sager <john@sager.me.uk 
>> <mailto:john@sager.me.uk>> wrote:
>>
>>     That's basically what I do. I set marks on outgoing traffic in 
>> the mangle
>>     table which are copied to connmark before egress. Then on ingress 
>> the
>>     connmark is restored to the packet and punted to ifb0 using 
>> 'action
>>     connmark
>>     action mirred egress redirect dev $IFB' as an ingress filter on 
>> the
>>     incoming
>>     interface (ppp0 in my case). Then I have HTB classes on ifb0 
>> which set rate
>>     limits for different traffic classes indicated by the marks. I 
>> have only 6
>>     traffic classes (I bundle all video into one class), but as marks 
>> are 32
>>     bits wide there is lots of scope for classes for individual IP addresses.
>>
>>     John
>>
>>     On 18/02/2021 19:28, Toke Høiland-Jørgensen via Cake wrote:
>>      > Peter Lepeska <bizzbyster@gmail.com 
>> <mailto:bizzbyster@gmail.com>>
>>     writes:
>>      >
>>      >> A user on the OpenWrt forum suggested hashlimit rules 
>> supported by
>>      >> iptables. How does that idea sound to you?
>>      >
>>      > That will result in a cliff-edge policer (i.e., as soon as a 
>> device goes
>>      > over its limits it will see every packet get dropped). This 
>> doesn't
>>      > interact too well with the burstiness of TCP, so you'll likely 
>> get
>>      > erratic behaviour of the traffic if you do that. Doing the 
>> same thing
>>      > with HTB means the router will queue+shape each class (and 
>> with FQ-CoDel
>>      > on the leaves, you'll get a nice AQM behaviour as well), so 
>> that will be
>>      > smoother and less prone to bloat :)
>>      >
>>      > -Toke
>>      > _______________________________________________
>>      > Cake mailing list
>>      > Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>>      > https://lists.bufferbloat.net/listinfo/cake
>>      >
>>     _______________________________________________
>>     Cake mailing list
>>     Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>>     https://lists.bufferbloat.net/listinfo/cake
>>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>