[Cerowrt-devel] squash/ignore DSCP and mangle table questions

[Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

[-- Attachment #1: Type: text/plain, Size: 4258 bytes --]

Apologies if this is the wrong list, but cerowrt-users is empty and I'm not
sure if I should spam the codel or sqm related lists.

I can't quite wrap my head around how the "Squash DSCP on inbound packets
(ingress):" and "Ignore DSCP on ingress" work under the SQM Luci page and
how this might affect my rules for --set-dscp

I'm trying to flag all packets on a certain port to/from a certain
interface (sw10) as DSCP 0x2e (or EF), which WMM should set as VoIP (for
mumble voice traffic, which the mobile phone app or the server doesn't seem
to set properly so the phone thinks it's regular traffic)

I took a look at the simple.qos script, as well as the mangle table but I'm
still a bit confused. Here's the relevant snippet from 'iptables -S -t
mangle' with it set to "DO NOT SQUASH" and "ALLOW"

-N QOS_MARK_ge00
-N fwmark
-A PREROUTING -i vtun+ -p tcp -j MARK --set-xmark 0x2/0xff
-A PREROUTING -i ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
-A PREROUTING -j fwmark
-A POSTROUTING -o ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
-A QOS_MARK_ge00 -j MARK --set-xmark 0x2/0xff
-A QOS_MARK_ge00 -m dscp --dscp 0x08 -j MARK --set-xmark 0x3/0xff
-A QOS_MARK_ge00 -m dscp --dscp 0x30 -j MARK --set-xmark 0x1/0xff
-A QOS_MARK_ge00 -m dscp --dscp 0x2e -j MARK --set-xmark 0x1/0xff
-A QOS_MARK_ge00 -m dscp --dscp 0x24 -j MARK --set-xmark 0x1/0xff
-A QOS_MARK_ge00 -m tos --tos 0x10/0x3f -j MARK --set-xmark 0x1/0xff

and the full iptables -vnL -t mangle output is here (to reduce spam):
https://pastebin.com/raw.php?i=uba2n1qw

From what I understand, this is just MARKing all incoming packets from ge00
for use in fq_codel based on it's DSCP tag? But does it change the DSCP
when coming in and going out? Also curious why this isn't enabled by
default, does it cause excessive CPU load? It seems like a gain to me.

Anyways, back to my other question. I'm trying to flag tcp/udp port 6484
going to/from sw10 and these rules seems to work

-t mangle -I <table> -i sw10 -p udp -m udp --dport 6484 -m comment
--comment "reclassify mumble traffic as VoIP" -j DSCP --set-dscp 0x2e
-t mangle -I <table> -i sw10 -p tcp -m tcp --dport 6484 -m comment
--comment "reclassify mumble traffic as VoIP" -j DSCP --set-dscp 0x2e
-t mangle -I <table> -o sw10 -p udp -m udp --sport 6484 -m comment
--comment "reclassify mumble traffic as VoIP" -j DSCP --set-dscp 0x2e
-t mangle -I <table> -o sw10 -p tcp -m tcp --sport 6484 -m comment
--comment "reclassify mumble traffic as VoIP" -j DSCP --set-dscp 0x2e

However, I'm a bit uncertain which mangle zone to insert them into. My
first thought would be to put them in the mangle FOWARD chain since it's
traffic passing through the router, which I made two new zones to jump
through if it sees traffic to/from sw10 but I'm not entirely certain it was
working in the FORWARD chain. I did see pkts and bytes counting upward but
I'm not sure it was catching ALL the traffic, or how it is related to
WMM/WME which I need to function properly. next I tried the mangle
PREROUTING, mangle FORWARD, and mangle POSTROUTING chains, which all 3
showed pkts and bytes but the most were in FORWARD. this is probably
redundant so perhaps it only needs PREROUTING/POSTROUTING? or only FORWARD?

Also, the "fwmark" chain doesn't appear to do anything unless I'm
misunderstanding it. I don't see anything else referencing the chain. what
is it's purpose? should this have the same MARKing rules as the
QOS_MARK_ge00 table, but for forwarding? Or does marking in the PREROUTING
table automatically apply to FORWARD traffic making that redundant?

Chain PREROUTING (policy ACCEPT 425K packets, 395M bytes)
 pkts bytes target     prot opt in     out     source
destination
    0     0 MARK       tcp  --  vtun+  *       0.0.0.0/0
0.0.0.0/0            MARK xset 0x2/0xff
1705K 2048M QOS_MARK_ge00  all  --  ge00   *       0.0.0.0/0
0.0.0.0/0           [goto]  mark match 0x0/0xff
 188K   86M fwmark     all  --  *      *       0.0.0.0/0
0.0.0.0/0


Thanks in advance, and I appreciate all the work Dave and everyone else
here is doing to improve the internet. I understand Cero is more of a proof
of concept and testing tool, but I like to stay more on the bleeding edge
of this sort of stuff due to my highly latency sensitive traffic
requirements.

[-- Attachment #2: Type: text/html, Size: 5555 bytes --]

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[Sebastian Moeller]

Hi,


On Apr 13, 2015, at 08:37 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:

> Apologies if this is the wrong list, but cerowrt-users is empty and I'm not sure if I should spam the codel or sqm related lists.
> 
> I can't quite wrap my head around how the "Squash DSCP on inbound packets (ingress):" and "Ignore DSCP on ingress" work under the SQM Luci page and how this might affect my rules for --set-dscp

	"Ignore DSCP on ingress”, this basically uses an 1-tier HTB shaper on ingress, DSCP marks in the packets are simply ignored, but not touched.
	"Squash DSCP on inbound packets (ingress):” this will replace all DSCP marks with 0x0 (I believe), but only after the ingress qdisc.
In essence this means you can actually interpret ingress DSCP marks from upstream ("Ignore DSCP on ingress” set to ALLOW) but wipe them after the ingress shaping (with "Squash DSCP on inbound packets (ingress);” active). So the default should be “Ignore DSCP on ingress” and Squash (the second can be argued, as long as no one bases routing decisions on the marks they do not hurt). The rest of your questions are beyond my expertise...

Best Regards
	Sebastian

> 
> I'm trying to flag all packets on a certain port to/from a certain interface (sw10) as DSCP 0x2e (or EF), which WMM should set as VoIP (for mumble voice traffic, which the mobile phone app or the server doesn't seem to set properly so the phone thinks it's regular traffic)
> 
> I took a look at the simple.qos script, as well as the mangle table but I'm still a bit confused. Here's the relevant snippet from 'iptables -S -t mangle' with it set to "DO NOT SQUASH" and "ALLOW"
> 
> -N QOS_MARK_ge00
> -N fwmark
> -A PREROUTING -i vtun+ -p tcp -j MARK --set-xmark 0x2/0xff
> -A PREROUTING -i ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
> -A PREROUTING -j fwmark
> -A POSTROUTING -o ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
> -A QOS_MARK_ge00 -j MARK --set-xmark 0x2/0xff
> -A QOS_MARK_ge00 -m dscp --dscp 0x08 -j MARK --set-xmark 0x3/0xff
> -A QOS_MARK_ge00 -m dscp --dscp 0x30 -j MARK --set-xmark 0x1/0xff
> -A QOS_MARK_ge00 -m dscp --dscp 0x2e -j MARK --set-xmark 0x1/0xff
> -A QOS_MARK_ge00 -m dscp --dscp 0x24 -j MARK --set-xmark 0x1/0xff
> -A QOS_MARK_ge00 -m tos --tos 0x10/0x3f -j MARK --set-xmark 0x1/0xff
> 
> and the full iptables -vnL -t mangle output is here (to reduce spam): https://pastebin.com/raw.php?i=uba2n1qw
> 
> From what I understand, this is just MARKing all incoming packets from ge00 for use in fq_codel based on it's DSCP tag? But does it change the DSCP when coming in and going out? Also curious why this isn't enabled by default, does it cause excessive CPU load? It seems like a gain to me.
> 
> Anyways, back to my other question. I'm trying to flag tcp/udp port 6484 going to/from sw10 and these rules seems to work
> 
> -t mangle -I <table> -i sw10 -p udp -m udp --dport 6484 -m comment --comment "reclassify mumble traffic as VoIP" -j DSCP --set-dscp 0x2e
> -t mangle -I <table> -i sw10 -p tcp -m tcp --dport 6484 -m comment --comment "reclassify mumble traffic as VoIP" -j DSCP --set-dscp 0x2e
> -t mangle -I <table> -o sw10 -p udp -m udp --sport 6484 -m comment --comment "reclassify mumble traffic as VoIP" -j DSCP --set-dscp 0x2e
> -t mangle -I <table> -o sw10 -p tcp -m tcp --sport 6484 -m comment --comment "reclassify mumble traffic as VoIP" -j DSCP --set-dscp 0x2e
> 
> However, I'm a bit uncertain which mangle zone to insert them into. My first thought would be to put them in the mangle FOWARD chain since it's traffic passing through the router, which I made two new zones to jump through if it sees traffic to/from sw10 but I'm not entirely certain it was working in the FORWARD chain. I did see pkts and bytes counting upward but I'm not sure it was catching ALL the traffic, or how it is related to WMM/WME which I need to function properly. next I tried the mangle PREROUTING, mangle FORWARD, and mangle POSTROUTING chains, which all 3 showed pkts and bytes but the most were in FORWARD. this is probably redundant so perhaps it only needs PREROUTING/POSTROUTING? or only FORWARD?
> 
> Also, the "fwmark" chain doesn't appear to do anything unless I'm misunderstanding it. I don't see anything else referencing the chain. what is it's purpose? should this have the same MARKing rules as the QOS_MARK_ge00 table, but for forwarding? Or does marking in the PREROUTING table automatically apply to FORWARD traffic making that redundant?
> 
> Chain PREROUTING (policy ACCEPT 425K packets, 395M bytes)
>  pkts bytes target     prot opt in     out     source               destination
>     0     0 MARK       tcp  --  vtun+  *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x2/0xff
> 1705K 2048M QOS_MARK_ge00  all  --  ge00   *       0.0.0.0/0            0.0.0.0/0           [goto]  mark match 0x0/0xff
>  188K   86M fwmark     all  --  *      *       0.0.0.0/0            0.0.0.0/0
> 
> 
> Thanks in advance, and I appreciate all the work Dave and everyone else here is doing to improve the internet. I understand Cero is more of a proof of concept and testing tool, but I like to stay more on the bleeding edge of this sort of stuff due to my highly latency sensitive traffic requirements.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

[-- Attachment #1: Type: text/plain, Size: 3080 bytes --]

Thanks for the reply,


On Mon, Apr 13, 2015 at 3:36 AM, Sebastian Moeller <moeller0@gmx.de> wrote:

>         "Ignore DSCP on ingress”, this basically uses an 1-tier HTB
shaper on ingress, DSCP marks in the packets are simply ignored, but not
touched.

So if I understand this correctly, this means they're kind of in
passthrough mode on ignore but the qdiscs don't act on them due to no
iptables MARKing? And If *not* ignoring DSCP on ingress, they're still
passed through untouched but prioritized internally with the qdiscs?

>         "Squash DSCP on inbound packets (ingress):” this will replace all
DSCP marks with 0x0 (I believe), but only after the ingress qdisc.
> In essence this means you can actually interpret ingress DSCP marks from
upstream ("Ignore DSCP on ingress” set to ALLOW) but wipe them after the
ingress shaping (with "Squash DSCP on inbound packets (ingress);” active).
So the default should be “Ignore DSCP on ingress” and Squash (the second
can be argued, as long as no one bases routing decisions on the marks they
do not hurt). The rest of your questions are beyond my expertise...

Hmm, why would we want to remove all DSCP on output then? I assume many
ISPs and routers will squash them anyways, but wouldn't it serve *some*
purpose to differentiate between different traffic types?


Also, regarding my mangling questions I think I figured out the correct
chains to insert my rules in. mangle PREROUTING and mangle POSTROUTING
because both of those have a jump at the end to a chain that MARKs for
qdiscs based on DSCP values. They appear to be working since I see the MARK
rules counting upward. I assume PRE and POST go through FORWARD anyways so
it's redundant, and using FORWARD only didn't appear to be working. I've
ended up with this:

# Mangle sw10 packets to reclassify mumble traffic. Hopefully this passes
through DSCP so WMM can recognize 0x2e as VoIP traffic
iptables -t mangle -N RECLASS_sw10
iptables -t mangle -A RECLASS_sw10 -p tcp -m tcp --sport 6484  -j DSCP
--set-dscp 0x2e
iptables -t mangle -A RECLASS_sw10 -p tcp -m tcp --dport 6484  -j DSCP
--set-dscp 0x2e
iptables -t mangle -A RECLASS_sw10 -p udp -m udp --sport 6484 -j DSCP
--set-dscp 0x2e
iptables -t mangle -A RECLASS_sw10 -p udp -m udp --dport 6484 -j DSCP
--set-dscp 0x2e
iptables -t mangle -I PREROUTING -i sw10 -j RECLASS_sw10
iptables -t mangle -I POSTROUTING -o sw10 -j RECLASS_sw10

# Mangle gw00 to reclassify guest as low priority
iptables -t mangle -N RECLASS_gw00
iptables -t mangle -A RECLASS_gw00 -j DSCP --set-dscp 0x08
iptables -t mangle -I PREROUTING -i gw00 -j RECLASS_gw00
iptables -t mangle -I POSTROUTING -o gw00 -j RECLASS_gw00


However, I'm not sure how the port specific rules handle
RELATED,ESTABLISHED because I believe the initial connection is done on
port 6484 but then it uses a random high port. I'm not sure how I could add
a RELATED,ESTABLISHED match to connections on port 6484 to my custom zone
that reclassifies to 0x2e to catch *all* traffic related to it.

[-- Attachment #2: Type: text/html, Size: 3493 bytes --]

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

[-- Attachment #1: Type: text/plain, Size: 4003 bytes --]

Actually I think I figured out the last part,

iptables -t mangle -A RECLASS_sw10 -p tcp -m tcp --sport 6484 -m conntrack
--ctstate NEW,RELATED,ESTABLISHED -j DSCP --set-dscp 0x2e
iptables -t mangle -A RECLASS_sw10 -p tcp -m tcp --dport 6484 -m conntrack
--ctstate NEW,RELATED,ESTABLISHED -j DSCP --set-dscp 0x2e
iptables -t mangle -A RECLASS_sw10 -p udp -m udp --sport 6484 -m conntrack
--ctstate NEW,RELATED,ESTABLISHED -j DSCP --set-dscp 0x2e
iptables -t mangle -A RECLASS_sw10 -p udp -m udp --dport 6484 -m conntrack
--ctstate NEW,RELATED,ESTABLISHED -j DSCP --set-dscp 0x2e

Probably not a good idea to do conntracking here, but it should only be
matching that specific interface and port anyways.

On Mon, Apr 13, 2015 at 6:27 AM, leetminiwheat <LeetMiniWheat@gmail.com>
wrote:

> Thanks for the reply,
>
>
> On Mon, Apr 13, 2015 at 3:36 AM, Sebastian Moeller <moeller0@gmx.de>
> wrote:
>
> >         "Ignore DSCP on ingress”, this basically uses an 1-tier HTB
> shaper on ingress, DSCP marks in the packets are simply ignored, but not
> touched.
>
> So if I understand this correctly, this means they're kind of in
> passthrough mode on ignore but the qdiscs don't act on them due to no
> iptables MARKing? And If *not* ignoring DSCP on ingress, they're still
> passed through untouched but prioritized internally with the qdiscs?
>
> >         "Squash DSCP on inbound packets (ingress):” this will replace
> all DSCP marks with 0x0 (I believe), but only after the ingress qdisc.
> > In essence this means you can actually interpret ingress DSCP marks from
> upstream ("Ignore DSCP on ingress” set to ALLOW) but wipe them after the
> ingress shaping (with "Squash DSCP on inbound packets (ingress);” active).
> So the default should be “Ignore DSCP on ingress” and Squash (the second
> can be argued, as long as no one bases routing decisions on the marks they
> do not hurt). The rest of your questions are beyond my expertise...
>
> Hmm, why would we want to remove all DSCP on output then? I assume many
> ISPs and routers will squash them anyways, but wouldn't it serve *some*
> purpose to differentiate between different traffic types?
>
>
> Also, regarding my mangling questions I think I figured out the correct
> chains to insert my rules in. mangle PREROUTING and mangle POSTROUTING
> because both of those have a jump at the end to a chain that MARKs for
> qdiscs based on DSCP values. They appear to be working since I see the MARK
> rules counting upward. I assume PRE and POST go through FORWARD anyways so
> it's redundant, and using FORWARD only didn't appear to be working. I've
> ended up with this:
>
> # Mangle sw10 packets to reclassify mumble traffic. Hopefully this passes
> through DSCP so WMM can recognize 0x2e as VoIP traffic
> iptables -t mangle -N RECLASS_sw10
> iptables -t mangle -A RECLASS_sw10 -p tcp -m tcp --sport 6484  -j DSCP
> --set-dscp 0x2e
> iptables -t mangle -A RECLASS_sw10 -p tcp -m tcp --dport 6484  -j DSCP
> --set-dscp 0x2e
> iptables -t mangle -A RECLASS_sw10 -p udp -m udp --sport 6484 -j DSCP
> --set-dscp 0x2e
> iptables -t mangle -A RECLASS_sw10 -p udp -m udp --dport 6484 -j DSCP
> --set-dscp 0x2e
> iptables -t mangle -I PREROUTING -i sw10 -j RECLASS_sw10
> iptables -t mangle -I POSTROUTING -o sw10 -j RECLASS_sw10
>
> # Mangle gw00 to reclassify guest as low priority
> iptables -t mangle -N RECLASS_gw00
> iptables -t mangle -A RECLASS_gw00 -j DSCP --set-dscp 0x08
> iptables -t mangle -I PREROUTING -i gw00 -j RECLASS_gw00
> iptables -t mangle -I POSTROUTING -o gw00 -j RECLASS_gw00
>
>
> However, I'm not sure how the port specific rules handle
> RELATED,ESTABLISHED because I believe the initial connection is done on
> port 6484 but then it uses a random high port. I'm not sure how I could add
> a RELATED,ESTABLISHED match to connections on port 6484 to my custom zone
> that reclassifies to 0x2e to catch *all* traffic related to it.
>
>

[-- Attachment #2: Type: text/html, Size: 4725 bytes --]

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

[-- Attachment #1: Type: text/plain, Size: 6736 bytes --]

On Mon, Apr 13, 2015 at 7:32 AM, Sebastian Moeller <moeller0@gmx.de> wrote:

> > >         "Squash DSCP on inbound packets (ingress):” this will replace
all DSCP marks with 0x0 (I believe), but only after the ingress qdisc.
> > > In essence this means you can actually interpret ingress DSCP marks
from upstream ("Ignore DSCP on ingress” set to ALLOW) but wipe them after
the ingress shaping (with "Squash DSCP on inbound packets (ingress);”
active). So the default should be “Ignore DSCP on ingress” and Squash (the
second can be argued, as long as no one bases routing decisions on the
marks they do not hurt). The rest of your questions are beyond my
expertise...
> >
> > Hmm, why would we want to remove all DSCP on output then?
>
>         Because often we do not want to trust the internet to do the
right thing and not game our classification? At least this is a common
argument made...
>
> > I assume many ISPs and routers will squash them anyways,
>
>         ISPs are free to set the DSCP values to whatever suits them, and
sometimes they do weird things, in essence per default we should not trust
them...
>
> > but wouldn't it serve *some* purpose to differentiate between different
traffic types?
>
>         Sure, if you know what you do setting reasonable DSCP values for
VoIP sounds like a good thing (but due to fq_codel’s inner working might
not be required). Alas iptables is only available to us after the packets
went through the IFB, so any resetting of DSCP values would be for internal
network nodes, our shaper unfortunately can not use this information…

Curious, if fq_codel runs after iptables and can't use DSCP information,
why does it mark packets in the mangle chain? I'm still trying to wrap my
head around the --set-xmark

relevant snippet here with squash disabled and ignore ingress set to allow.
Can anyone answer why it's matching DSCP marks and what is it doing with
xset? I assumed it was marking packets for use in QOS, such as the chain
suggests "Chain QOS_MARK_ge00" but this is a jump target from both
PREROUTING and POSTROUTING so it should hit FORWARD too,

#####################################################################################################
#iptables -nL -t mangle
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
MARK       tcp  --  0.0.0.0/0            0.0.0.0/0            MARK xset
0x2/0xff
fwmark     all  --  0.0.0.0/0            0.0.0.0/0
QOS_MARK_ge00  all  --  0.0.0.0/0            0.0.0.0/0           [goto]
 mark match 0x0/0xff

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
mssfix     all  --  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
DSCP       udp  --  0.0.0.0/0            0.0.0.0/0            multiport
ports 123,53 DSCP set 0x24

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
QOS_MARK_ge00  all  --  0.0.0.0/0            0.0.0.0/0           [goto]
 mark match 0x0/0xff

Chain QOS_MARK_ge00 (2 references)
target     prot opt source               destination
MARK       all  --  0.0.0.0/0            0.0.0.0/0            MARK xset
0x2/0xff
MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match
0x08 MARK xset 0x3/0xff
MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match
0x30 MARK xset 0x1/0xff
MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match
0x2e MARK xset 0x1/0xff
MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match
0x24 MARK xset 0x1/0xff
MARK       all  --  0.0.0.0/0            0.0.0.0/0            tos
match0x10/0x3f MARK xset 0x1/0xff

Chain fwmark (1 references)
target     prot opt source               destination

Chain mssfix (1 references)
target     prot opt source               destination
TCPMSS     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp
flags:0x06/0x02 /* wan (mtu_fix) */ TCPMSS clamp to PMTU
#####################################################################################################
#iptables -S -t mangle
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N fwmark
-N mssfix
-A PREROUTING -i vtun+ -p tcp -j MARK --set-xmark 0x2/0xff
-A PREROUTING -j fwmark
-A PREROUTING -i ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
-A FORWARD -j mssfix
-A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP --set-dscp 0x24
-A POSTROUTING -o ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
-A QOS_MARK_ge00 -j MARK --set-xmark 0x2/0xff
-A QOS_MARK_ge00 -m dscp --dscp 0x08 -j MARK --set-xmark 0x3/0xff
-A QOS_MARK_ge00 -m dscp --dscp 0x30 -j MARK --set-xmark 0x1/0xff
-A QOS_MARK_ge00 -m dscp --dscp 0x2e -j MARK --set-xmark 0x1/0xff
-A QOS_MARK_ge00 -m dscp --dscp 0x24 -j MARK --set-xmark 0x1/0xff
-A QOS_MARK_ge00 -m tos --tos 0x10/0x3f -j MARK --set-xmark 0x1/0xff
-A mssfix -o ge00 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment
--comment "wan (mtu_fix)" -j TCPMSS --clamp-mss-to-pmtu
#####################################################################################################

I've inserted my custom --set-dscp user created chains into PREROUTING and
POSTROUTING, which I excluded from the above but I still don't understand
what exactly these marks are doing, even after reading the documentation.
It seems like it's still replacing DSCP from what I can tell.

------
Also, another unrelated question regarding queue buffers;

All my ifconfig interfaces show txqueuelen:1000 except for ifb4ge00 and
ifb4gw00, both of which I have SQM rate limiting on. wasn't this supposed
to be tweaked by debloat scripts? /etc/config/debloat says obsoleted
by /etc/hotplug.d/iface/02-debloat but that file is empty.

uci seems to show the txqueuelen options disabled. Did something change,
are these not needed anymore, or did I screw up a config somewhere?
 # uci show | grep debloat
debloat.@wireless[0]=wireless
debloat.@wireless[0].txqueuelen=4
debloat.@wirelessn[0]=wirelessn
debloat.@wirelessn[0].txqueuelen=16
debloat.@wired10[0]=wired10
debloat.@wired10[0].txqueuelen=4
debloat.@wired100[0]=wired100
debloat.@wired100[0].txqueuelen=16
debloat.@wired1000[0]=wired1000
debloat.@wired1000[0].txqueuelen=32
uci: Entry not found


Thanks for your time, and my apologies if these questions seem dumb or
regarded as unnecessary mailing list spam - I'm still learning and tweaking
things. I do my best to search the wikis and google before asking here.

[-- Attachment #2: Type: text/html, Size: 9031 bytes --]

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

[-- Attachment #1: Type: text/plain, Size: 7527 bytes --]

Upon further inspection, the SQM log says:

SQM: Keeping differentiated services code points (DSCP) from ingress.
SQM: Perform DSCP based filtering on ingress. (3-tier classification),

so perhaps the shaper can use this information? but only on ingress?

I'm using the latest SQM scripts from git. Could this also be why my
interfaces are remaining at 1000 txqueuelen due to new scripts? (I'm on
3.10.50-1)

Still confused about what the xset stuff is doing though.

On Mon, Apr 13, 2015 at 9:43 AM, leetminiwheat <LeetMiniWheat@gmail.com>
wrote:
>
>
>
> On Mon, Apr 13, 2015 at 7:32 AM, Sebastian Moeller <moeller0@gmx.de>
wrote:
>
> > > >         "Squash DSCP on inbound packets (ingress):” this will
replace all DSCP marks with 0x0 (I believe), but only after the ingress
qdisc.
> > > > In essence this means you can actually interpret ingress DSCP marks
from upstream ("Ignore DSCP on ingress” set to ALLOW) but wipe them after
the ingress shaping (with "Squash DSCP on inbound packets (ingress);”
active). So the default should be “Ignore DSCP on ingress” and Squash (the
second can be argued, as long as no one bases routing decisions on the
marks they do not hurt). The rest of your questions are beyond my
expertise...
> > >
> > > Hmm, why would we want to remove all DSCP on output then?
> >
> >         Because often we do not want to trust the internet to do the
right thing and not game our classification? At least this is a common
argument made...
> >
> > > I assume many ISPs and routers will squash them anyways,
> >
> >         ISPs are free to set the DSCP values to whatever suits them,
and sometimes they do weird things, in essence per default we should not
trust them...
> >
> > > but wouldn't it serve *some* purpose to differentiate between
different traffic types?
> >
> >         Sure, if you know what you do setting reasonable DSCP values
for VoIP sounds like a good thing (but due to fq_codel’s inner working
might not be required). Alas iptables is only available to us after the
packets went through the IFB, so any resetting of DSCP values would be for
internal network nodes, our shaper unfortunately can not use this
information…
>
> Curious, if fq_codel runs after iptables and can't use DSCP information,
why does it mark packets in the mangle chain? I'm still trying to wrap my
head around the --set-xmark
>
> relevant snippet here with squash disabled and ignore ingress set to
allow. Can anyone answer why it's matching DSCP marks and what is it doing
with xset? I assumed it was marking packets for use in QOS, such as the
chain suggests "Chain QOS_MARK_ge00" but this is a jump target from both
PREROUTING and POSTROUTING so it should hit FORWARD too,
>
>
#####################################################################################################
> #iptables -nL -t mangle
> Chain PREROUTING (policy ACCEPT)
> target     prot opt source               destination
> MARK       tcp  --  0.0.0.0/0            0.0.0.0/0            MARK xset
0x2/0xff
> fwmark     all  --  0.0.0.0/0            0.0.0.0/0
> QOS_MARK_ge00  all  --  0.0.0.0/0            0.0.0.0/0           [goto]
 mark match 0x0/0xff
>
> Chain INPUT (policy ACCEPT)
> target     prot opt source               destination
>
> Chain FORWARD (policy ACCEPT)
> target     prot opt source               destination
> mssfix     all  --  0.0.0.0/0            0.0.0.0/0
>
> Chain OUTPUT (policy ACCEPT)
> target     prot opt source               destination
> DSCP       udp  --  0.0.0.0/0            0.0.0.0/0            multiport
ports 123,53 DSCP set 0x24
>
> Chain POSTROUTING (policy ACCEPT)
> target     prot opt source               destination
> QOS_MARK_ge00  all  --  0.0.0.0/0            0.0.0.0/0           [goto]
 mark match 0x0/0xff
>
> Chain QOS_MARK_ge00 (2 references)
> target     prot opt source               destination
> MARK       all  --  0.0.0.0/0            0.0.0.0/0            MARK xset
0x2/0xff
> MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match
0x08 MARK xset 0x3/0xff
> MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match
0x30 MARK xset 0x1/0xff
> MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match
0x2e MARK xset 0x1/0xff
> MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match
0x24 MARK xset 0x1/0xff
> MARK       all  --  0.0.0.0/0            0.0.0.0/0            tos
match0x10/0x3f MARK xset 0x1/0xff
>
> Chain fwmark (1 references)
> target     prot opt source               destination
>
> Chain mssfix (1 references)
> target     prot opt source               destination
> TCPMSS     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp
flags:0x06/0x02 /* wan (mtu_fix) */ TCPMSS clamp to PMTU
>
#####################################################################################################
> #iptables -S -t mangle
> -P PREROUTING ACCEPT
> -P INPUT ACCEPT
> -P FORWARD ACCEPT
> -P OUTPUT ACCEPT
> -P POSTROUTING ACCEPT
> -N fwmark
> -N mssfix
> -A PREROUTING -i vtun+ -p tcp -j MARK --set-xmark 0x2/0xff
> -A PREROUTING -j fwmark
> -A PREROUTING -i ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
> -A FORWARD -j mssfix
> -A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP --set-dscp 0x24
> -A POSTROUTING -o ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
> -A QOS_MARK_ge00 -j MARK --set-xmark 0x2/0xff
> -A QOS_MARK_ge00 -m dscp --dscp 0x08 -j MARK --set-xmark 0x3/0xff
> -A QOS_MARK_ge00 -m dscp --dscp 0x30 -j MARK --set-xmark 0x1/0xff
> -A QOS_MARK_ge00 -m dscp --dscp 0x2e -j MARK --set-xmark 0x1/0xff
> -A QOS_MARK_ge00 -m dscp --dscp 0x24 -j MARK --set-xmark 0x1/0xff
> -A QOS_MARK_ge00 -m tos --tos 0x10/0x3f -j MARK --set-xmark 0x1/0xff
> -A mssfix -o ge00 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment
--comment "wan (mtu_fix)" -j TCPMSS --clamp-mss-to-pmtu
>
#####################################################################################################
>
> I've inserted my custom --set-dscp user created chains into PREROUTING
and POSTROUTING, which I excluded from the above but I still don't
understand what exactly these marks are doing, even after reading the
documentation. It seems like it's still replacing DSCP from what I can tell.
>
> ------
> Also, another unrelated question regarding queue buffers;
>
> All my ifconfig interfaces show txqueuelen:1000 except for ifb4ge00 and
ifb4gw00, both of which I have SQM rate limiting on. wasn't this supposed
to be tweaked by debloat scripts? /etc/config/debloat says obsoleted by
/etc/hotplug.d/iface/02-debloat but that file is empty.
>
> uci seems to show the txqueuelen options disabled. Did something change,
are these not needed anymore, or did I screw up a config somewhere?
>  # uci show | grep debloat
> debloat.@wireless[0]=wireless
> debloat.@wireless[0].txqueuelen=4
> debloat.@wirelessn[0]=wirelessn
> debloat.@wirelessn[0].txqueuelen=16
> debloat.@wired10[0]=wired10
> debloat.@wired10[0].txqueuelen=4
> debloat.@wired100[0]=wired100
> debloat.@wired100[0].txqueuelen=16
> debloat.@wired1000[0]=wired1000
> debloat.@wired1000[0].txqueuelen=32
> uci: Entry not found
>
>
> Thanks for your time, and my apologies if these questions seem dumb or
regarded as unnecessary mailing list spam - I'm still learning and tweaking
things. I do my best to search the wikis and google before asking here.
>
>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 9521 bytes --]

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[Sebastian Moeller]

Hi leetminiwheat,

On Apr 13, 2015, at 16:16 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:

> Upon further inspection, the SQM log says: 
> 
> SQM: Keeping differentiated services code points (DSCP) from ingress.
> SQM: Perform DSCP based filtering on ingress. (3-tier classification), 
> 
> so perhaps the shaper can use this information? but only on ingress?

	Well on egress simple.qos can and will use (some) DSCP values to steer packets into one of the three tiers; on ingress this only works if “Ignore DSCP on ingress” is set to allow. “Keeping differentiated services code points (DSCP) from ingress.” just tells you that you did not request that the TOS bits be zeroed (excluding the ECN bits).


> 
> I'm using the latest SQM scripts from git. Could this also be why my interfaces are remaining at 1000 txqueuelen due to new scripts? (I'm on 3.10.50-1)

	Could you post the output of calling the following commands on your router please:

/etc/init.d/sqm stop
/etc/init.d/sqm start
tc -d qdisc

Best Regards
	Sebastian


> 
> Still confused about what the xset stuff is doing though.
> 
> On Mon, Apr 13, 2015 at 9:43 AM, leetminiwheat <LeetMiniWheat@gmail.com> wrote:
> >
> >
> >
> > On Mon, Apr 13, 2015 at 7:32 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
> >
> > > > >         "Squash DSCP on inbound packets (ingress):” this will replace all DSCP marks with 0x0 (I believe), but only after the ingress qdisc.
> > > > > In essence this means you can actually interpret ingress DSCP marks from upstream ("Ignore DSCP on ingress” set to ALLOW) but wipe them after the ingress shaping (with "Squash DSCP on inbound packets (ingress);” active). So the default should be “Ignore DSCP on ingress” and Squash (the second can be argued, as long as no one bases routing decisions on the marks they do not hurt). The rest of your questions are beyond my expertise...
> > > >
> > > > Hmm, why would we want to remove all DSCP on output then?
> > >
> > >         Because often we do not want to trust the internet to do the right thing and not game our classification? At least this is a common argument made...
> > >
> > > > I assume many ISPs and routers will squash them anyways,
> > >
> > >         ISPs are free to set the DSCP values to whatever suits them, and sometimes they do weird things, in essence per default we should not trust them...
> > >
> > > > but wouldn't it serve *some* purpose to differentiate between different traffic types?
> > >
> > >         Sure, if you know what you do setting reasonable DSCP values for VoIP sounds like a good thing (but due to fq_codel’s inner working might not be required). Alas iptables is only available to us after the packets went through the IFB, so any resetting of DSCP values would be for internal network nodes, our shaper unfortunately can not use this information…
> >
> > Curious, if fq_codel runs after iptables and can't use DSCP information, why does it mark packets in the mangle chain? I'm still trying to wrap my head around the --set-xmark
> >
> > relevant snippet here with squash disabled and ignore ingress set to allow. Can anyone answer why it's matching DSCP marks and what is it doing with xset? I assumed it was marking packets for use in QOS, such as the chain suggests "Chain QOS_MARK_ge00" but this is a jump target from both PREROUTING and POSTROUTING so it should hit FORWARD too,
> >
> > #####################################################################################################
> > #iptables -nL -t mangle
> > Chain PREROUTING (policy ACCEPT)
> > target     prot opt source               destination
> > MARK       tcp  --  0.0.0.0/0            0.0.0.0/0            MARK xset 0x2/0xff
> > fwmark     all  --  0.0.0.0/0            0.0.0.0/0
> > QOS_MARK_ge00  all  --  0.0.0.0/0            0.0.0.0/0           [goto]  mark match 0x0/0xff
> >
> > Chain INPUT (policy ACCEPT)
> > target     prot opt source               destination
> >
> > Chain FORWARD (policy ACCEPT)
> > target     prot opt source               destination
> > mssfix     all  --  0.0.0.0/0            0.0.0.0/0
> >
> > Chain OUTPUT (policy ACCEPT)
> > target     prot opt source               destination
> > DSCP       udp  --  0.0.0.0/0            0.0.0.0/0            multiport ports 123,53 DSCP set 0x24
> >
> > Chain POSTROUTING (policy ACCEPT)
> > target     prot opt source               destination
> > QOS_MARK_ge00  all  --  0.0.0.0/0            0.0.0.0/0           [goto]  mark match 0x0/0xff
> >
> > Chain QOS_MARK_ge00 (2 references)
> > target     prot opt source               destination
> > MARK       all  --  0.0.0.0/0            0.0.0.0/0            MARK xset 0x2/0xff
> > MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match 0x08 MARK xset 0x3/0xff
> > MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match 0x30 MARK xset 0x1/0xff
> > MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match 0x2e MARK xset 0x1/0xff
> > MARK       all  --  0.0.0.0/0            0.0.0.0/0            DSCP match 0x24 MARK xset 0x1/0xff
> > MARK       all  --  0.0.0.0/0            0.0.0.0/0            tos match0x10/0x3f MARK xset 0x1/0xff
> >
> > Chain fwmark (1 references)
> > target     prot opt source               destination
> >
> > Chain mssfix (1 references)
> > target     prot opt source               destination
> > TCPMSS     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* wan (mtu_fix) */ TCPMSS clamp to PMTU
> > #####################################################################################################
> > #iptables -S -t mangle
> > -P PREROUTING ACCEPT
> > -P INPUT ACCEPT
> > -P FORWARD ACCEPT
> > -P OUTPUT ACCEPT
> > -P POSTROUTING ACCEPT
> > -N fwmark
> > -N mssfix
> > -A PREROUTING -i vtun+ -p tcp -j MARK --set-xmark 0x2/0xff
> > -A PREROUTING -j fwmark
> > -A PREROUTING -i ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
> > -A FORWARD -j mssfix
> > -A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP --set-dscp 0x24
> > -A POSTROUTING -o ge00 -m mark --mark 0x0/0xff -g QOS_MARK_ge00
> > -A QOS_MARK_ge00 -j MARK --set-xmark 0x2/0xff
> > -A QOS_MARK_ge00 -m dscp --dscp 0x08 -j MARK --set-xmark 0x3/0xff
> > -A QOS_MARK_ge00 -m dscp --dscp 0x30 -j MARK --set-xmark 0x1/0xff
> > -A QOS_MARK_ge00 -m dscp --dscp 0x2e -j MARK --set-xmark 0x1/0xff
> > -A QOS_MARK_ge00 -m dscp --dscp 0x24 -j MARK --set-xmark 0x1/0xff
> > -A QOS_MARK_ge00 -m tos --tos 0x10/0x3f -j MARK --set-xmark 0x1/0xff
> > -A mssfix -o ge00 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "wan (mtu_fix)" -j TCPMSS --clamp-mss-to-pmtu
> > #####################################################################################################
> >
> > I've inserted my custom --set-dscp user created chains into PREROUTING and POSTROUTING, which I excluded from the above but I still don't understand what exactly these marks are doing, even after reading the documentation. It seems like it's still replacing DSCP from what I can tell.
> >
> > ------
> > Also, another unrelated question regarding queue buffers;
> >
> > All my ifconfig interfaces show txqueuelen:1000 except for ifb4ge00 and ifb4gw00, both of which I have SQM rate limiting on. wasn't this supposed to be tweaked by debloat scripts? /etc/config/debloat says obsoleted by /etc/hotplug.d/iface/02-debloat but that file is empty. 
> >
> > uci seems to show the txqueuelen options disabled. Did something change, are these not needed anymore, or did I screw up a config somewhere?
> >  # uci show | grep debloat
> > debloat.@wireless[0]=wireless
> > debloat.@wireless[0].txqueuelen=4
> > debloat.@wirelessn[0]=wirelessn
> > debloat.@wirelessn[0].txqueuelen=16
> > debloat.@wired10[0]=wired10
> > debloat.@wired10[0].txqueuelen=4
> > debloat.@wired100[0]=wired100
> > debloat.@wired100[0].txqueuelen=16
> > debloat.@wired1000[0]=wired1000
> > debloat.@wired1000[0].txqueuelen=32
> > uci: Entry not found
> >
> >
> > Thanks for your time, and my apologies if these questions seem dumb or regarded as unnecessary mailing list spam - I'm still learning and tweaking things. I do my best to search the wikis and google before asking here.
> >
> >
> >
> >
> >
> >

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

[-- Attachment #1: Type: text/plain, Size: 8176 bytes --]

On Mon, Apr 13, 2015 at 10:23 AM, Sebastian Moeller <moeller0@gmx.de> wrote:

>         Could you post the output of calling the following commands on
your router please:
>
> /etc/init.d/sqm stop
> /etc/init.d/sqm start
> tc -d qdisc

I had some tweaks in /etc/rc.local I had to remove, so this is the output
after a fresh reboot and running those commands:

 ~ # /etc/init.d/sqm stop
 ~ # /etc/init.d/sqm start
 ~ # tc -d qdisc
qdisc fq_codel a: dev se00 root refcnt 2 limit 1000p flows 1024 quantum
1514 target 5.0ms interval 100.0ms ecn
qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 12 direct_packets_stat 1
ver 3.17 direct_qlen 1000
qdisc fq_codel 110: dev ge00 parent 1:11 limit 1001p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 120: dev ge00 parent 1:12 limit 1001p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 130: dev ge00 parent 1:13 limit 1001p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
qdisc mq 1: dev sw10 root
qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum 500
target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum 300
target 5.0ms interval 100.0ms
qdisc mq 1: dev sw00 root
qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum 500
target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum 300
target 5.0ms interval 100.0ms
qdisc htb 1: dev gw00 root refcnt 5 r2q 10 default 12 direct_packets_stat 0
ver 3.17 direct_qlen 1000
qdisc fq_codel 110: dev gw00 parent 1:11 limit 1001p flows 1024 quantum 300
target 8.0ms interval 103.0ms
qdisc fq_codel 120: dev gw00 parent 1:12 limit 1001p flows 1024 quantum 300
target 8.0ms interval 103.0ms
qdisc fq_codel 130: dev gw00 parent 1:13 limit 1001p flows 1024 quantum 300
target 8.0ms interval 103.0ms
qdisc ingress ffff: dev gw00 parent ffff:fff1 ----------------
qdisc htb 1: dev ifb4ge00 root refcnt 2 r2q 10 default 12
direct_packets_stat 0 ver 3.17 direct_qlen 32
qdisc fq_codel 110: dev ifb4ge00 parent 1:11 limit 1001p flows 1024 quantum
500 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 120: dev ifb4ge00 parent 1:12 limit 1001p flows 1024 quantum
1500 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 130: dev ifb4ge00 parent 1:13 limit 1001p flows 1024 quantum
300 target 5.0ms interval 100.0ms ecn
qdisc htb 1: dev ifb4gw00 root refcnt 2 r2q 10 default 12
direct_packets_stat 0 ver 3.17 direct_qlen 32
qdisc fq_codel 110: dev ifb4gw00 parent 1:11 limit 1001p flows 1024 quantum
500 target 10.9ms interval 105.9ms ecn
qdisc fq_codel 120: dev ifb4gw00 parent 1:12 limit 1001p flows 1024 quantum
1500 target 10.9ms interval 105.9ms ecn
qdisc fq_codel 130: dev ifb4gw00 parent 1:13 limit 1001p flows 1024 quantum
300 target 10.9ms interval 105.9ms ecn

 ~ # ifconfig
ge00        Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5D
               collisions:0 txqueuelen:1000

gw00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
               collisions:0 txqueuelen:1000


ifb4ge00   Link encap:Ethernet  HWaddr C2:00:73:A7:1E:30
               collisions:0 txqueuelen:32


ifb4gw00  Link encap:Ethernet  HWaddr 9A:BA:17:27:79:A5
               collisions:0 txqueuelen:32

lo            Link encap:Local Loopback
              collisions:0 txqueuelen:0

se00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
              collisions:0 txqueuelen:1000

sw00      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5C
             collisions:0 txqueuelen:1000

sw10      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5E
             collisions:0 txqueuelen:1000

and after rebooting and running my commands (still testing different
values. should the IFBs txqueuelen be left alone? I left at default):
 ~ # ifconfig ge00 txqueuelen 8
 ~ # ifconfig se00 txqueuelen 8
 ~ # ifconfig gw00 txqueuelen 4
 ~ # ifconfig sw00 txqueuelen 4
 ~ # ifconfig sw10 txqueuelen 4
followed by a sqm restart
 ~ # /etc/init.d/sqm stop
 ~ # /etc/init.d/sqm start
 ~ # tc -d qdisc
qdisc fq_codel a: dev se00 root refcnt 2 limit 1000p flows 1024 quantum
1514 target 5.0ms interval 100.0ms ecn
qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 12 direct_packets_stat 0
ver 3.17 direct_qlen 8
qdisc fq_codel 110: dev ge00 parent 1:11 limit 1001p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 120: dev ge00 parent 1:12 limit 1001p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 130: dev ge00 parent 1:13 limit 1001p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
qdisc mq 1: dev sw00 root
qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum 500
target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum 300
target 5.0ms interval 100.0ms
qdisc mq 1: dev sw10 root
qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum 500
target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum 300
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum 300
target 5.0ms interval 100.0ms
qdisc htb 1: dev gw00 root refcnt 5 r2q 10 default 12 direct_packets_stat 0
ver 3.17 direct_qlen 4
qdisc fq_codel 110: dev gw00 parent 1:11 limit 1001p flows 1024 quantum 300
target 8.0ms interval 103.0ms
qdisc fq_codel 120: dev gw00 parent 1:12 limit 1001p flows 1024 quantum 300
target 8.0ms interval 103.0ms
qdisc fq_codel 130: dev gw00 parent 1:13 limit 1001p flows 1024 quantum 300
target 8.0ms interval 103.0ms
qdisc ingress ffff: dev gw00 parent ffff:fff1 ----------------
qdisc htb 1: dev ifb4ge00 root refcnt 2 r2q 10 default 12
direct_packets_stat 0 ver 3.17 direct_qlen 32
qdisc fq_codel 110: dev ifb4ge00 parent 1:11 limit 1001p flows 1024 quantum
500 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 120: dev ifb4ge00 parent 1:12 limit 1001p flows 1024 quantum
1500 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 130: dev ifb4ge00 parent 1:13 limit 1001p flows 1024 quantum
300 target 5.0ms interval 100.0ms ecn
qdisc htb 1: dev ifb4gw00 root refcnt 2 r2q 10 default 12
direct_packets_stat 0 ver 3.17 direct_qlen 32
qdisc fq_codel 110: dev ifb4gw00 parent 1:11 limit 1001p flows 1024 quantum
500 target 10.9ms interval 105.9ms ecn
qdisc fq_codel 120: dev ifb4gw00 parent 1:12 limit 1001p flows 1024 quantum
1500 target 10.9ms interval 105.9ms ecn
qdisc fq_codel 130: dev ifb4gw00 parent 1:13 limit 1001p flows 1024 quantum
300 target 10.9ms interval 105.9ms ecn


 ~ # ifconfig
ge00        Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5D
               collisions:0 txqueuelen:8

gw00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
               collisions:0 txqueuelen:4


ifb4ge00   Link encap:Ethernet  HWaddr C2:00:73:A7:1E:30
               collisions:0 txqueuelen:32


ifb4gw00  Link encap:Ethernet  HWaddr 9A:BA:17:27:79:A5
               collisions:0 txqueuelen:32

lo            Link encap:Local Loopback
              collisions:0 txqueuelen:0

se00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
              collisions:0 txqueuelen: 8

sw00      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5C
             collisions:0 txqueuelen: 4

sw10      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5E
             collisions:0 txqueuelen: 4

[-- Attachment #2: Type: text/html, Size: 9972 bytes --]

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[Sebastian Moeller]

Hi there,

On Apr 13, 2015, at 18:48 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:

> 
> 
> On Mon, Apr 13, 2015 at 10:23 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> >         Could you post the output of calling the following commands on your router please:
> >
> > /etc/init.d/sqm stop
> > /etc/init.d/sqm start
> > tc -d qdisc
> 
> I had some tweaks in /etc/rc.local I had to remove, so this is the output after a fresh reboot and running those commands:

Thanks.

> 
>  ~ # /etc/init.d/sqm stop
>  ~ # /etc/init.d/sqm start
>  ~ # tc -d qdisc
> qdisc fq_codel a: dev se00 root refcnt 2 limit 1000p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
> qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 12 direct_packets_stat 1 ver 3.17 direct_qlen 1000
> qdisc fq_codel 110: dev ge00 parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 120: dev ge00 parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 130: dev ge00 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
> qdisc mq 1: dev sw10 root
> qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
> qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
> qdisc mq 1: dev sw00 root
> qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
> qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
> qdisc htb 1: dev gw00 root refcnt 5 r2q 10 default 12 direct_packets_stat 0 ver 3.17 direct_qlen 1000
> qdisc fq_codel 110: dev gw00 parent 1:11 limit 1001p flows 1024 quantum 300 target 8.0ms interval 103.0ms
> qdisc fq_codel 120: dev gw00 parent 1:12 limit 1001p flows 1024 quantum 300 target 8.0ms interval 103.0ms
> qdisc fq_codel 130: dev gw00 parent 1:13 limit 1001p flows 1024 quantum 300 target 8.0ms interval 103.0ms
> qdisc ingress ffff: dev gw00 parent ffff:fff1 ----------------
> qdisc htb 1: dev ifb4ge00 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 ver 3.17 direct_qlen 32
> qdisc fq_codel 110: dev ifb4ge00 parent 1:11 limit 1001p flows 1024 quantum 500 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 120: dev ifb4ge00 parent 1:12 limit 1001p flows 1024 quantum 1500 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 130: dev ifb4ge00 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc htb 1: dev ifb4gw00 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 ver 3.17 direct_qlen 32
> qdisc fq_codel 110: dev ifb4gw00 parent 1:11 limit 1001p flows 1024 quantum 500 target 10.9ms interval 105.9ms ecn
> qdisc fq_codel 120: dev ifb4gw00 parent 1:12 limit 1001p flows 1024 quantum 1500 target 10.9ms interval 105.9ms ecn
> qdisc fq_codel 130: dev ifb4gw00 parent 1:13 limit 1001p flows 1024 quantum 300 target 10.9ms interval 105.9ms ecn

	This looks reasonable.

> 
>  ~ # ifconfig
> ge00        Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5D
>                collisions:0 txqueuelen:1000
> 
> gw00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
>                collisions:0 txqueuelen:1000
> 
> 
> ifb4ge00   Link encap:Ethernet  HWaddr C2:00:73:A7:1E:30 
>                collisions:0 txqueuelen:32
> 
> 
> ifb4gw00  Link encap:Ethernet  HWaddr 9A:BA:17:27:79:A5     
>                collisions:0 txqueuelen:32
> 
> lo            Link encap:Local Loopback
>               collisions:0 txqueuelen:0
> 
> se00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
>               collisions:0 txqueuelen:1000
> 
> sw00      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5C
>              collisions:0 txqueuelen:1000
> 
> sw10      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5E
>              collisions:0 txqueuelen:1000

	I had a look at my cerowrt router and I also see txqueuelen at 1000, but IIRC that does not matter much anymore, since the wndr37/800 support BQL:
cat /sys/class/net/ge00/queues/tx-0/byte_queue_limits/limit_max
3000
So even with txqueuelen = 1000 the tx queue will only hold 3000 bytes. For fib and friends it does not really matter as far as I can tell.

> 
> and after rebooting and running my commands (still testing different values. should the IFBs txqueuelen be left alone? I left at default):
>  ~ # ifconfig ge00 txqueuelen 8
>  ~ # ifconfig se00 txqueuelen 8
>  ~ # ifconfig gw00 txqueuelen 4
>  ~ # ifconfig sw00 txqueuelen 4
>  ~ # ifconfig sw10 txqueuelen 4
> followed by a sqm restart
>  ~ # /etc/init.d/sqm stop
>  ~ # /etc/init.d/sqm start
>  ~ # tc -d qdisc
> qdisc fq_codel a: dev se00 root refcnt 2 limit 1000p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
> qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 ver 3.17 direct_qlen 8
> qdisc fq_codel 110: dev ge00 parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 120: dev ge00 parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 130: dev ge00 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
> qdisc mq 1: dev sw00 root
> qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
> qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
> qdisc mq 1: dev sw10 root
> qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
> qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
> qdisc htb 1: dev gw00 root refcnt 5 r2q 10 default 12 direct_packets_stat 0 ver 3.17 direct_qlen 4
> qdisc fq_codel 110: dev gw00 parent 1:11 limit 1001p flows 1024 quantum 300 target 8.0ms interval 103.0ms
> qdisc fq_codel 120: dev gw00 parent 1:12 limit 1001p flows 1024 quantum 300 target 8.0ms interval 103.0ms
> qdisc fq_codel 130: dev gw00 parent 1:13 limit 1001p flows 1024 quantum 300 target 8.0ms interval 103.0ms
> qdisc ingress ffff: dev gw00 parent ffff:fff1 ----------------
> qdisc htb 1: dev ifb4ge00 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 ver 3.17 direct_qlen 32
> qdisc fq_codel 110: dev ifb4ge00 parent 1:11 limit 1001p flows 1024 quantum 500 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 120: dev ifb4ge00 parent 1:12 limit 1001p flows 1024 quantum 1500 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 130: dev ifb4ge00 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc htb 1: dev ifb4gw00 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 ver 3.17 direct_qlen 32
> qdisc fq_codel 110: dev ifb4gw00 parent 1:11 limit 1001p flows 1024 quantum 500 target 10.9ms interval 105.9ms ecn
> qdisc fq_codel 120: dev ifb4gw00 parent 1:12 limit 1001p flows 1024 quantum 1500 target 10.9ms interval 105.9ms ecn
> qdisc fq_codel 130: dev ifb4gw00 parent 1:13 limit 1001p flows 1024 quantum 300 target 10.9ms interval 105.9ms ecn
> 
> 
>  ~ # ifconfig
> ge00        Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5D
>                collisions:0 txqueuelen:8
> 
> gw00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
>                collisions:0 txqueuelen:4
> 
> 
> ifb4ge00   Link encap:Ethernet  HWaddr C2:00:73:A7:1E:30 
>                collisions:0 txqueuelen:32
> 
> 
> ifb4gw00  Link encap:Ethernet  HWaddr 9A:BA:17:27:79:A5     
>                collisions:0 txqueuelen:32
> 
> lo            Link encap:Local Loopback
>               collisions:0 txqueuelen:0
> 
> se00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
>               collisions:0 txqueuelen: 8
> 
> sw00      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5C
>              collisions:0 txqueuelen: 4
> 
> sw10      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5E
>              collisions:0 txqueuelen: 4
> 
> 

If you have time and netperf-wrapper it would be good to convince yourself and us again, that txqueuelen really does not matter for BQL’d interfaces by running RRUL tests with and without your modifications….

Best Regards
	Sebastian

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

On Mon, Apr 13, 2015 at 5:39 PM, Sebastian Moeller <moeller0@gmx.de> wrote:
>         This looks reasonable.
>
> >
> >  ~ # ifconfig
> > ge00        Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5D
> >                collisions:0 txqueuelen:1000
> >
> > gw00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
> >                collisions:0 txqueuelen:1000
> >
> >
> > ifb4ge00   Link encap:Ethernet  HWaddr C2:00:73:A7:1E:30
> >                collisions:0 txqueuelen:32
> >
> >
> > ifb4gw00  Link encap:Ethernet  HWaddr 9A:BA:17:27:79:A5
> >                collisions:0 txqueuelen:32
> >
> > lo            Link encap:Local Loopback
> >               collisions:0 txqueuelen:0
> >
> > se00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
> >               collisions:0 txqueuelen:1000
> >
> > sw00      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5C
> >              collisions:0 txqueuelen:1000
> >
> > sw10      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5E
> >              collisions:0 txqueuelen:1000
>
>         I had a look at my cerowrt router and I also see txqueuelen at 1000, but IIRC that does not matter much anymore, since the wndr37/800 support BQL:
> cat /sys/class/net/ge00/queues/tx-0/byte_queue_limits/limit_max
> 3000
> So even with txqueuelen = 1000 the tx queue will only hold 3000 bytes. For fib and friends it does not really matter as far as I can tell.

Interesting, I had been reading about BQL but didn't fully understand
it until now. This clears up a lot of confusion, thank you.

I assume tweaking ring parameters from default RX:128 and TX:32
doesn't matter anymore thenr?

> > and after rebooting and running my [ethtool/txqueuelen]commands (still testing different values. should the IFBs txqueuelen be left alone? I left at default):
> >
> >  ~ # ifconfig
> > ge00        Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5D
> >                collisions:0 txqueuelen:8
> >
> > gw00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
> >                collisions:0 txqueuelen:4
> >
> >
> > ifb4ge00   Link encap:Ethernet  HWaddr C2:00:73:A7:1E:30
> >                collisions:0 txqueuelen:32
> >
> >
> > ifb4gw00  Link encap:Ethernet  HWaddr 9A:BA:17:27:79:A5
> >                collisions:0 txqueuelen:32
> >
> > lo            Link encap:Local Loopback
> >               collisions:0 txqueuelen:0
> >
> > se00       Link encap:Ethernet  HWaddr 22:4E:7F:91:9F:5C
> >               collisions:0 txqueuelen: 8
> >
> > sw00      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5C
> >              collisions:0 txqueuelen: 4
> >
> > sw10      Link encap:Ethernet  HWaddr 20:4E:7F:91:9F:5E
> >              collisions:0 txqueuelen: 4
> >
> >
>
> If you have time and netperf-wrapper it would be good to convince yourself and us again, that txqueuelen really does not matter for BQL’d interfaces by running RRUL tests with and without your modifications….

Will do if I can get a friend to set up a netperf server in a node at
his datacenter he works at, though I believe he's using Debian Jessy
without any kind of QoS, but his datacenter routers might screw with
the packets. Maybe I can hook a spare box up to the WAN port
temporarily to test with.


############################################################################

Off topic a bit, if I might pick some of your brains - does anyone
have experience with Intel gigE NICs and any advice on how they might
be tweaked for lowest latency possible when connected to a debloated
router? Most of the websites on this topic are inexperienced,
outdated, or have no evidence to back their assumptions.

Throughput is not a concern, as the machine in question is for gaming
only, on Windows 8.1 (though in linux it has similarly named
settings), connected via gigE to CeroWRT/WNDR3800 gateway which is
directly connected via gigE to fiber ONT providing 32/25 (Cero's SQM
capped to 28.8/22.5). ((bittorrent traffic still has latency issues
though even with really low bandwidth caps, measured from a linux box,
perhaps ISP related))

Intel Network Interface is forced into MSI mode (message signaled interrupts)

My adapter settings settings below, those with an asterisk have been modified

*Adaptive Inter-Frame Spacing: Disabled
*Flow Control: Disabled (Options: Disabled, Rx & Tx Enabled, Rx
Enabled, Tx Enabled)
## Default is Rx Enabled. I really don't see a benefit to pause frames
when we have QoS algorithms.
Receive Buffers: 256 (value: 80-2048, this is the equivalent of Ring
parameters in Linux)
Transmit Buffers: 512 (value: 80-2048, this is the equivalent of Ring
parameters in Linux)
## Buffer thoughts: maybe a lower ratio such as 128 and 256 or 64 and
128? Documentation says:
##   each descriptor comes with a 2048 byte buffer.
##   Why more TX than RX?  Our architecture favors the
nondeterministic RX side for priority so there is more turnover of
descriptors
##   than the TX side.  Plus the O/S can sometimes not return TX
assets back to the driver in a timely fashion.
IPv4 Checksum Offload: Rx & Tx Enabled
Protocol ARP Offload: Enabled
Protocol NS Offload: Enabled
TCP Checksum Offload (IPv4): Rx & Tx Enabled (Options: Disabled, Rx
Enabled, Tx Enabled)
TCP Checksum Offload (IPv6): Rx & Tx Enabled (Options: Disabled, Rx
Enabled, Tx Enabled)
UDP Checksum Offload (IPv4): Rx & Tx Enabled (Options: Disabled, Rx
Enabled, Tx Enabled)
UDP Checksum Offload (IPv6): Rx & Tx Enabled (Options: Disabled, Rx
Enabled, Tx Enabled)
Large Send Offload V2 (IPv4): Enabled (this is called TSO, TCP
Segmentation Offload, in linux)
Large Send Offload V2 (IPv6): Enabled (this is called TSO, TCP
Segmentation Offload, in linux)
## Offload thoughts: Should any offloads be disabled if the host
machine is fast enough? do these cause delays?
*Jumbo Packet: Disabled
## Probably don't want anything potentially latency sensitive being
clumped up into a huge packet,
     especially when the destination is WAN which the router would
have to segment anyways
*Interrupt Moderation: Disabled (This is probably similar to large
receive offload (LRO), description says it reduces interrupts.)
*Interrupt Moderation Rate: Off (Options: Adaptive, Extreme, High,
Low, Medium, Minimal, Off)
## Presumably Interrupt Moderation might cause latency by aggregating
packets into a buffer?
Receive Side Scaling: Enabled (Documentation says this makes
processing scale to multiple CPU cores)
Maximum Number of RSS Queues: 2 Queue (Options: 1 Queue, 2 Queue)
## Scaling up to 2 cpu cores? unsure if this would cause latency
Gigabit Master Slave Mode: Auto Detect (Options: Auto Detect, Force
Master Mode, Force Slave Mode)
Enable PME: Disabled
Energy Efficient Ethernet: Disabled
System Idle Power Saver: Disabled

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[Sebastian Moeller]

Hi leetminiwheat,

On Apr 15, 2015, at 03:35 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:

> On Mon, Apr 13, 2015 at 5:39 PM, Sebastian Moeller <moeller0@gmx.de> wrote:
>>        This looks reasonable.
>> 
>>> [...]
>> 
>>        I had a look at my cerowrt router and I also see txqueuelen at 1000, but IIRC that does not matter much anymore, since the wndr37/800 support BQL:
>> cat /sys/class/net/ge00/queues/tx-0/byte_queue_limits/limit_max
>> 3000
>> So even with txqueuelen = 1000 the tx queue will only hold 3000 bytes. For fib and friends it does not really matter as far as I can tell.
> 
> Interesting, I had been reading about BQL but didn't fully understand
> it until now. This clears up a lot of confusion, thank you.
> 
> I assume tweaking ring parameters from default RX:128 and TX:32
> doesn't matter anymore thenr?

	As far as I know we leave that alone, see: http://www.bufferbloat.net/projects/bloat/wiki/Linux_Tips:
“Set the size of the ring buffer for the network interface

NOTE: THIS HACK IS NO LONGER NEEDED on many ethernet drivers in Linux 3.3, which has Byte Queue Limits instead, which does a far better job."


> 
>> [...]
>> If you have time and netperf-wrapper it would be good to convince yourself and us again, that txqueuelen really does not matter for BQL’d interfaces by running RRUL tests with and without your modifications….
> 
> Will do if I can get a friend to set up a netperf server in a node at
> his datacenter he works at, though I believe he's using Debian Jessy
> without any kind of QoS, but his datacenter routers might screw with
> the packets. Maybe I can hook a spare box up to the WAN port
> temporarily to test with.

	Ah, there are three netperf servers out there that, as far as I know, we are allowed to use:
netperf-west.bufferbloat.net
netperf-east.bufferbloat.net
netperf-eu.bufferbloat.net

	My guess is the last one should be closest to you, and should do at least for initial tests. Having a faster closer netperf server available certainly is even more attractive.


Best Regards
	Sebastian

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

Sorry this is getting a bit off-topic here.

> On Wed, Apr 15, 2015 at 5:05 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>
>> On Apr 15, 2015, at 03:35 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:
>
>> I assume tweaking ring parameters from default RX:128 and TX:32
>> doesn't matter anymore thenr?
>
>         As far as I know we leave that alone, see: http://www.bufferbloat.net/projects/bloat/wiki/Linux_Tips:
> “Set the size of the ring buffer for the network interface
>
> NOTE: THIS HACK IS NO LONGER NEEDED on many ethernet drivers in Linux 3.3, which has Byte Queue Limits instead, which does a far better job."
>
I noticed Dave mentioned on a mailing list that changing tx ring still
does have some benefit, and his notes in debloat script suggest BQL
doesn't always work as implied.
>
>>
>>> [...]
>>> If you have time and netperf-wrapper it would be good to convince yourself and us again, that txqueuelen really does not matter for BQL’d interfaces by running RRUL tests with and without your modifications….

Thanks, after extensive RRUL testing.... I've come to the same
conclusion Dave did, that changing tx perameters just isn't worth it
and causes instability. I noticed on 120s tests my WAN connection
would reset with ath9k: pll_reg and latencies would skyrocket
thereafter. I don't quite have a producible error, but it seemed like
associating/diassociating wireless clients might be related to it
(with Revert "debloat: stop changing wifi qlen") but I was also
changing txring on ethernet for testing at 4, 8, 16, etc.

Also, I tested some custom HFSC+fq_codel qos scripts here:
https://github.com/zcecc22/qos-nxt
He defaults to 90% (meaning you have to adjust your b/w limits), and
the 2-bin codel doesn't seem to work very well. Seemed like an
interesting compromise between simple and simplest, but the results
were pretty terrible. I'd like to test CAKE more, but it seems
3.10.50-1 doesn't have the required kernel support.

Recent changes in barrier breaker to txring seem pretty dumb, they
default to 256 txring now I believe, ticket here was closed with
"worksforme" https://dev.openwrt.org/ticket/13072 so I'm reluctant to
upgrade, plus I don't fully understand the extent of which Dave's
kernel hacks impact things. Closer inspection/comparison/diffs are
needed if I'm to upgrade and integrate Cero's tweaks.

Oddly enough, simplest.qos on WAN gives me better throughput/latency
at times (likely due to less overhead), but other times simple.qos is
doing what it should and giving more throughput and lower latency to
higher priority traffic. I seem to get better RRUL tests with LIMIT=
blank, and ILIMIT/ELIMIT set to auto which results in this:

qdisc fq_codel a: dev se00 root refcnt 2 limit 1514p flows 1024
quantum 1514 target 5.0ms interval 100.0ms ecn
qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 12
direct_packets_stat 0 direct_qlen 1000
qdisc fq_codel 110: dev ge00 parent 1:11 limit 1024p flows 1024
quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 120: dev ge00 parent 1:12 limit 1024p flows 1024
quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 130: dev ge00 parent 1:13 limit 1024p flows 1024
quantum 300 target 5.0ms interval 100.0ms ecn
qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
qdisc mq 1: dev sw10 root
qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum
500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum
300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum
300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum
300 target 5.0ms interval 100.0ms
qdisc mq 1: dev sw00 root
qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum
500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum
300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum
300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum
300 target 5.0ms interval 100.0ms
qdisc htb 1: dev ifb4ge00 root refcnt 2 r2q 10 default 12
direct_packets_stat 0 direct_qlen 32
qdisc fq_codel 110: dev ifb4ge00 parent 1:11 limit 1024p flows 1024
quantum 500 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 120: dev ifb4ge00 parent 1:12 limit 1024p flows 1024
quantum 1500 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 130: dev ifb4ge00 parent 1:13 limit 1024p flows 1024
quantum 300 target 5.0ms interval 100.0ms ecn
qdisc htb 1: dev ifb4gw00 root refcnt 2 r2q 10 default 12
direct_packets_stat 0 direct_qlen 32
qdisc fq_codel 110: dev ifb4gw00 parent 1:11 limit 1024p flows 1024
quantum 500 target 10.3ms interval 105.3ms ecn
qdisc fq_codel 120: dev ifb4gw00 parent 1:12 limit 1024p flows 1024
quantum 1500 target 10.3ms interval 105.3ms ecn
qdisc fq_codel 130: dev ifb4gw00 parent 1:13 limit 1024p flows 1024
quantum 300 target 10.3ms interval 105.3ms ecn

image of RRUL 45s graph here with simple.qos, no tx changes, auto
LIMIT on FiOS 32/25 (30Mb/22.5Mb QoS): https://screencloud.net/v/tVV0
- looks pretty good to me, but I should set up more MARK or DSCP
classifications for my important/unimportant traffic. MARK is probably
a better idea since it won't unnecessarily mis-flag outgoing traffic.
I assume QOS_MARK_ge00 sees marks from other interfaces too.

I'm still unsure whether to apply simple/simplest to my secure
wireless or leave it alone, Dave's debloat script seems to have
wireless-specific optimizations when left on auto, does simple.qos
handle VO/VI/BE/BK queues as efficiently? I never top out my wireless
since it's used only for mobile phones anyways and I run HT20 which
seems to be more reliable/less latency. however my guest wifi I do
need to limit and segregate via firewall so I have it enabled.

P.S. I learned the hard way NEVER to enable port 4 on the switch,
results in broken ethernet. port4 is unused and likely internally
reserved for unknown purposes. I'm still trying to figure out how it
maps an interface to an actual port, since I'd like to assign a single
switch switch port to it's own subnet for my FiOS router instead of
having to use a secondary router to clone the ge00 interface on the
backend router to forward FiOS ports to the verizon/FiOS MOCA bridge
router in order for alerts to display on set-top boxes such as caller
ID. There has to be a way of doing this without needing 3 routers...
My current thoughts are to remove the port (port3 in this case) from
the switch and make a new switch config with just 4 and 5t and somehow
make a new interface on that for the FiOS router, but assigning the
same ip address as the default gateway/route from ge00 on that port
might confuse routing. More information on their rather complicated
and seemingly unnecessary config with a backend router is located
here: http://www.dslreports.com/faq/verizonfios/3.0_Networking#16710

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

Correcton on P.S. section: 3 and 5t not 4 and 5t.

also regarding my tc qdisc output: i clearly need to reboot or
something to fix the duplicate IFBs after testing a bunch of QoS. I
currently have to hand-mix the latest ones with Cero since I don't
have an updated luci-sqm

On Tue, Apr 21, 2015 at 8:19 PM, leetminiwheat <LeetMiniWheat@gmail.com> wrote:
> Sorry this is getting a bit off-topic here.
>
>> On Wed, Apr 15, 2015 at 5:05 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>>
>>> On Apr 15, 2015, at 03:35 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:
>>
>>> I assume tweaking ring parameters from default RX:128 and TX:32
>>> doesn't matter anymore thenr?
>>
>>         As far as I know we leave that alone, see: http://www.bufferbloat.net/projects/bloat/wiki/Linux_Tips:
>> “Set the size of the ring buffer for the network interface
>>
>> NOTE: THIS HACK IS NO LONGER NEEDED on many ethernet drivers in Linux 3.3, which has Byte Queue Limits instead, which does a far better job."
>>
> I noticed Dave mentioned on a mailing list that changing tx ring still
> does have some benefit, and his notes in debloat script suggest BQL
> doesn't always work as implied.
>>
>>>
>>>> [...]
>>>> If you have time and netperf-wrapper it would be good to convince yourself and us again, that txqueuelen really does not matter for BQL’d interfaces by running RRUL tests with and without your modifications….
>
> Thanks, after extensive RRUL testing.... I've come to the same
> conclusion Dave did, that changing tx perameters just isn't worth it
> and causes instability. I noticed on 120s tests my WAN connection
> would reset with ath9k: pll_reg and latencies would skyrocket
> thereafter. I don't quite have a producible error, but it seemed like
> associating/diassociating wireless clients might be related to it
> (with Revert "debloat: stop changing wifi qlen") but I was also
> changing txring on ethernet for testing at 4, 8, 16, etc.
>
> Also, I tested some custom HFSC+fq_codel qos scripts here:
> https://github.com/zcecc22/qos-nxt
> He defaults to 90% (meaning you have to adjust your b/w limits), and
> the 2-bin codel doesn't seem to work very well. Seemed like an
> interesting compromise between simple and simplest, but the results
> were pretty terrible. I'd like to test CAKE more, but it seems
> 3.10.50-1 doesn't have the required kernel support.
>
> Recent changes in barrier breaker to txring seem pretty dumb, they
> default to 256 txring now I believe, ticket here was closed with
> "worksforme" https://dev.openwrt.org/ticket/13072 so I'm reluctant to
> upgrade, plus I don't fully understand the extent of which Dave's
> kernel hacks impact things. Closer inspection/comparison/diffs are
> needed if I'm to upgrade and integrate Cero's tweaks.
>
> Oddly enough, simplest.qos on WAN gives me better throughput/latency
> at times (likely due to less overhead), but other times simple.qos is
> doing what it should and giving more throughput and lower latency to
> higher priority traffic. I seem to get better RRUL tests with LIMIT=
> blank, and ILIMIT/ELIMIT set to auto which results in this:
>
> qdisc fq_codel a: dev se00 root refcnt 2 limit 1514p flows 1024
> quantum 1514 target 5.0ms interval 100.0ms ecn
> qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 12
> direct_packets_stat 0 direct_qlen 1000
> qdisc fq_codel 110: dev ge00 parent 1:11 limit 1024p flows 1024
> quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 120: dev ge00 parent 1:12 limit 1024p flows 1024
> quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 130: dev ge00 parent 1:13 limit 1024p flows 1024
> quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
> qdisc mq 1: dev sw10 root
> qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum
> 500 target 10.0ms interval 100.0ms
> qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms
> qdisc mq 1: dev sw00 root
> qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum
> 500 target 10.0ms interval 100.0ms
> qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms
> qdisc htb 1: dev ifb4ge00 root refcnt 2 r2q 10 default 12
> direct_packets_stat 0 direct_qlen 32
> qdisc fq_codel 110: dev ifb4ge00 parent 1:11 limit 1024p flows 1024
> quantum 500 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 120: dev ifb4ge00 parent 1:12 limit 1024p flows 1024
> quantum 1500 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 130: dev ifb4ge00 parent 1:13 limit 1024p flows 1024
> quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc htb 1: dev ifb4gw00 root refcnt 2 r2q 10 default 12
> direct_packets_stat 0 direct_qlen 32
> qdisc fq_codel 110: dev ifb4gw00 parent 1:11 limit 1024p flows 1024
> quantum 500 target 10.3ms interval 105.3ms ecn
> qdisc fq_codel 120: dev ifb4gw00 parent 1:12 limit 1024p flows 1024
> quantum 1500 target 10.3ms interval 105.3ms ecn
> qdisc fq_codel 130: dev ifb4gw00 parent 1:13 limit 1024p flows 1024
> quantum 300 target 10.3ms interval 105.3ms ecn
>
> image of RRUL 45s graph here with simple.qos, no tx changes, auto
> LIMIT on FiOS 32/25 (30Mb/22.5Mb QoS): https://screencloud.net/v/tVV0
> - looks pretty good to me, but I should set up more MARK or DSCP
> classifications for my important/unimportant traffic. MARK is probably
> a better idea since it won't unnecessarily mis-flag outgoing traffic.
> I assume QOS_MARK_ge00 sees marks from other interfaces too.
>
> I'm still unsure whether to apply simple/simplest to my secure
> wireless or leave it alone, Dave's debloat script seems to have
> wireless-specific optimizations when left on auto, does simple.qos
> handle VO/VI/BE/BK queues as efficiently? I never top out my wireless
> since it's used only for mobile phones anyways and I run HT20 which
> seems to be more reliable/less latency. however my guest wifi I do
> need to limit and segregate via firewall so I have it enabled.
>
> P.S. I learned the hard way NEVER to enable port 4 on the switch,
> results in broken ethernet. port4 is unused and likely internally
> reserved for unknown purposes. I'm still trying to figure out how it
> maps an interface to an actual port, since I'd like to assign a single
> switch switch port to it's own subnet for my FiOS router instead of
> having to use a secondary router to clone the ge00 interface on the
> backend router to forward FiOS ports to the verizon/FiOS MOCA bridge
> router in order for alerts to display on set-top boxes such as caller
> ID. There has to be a way of doing this without needing 3 routers...
> My current thoughts are to remove the port (port3 in this case) from
> the switch and make a new switch config with just 4 and 5t and somehow
> make a new interface on that for the FiOS router, but assigning the
> same ip address as the default gateway/route from ge00 on that port
> might confuse routing. More information on their rather complicated
> and seemingly unnecessary config with a backend router is located
> here: http://www.dslreports.com/faq/verizonfios/3.0_Networking#16710

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[Sebastian Moeller]

Hi leetminiwheat,


On Apr 22, 2015, at 02:19 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:

> Sorry this is getting a bit off-topic here.
> 
>> On Wed, Apr 15, 2015 at 5:05 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>>> On Apr 15, 2015, at 03:35 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:
>> 
>>> I assume tweaking ring parameters from default RX:128 and TX:32
>>> doesn't matter anymore thenr?
>> 
>>        As far as I know we leave that alone, see: http://www.bufferbloat.net/projects/bloat/wiki/Linux_Tips:
>> “Set the size of the ring buffer for the network interface
>> 
>> NOTE: THIS HACK IS NO LONGER NEEDED on many ethernet drivers in Linux 3.3, which has Byte Queue Limits instead, which does a far better job."
>> 
> I noticed Dave mentioned on a mailing list that changing tx ring still
> does have some benefit, and his notes in debloat script suggest BQL
> doesn't always work as implied.

	Could you cite that note please? I can not find it, @Dave maybe you could comment on the notes applicability?

>> 
>>> 
>>>> [...]
>>>> If you have time and netperf-wrapper it would be good to convince yourself and us again, that txqueuelen really does not matter for BQL’d interfaces by running RRUL tests with and without your modifications….
> 
> Thanks, after extensive RRUL testing.... I've come to the same
> conclusion Dave did, that changing tx perameters just isn't worth it
> and causes instability. I noticed on 120s tests my WAN connection
> would reset with ath9k: pll_reg and latencies would skyrocket
> thereafter. I don't quite have a producible error, but it seemed like
> associating/diassociating wireless clients might be related to it
> (with Revert "debloat: stop changing wifi qlen") but I was also
> changing txring on ethernet for testing at 4, 8, 16, etc.
> 
> Also, I tested some custom HFSC+fq_codel qos scripts here:
> https://github.com/zcecc22/qos-nxt

	Inteeresting, does his never sqm-qos work better for you?

> He defaults to 90% (meaning you have to adjust your b/w limits), and
> the 2-bin codel doesn't seem to work very well. Seemed like an
> interesting compromise between simple and simplest, but the results
> were pretty terrible.

	If you have a RRUL plot to share that would be helpful.

> I'd like to test CAKE more, but it seems
> 3.10.50-1 doesn't have the required kernel support.
> 
> Recent changes in barrier breaker to txring seem pretty dumb, they
> default to 256 txring now I believe, ticket here was closed with
> "worksforme" https://dev.openwrt.org/ticket/13072 so I'm reluctant to
> upgrade, plus I don't fully understand the extent of which Dave's
> kernel hacks impact things. Closer inspection/comparison/diffs are
> needed if I'm to upgrade and integrate Cero's tweaks.

	I am probably off here, but I assume that with a properly sizes BQL the actual tx ring does not matter for latency anymore and can be selected to keep the hardware happy ;)

> 
> Oddly enough, simplest.qos on WAN gives me better throughput/latency
> at times (likely due to less overhead), but other times simple.qos is
> doing what it should and giving more throughput and lower latency to
> higher priority traffic. I seem to get better RRUL tests with LIMIT=
> blank, and ILIMIT/ELIMIT set to auto which results in this:

	Hmm, LIMIT defaults to 1001 basically so we can see that it was not set, but left at its default. I believe that at one time limit defaulted to 10240 packets which was to large for a wndr3700, that’s why I changed that to 1001, but I have a hard time believing that the difference between 1001 and 1024 packets should affect RRUL noticeably… but as always if the data supports that notion I am willing to adapt...

> 
> qdisc fq_codel a: dev se00 root refcnt 2 limit 1514p flows 1024
> quantum 1514 target 5.0ms interval 100.0ms ecn
> qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 12
> direct_packets_stat 0 direct_qlen 1000
> qdisc fq_codel 110: dev ge00 parent 1:11 limit 1024p flows 1024
> quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 120: dev ge00 parent 1:12 limit 1024p flows 1024
> quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 130: dev ge00 parent 1:13 limit 1024p flows 1024
> quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
> qdisc mq 1: dev sw10 root
> qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum
> 500 target 10.0ms interval 100.0ms
> qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms
> qdisc mq 1: dev sw00 root
> qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum
> 500 target 10.0ms interval 100.0ms
> qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum
> 300 target 5.0ms interval 100.0ms
> qdisc htb 1: dev ifb4ge00 root refcnt 2 r2q 10 default 12
> direct_packets_stat 0 direct_qlen 32
> qdisc fq_codel 110: dev ifb4ge00 parent 1:11 limit 1024p flows 1024
> quantum 500 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 120: dev ifb4ge00 parent 1:12 limit 1024p flows 1024
> quantum 1500 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 130: dev ifb4ge00 parent 1:13 limit 1024p flows 1024
> quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc htb 1: dev ifb4gw00 root refcnt 2 r2q 10 default 12
> direct_packets_stat 0 direct_qlen 32

	I believe this was left-over as gw00 (temporarily) went away and hence the egress interface disappeared. More recent versions of sqm-scripts try to handle this by reacting to the ifup hotplug events. You might want to try the most recent version of sqm-scripts.


> qdisc fq_codel 110: dev ifb4gw00 parent 1:11 limit 1024p flows 1024
> quantum 500 target 10.3ms interval 105.3ms ecn
> qdisc fq_codel 120: dev ifb4gw00 parent 1:12 limit 1024p flows 1024
> quantum 1500 target 10.3ms interval 105.3ms ecn
> qdisc fq_codel 130: dev ifb4gw00 parent 1:13 limit 1024p flows 1024
> quantum 300 target 10.3ms interval 105.3ms ecn
> 
> image of RRUL 45s graph here with simple.qos, no tx changes, auto
> LIMIT on FiOS 32/25 (30Mb/22.5Mb QoS): https://screencloud.net/v/tVV0
> - looks pretty good to me,

	So what is a bit weird is that you see the priority banding in the download direction (upper plot) but not in the egress direction, for IPv4 this typically looks different, were you by any chance using IPv6 for this test?

> but I should set up more MARK or DSCP
> classifications for my important/unimportant traffic. MARK is probably
> a better idea since it won't unnecessarily mis-flag outgoing traffic.
> I assume QOS_MARK_ge00 sees marks from other interfaces too.
> 
> I'm still unsure whether to apply simple/simplest to my secure
> wireless or leave it alone, Dave's debloat script seems to have
> wireless-specific optimizations when left on auto, does simple.qos
> handle VO/VI/BE/BK queues as efficiently?

	No sqm-scripts does not touch your interfaces to that level of detail. Setting up sqm-scripts on a wireless interface works okay, as long as you have a good estimate what the lowest on-air rate is going to be (and the shape to 50% of that rate ;)), sqm-scripts does not really handle the half-duplexicity of wifi too well… . But if you are willing to shape both ingress and egress aggressively enough you should see stable low(er) latencies under load, until your wifi card will exercise the VO/VI/BE/BK queues, then all bets are off. I find wifi with its highly variable link-rates a bit tedious to shape, either you sacrifice a lot of bandwidth or you end up with a shaper that only works half of the time… I am rooting for Dave to get-wifi-fast soon ;)

> I never top out my wireless
> since it's used only for mobile phones anyways and I run HT20 which
> seems to be more reliable/less latency.

	As compared to ht40? On 5GHz or on 2.4GHz?

> however my guest wifi I do
> need to limit and segregate via firewall so I have it enabled.

	Silly question, why do you need to limit your guest wifi? At most I would disallow the guest the use of the highest priority queue (sort of keeping it as a poor man’s control plane), but otherwise let them have their fair share of the full bandwidth after all they are my guests ;) (or alternatively put that traffic into the BK queue so they will never really delay your traffic)

> 
> P.S. I learned the hard way NEVER to enable port 4 on the switch,
> results in broken ethernet.

	Not sure what hardware we are talking about, but on my wndr3700v2 all ports seem to work okay, that is to say the CPU talks to the switch and I can plug in a cable in all 4 exposed lan ports and get working connectivity, so I am surely missing something.

> port4 is unused and likely internally
> reserved for unknown purposes. I'm still trying to figure out how it
> maps an interface to an actual port, since I'd like to assign a single
> switch switch port to it's own subnet for my FiOS router instead of
> having to use a secondary router to clone the ge00 interface on the
> backend router to forward FiOS ports to the verizon/FiOS MOCA bridge
> router in order for alerts to display on set-top boxes such as caller
> ID. There has to be a way of doing this without needing 3 routers...
> My current thoughts are to remove the port (port3 in this case) from
> the switch and make a new switch config with just 4 and 5t and somehow
> make a new interface on that for the FiOS router, but assigning the
> same ip address as the default gateway/route from ge00 on that port
> might confuse routing. More information on their rather complicated
> and seemingly unnecessary config with a backend router is located
> here: http://www.dslreports.com/faq/verizonfios/3.0_Networking#16710

	Again, this is far away from my area of expertise, but I would simple use a switch as poor man’s aggregation network between the ONT’s ethernet-port and the main-router; and then I would connrect the actiontec’s wan port to the same switch. With a bit of vlan? configuration it should be possible to have the actiontec only see the main-router (to not confuse the ONT, but maybe that is not necessary), and use a fixed route from the main-router to the actiontec’s network with the devices you want to access. I guess at that level of abstraction I avoid all the pesky issues cropping up when trying to implement that ;)


Best Regards
	Sebastian

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 1134 bytes --]

Hi leetminiwheat,


On Apr 22, 2015, at 02:28 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:

> Correcton on P.S. section: 3 and 5t not 4 and 5t.

	This is confusing me ;)

> 
> also regarding my tc qdisc output: i clearly need to reboot or
> something to fix the duplicate IFBs after testing a bunch of QoS.

	You could also try to run /usb/lib/sqm/stop.sh $NAME_OF_THE_INTERFACE (so not ifb5gw00 but rather gw00) that hopefully cleans up the left, let me know how that works...

> I currently have to hand-mix the latest ones with Cero since I don't
> have an updated luci-sqm
> 

Please find attached the most recent sqm-scripts and the most recent luci-sqm (the relevant script) please copy sqm-cbi.lua to:
/usr/lib/lua/luci/model/cbi/sqm.lua and you should be as up to date as you can be. Make sure to also move all the files from the attached sqm-scripts folder to the matching folders on your cerowrt router; that should hopefully fix the leftover IFB issue to some degree (we currently do not clean up when an interface goes away, only when the interface gets upped again)


Best Regards
	Sebastian

[-- Attachment #2: sqm-cbi.lua --]
[-- Type: application/octet-stream, Size: 8144 bytes --]

--[[
LuCI - Lua Configuration Interface

Copyright 2014 Steven Barth <steven@midlink.org>
Copyright 2014 Dave Taht <dave.taht@bufferbloat.net>

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

$Id$
]]--

local wa = require "luci.tools.webadmin"
local fs = require "nixio.fs"
local net = require "luci.model.network".init()
local sys = require "luci.sys"
--local ifaces = net:get_interfaces()
local ifaces = sys.net:devices()
local path = "/usr/lib/sqm"

m = Map("sqm", translate("Smart Queue Management"),
	translate("With <abbr title=\"Smart Queue Management\">SQM</abbr> you " ..
		"can enable traffic shaping, better mixing (Fair Queueing)," ..
		" active queue length management (AQM) " ..
		" and prioritisation on one " ..
		"network interface."))

s = m:section(TypedSection, "queue", translate("Queues"))
s:tab("tab_basic", translate("Basic Settings"))
s:tab("tab_qdisc", translate("Queue Discipline"))
s:tab("tab_linklayer", translate("Link Layer Adaptation"))
s.addremove = true -- set to true to allow adding SQM instances in the GUI
s.anonymous = true

-- BASIC
e = s:taboption("tab_basic", Flag, "enabled", translate("Enable"))
e.rmempty = false

n = s:taboption("tab_basic", ListValue, "interface", translate("Interface name"))
-- sm lifted from luci-app-wol, the original implementation failed to show pppoe-ge00 type interface names
for _, iface in ipairs(ifaces) do
--     if iface:is_up() then
--	n:value(iface:name())
--     end
	if iface ~= "lo" then 
		n:value(iface) 
	end
end
n.rmempty = false


dl = s:taboption("tab_basic", Value, "download", translate("Download speed (kbit/s) (ingress):"))
dl.datatype = "and(uinteger,min(0))"
dl.rmempty = false

ul = s:taboption("tab_basic", Value, "upload", translate("Upload speed (kbit/s) (egress):"))
ul.datatype = "and(uinteger,min(0))"
ul.rmempty = false

-- QDISC

c = s:taboption("tab_qdisc", ListValue, "qdisc", translate("Queueing discipline"))
c:value("fq_codel", "fq_codel ("..translate("default")..")")
c:value("efq_codel")
c:value("nfq_codel")
c:value("sfq")
c:value("codel")
c:value("ns2_codel")
c:value("pie")
c:value("sfq")
c.default = "fq_codel"
c.rmempty = false

local qos_desc = ""
sc = s:taboption("tab_qdisc", ListValue, "script", translate("Queue setup script"))
for file in fs.dir(path) do
  if string.find(file, ".qos$") then
    sc:value(file)
  end
  if string.find(file, ".qos.help$") then
    fh = io.open(path .. "/" .. file, "r")
    qos_desc = qos_desc .. "<p><b>" .. file:gsub(".help$", "") .. ":</b><br />" .. fh:read("*a") .. "</p>"
  end
end
sc.default = "simple.qos"
sc.rmempty = false
sc.description = qos_desc

ad = s:taboption("tab_qdisc", Flag, "qdisc_advanced", translate("Show and Use Advanced Configuration"))
ad.default = false
ad.rmempty = true

squash_dscp  = s:taboption("tab_qdisc", ListValue, "squash_dscp", translate("Squash DSCP on inbound packets (ingress):"))
squash_dscp:value("1", "SQUASH")
squash_dscp:value("0", "DO NOT SQUASH")
squash_dscp.default = "1"
squash_dscp.rmempty = true
squash_dscp:depends("qdisc_advanced", "1")

squash_ingress = s:taboption("tab_qdisc", ListValue, "squash_ingress", translate("Ignore DSCP on ingress:"))
squash_ingress:value("1", "Ignore")
squash_ingress:value("0", "Allow")
squash_ingress.default = "1"
squash_ingress.rmempty = true
squash_ingress:depends("qdisc_advanced", "1")

iecn = s:taboption("tab_qdisc", ListValue, "ingress_ecn", translate("Explicit congestion notification (ECN) status on inbound packets (ingress):"))
iecn:value("ECN", "ECN ("..translate("default")..")")
iecn:value("NOECN")
iecn.default = "ECN"
iecn.rmempty = true
iecn:depends("qdisc_advanced", "1")

eecn = s:taboption("tab_qdisc", ListValue, "egress_ecn", translate("Explicit congestion notification (ECN) status on outbound packets (egress)."))
eecn:value("NOECN", "NOECN ("..translate("default")..")")
eecn:value("ECN")
eecn.default = "NOECN"
eecn.rmempty = true
eecn:depends("qdisc_advanced", "1")

ad2 = s:taboption("tab_qdisc", Flag, "qdisc_really_really_advanced", translate("Show and Use Dangerous Configuration"))
ad2.default = false
ad2.rmempty = true
ad2:depends("qdisc_advanced", "1")

ilim = s:taboption("tab_qdisc", Value, "ilimit", translate("Hard limit on ingress queues; leave empty for default."))
-- ilim.default = 1000
ilim.isnumber = true
ilim.datatype = "and(uinteger,min(0))"
ilim.rmempty = true
ilim:depends("qdisc_really_really_advanced", "1")

elim = s:taboption("tab_qdisc", Value, "elimit", translate("Hard limit on egress queues; leave empty for default."))
-- elim.default = 1000
elim.datatype = "and(uinteger,min(0))"
elim.rmempty = true
elim:depends("qdisc_really_really_advanced", "1")


itarg = s:taboption("tab_qdisc", Value, "itarget", translate("Latency target for ingress, e.g 5ms [units: s, ms, or  us]; leave empty for automatic selection, put in the word default for the qdisc's default."))
itarg.datatype = "string"
itarg.rmempty = true
itarg:depends("qdisc_really_really_advanced", "1")

etarg = s:taboption("tab_qdisc", Value, "etarget", translate("Latency target for egress, e.g. 5ms [units: s, ms, or  us]; leave empty for automatic selection, put in the word default for the qdisc's default."))
etarg.datatype = "string"
etarg.rmempty = true
etarg:depends("qdisc_really_really_advanced", "1")



iqdisc_opts = s:taboption("tab_qdisc", Value, "iqdisc_opts", translate("Advanced option string to pass to the ingress queueing disciplines; no error checking, use very carefully."))
iqdisc_opts.rmempty = true
iqdisc_opts:depends("qdisc_really_really_advanced", "1")

eqdisc_opts = s:taboption("tab_qdisc", Value, "eqdisc_opts", translate("Advanced option string to pass to the egress queueing disciplines; no error checking, use very carefully."))
eqdisc_opts.rmempty = true
eqdisc_opts:depends("qdisc_really_really_advanced", "1")

-- LINKLAYER
ll = s:taboption("tab_linklayer", ListValue, "linklayer", translate("Which link layer to account for:"))
ll:value("none", "none ("..translate("default")..")")
ll:value("ethernet", "Ethernet with overhead: select for e.g. VDSL2.")
ll:value("atm", "ATM: select for e.g. ADSL1, ADSL2, ADSL2+.")
-- ll:value("adsl")	-- reduce the options
ll.default = "none"

po = s:taboption("tab_linklayer", Value, "overhead", translate("Per Packet Overhead (byte):"))
po.datatype = "and(integer,min(-1500))"
po.default = 0
po.isnumber = true
po.rmempty = true
po:depends("linklayer", "ethernet")
-- po:depends("linklayer", "adsl")
po:depends("linklayer", "atm")


adll = s:taboption("tab_linklayer", Flag, "linklayer_advanced", translate("Show Advanced Linklayer Options, (only needed if MTU > 1500)"))
adll.rmempty = true
adll:depends("linklayer", "ethernet")
-- adll:depends("linklayer", "adsl")
adll:depends("linklayer", "atm")

smtu = s:taboption("tab_linklayer", Value, "tcMTU", translate("Maximal Size for size and rate calculations, tcMTU (byte); needs to be >= interface MTU + overhead:"))
smtu.datatype = "and(uinteger,min(0))"
smtu.default = 2047
smtu.isnumber = true
smtu.rmempty = true
smtu:depends("linklayer_advanced", "1")

stsize = s:taboption("tab_linklayer", Value, "tcTSIZE", translate("Number of entries in size/rate tables, TSIZE; for ATM choose TSIZE = (tcMTU + 1) / 16:"))
stsize.datatype = "and(uinteger,min(0))"
stsize.default = 128
stsize.isnumber = true
stsize.rmempty = true
stsize:depends("linklayer_advanced", "1")

smpu = s:taboption("tab_linklayer", Value, "tcMPU", translate("Minimal packet size, MPU (byte); needs to be > 0 for ethernet size tables:"))
smpu.datatype = "and(uinteger,min(0))"
smpu.default = 0
smpu.isnumber = true
smpu.rmempty = true
smpu:depends("linklayer_advanced", "1")

lla = s:taboption("tab_linklayer", ListValue, "linklayer_adaptation_mechanism", translate("Which linklayer adaptation mechanism to use; for testing only"))
lla:value("htb_private")
lla:value("tc_stab", "tc_stab ("..translate("default")..")")
lla.default = "tc_stab"
lla.rmempty = true
lla:depends("linklayer_advanced", "1")

-- PRORITIES?

return m

[-- Attachment #3: sqm-scripts.zip --]
[-- Type: application/zip, Size: 27965 bytes --]

[-- Attachment #4: Type: text/plain, Size: 7617 bytes --]

> 
> On Tue, Apr 21, 2015 at 8:19 PM, leetminiwheat <LeetMiniWheat@gmail.com> wrote:
>> Sorry this is getting a bit off-topic here.
>> 
>>> On Wed, Apr 15, 2015 at 5:05 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>>> 
>>>> On Apr 15, 2015, at 03:35 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:
>>> 
>>>> I assume tweaking ring parameters from default RX:128 and TX:32
>>>> doesn't matter anymore thenr?
>>> 
>>>        As far as I know we leave that alone, see: http://www.bufferbloat.net/projects/bloat/wiki/Linux_Tips:
>>> “Set the size of the ring buffer for the network interface
>>> 
>>> NOTE: THIS HACK IS NO LONGER NEEDED on many ethernet drivers in Linux 3.3, which has Byte Queue Limits instead, which does a far better job."
>>> 
>> I noticed Dave mentioned on a mailing list that changing tx ring still
>> does have some benefit, and his notes in debloat script suggest BQL
>> doesn't always work as implied.
>>> 
>>>> 
>>>>> [...]
>>>>> If you have time and netperf-wrapper it would be good to convince yourself and us again, that txqueuelen really does not matter for BQL’d interfaces by running RRUL tests with and without your modifications….
>> 
>> Thanks, after extensive RRUL testing.... I've come to the same
>> conclusion Dave did, that changing tx perameters just isn't worth it
>> and causes instability. I noticed on 120s tests my WAN connection
>> would reset with ath9k: pll_reg and latencies would skyrocket
>> thereafter. I don't quite have a producible error, but it seemed like
>> associating/diassociating wireless clients might be related to it
>> (with Revert "debloat: stop changing wifi qlen") but I was also
>> changing txring on ethernet for testing at 4, 8, 16, etc.
>> 
>> Also, I tested some custom HFSC+fq_codel qos scripts here:
>> https://github.com/zcecc22/qos-nxt
>> He defaults to 90% (meaning you have to adjust your b/w limits), and
>> the 2-bin codel doesn't seem to work very well. Seemed like an
>> interesting compromise between simple and simplest, but the results
>> were pretty terrible. I'd like to test CAKE more, but it seems
>> 3.10.50-1 doesn't have the required kernel support.
>> 
>> Recent changes in barrier breaker to txring seem pretty dumb, they
>> default to 256 txring now I believe, ticket here was closed with
>> "worksforme" https://dev.openwrt.org/ticket/13072 so I'm reluctant to
>> upgrade, plus I don't fully understand the extent of which Dave's
>> kernel hacks impact things. Closer inspection/comparison/diffs are
>> needed if I'm to upgrade and integrate Cero's tweaks.
>> 
>> Oddly enough, simplest.qos on WAN gives me better throughput/latency
>> at times (likely due to less overhead), but other times simple.qos is
>> doing what it should and giving more throughput and lower latency to
>> higher priority traffic. I seem to get better RRUL tests with LIMIT=
>> blank, and ILIMIT/ELIMIT set to auto which results in this:
>> 
>> qdisc fq_codel a: dev se00 root refcnt 2 limit 1514p flows 1024
>> quantum 1514 target 5.0ms interval 100.0ms ecn
>> qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 12
>> direct_packets_stat 0 direct_qlen 1000
>> qdisc fq_codel 110: dev ge00 parent 1:11 limit 1024p flows 1024
>> quantum 300 target 5.0ms interval 100.0ms ecn
>> qdisc fq_codel 120: dev ge00 parent 1:12 limit 1024p flows 1024
>> quantum 300 target 5.0ms interval 100.0ms ecn
>> qdisc fq_codel 130: dev ge00 parent 1:13 limit 1024p flows 1024
>> quantum 300 target 5.0ms interval 100.0ms ecn
>> qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
>> qdisc mq 1: dev sw10 root
>> qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum
>> 500 target 10.0ms interval 100.0ms
>> qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum
>> 300 target 5.0ms interval 100.0ms ecn
>> qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum
>> 300 target 5.0ms interval 100.0ms ecn
>> qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum
>> 300 target 5.0ms interval 100.0ms
>> qdisc mq 1: dev sw00 root
>> qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum
>> 500 target 10.0ms interval 100.0ms
>> qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum
>> 300 target 5.0ms interval 100.0ms ecn
>> qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum
>> 300 target 5.0ms interval 100.0ms ecn
>> qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum
>> 300 target 5.0ms interval 100.0ms
>> qdisc htb 1: dev ifb4ge00 root refcnt 2 r2q 10 default 12
>> direct_packets_stat 0 direct_qlen 32
>> qdisc fq_codel 110: dev ifb4ge00 parent 1:11 limit 1024p flows 1024
>> quantum 500 target 5.0ms interval 100.0ms ecn
>> qdisc fq_codel 120: dev ifb4ge00 parent 1:12 limit 1024p flows 1024
>> quantum 1500 target 5.0ms interval 100.0ms ecn
>> qdisc fq_codel 130: dev ifb4ge00 parent 1:13 limit 1024p flows 1024
>> quantum 300 target 5.0ms interval 100.0ms ecn
>> qdisc htb 1: dev ifb4gw00 root refcnt 2 r2q 10 default 12
>> direct_packets_stat 0 direct_qlen 32
>> qdisc fq_codel 110: dev ifb4gw00 parent 1:11 limit 1024p flows 1024
>> quantum 500 target 10.3ms interval 105.3ms ecn
>> qdisc fq_codel 120: dev ifb4gw00 parent 1:12 limit 1024p flows 1024
>> quantum 1500 target 10.3ms interval 105.3ms ecn
>> qdisc fq_codel 130: dev ifb4gw00 parent 1:13 limit 1024p flows 1024
>> quantum 300 target 10.3ms interval 105.3ms ecn
>> 
>> image of RRUL 45s graph here with simple.qos, no tx changes, auto
>> LIMIT on FiOS 32/25 (30Mb/22.5Mb QoS): https://screencloud.net/v/tVV0
>> - looks pretty good to me, but I should set up more MARK or DSCP
>> classifications for my important/unimportant traffic. MARK is probably
>> a better idea since it won't unnecessarily mis-flag outgoing traffic.
>> I assume QOS_MARK_ge00 sees marks from other interfaces too.
>> 
>> I'm still unsure whether to apply simple/simplest to my secure
>> wireless or leave it alone, Dave's debloat script seems to have
>> wireless-specific optimizations when left on auto, does simple.qos
>> handle VO/VI/BE/BK queues as efficiently? I never top out my wireless
>> since it's used only for mobile phones anyways and I run HT20 which
>> seems to be more reliable/less latency. however my guest wifi I do
>> need to limit and segregate via firewall so I have it enabled.
>> 
>> P.S. I learned the hard way NEVER to enable port 4 on the switch,
>> results in broken ethernet. port4 is unused and likely internally
>> reserved for unknown purposes. I'm still trying to figure out how it
>> maps an interface to an actual port, since I'd like to assign a single
>> switch switch port to it's own subnet for my FiOS router instead of
>> having to use a secondary router to clone the ge00 interface on the
>> backend router to forward FiOS ports to the verizon/FiOS MOCA bridge
>> router in order for alerts to display on set-top boxes such as caller
>> ID. There has to be a way of doing this without needing 3 routers...
>> My current thoughts are to remove the port (port3 in this case) from
>> the switch and make a new switch config with just 4 and 5t and somehow
>> make a new interface on that for the FiOS router, but assigning the
>> same ip address as the default gateway/route from ge00 on that port
>> might confuse routing. More information on their rather complicated
>> and seemingly unnecessary config with a backend router is located
>> here: http://www.dslreports.com/faq/verizonfios/3.0_Networking#16710

Re: [Cerowrt-devel] squash/ignore DSCP and mangle table questions

[leetminiwheat]

On Wed, Apr 22, 2015 at 5:19 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>
> On Apr 22, 2015, at 02:28 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:
>
>> Correcton on P.S. section: 3 and 5t not 4 and 5t.
>
>         This is confusing me ;)

I just meant the switch vlan config, configuring port 4 to anything
seems to break it. default is 0 1 2 3 5t

my idea was to segregate a vlan on 3. I'll have to give it some more thought.


> Please find attached the most recent sqm-scripts and the most recent luci-sqm (the relevant script) please copy sqm-cbi.lua to:
> /usr/lib/lua/luci/model/cbi/sqm.lua and you should be as up to date as you can be. Make sure to also move all the files from the attached sqm-scripts folder to the matching folders on your cerowrt router; that should hopefully fix the leftover IFB issue to some degree (we currently do not clean up when an interface goes away, only when the interface gets upped again)

Thanks! much appreciated and easier than cherry picking from git,
especially since I don't have a buildroot set up at the moment

On Wed, Apr 22, 2015 at 5:03 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>
> On Apr 22, 2015, at 02:19 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:
>
>> Sorry this is getting a bit off-topic here.
>>
>>> On Wed, Apr 15, 2015 at 5:05 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>>>
>>>> On Apr 15, 2015, at 03:35 , leetminiwheat <LeetMiniWheat@gmail.com> wrote:
>>>
>>>> I assume tweaking ring parameters from default RX:128 and TX:32
>>>> doesn't matter anymore thenr?
>>>
>>>        As far as I know we leave that alone, see: http://www.bufferbloat.net/projects/bloat/wiki/Linux_Tips:
>>> “Set the size of the ring buffer for the network interface
>>>
>>> NOTE: THIS HACK IS NO LONGER NEEDED on many ethernet drivers in Linux 3.3, which has Byte Queue Limits instead, which does a far better job."
>>>
>> I noticed Dave mentioned on a mailing list that changing tx ring still
>> does have some benefit, and his notes in debloat script suggest BQL
>> doesn't always work as implied.
>
>         Could you cite that note please? I can not find it, @Dave maybe you could comment on the notes applicability?

/usr/sbin/debloat
{
 * Byte Queue Limits is supposed to have a rate limiter that works.

It is not very effective at less than 100Mbit. I get ~32k peak there
and with GSO on, at 100Mbit, I have seen latency spikes of up to 70ms.

   (Not recently tested, however)

A per queue limit of 2-3 large packets appears to be the best
compromise at 100Mbit and below. So typically I hammer down BQL to
4.5k or 3k at < 100Mbit, and turn GSO/TSO off, and as a result see
ping against load latencies in the 1 to 2ms range, which is about
what you would expect. I have tried 1500 bytes, which limited the top
end performance to about 84Mbit.
}

Can't seem to find the mailing list archive where Dave mentioned tx
ring still having some further benefit in addition to BQL.

>> Also, I tested some custom HFSC+fq_codel qos scripts here:
>> https://github.com/zcecc22/qos-nxt
>
>         Inteeresting, does his never sqm-qos work better for you?
>
>> He defaults to 90% (meaning you have to adjust your b/w limits), and
>> the 2-bin codel doesn't seem to work very well. Seemed like an
>> interesting compromise between simple and simplest, but the results
>> were pretty terrible.
>
>         If you have a RRUL plot to share that would be helpful.

Actually, I linked the wrong QoS scripts, those were his old ones
which I haven't tested. These are the newer simplified ones:
https://github.com/zcecc22/sqm-qos

nxt.qos plot: http://screencloud.net/v/jHza
nxt_v2.qos plot: http://screencloud.net/v/oe8x

Note: adjusted QoS caps to my full provisioned rate since these
scripts limit to 90% anyways (and I use 90% in
simple.qos/simplest.qos)

as you can see, lots of latency spikes. I'm not sure what these
scripts are intended to accomplish, perhaps they're more optimized for
lower speed connections. I haven't tested his older scripts but they
looked more advanced and even changed the tx rings and such, they're
located here:

>
>> I'd like to test CAKE more, but it seems
>> 3.10.50-1 doesn't have the required kernel support.
>>
>> Recent changes in barrier breaker to txring seem pretty dumb, they
>> default to 256 txring now I believe, ticket here was closed with
>> "worksforme" https://dev.openwrt.org/ticket/13072 so I'm reluctant to
>> upgrade, plus I don't fully understand the extent of which Dave's
>> kernel hacks impact things. Closer inspection/comparison/diffs are
>> needed if I'm to upgrade and integrate Cero's tweaks.
>
>         I am probably off here, but I assume that with a properly sizes BQL the actual tx ring does not matter for latency anymore and can be selected to keep the hardware happy ;)
>
>>
>> Oddly enough, simplest.qos on WAN gives me better throughput/latency
>> at times (likely due to less overhead), but other times simple.qos is
>> doing what it should and giving more throughput and lower latency to
>> higher priority traffic. I seem to get better RRUL tests with LIMIT=
>> blank, and ILIMIT/ELIMIT set to auto which results in this:
>
>         Hmm, LIMIT defaults to 1001 basically so we can see that it was not set, but left at its default. I believe that at one time limit defaulted to 10240 packets which was to large for a wndr3700, that’s why I changed that to 1001, but I have a hard time believing that the difference between 1001 and 1024 packets should affect RRUL noticeably… but as always if the data supports that notion I am willing to adapt...

Yeah I'm not entirely sure... I ran about 20 tests last night with
different settings but didn't extensively test auto vs 1001, i do see
this in the output though but likely the wireless:
SQM: get_limit: CURLIMIT:
SQM: cur_target: auto cur_bandwidth: 1536
SQM: get_limit: CURLIMIT:
SQM: cur_target: auto cur_bandwidth: 1536
SQM: get_limit: CURLIMIT:
SQM: cur_target: auto cur_bandwidth: 1536
SQM: egress shaping activated
SQM: Perform DSCP based filtering on ingress. (3-tier classification)
SQM: get_limit: CURLIMIT:
SQM: cur_target: auto cur_bandwidth: 1192
SQM: get_limit: CURLIMIT:
SQM: cur_target: auto cur_bandwidth: 1192
SQM: get_limit: CURLIMIT:
SQM: cur_target: auto cur_bandwidth: 1192


>> image of RRUL 45s graph here with simple.qos, no tx changes, auto
>> LIMIT on FiOS 32/25 (30Mb/22.5Mb QoS): https://screencloud.net/v/tVV0
>> - looks pretty good to me,
>
>         So what is a bit weird is that you see the priority banding in the download direction (upper plot) but not in the egress direction, for IPv4 this typically looks different, were you by any chance using IPv6 for this test?

This was purely ipv4, verizon still has not rolled out ipv6 and I try
to remove as much of it from Cero as possible. I did notice after the
fact that the upload was a little inconsistent but not entirely sure
what that can be attributed to. I speedtest at 25mbps upload, and rate
limit to 22.5mbps, but verizon could be screwing with it. I did test
20mbps as well but didn't see any noticeable differences. I was mostly
looking at the latency and separation between priority queues. I may
run some more at lower speed again, my upload can be inconsistent
depending on destination (verizon often routes me through really cheap
backbone routes, as a result I need to ssh tunnel my gaming traffic
through a friend's server at the datacenter he works at)

>> I never top out my wireless
>> since it's used only for mobile phones anyways and I run HT20 which
>> seems to be more reliable/less latency.
>
>         As compared to ht40? On 5GHz or on 2.4GHz?

probably applies to both, but HT40 has more overhead.

I heard some talk about cleaning up WiFi recently though, including
fast-tx paths but at the moment it only works in client/ap mode with
single client. recent kernel changes regarding WiFi seem based on some
really bad assumptions though.


>> however my guest wifi I do
>> need to limit and segregate via firewall so I have it enabled.
>
>         Silly question, why do you need to limit your guest wifi? At most I would disallow the guest the use of the highest priority queue (sort of keeping it as a poor man’s control plane), but otherwise let them have their fair share of the full bandwidth after all they are my guests ;) (or alternatively put that traffic into the BK queue so they will never really delay your traffic)

prioritization doesn't always work, as we can see with bittorrent
traffic. and it's not so much for "guests" but more of a hotspot since
I'm in a densely populated suburban area. Also I don't want to
encourage them to leach off me if they have their own faster WiFi, or
can just get their own home internet. I know some may disagree with me
but it IS my portion of the internet that *I* pay for. I pay probably
way more than I should for my fiber service and it seems to have
latency issues upstream with >200 simultaneous connections sending or
receiving data, something that might not ever be avoidable using QoS
on my end. I know I'm on GPON probably on a fiber switch/hub thingy
(forget the name) with 32-64 other customers, about a mile from the
nearest main office.


>> port4 is unused and likely internally
>> reserved for unknown purposes. I'm still trying to figure out how it
>> maps an interface to an actual port, since I'd like to assign a single
>> switch switch port to it's own subnet for my FiOS router instead of
>> having to use a secondary router to clone the ge00 interface on the
>> backend router to forward FiOS ports to the verizon/FiOS MOCA bridge
>> router in order for alerts to display on set-top boxes such as caller
>> ID. There has to be a way of doing this without needing 3 routers...
>> My current thoughts are to remove the port (port3 in this case) from
>> the switch and make a new switch config with just 4 and 5t and somehow
>> make a new interface on that for the FiOS router, but assigning the
>> same ip address as the default gateway/route from ge00 on that port
>> might confuse routing. More information on their rather complicated
>> and seemingly unnecessary config with a backend router is located
>> here: http://www.dslreports.com/faq/verizonfios/3.0_Networking#16710
>
>         Again, this is far away from my area of expertise, but I would simple use a switch as poor man’s aggregation network between the ONT’s ethernet-port and the main-router; and then I would connrect the actiontec’s wan port to the same switch. With a bit of vlan? configuration it should be possible to have the actiontec only see the main-router (to not confuse the ONT, but maybe that is not necessary), and use a fixed route from the main-router to the actiontec’s network with the devices you want to access. I guess at that level of abstraction I avoid all the pesky issues cropping up when trying to implement that ;)

Yeah that was another option I saw, of using a switch with dedicated
VLANs but I don't really have the hardware to do that. putting both on
a simple dumb switch sounds bad if they both request an IP upstream. I
will give it some more thought to somehow create a separate virtual
network inside the WNDR3800 just for the ISP router. somehow just port
forwarding alone doesn't do it.