[Cerowrt-devel] Possible Bug(s) in Cero 3.10.50-1

[Cerowrt-devel] Possible Bug(s) in Cero 3.10.50-1

[Richard]

Hi, all. End user here. Just thought I'd post a few possible bugs I've run
into since updating to 3.10.50-1. I'm not exactly sure if these have been
reported or are intentional, but I figured it couldn't hurt to post them anyway.

1) When using PPPoE on the outbound interface, traffic skips classification
MARKS set by iptables in the QOS_MARK_ge00 chain entirely. This is whilst
using simple.qos. Everything is placed in the 1:12 class in HTB in both
ingress and egress regardless of rules set. This was tried using 3.10.34-4
and then a fresh install of 3.10.50-1.

2) In 3.10.50-1, whilst running multiple Intermediate Functional Blocks,
restarting SQM often has a chance to not close IFBs after the first IFB. i.e
Anything after ifb0 has a chance to not close. Cero then creates a new
Block(s) after the ones that haven't closed as it believes they are still in
use. Doing this enough eventually fills up all available Blocks and then
ingress shaping fails to start.

Workaround for me has been to SSH in, stop SQM completely, and then start it
back up again whenever I change settings as that ensures any lingering IFBs
are closed down. 


Unfortunately, I foolishly forgot to keep any logs using cerostats.sh and no
longer have a modem to test PPPoE on; the one I had couldn't hold the DSL
line for very long and was subsequently returned. I also ran into something
which I thought was Bug #442 after updating to 3.10.50-1. I had moved from
3.10.34-4 using the sysupgrade image.

The router seemed to lock up twice within the first 15mins after boot and
again after reboot. Only the 2.4Ghz network went nuts while 5Ghz remained
fine. Everything on the 2.4Ghz network was still connected, yet nothing on
2.4 could get through - both to the internet and to the router itself. I
then decided to do a clean install and haven't run into it since. This is
something which has happened to me before on an earlier release and I only
ever seem to run into this bug whenever I use a sysupgrade image, or restore
my settings from an archive. 

Something I've noticed is that #442 (or something similar) never seems
happen if I do a clean install and rewrite my settings from scratch... 
Just a thought.

I think that's about it.


And if anyone's willing to answer this, I know this isn't exactly the place
ask this, but, aside from having Cero handle external ICMPs requests, is
there any inherent performance/security/bufferbloat benefits from having
Cero handle my external ip over a gateway --> router combo?

Right now, my setup consist of a gateway and I'm unable to put it in bridge
mode. The gateway does NAT, has SPI disabled, and has a static route and DMZ
defined towards Cero. Cero is connected to the end of it with Masquerading
disabled and the firewall still up. Every device we have runs through Cero.

I'd like to know anything at all before I decide to go looking for another
dedicated modem, or if I should even bother to go looking in the first place.

Hope this helps!
—Regards, Richard

Re: [Cerowrt-devel] Possible Bug(s) in Cero 3.10.50-1

[Sebastian Moeller]

Hi Richard,


On Sep 13, 2014, at 20:06 , Richard <rocon46@hotmail.com> wrote:

> Hi, all. End user here. Just thought I'd post a few possible bugs I've run
> into since updating to 3.10.50-1. I'm not exactly sure if these have been
> reported or are intentional, but I figured it couldn't hurt to post them anyway.
> 
> 1) When using PPPoE on the outbound interface, traffic skips classification
> MARKS set by iptables in the QOS_MARK_ge00 chain entirely. This is whilst
> using simple.qos. Everything is placed in the 1:12 class in HTB in both
> ingress and egress regardless of rules set. This was tried using 3.10.34-4
> and then a fresh install of 3.10.50-1.
> 
> 2) In 3.10.50-1, whilst running multiple Intermediate Functional Blocks,
> restarting SQM often has a chance to not close IFBs after the first IFB. i.e
> Anything after ifb0 has a chance to not close. Cero then creates a new
> Block(s) after the ones that haven't closed as it believes they are still in
> use. Doing this enough eventually fills up all available Blocks and then
> ingress shaping fails to start.

	Ah, so how do you get into this situation from the GUI exactly? If you have a simple set of steps to get to this failure mode I would appreciate to learn. So that I can go fix the IFB detection in cerowrt (one of the issues with IFBs is that I can figure out if (and which) fib is attached to a real interface, but I have found no way to query an IFB to figure out which interface it is attached to; so SQM currently tries to reuse its own IFB devices, but that fails if they are detached from their real interface before sqm-scripts sees them. Also SQM takes IFB down after use and avoid to reassign up IFBs that are not attached to interfaces for which SQM is active…)
       BTW, I tend to deselect the “enable” box in the GUI and then hit the “Save & Apply” button which does not lead to an excess in generated/blocked IFBs for me, but maybe a different approach to interact with the GUI leads to the IFB inflation. But no matter what the cause, I would like to fix that… (the whole idea of the complicated IFB handling was to allow SQM to coexist nicely with manually configured IFBs so the hands-off approach if SQM is not sure who owns an IFB)

> 
> Workaround for me has been to SSH in, stop SQM completely, and then start it
> back up again whenever I change settings as that ensures any lingering IFBs
> are closed down. 
> 
> 
> Unfortunately, I foolishly forgot to keep any logs using cerostats.sh and no
> longer have a modem to test PPPoE on; the one I had couldn't hold the DSL
> line for very long and was subsequently returned. I also ran into something
> which I thought was Bug #442 after updating to 3.10.50-1. I had moved from
> 3.10.34-4 using the sysupgrade image.
> 
> The router seemed to lock up twice within the first 15mins after boot and
> again after reboot. Only the 2.4Ghz network went nuts while 5Ghz remained
> fine. Everything on the 2.4Ghz network was still connected, yet nothing on
> 2.4 could get through - both to the internet and to the router itself.

	Interestingly, I had similar things happen on the same radio, in may case disabling an re-enabling the wifi interfaces for the 2.4GHz radio solved to issue, but that was before 3.10.50-1, no lock ups with that. I always re-entered the network configuration via the GUI on each upgrade (but only used sysupgrade images for ages, with the “-n” flag specified to not keep old config files…)

> I
> then decided to do a clean install and haven't run into it since. This is
> something which has happened to me before on an earlier release and I only
> ever seem to run into this bug whenever I use a sysupgrade image, or restore
> my settings from an archive. 
> 
> Something I've noticed is that #442 (or something similar) never seems
> happen if I do a clean install and rewrite my settings from scratch... 
> Just a thought.
> 
> I think that's about it.
> 
> 
> And if anyone's willing to answer this, I know this isn't exactly the place
> ask this, but, aside from having Cero handle external ICMPs requests, is
> there any inherent performance/security/bufferbloat benefits from having
> Cero handle my external ip over a gateway --> router combo?

	Potentially you could avoid double NAT by having cerowrt handle the pppoe session and still keep cerowrt’s nice numbering scheme...

> 
> Right now, my setup consist of a gateway and I'm unable to put it in bridge
> mode. The gateway does NAT, has SPI disabled, and has a static route and DMZ
> defined towards Cero. Cero is connected to the end of it with Masquerading
> disabled and the firewall still up. Every device we have runs through Cero.

	So I have something similar, except the ISP router does not allow to disable the firewall or configure a DMZ or enable ICMP echo requests from the outside, but all end devices are connected via cerowrt in default mode (NAT and firewall active), so basically double NATed. But so far everything work just fine. On the plus side, if I configure/upgrade cerowrt in un-usable state there is still the ISP router to fall back to…


Best Regards
	Sebastian

> 
> I'd like to know anything at all before I decide to go looking for another
> dedicated modem, or if I should even bother to go looking in the first place.
> 
> Hope this helps!
> —Regards, Richard
> 
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Possible Bug(s) in Cero 3.10.50-1

[Richard]

Sebastian Moeller <moeller0 <at> gmx.de> writes:

> 
> Hi Richard,
> 
> On Sep 13, 2014, at 20:06 , Richard <rocon46 <at> hotmail.com> wrote:
> 
> > Hi, all. End user here. Just thought I'd post a few possible bugs I've run
> > into since updating to 3.10.50-1. I'm not exactly sure if these have been
> > reported or are intentional, but I figured it couldn't hurt to post them
anyway.
> > 
> > 1) When using PPPoE on the outbound interface, traffic skips classification
> > MARKS set by iptables in the QOS_MARK_ge00 chain entirely. This is whilst
> > using simple.qos. Everything is placed in the 1:12 class in HTB in both
> > ingress and egress regardless of rules set. This was tried using 3.10.34-4
> > and then a fresh install of 3.10.50-1.
> > 
> > 2) In 3.10.50-1, whilst running multiple Intermediate Functional Blocks,
> > restarting SQM often has a chance to not close IFBs after the first IFB. i.e
> > Anything after ifb0 has a chance to not close. Cero then creates a new
> > Block(s) after the ones that haven't closed as it believes they are still in
> > use. Doing this enough eventually fills up all available Blocks and then
> > ingress shaping fails to start.
> 
> 	Ah, so how do you get into this situation from the GUI exactly? If you
have a simple set of steps to get to this
> failure mode I would appreciate to learn. So that I can go fix the IFB
detection in cerowrt (one of the issues
> with IFBs is that I can figure out if (and which) fib is attached to a
real interface, but I have found no way to
> query an IFB to figure out which interface it is attached to; so SQM
currently tries to reuse its own IFB
> devices, but that fails if they are detached from their real interface
before sqm-scripts sees them. Also
> SQM takes IFB down after use and avoid to reassign up IFBs that are not
attached to interfaces for which SQM
> is active…)


I had a script I ran which just added DSCP MARKs in the -t mangle FORWARD
table. To save me from deleting them if I made any changes, the script
restarted both the firewall and SQM before adding my rules. It restarted
them using the "/etc/init.d/firewall restart" and "/etc/init.d/sqm restart"
commands. Messing about, using both the gui and reverting the script to use
'restart' instead of 'stop' and 'start', to try and trigger the problem
again hasn't been fruitful. I've been unable to recreate it I'm afraid.



>        BTW, I tend to deselect the “enable” box in the GUI and then hit
the “Save & Apply” button which does
> not lead to an excess in generated/blocked IFBs for me, but maybe a
different approach to interact with the
> GUI leads to the IFB inflation. But no matter what the cause, I would like
to fix that… (the whole idea of
> the complicated IFB handling was to allow SQM to coexist nicely with
manually configured IFBs so the
> hands-off approach if SQM is not sure who owns an IFB)
> 
> > 
> > Workaround for me has been to SSH in, stop SQM completely, and then start it
> > back up again whenever I change settings as that ensures any lingering IFBs
> > are closed down. 
> > 
> > 
> > Unfortunately, I foolishly forgot to keep any logs using cerostats.sh and no
> > longer have a modem to test PPPoE on; the one I had couldn't hold the DSL
> > line for very long and was subsequently returned. I also ran into something
> > which I thought was Bug #442 after updating to 3.10.50-1. I had moved from
> > 3.10.34-4 using the sysupgrade image.
> > 
> > The router seemed to lock up twice within the first 15mins after boot and
> > again after reboot. Only the 2.4Ghz network went nuts while 5Ghz remained
> > fine. Everything on the 2.4Ghz network was still connected, yet nothing on
> > 2.4 could get through - both to the internet and to the router itself.
> 
> 	Interestingly, I had similar things happen on the same radio, in may case
disabling an re-enabling the
> wifi interfaces for the 2.4GHz radio solved to issue, but that was before
3.10.50-1, no lock ups with that.


Great! I'll keep that in mind next time instead of doing a hard reset.


> I always re-entered the network configuration via the GUI on each upgrade
(but only used sysupgrade
> images for ages, with the “-n” flag specified to not keep old config files…)


Wasn't aware of the -n flag. At least I don't have to TFTP to update the
firmware anymore...


> 
> > I
> > then decided to do a clean install and haven't run into it since. This is
> > something which has happened to me before on an earlier release and I only
> > ever seem to run into this bug whenever I use a sysupgrade image, or restore
> > my settings from an archive. 
> > 
> > Something I've noticed is that #442 (or something similar) never seems
> > happen if I do a clean install and rewrite my settings from scratch... 
> > Just a thought.
> > 
> > I think that's about it.
> > 
> > 
> > And if anyone's willing to answer this, I know this isn't exactly the place
> > ask this, but, aside from having Cero handle external ICMPs requests, is
> > there any inherent performance/security/bufferbloat benefits from having
> > Cero handle my external ip over a gateway --> router combo?
> 
> 	Potentially you could avoid double NAT by having cerowrt handle the pppoe
session and still keep
> cerowrt’s nice numbering scheme...
> 
> > 
> > Right now, my setup consist of a gateway and I'm unable to put it in bridge
> > mode. The gateway does NAT, has SPI disabled, and has a static route and DMZ
> > defined towards Cero. Cero is connected to the end of it with Masquerading
> > disabled and the firewall still up. Every device we have runs through Cero.
> 
> 	So I have something similar, except the ISP router does not allow to
disable the firewall or configure a DMZ
> or enable ICMP echo requests from the outside, but all end devices are
connected via cerowrt in default
> mode (NAT and firewall active), so basically double NATed. But so far
everything work just fine. On the
> plus side, if I configure/upgrade cerowrt in un-usable state there is
still the ISP router to fall back to…
> 

Ah, well, I guess I have nothing to worry about then. I was worried that
having Cero not handle my external address mainly might 'cause fq_codel not
to work as well as it should. Hearing that you run a double NAT scenario
with everything still working eases that worry.

I have chosen to disable ICMP echo requests on the gateway, though, as it's
something I can't seem to forward to Cero. Excessive requests might fudge
the traffic shaping as it's something Cerowrt can't take into account. I'm
aware it might break some websites that use MTU path discovery, but I
figured using MSS clamping in Cero would solve any problems that might
incur. That's my reasoning for it, anyway. 

Thanks for the input.
	Richard

> Best Regards
> 	Sebastian
> 
> > 
> > I'd like to know anything at all before I decide to go looking for another
> > dedicated modem, or if I should even bother to go looking in the first
place.
> > 
> > Hope this helps!
> > —Regards, Richard
> > 
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel <at> lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 
> 

Re: [Cerowrt-devel] Possible Bug(s) in Cero 3.10.50-1

[Sebastian Moeller]

Hi Richard,

On Sep 14, 2014, at 09:45 , Richard <rocon46@hotmail.com> wrote:

> Sebastian Moeller <moeller0 <at> gmx.de> writes:
> 
>> 
>> Hi Richard,
>> 
>> On Sep 13, 2014, at 20:06 , Richard <rocon46 <at> hotmail.com> wrote:
>> 
>>> Hi, all. End user here. Just thought I'd post a few possible bugs I've run
>>> into since updating to 3.10.50-1. I'm not exactly sure if these have been
>>> reported or are intentional, but I figured it couldn't hurt to post them
> anyway.
>>> 
>>> 1) When using PPPoE on the outbound interface, traffic skips classification
>>> MARKS set by iptables in the QOS_MARK_ge00 chain entirely. This is whilst
>>> using simple.qos. Everything is placed in the 1:12 class in HTB in both
>>> ingress and egress regardless of rules set. This was tried using 3.10.34-4
>>> and then a fresh install of 3.10.50-1.
>>> 
>>> 2) In 3.10.50-1, whilst running multiple Intermediate Functional Blocks,
>>> restarting SQM often has a chance to not close IFBs after the first IFB. i.e
>>> Anything after ifb0 has a chance to not close. Cero then creates a new
>>> Block(s) after the ones that haven't closed as it believes they are still in
>>> use. Doing this enough eventually fills up all available Blocks and then
>>> ingress shaping fails to start.
>> 
>> 	Ah, so how do you get into this situation from the GUI exactly? If you
> have a simple set of steps to get to this
>> failure mode I would appreciate to learn. So that I can go fix the IFB
> detection in cerowrt (one of the issues
>> with IFBs is that I can figure out if (and which) fib is attached to a
> real interface, but I have found no way to
>> query an IFB to figure out which interface it is attached to; so SQM
> currently tries to reuse its own IFB
>> devices, but that fails if they are detached from their real interface
> before sqm-scripts sees them. Also
>> SQM takes IFB down after use and avoid to reassign up IFBs that are not
> attached to interfaces for which SQM
>> is active…)
> 
> 
> I had a script I ran which just added DSCP MARKs in the -t mangle FORWARD
> table. To save me from deleting them if I made any changes, the script
> restarted both the firewall and SQM before adding my rules. It restarted
> them using the "/etc/init.d/firewall restart" and "/etc/init.d/sqm restart"
> commands. Messing about, using both the gui and reverting the script to use
> 'restart' instead of 'stop' and 'start', to try and trigger the problem
> again hasn't been fruitful. I've been unable to recreate it I'm afraid.

	Too bad, in case you find a way to force this failure mode, please let me know so I can have a go at fixing it...

> 
> 
> 
>>       BTW, I tend to deselect the “enable” box in the GUI and then hit
> the “Save & Apply” button which does
>> not lead to an excess in generated/blocked IFBs for me, but maybe a
> different approach to interact with the
>> GUI leads to the IFB inflation. But no matter what the cause, I would like
> to fix that… (the whole idea of
>> the complicated IFB handling was to allow SQM to coexist nicely with
> manually configured IFBs so the
>> hands-off approach if SQM is not sure who owns an IFB)
>> 
>>> 
>>> Workaround for me has been to SSH in, stop SQM completely, and then start it
>>> back up again whenever I change settings as that ensures any lingering IFBs
>>> are closed down. 
>>> 
>>> 
>>> Unfortunately, I foolishly forgot to keep any logs using cerostats.sh and no
>>> longer have a modem to test PPPoE on; the one I had couldn't hold the DSL
>>> line for very long and was subsequently returned. I also ran into something
>>> which I thought was Bug #442 after updating to 3.10.50-1. I had moved from
>>> 3.10.34-4 using the sysupgrade image.
>>> 
>>> The router seemed to lock up twice within the first 15mins after boot and
>>> again after reboot. Only the 2.4Ghz network went nuts while 5Ghz remained
>>> fine. Everything on the 2.4Ghz network was still connected, yet nothing on
>>> 2.4 could get through - both to the internet and to the router itself.
>> 
>> 	Interestingly, I had similar things happen on the same radio, in may case
> disabling an re-enabling the
>> wifi interfaces for the 2.4GHz radio solved to issue, but that was before
> 3.10.50-1, no lock ups with that.
> 
> 
> Great! I'll keep that in mind next time instead of doing a hard reset.
> 
> 
>> I always re-entered the network configuration via the GUI on each upgrade
> (but only used sysupgrade
>> images for ages, with the “-n” flag specified to not keep old config files…)
> 
> 
> Wasn't aware of the -n flag. At least I don't have to TFTP to update the
> firmware anymore…

	I think a number of people have been quite happy with the sysupgrade via the GUI (just de-select the “keep sttings” check box and you should be fine). I think what really could cause the issues observed is when SQM's stop.sh fails half-way through (I will try to have a look at that later...)

> 
> 
>> 
>>> I
>>> then decided to do a clean install and haven't run into it since. This is
>>> something which has happened to me before on an earlier release and I only
>>> ever seem to run into this bug whenever I use a sysupgrade image, or restore
>>> my settings from an archive. 
>>> 
>>> Something I've noticed is that #442 (or something similar) never seems
>>> happen if I do a clean install and rewrite my settings from scratch... 
>>> Just a thought.
>>> 
>>> I think that's about it.
>>> 
>>> 
>>> And if anyone's willing to answer this, I know this isn't exactly the place
>>> ask this, but, aside from having Cero handle external ICMPs requests, is
>>> there any inherent performance/security/bufferbloat benefits from having
>>> Cero handle my external ip over a gateway --> router combo?
>> 
>> 	Potentially you could avoid double NAT by having cerowrt handle the pppoe
> session and still keep
>> cerowrt’s nice numbering scheme...
>> 
>>> 
>>> Right now, my setup consist of a gateway and I'm unable to put it in bridge
>>> mode. The gateway does NAT, has SPI disabled, and has a static route and DMZ
>>> defined towards Cero. Cero is connected to the end of it with Masquerading
>>> disabled and the firewall still up. Every device we have runs through Cero.
>> 
>> 	So I have something similar, except the ISP router does not allow to
> disable the firewall or configure a DMZ
>> or enable ICMP echo requests from the outside, but all end devices are
> connected via cerowrt in default
>> mode (NAT and firewall active), so basically double NATed. But so far
> everything work just fine. On the
>> plus side, if I configure/upgrade cerowrt in un-usable state there is
> still the ISP router to fall back to…
>> 
> 
> Ah, well, I guess I have nothing to worry about then. I was worried that
> having Cero not handle my external address mainly might 'cause fq_codel not
> to work as well as it should. Hearing that you run a double NAT scenario
> with everything still working eases that worry.

	In theory fq_codel could not care less, as long as it is in control of the bottleneck all should be fine, and latency under load should remain well-behaved.

> 
> I have chosen to disable ICMP echo requests on the gateway, though, as it's
> something I can't seem to forward to Cero.

	Well, my ISP router does not respond to echo requests and does not allow me to forward them to downstream devices and yet things just work (note I am still ruing IPv4)

> Excessive requests might fudge
> the traffic shaping as it's something Cerowrt can't take into account.

	But that is a systematic problem with downstream shaping after the the bottleneck-link (by creation of an artificial bottleneck), too much incoming traffic of any kind will cause backlog in the buffers of the device at the upstream end of the real bottleneck-link… So even if the ICMP packets reach cerowrt this might still hose the downstream latency under load...

> I'm
> aware it might break some websites that use MTU path discovery, but I
> figured using MSS clamping in Cero would solve any problems that might
> incur. That's my reasoning for it, anyway. 

	I would hope that the real xdsl-router handles either the MSS clamping for us or better yet properly handles the IPv6 path mtu discovery (and at least in may case that works well)

Best Regards
	Sebastian

> 
> Thanks for the input.
> 	Richard
> 
>> Best Regards
>> 	Sebastian
>> 
>>> 
>>> I'd like to know anything at all before I decide to go looking for another
>>> dedicated modem, or if I should even bother to go looking in the first
> place.
>>> 
>>> Hope this helps!
>>> —Regards, Richard
>>> 
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel <at> lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> 
>> 
> 
> 
> 
> 
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Possible Bug(s) in Cero 3.10.50-1

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 4250 bytes --]

One of the features of the work going on in the ubnt beta forums was the
discovery that you can create named ifb interfaces. So we could switch sqm
to a 1 to 1 mapping of ge00-ifb, se00-ifb, etc. and thus have an easier
time tearing them down.

I figure that QoS chain needs to be applied to the pppoe interface not the
ge00 interface?

I generally have encouraged folk to always reinstall from scratch. Now that
we are maturing and getting stabler, in place upgrades are becoming more
interesting...

I generally have more faith in cero's fire walling and nat handling than
most third party equipment. So bridging is often better. But what I'd like
most to happen for dsl is finding a good openwrt compatible dsl/wifi modem
and have that as something to recommend to debloat ers on that tech.
On Sep 13, 2014 11:07 AM, "Richard" <rocon46@hotmail.com> wrote:

> Hi, all. End user here. Just thought I'd post a few possible bugs I've run
> into since updating to 3.10.50-1. I'm not exactly sure if these have been
> reported or are intentional, but I figured it couldn't hurt to post them
> anyway.
>
> 1) When using PPPoE on the outbound interface, traffic skips classification
> MARKS set by iptables in the QOS_MARK_ge00 chain entirely. This is whilst
> using simple.qos. Everything is placed in the 1:12 class in HTB in both
> ingress and egress regardless of rules set. This was tried using 3.10.34-4
> and then a fresh install of 3.10.50-1.
>
> 2) In 3.10.50-1, whilst running multiple Intermediate Functional Blocks,
> restarting SQM often has a chance to not close IFBs after the first IFB.
> i.e
> Anything after ifb0 has a chance to not close. Cero then creates a new
> Block(s) after the ones that haven't closed as it believes they are still
> in
> use. Doing this enough eventually fills up all available Blocks and then
> ingress shaping fails to start.
>
> Workaround for me has been to SSH in, stop SQM completely, and then start
> it
> back up again whenever I change settings as that ensures any lingering IFBs
> are closed down.
>
>
> Unfortunately, I foolishly forgot to keep any logs using cerostats.sh and
> no
> longer have a modem to test PPPoE on; the one I had couldn't hold the DSL
> line for very long and was subsequently returned. I also ran into something
> which I thought was Bug #442 after updating to 3.10.50-1. I had moved from
> 3.10.34-4 using the sysupgrade image.
>
> The router seemed to lock up twice within the first 15mins after boot and
> again after reboot. Only the 2.4Ghz network went nuts while 5Ghz remained
> fine. Everything on the 2.4Ghz network was still connected, yet nothing on
> 2.4 could get through - both to the internet and to the router itself. I
> then decided to do a clean install and haven't run into it since. This is
> something which has happened to me before on an earlier release and I only
> ever seem to run into this bug whenever I use a sysupgrade image, or
> restore
> my settings from an archive.
>
> Something I've noticed is that #442 (or something similar) never seems
> happen if I do a clean install and rewrite my settings from scratch...
> Just a thought.
>
> I think that's about it.
>
>
> And if anyone's willing to answer this, I know this isn't exactly the place
> ask this, but, aside from having Cero handle external ICMPs requests, is
> there any inherent performance/security/bufferbloat benefits from having
> Cero handle my external ip over a gateway --> router combo?
>
> Right now, my setup consist of a gateway and I'm unable to put it in bridge
> mode. The gateway does NAT, has SPI disabled, and has a static route and
> DMZ
> defined towards Cero. Cero is connected to the end of it with Masquerading
> disabled and the firewall still up. Every device we have runs through Cero.
>
> I'd like to know anything at all before I decide to go looking for another
> dedicated modem, or if I should even bother to go looking in the first
> place.
>
> Hope this helps!
> —Regards, Richard
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

[-- Attachment #2: Type: text/html, Size: 4846 bytes --]

Re: [Cerowrt-devel] Possible Bug(s) in Cero 3.10.50-1

[Sebastian Moeller]

Hi Dave, hi List,

On Sep 14, 2014, at 12:45 , Dave Taht <dave.taht@gmail.com> wrote:

> One of the features of the work going on in the ubnt beta forums was the discovery that you can create named ifb interfaces. So we could switch sqm to a 1 to 1 mapping of ge00-ifb, se00-ifb, etc. and thus have an easier time tearing them down.

     Might be a solution, let m think about it (wist case IFB-SQM_ge00 should be baroque enough to not be accidentally used by other people ;) )

> 
> I figure that QoS chain needs to be applied to the pppoe interface not the ge00 interface?

	Probably, but I think the pppoe interface does not appear in the SQM  interface name drop down box (I have not managed to make pppoe  from cerowrt work at all so I never could test this)

> 
> I generally have encouraged folk to always reinstall from scratch. Now that we are maturing and getting stabler, in place upgrades are becoming more interesting...
> 
> I generally have more faith in cero's fire walling and nat handling than most third party equipment. So bridging is often better. But what I'd like most to happen for dsl is finding a good openwrt compatible dsl/wifi modem and have that as something to recommend to debloat ers on that tech.

	Oh, I am all for it. It seems there is a open source driver for some of the lantiq del chips that should support the ADSLs (1, 2, 2+) and VDSL2 so that might be a decent starting point. Alas, in VDSL2-land currently there is a big push to enable vectoring (central office side crosstalk elimination by modifying the signals that they have the desired waveform after cross-talk has happened, nifty technology) and I am not sure whether the lantiq-chips supported by open source drivers support that… (in Germany the incumbent plans to only offer VDSL2 to vectoring capable modems, other modems will fall back to ADSL2+)

Best Regards
	Sebastian


> 
> On Sep 13, 2014 11:07 AM, "Richard" <rocon46@hotmail.com> wrote:
> Hi, all. End user here. Just thought I'd post a few possible bugs I've run
> into since updating to 3.10.50-1. I'm not exactly sure if these have been
> reported or are intentional, but I figured it couldn't hurt to post them anyway.
> 
> 1) When using PPPoE on the outbound interface, traffic skips classification
> MARKS set by iptables in the QOS_MARK_ge00 chain entirely. This is whilst
> using simple.qos. Everything is placed in the 1:12 class in HTB in both
> ingress and egress regardless of rules set. This was tried using 3.10.34-4
> and then a fresh install of 3.10.50-1.
> 
> 2) In 3.10.50-1, whilst running multiple Intermediate Functional Blocks,
> restarting SQM often has a chance to not close IFBs after the first IFB. i.e
> Anything after ifb0 has a chance to not close. Cero then creates a new
> Block(s) after the ones that haven't closed as it believes they are still in
> use. Doing this enough eventually fills up all available Blocks and then
> ingress shaping fails to start.
> 
> Workaround for me has been to SSH in, stop SQM completely, and then start it
> back up again whenever I change settings as that ensures any lingering IFBs
> are closed down.
> 
> 
> Unfortunately, I foolishly forgot to keep any logs using cerostats.sh and no
> longer have a modem to test PPPoE on; the one I had couldn't hold the DSL
> line for very long and was subsequently returned. I also ran into something
> which I thought was Bug #442 after updating to 3.10.50-1. I had moved from
> 3.10.34-4 using the sysupgrade image.
> 
> The router seemed to lock up twice within the first 15mins after boot and
> again after reboot. Only the 2.4Ghz network went nuts while 5Ghz remained
> fine. Everything on the 2.4Ghz network was still connected, yet nothing on
> 2.4 could get through - both to the internet and to the router itself. I
> then decided to do a clean install and haven't run into it since. This is
> something which has happened to me before on an earlier release and I only
> ever seem to run into this bug whenever I use a sysupgrade image, or restore
> my settings from an archive.
> 
> Something I've noticed is that #442 (or something similar) never seems
> happen if I do a clean install and rewrite my settings from scratch...
> Just a thought.
> 
> I think that's about it.
> 
> 
> And if anyone's willing to answer this, I know this isn't exactly the place
> ask this, but, aside from having Cero handle external ICMPs requests, is
> there any inherent performance/security/bufferbloat benefits from having
> Cero handle my external ip over a gateway --> router combo?
> 
> Right now, my setup consist of a gateway and I'm unable to put it in bridge
> mode. The gateway does NAT, has SPI disabled, and has a static route and DMZ
> defined towards Cero. Cero is connected to the end of it with Masquerading
> disabled and the firewall still up. Every device we have runs through Cero.
> 
> I'd like to know anything at all before I decide to go looking for another
> dedicated modem, or if I should even bother to go looking in the first place.
> 
> Hope this helps!
> —Regards, Richard
> 
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Possible Bug(s) in Cero 3.10.50-1

[Richard]

Sebastian Moeller <moeller0 <at> gmx.de> writes:


> 	Too bad, in case you find a way to force this failure mode, please let me
know so I can have a go at fixing it...
> 

Will do.


> 
> 	I think a number of people have been quite happy with the sysupgrade via
the GUI (just de-select the “keep
> sttings” check box and you should be fine). I think what really could
cause the issues observed is when
> SQM's stop.sh fails half-way through (I will try to have a look at that
later...)

I can't believe I've manage to overlook that a few times. Ugh, I guess I
should pay more attention when going through the menus next time.


> 
> 	But that is a systematic problem with downstream shaping after the the
bottleneck-link (by creation of an
> artificial bottleneck), too much incoming traffic of any kind will cause
backlog in the buffers of the
> device at the upstream end of the real bottleneck-link… So even if the
ICMP packets reach cerowrt this
> might still hose the downstream latency under load...

Yeah, you're right about that. I'm just saying that dropping the ICMP echo
requests may help in it's own small insignificant way by simply having less
request being made by random bots on the net due to the lack of response. Of
course if they know you're there, do a port scan, or aren't phased by a lack
of response, it doesn't make a lick of difference as they're still filling
the downstream pipe. But I get what you mean, excessive inbound traffic
which fill up the buffers on the real bottleneck device will reek havoc on
latency.

 	Dave Taht
 	I figure that QoS chain needs to be applied to the pppoe interface not the
ge00 interface?

I thought so too, but I'm just saying you can't apply the QoS chain directly
to a PPPoE session through the GUI as SQM's GUI page only let's you select
the interface it's running through (which was ge00 in my case). 

The more I think about it, I should've probably tried editing the SQM config
files and directly point it to the PPPoE interface just to see if it worked
or not. I wish I could've messed around with it more, but I no longer have a
modem to play with. But you're probably right.

> 
> > I'm
> > aware it might break some websites that use MTU path discovery, but I
> > figured using MSS clamping in Cero would solve any problems that might
> > incur. That's my reasoning for it, anyway. 
> 
> 	I would hope that the real xdsl-router handles either the MSS clamping
for us or better yet properly
> handles the IPv6 path mtu discovery (and at least in may case that works well)
> 
> Best Regards
> 	Sebastian
> 
> > 
> > Thanks for the input.
> > 	Richard
> > 
> >> Best Regards
> >> 	Sebastian
> >> 
> >>> 
> >>> I'd like to know anything at all before I decide to go looking for another
> >>> dedicated modem, or if I should even bother to go looking in the first
> > place.
> >>> 
> >>> Hope this helps!
> >>> —Regards, Richard
> >>> 
> >>> _______________________________________________
> >>> Cerowrt-devel mailing list
> >>> Cerowrt-devel <at> lists.bufferbloat.net
> >>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> >> 
> >> 
> > 
> > 
> > 
> > 
> > _______________________________________________
> > Cerowrt-devel mailing list
> > Cerowrt-devel <at> lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 
> 

Re: [Cerowrt-devel] Possible Bug(s) in Cero 3.10.50-1

[Sebastian Moeller]

Hi Dave, hi List,



On Sep 14, 2014, at 12:45 , Dave Taht <dave.taht@gmail.com> wrote:

> One of the features of the work going on in the ubnt beta forums was the discovery that you can create named ifb interfaces. So we could switch sqm to a 1 to 1 mapping of ge00-ifb, se00-ifb, etc. and thus have an easier time tearing them down.

	So this might actually be saner than the current approach (I inflicted upon cerowrt) but I think it relies on iproute2’s ip binary, which we have in cerowrt but which does not seem to be standard in openwrt. Once I can find some time I will se what I can come up with. (A quick first test allowed my to create an additional 17 named fib devices, so maybe we do not actually be selective about reusing IFBs already in existence)… I guess I need to implement this to see how robust it is.

Best Regards
	Sebastian

> 
> I figure that QoS chain needs to be applied to the pppoe interface not the ge00 interface?
> 
> I generally have encouraged folk to always reinstall from scratch. Now that we are maturing and getting stabler, in place upgrades are becoming more interesting...
> 
> I generally have more faith in cero's fire walling and nat handling than most third party equipment. So bridging is often better. But what I'd like most to happen for dsl is finding a good openwrt compatible dsl/wifi modem and have that as something to recommend to debloat ers on that tech.
> 
> On Sep 13, 2014 11:07 AM, "Richard" <rocon46@hotmail.com> wrote:
> Hi, all. End user here. Just thought I'd post a few possible bugs I've run
> into since updating to 3.10.50-1. I'm not exactly sure if these have been
> reported or are intentional, but I figured it couldn't hurt to post them anyway.
> 
> 1) When using PPPoE on the outbound interface, traffic skips classification
> MARKS set by iptables in the QOS_MARK_ge00 chain entirely. This is whilst
> using simple.qos. Everything is placed in the 1:12 class in HTB in both
> ingress and egress regardless of rules set. This was tried using 3.10.34-4
> and then a fresh install of 3.10.50-1.
> 
> 2) In 3.10.50-1, whilst running multiple Intermediate Functional Blocks,
> restarting SQM often has a chance to not close IFBs after the first IFB. i.e
> Anything after ifb0 has a chance to not close. Cero then creates a new
> Block(s) after the ones that haven't closed as it believes they are still in
> use. Doing this enough eventually fills up all available Blocks and then
> ingress shaping fails to start.
> 
> Workaround for me has been to SSH in, stop SQM completely, and then start it
> back up again whenever I change settings as that ensures any lingering IFBs
> are closed down.
> 
> 
> Unfortunately, I foolishly forgot to keep any logs using cerostats.sh and no
> longer have a modem to test PPPoE on; the one I had couldn't hold the DSL
> line for very long and was subsequently returned. I also ran into something
> which I thought was Bug #442 after updating to 3.10.50-1. I had moved from
> 3.10.34-4 using the sysupgrade image.
> 
> The router seemed to lock up twice within the first 15mins after boot and
> again after reboot. Only the 2.4Ghz network went nuts while 5Ghz remained
> fine. Everything on the 2.4Ghz network was still connected, yet nothing on
> 2.4 could get through - both to the internet and to the router itself. I
> then decided to do a clean install and haven't run into it since. This is
> something which has happened to me before on an earlier release and I only
> ever seem to run into this bug whenever I use a sysupgrade image, or restore
> my settings from an archive.
> 
> Something I've noticed is that #442 (or something similar) never seems
> happen if I do a clean install and rewrite my settings from scratch...
> Just a thought.
> 
> I think that's about it.
> 
> 
> And if anyone's willing to answer this, I know this isn't exactly the place
> ask this, but, aside from having Cero handle external ICMPs requests, is
> there any inherent performance/security/bufferbloat benefits from having
> Cero handle my external ip over a gateway --> router combo?
> 
> Right now, my setup consist of a gateway and I'm unable to put it in bridge
> mode. The gateway does NAT, has SPI disabled, and has a static route and DMZ
> defined towards Cero. Cero is connected to the end of it with Masquerading
> disabled and the firewall still up. Every device we have runs through Cero.
> 
> I'd like to know anything at all before I decide to go looking for another
> dedicated modem, or if I should even bother to go looking in the first place.
> 
> Hope this helps!
> —Regards, Richard
> 
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel