Development issues regarding the cerowrt test router project
 help / color / mirror / Atom feed
From: "Richard E. Brown" <richb.hanover@gmail.com>
To: cerowrt-devel@lists.bufferbloat.net
Subject: Re: [Cerowrt-devel] CeroWrt 3.10.18-1 Field Report
Date: Tue, 12 Nov 2013 23:21:12 -0500	[thread overview]
Message-ID: <8DD9AC0D-6606-4E7A-A60B-59D126D8D847@gmail.com> (raw)
In-Reply-To: <47AA0EBF-0886-444B-9A04-EB23105A8A3B@gmx.de>


On Nov 12, 2013, at 4:11 AM, Sebastian Moeller <moeller0@gmx.de> wrote:

>> - The kernel.log shows lots of the stack traces below: 2-5 per second on a long-term basis. 
> 
> 	These look quite weird, the error is a slow patch warning from hfsc_schedule_watchdog . But, hfsc is the queuing discipline used by stock OpenWrt, cerowrt , so far, has only used HTB (last I checked was cerowrt 3.10.11-3). So my guess is that you were running the default QOS system instead (or worse in addition) to cerowrt's. It would be great to see the output of:
> tc -d qdisc ; tc -s class show dev ifb0 ; tc -s class show dev ge00
> to check what is up with the AQM system...

Here’s the output as requested from my router that started out as 3.7.5-2 with QoS enabled, and then was flashed via sysupgrade through the web GUI to 3.10.18-1, preserving the settings. 

Rich
 -----------------------------------------------------

root@cerowrt:~# tc -d qdisc ; tc -s class show dev ifb0 ; tc -s class show dev ge00
qdisc fq_codel a: dev se00 root refcnt 2 limit 1000p flows 1024 quantum 1000 target 5.0ms interval 100.0ms ecn
qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 10 direct_packets_stat 0 ver 3.17
qdisc fq_codel 110: dev ge00 parent 1:10 limit 600p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
qdisc mq 1: dev sw00 root
qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc mq 1: dev gw01 root
qdisc fq_codel 10: dev gw01 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev gw01 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev gw01 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev gw01 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc mq 1: dev gw00 root
qdisc fq_codel 10: dev gw00 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev gw00 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev gw00 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev gw00 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc hfsc 1: dev ifb0 root refcnt 2 default 30
qdisc fq_codel 100: dev ifb0 parent 1:10 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 200: dev ifb0 parent 1:20 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 300: dev ifb0 parent 1:30 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 400: dev ifb0 parent 1:40 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc mq 1: dev sw10 root
qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc mq 1: dev gw11 root
qdisc fq_codel 10: dev gw11 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev gw11 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev gw11 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev gw11 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc mq 1: dev gw10 root
qdisc fq_codel 10: dev gw10 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev gw10 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev gw10 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev gw10 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
class hfsc 1: root
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 period 0 level 2

class hfsc 1:1 parent 1: sc m1 0bit d 0us m2 6000Kbit ul m1 0bit d 0us m2 6000Kbit
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 period 2 work 260 bytes level 1

class hfsc 1:10 parent 1:1 leaf 100: rt m1 1360Kbit d 325us m2 600000bit ls m1 1360Kbit d 325us m2 3333Kbit ul m1 0bit d 0us m2 6000Kbit
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 period 0 level 0

class hfsc 1:20 parent 1:1 leaf 200: rt m1 3126Kbit d 325us m2 3000Kbit ls m1 3126Kbit d 325us m2 1666Kbit ul m1 0bit d 0us m2 6000Kbit
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 period 0 level 0

class hfsc 1:30 parent 1:1 leaf 300: ls m1 0bit d 100.0ms m2 833000bit ul m1 0bit d 0us m2 6000Kbit
 Sent 260 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 period 2 work 260 bytes level 0

class hfsc 1:40 parent 1:1 leaf 400: ls m1 0bit d 200.0ms m2 166000bit ul m1 0bit d 0us m2 6000Kbit
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 period 0 level 0

class fq_codel 300:22f parent 300:
 (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  deficit 40 count 0 lastcount 0 ldelay 15us
class htb 1:10 parent 1:1 leaf 110: prio 0 rate 650000bit ceil 650000bit burst 1599b cburst 1599b
 Sent 54458 bytes 263 pkt (dropped 0, overlimits 0 requeues 0)
 rate 1984bit 1pps backlog 0b 0p requeues 0
 lended: 263 borrowed: 0 giants: 0
 tokens: 244610 ctokens: 244610

class htb 1:1 root rate 650000bit ceil 650000bit burst 1599b cburst 1599b
 Sent 54458 bytes 263 pkt (dropped 0, overlimits 0 requeues 0)
 rate 1984bit 1pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: 244610 ctokens: 244610

class fq_codel 110:2e7 parent 110:
 (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  deficit -55 count 0 lastcount 0 ldelay 12us
root@cerowrt:~#

I then reflashed 3.7.5-2, and disabled the QoS setting from the web GUI, and then flashed via sysupgrade through the web GUI to 3.10.18-1, again preserving the settings. I got these results:

root@cerowrt:~# tc -d qdisc ; tc -s class show dev ifb0 ; tc -s class show dev ge00
qdisc fq_codel a: dev se00 root refcnt 2 limit 1000p flows 1024 quantum 1000 target 5.0ms interval 100.0ms ecn
qdisc htb 1: dev ge00 root refcnt 2 r2q 10 default 10 direct_packets_stat 0 ver 3.17
qdisc fq_codel 110: dev ge00 parent 1:10 limit 600p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc ingress ffff: dev ge00 parent ffff:fff1 ----------------
qdisc htb 1: dev ifb0 root refcnt 2 r2q 10 default 10 direct_packets_stat 0 ver 3.17
qdisc fq_codel 110: dev ifb0 parent 1:10 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc mq 1: dev sw00 root
qdisc fq_codel 10: dev sw00 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw00 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw00 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw00 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc mq 1: dev gw01 root
qdisc fq_codel 10: dev gw01 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev gw01 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev gw01 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev gw01 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc fq_codel a: dev pppoe-ge00 root refcnt 2 limit 1000p flows 1024 quantum 1000 target 5.0ms interval 100.0ms ecn
qdisc mq 1: dev gw00 root
qdisc fq_codel 10: dev gw00 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev gw00 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev gw00 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev gw00 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc mq 1: dev sw10 root
qdisc fq_codel 10: dev sw10 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev sw10 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev sw10 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev sw10 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc mq 1: dev gw11 root
qdisc fq_codel 10: dev gw11 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev gw11 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev gw11 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev gw11 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
qdisc mq 1: dev gw10 root
qdisc fq_codel 10: dev gw10 parent 1:1 limit 800p flows 1024 quantum 500 target 10.0ms interval 100.0ms
qdisc fq_codel 20: dev gw10 parent 1:2 limit 800p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 30: dev gw10 parent 1:3 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 40: dev gw10 parent 1:4 limit 1000p flows 1024 quantum 300 target 5.0ms interval 100.0ms
class htb 1:10 parent 1:1 leaf 110: prio 0 rate 6000Kbit ceil 6000Kbit burst 1599b cburst 1599b
 Sent 69125382 bytes 210311 pkt (dropped 0, overlimits 0 requeues 0)
 rate 2254Kbit 991pps backlog 0b 0p requeues 0
 lended: 210311 borrowed: 0 giants: 0
 tokens: 4767 ctokens: 4767

class htb 1:1 root rate 6000Kbit ceil 6000Kbit burst 1599b cburst 1599b
 Sent 69125382 bytes 210311 pkt (dropped 0, overlimits 0 requeues 0)
 rate 2254Kbit 991pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: 4767 ctokens: 4767

class fq_codel 110:292 parent 110:
 (dropped 26, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  deficit -709 count 1 lastcount 1 ldelay 10us
class htb 1:10 parent 1:1 leaf 110: prio 0 rate 650000bit ceil 650000bit burst 1599b cburst 1599b
 Sent 14582488 bytes 192897 pkt (dropped 5, overlimits 0 requeues 0)
 rate 541856bit 909pps backlog 0b 0p requeues 0
 lended: 192897 borrowed: 0 giants: 0
 tokens: 276028 ctokens: 276028

class htb 1:1 root rate 650000bit ceil 650000bit burst 1599b cburst 1599b
 Sent 14582488 bytes 192897 pkt (dropped 0, overlimits 0 requeues 0)
 rate 541856bit 909pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: 276028 ctokens: 276028

class fq_codel 110:e4 parent 110:
 (dropped 8911, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  deficit 26 count 82 lastcount 49 ldelay 10us
root@cerowrt:~#

  parent reply	other threads:[~2013-11-13  4:21 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-11-12  5:47 Richard E. Brown
2013-11-12  9:11 ` Sebastian Moeller
     [not found]   ` <51BF9432-6FC2-4A14-B147-13F1E779CA93@gmail.com>
2013-11-12 17:26     ` Richard E. Brown
2013-11-12 21:17       ` Sebastian Moeller
2013-11-12 23:06         ` Sebastian Moeller
2013-11-12 23:11           ` Sebastian Moeller
2013-11-15 12:35             ` Sebastian Moeller
2013-11-13  4:21   ` Richard E. Brown [this message]
2013-11-13 13:56     ` Sebastian Moeller
2013-11-13 15:53       ` Richard E. Brown
2013-11-12  9:40 ` Fred Stratton
2013-11-12 17:24   ` Richard E. Brown
2013-11-15  1:56 ` Richard E. Brown
2013-11-15  2:27   ` David Personette

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.bufferbloat.net/postorius/lists/cerowrt-devel.lists.bufferbloat.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8DD9AC0D-6606-4E7A-A60B-59D126D8D847@gmail.com \
    --to=richb.hanover@gmail.com \
    --cc=cerowrt-devel@lists.bufferbloat.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox