[Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

I transitioned from ceroWRT to lupin, now at undisclosed version 4, to 
try cake. I have a renown poor ADSL line, connected to TalkTalk. After 
getting a 60 GBP discount by negotiating with a woman and her imaginary 
Manager friend in Bangalore a month ago, I have no intention of moving 
to fibre.

The wiki page gives insufficient detail.

I am currently using a bridged device to connect to the internet, and so 
have a pppoe-wan interface on the WNDR3800. I do not use PPPoA ever.

Whatever options I use with cake, the flow is either raw, or atm 
overhead is listed as 0.

I do not know what commands - or probably, more correctly, options - can 
be used, and in which order

incoming

... cake bandwidth 11500kbit besteffort

can you add atm overhead 38 (does not work, returns 0)

or are besteffort and atm mutually exclusive? Do I need to use the term 
'flows' somewhere?

do terms such as 'pppoe-vcmux; need to be preceded by 'atm'?

More clarity is needed, I suggest, or a man page equivalent.

On June 20th what I am experiencing was described by Alan Jenkins on the 
cake list.

I note the latest posting by DT on this Cero-devel list apparently using 
an ADSL line as a source example, only uses raw flows.

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 563 bytes --]

You're already using correct syntax - I've written it to be quite lenient
and use sensible defaults for missing information. There are several sets
of keywords and parameters which are mutually orthogonal, and don't depend
on each other, so "besteffort" has nothing to do with "overhead" or "atm".

What's probably happening is that you're using a slightly old version of
the cake kernel module which lacks the overhead parameter entirely, but a
more up to date tc which does support it. We've seen this combination crop
up ourselves recently.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 676 bytes --]

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

I am using the latest version compiled by DT, after you last made this 
incompatibility comment on the cake list.

I happen to have compiled separately a build with your changes of 22 
days ago incorporated, but based on lupin undisclosed version 2. I shall 
try that.

I am making an assumption throughout this that qdiscs should be used on 
the WAN side always, effectively before the firewall. Is this assumption 
correct?



On 10/07/15 13:57, Jonathan Morton wrote:
>
> You're already using correct syntax - I've written it to be quite 
> lenient and use sensible defaults for missing information. There are 
> several sets of keywords and parameters which are mutually orthogonal, 
> and don't depend on each other, so "besteffort" has nothing to do with 
> "overhead" or "atm".
>
> What's probably happening is that you're using a slightly old version 
> of the cake kernel module which lacks the overhead parameter entirely, 
> but a more up to date tc which does support it. We've seen this 
> combination crop up ourselves recently.
>
> - Jonathan Morton
>

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 149 bytes --]

Qdiscs should be used on any link that might become a bottleneck. In most
consumer cases, that will indeed be your WAN interface.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 188 bytes --]

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

You are absolutely correct.

I tried both a numeric overhead value, and alternatively 'pppoe-vcmux' 
and 'ether-fcs' in the build I crafted based on r46006, which is lupin 
undeclared version 2. Everything works as stated.

On lupin undeclared version 4, the current release based on r46117, the 
values were not recognised.

Thank you.

I had cake running on a Lantiq ADSL gateway running the same r46006 
build. Unfortunately this was bricked by attempts to get homenet 
working, so I have nothing to report about gateway usage at present.



On 10/07/15 13:57, Jonathan Morton wrote:
>
> You're already using correct syntax - I've written it to be quite 
> lenient and use sensible defaults for missing information. There are 
> several sets of keywords and parameters which are mutually orthogonal, 
> and don't depend on each other, so "besteffort" has nothing to do with 
> "overhead" or "atm".
>
> What's probably happening is that you're using a slightly old version 
> of the cake kernel module which lacks the overhead parameter entirely, 
> but a more up to date tc which does support it. We've seen this 
> combination crop up ourselves recently.
>
> - Jonathan Morton
>

[Cerowrt-devel] "Lupin undeclared"?

[Rich Brown]

Hi Fred,

I'm not familiar with "lupin undeclared" - could you send a pointer/link? Thanks.

Rich

On Jul 10, 2015, at 10:52 AM, Fred Stratton <fredstratton@imap.cc> wrote:

> 
> You are absolutely correct.
> 
> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux' and 'ether-fcs' in the build I crafted based on r46006, which is lupin undeclared version 2. Everything works as stated.
> 
> On lupin undeclared version 4, the current release based on r46117, the values were not recognised.
> 
> Thank you.
> 
> I had cake running on a Lantiq ADSL gateway running the same r46006 build. Unfortunately this was bricked by attempts to get homenet working, so I have nothing to report about gateway usage at present.
> 
> 
> 
> On 10/07/15 13:57, Jonathan Morton wrote:
>> 
>> You're already using correct syntax - I've written it to be quite lenient and use sensible defaults for missing information. There are several sets of keywords and parameters which are mutually orthogonal, and don't depend on each other, so "besteffort" has nothing to do with "overhead" or "atm".
>> 
>> What's probably happening is that you're using a slightly old version of the cake kernel module which lacks the overhead parameter entirely, but a more up to date tc which does support it. We've seen this combination crop up ourselves recently.
>> 
>> - Jonathan Morton
>> 
> 
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] "Lupin undeclared"?

[Alan Jenkins]

On 10/07/15 16:16, Rich Brown wrote:
> Hi Fred,
>
> I'm not familiar with "lupin undeclared" - could you send a pointer/link? Thanks.
>
> Rich

The unversioned testing build Dave posted at

http://snapon.lab.bufferbloat.net/~cero3/lupin/ar71xx/

which is not declared on the CeroWrt homepage.  (Presumably to avoid 
giving a false impression about how much life and commitment there is in 
those builds).

> On Jul 10, 2015, at 10:52 AM, Fred Stratton <fredstratton@imap.cc> wrote:
>
>> You are absolutely correct.
>>
>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux' and 'ether-fcs' in the build I crafted based on r46006, which is lupin undeclared version 2. Everything works as stated.
>>
>> On lupin undeclared version 4, the current release based on r46117, the values were not recognised.
>>
>> Thank you.
>>
>> I had cake running on a Lantiq ADSL gateway running the same r46006 build. Unfortunately this was bricked by attempts to get homenet working, so I have nothing to report about gateway usage at present.

unrelated, but ow, you have my sympathy.  I'd love to have a working 
lantiq with openwrt and fq_codel.

>>
>>
>>
>> On 10/07/15 13:57, Jonathan Morton wrote:
>>> You're already using correct syntax - I've written it to be quite lenient and use sensible defaults for missing information. There are several sets of keywords and parameters which are mutually orthogonal, and don't depend on each other, so "besteffort" has nothing to do with "overhead" or "atm".
>>>
>>> What's probably happening is that you're using a slightly old version of the cake kernel module which lacks the overhead parameter entirely, but a more up to date tc which does support it. We've seen this combination crop up ourselves recently.
>>>
>>> - Jonathan Morton
>>>
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] "Lupin undeclared"?

[Rich Brown]

Ahah! I had not noticed the new code name. It's well protected - the Googles didn't find anything.

[But now it will... Oh dear... :-) ]

Rich

On Jul 10, 2015, at 11:39 AM, Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:

> The unversioned testing build Dave posted at
> 
> http://snapon.lab.bufferbloat.net/~cero3/lupin/ar71xx/
> 
> which is not declared on the CeroWrt homepage.

Re: [Cerowrt-devel] "Lupin undeclared"?

[Dave Taht]

"lupin undeclared" is a build I managed to deploy at the campground
campus to about 8 of the 30 or so routers, including a few critical
paths. So far it is working spectacularly well in deployment there -
no forced reboots, aps, routing, dnssec, source specific stuff,
minstrel-blues, minstrel-variance... even cake.

root@pool2:~# uptime

 05:40:49 up 14 days, 22 min,  load average: 0.15, 0.07, 0.05

But I chickened out on dragging ipv6 to the pool. (5 hops)

Anyway cake is working as well as it can on wifi, running on a
picostation, I do not see any memory growth, it drops and marks...

http://pastebin.com/wbcXuNFM

qdisc cake 8004: root refcnt 5 unlimited diffserv4 flows raw
 Sent 107577100738 bytes 79678911 pkt (dropped 152068, overlimits 0
requeues 757808)
 backlog 0b 0p requeues 757808

and I guess we have to improve the statistics generated by tc to use
"k" and "m" to keep the cake columns aligned.

I am keeping detailed statistics on this portion of the network while
I am away and trying to design more.

I certainly did not intend for others to try these builds all that much.

My goal in life for the lupin build was to have something that did not
crash. Ever. It is a pita to climb trees and reflash the firmware
where these core wifi routers are placed.

It is kind of my hope to get out a new build during battlemesh or
shortly thereafter, but I plan to stick with this one for battlemesh.
Would like to sink some time into the ac1200...

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Alan Jenkins]

I'm glad to hear there's a working version (even if it's not in the 
current build :).

Do you have measurable improvements with overhead configured (v.s. 
unconfigured)?

I've used netperfrunner from CeroWrtScripts, e.g.

sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER

I believe accounting for overhead helps on this two-way test, because a) 
it saturates the uplink b) about half that bandwidth is tiny ack packets 
(depending on bandwidth asymmetry).  And small packets have 
proportionally high overhead.

(But it seems to only make a small difference for me, which always 
surprises Seb).

Alan

On 10/07/15 15:52, Fred Stratton wrote:
>
> You are absolutely correct.
>
> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
> undeclared version 2. Everything works as stated.
>
> On lupin undeclared version 4, the current release based on r46117, the
> values were not recognised.
>
> Thank you.
>
> I had cake running on a Lantiq ADSL gateway running the same r46006
> build. Unfortunately this was bricked by attempts to get homenet
> working, so I have nothing to report about gateway usage at present.
>
>
>
> On 10/07/15 13:57, Jonathan Morton wrote:
>>
>> You're already using correct syntax - I've written it to be quite
>> lenient and use sensible defaults for missing information. There are
>> several sets of keywords and parameters which are mutually orthogonal,
>> and don't depend on each other, so "besteffort" has nothing to do with
>> "overhead" or "atm".
>>
>> What's probably happening is that you're using a slightly old version
>> of the cake kernel module which lacks the overhead parameter entirely,
>> but a more up to date tc which does support it. We've seen this
>> combination crop up ourselves recently.
>>
>> - Jonathan Morton
>>
>

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

I have yet to undertake formal testing.

I moved from using a Buffalo WMBR-G300 running Barrier Breaker to a 
bridged device when I broke the former tying to install Homenet.

I omitted to recraft the cake script I was using to incorporate the 
correct interface, now pppoe-wan, and was unaware of the error until 
today, a fortnight later.

I am therefore not expecting significant improvements.

On 10/07/15 16:19, Alan Jenkins wrote:
>
> I'm glad to hear there's a working version (even if it's not in the 
> current build :).
>
> Do you have measurable improvements with overhead configured (v.s. 
> unconfigured)?
>
> I've used netperfrunner from CeroWrtScripts, e.g.
>
> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>
> I believe accounting for overhead helps on this two-way test, because 
> a) it saturates the uplink b) about half that bandwidth is tiny ack 
> packets (depending on bandwidth asymmetry).  And small packets have 
> proportionally high overhead.
>
> (But it seems to only make a small difference for me, which always 
> surprises Seb).
>
> Alan
>
> On 10/07/15 15:52, Fred Stratton wrote:
>>
>> You are absolutely correct.
>>
>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>> undeclared version 2. Everything works as stated.
>>
>> On lupin undeclared version 4, the current release based on r46117, the
>> values were not recognised.
>>
>> Thank you.
>>
>> I had cake running on a Lantiq ADSL gateway running the same r46006
>> build. Unfortunately this was bricked by attempts to get homenet
>> working, so I have nothing to report about gateway usage at present.
>>
>>
>>
>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>
>>> You're already using correct syntax - I've written it to be quite
>>> lenient and use sensible defaults for missing information. There are
>>> several sets of keywords and parameters which are mutually orthogonal,
>>> and don't depend on each other, so "besteffort" has nothing to do with
>>> "overhead" or "atm".
>>>
>>> What's probably happening is that you're using a slightly old version
>>> of the cake kernel module which lacks the overhead parameter entirely,
>>> but a more up to date tc which does support it. We've seen this
>>> combination crop up ourselves recently.
>>>
>>> - Jonathan Morton
>>>
>>
>

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

By your command
Rebooted to rerun qdisc script, rather than changing qdiscs from the 
command-line, so suboptimal process as end-point changed.

script configuring qdiscs and overhead 40 on

sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 
streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
  Download:  6.73 Mbps
    Upload:  0.58 Mbps
   Latency: (in msec, 62 pings, 0.00% packet loss)
       Min: 24.094
     10pct: 172.654
    Median: 260.563
       Avg: 253.580
     90pct: 330.003
       Max: 411.145

script configuring qdiscs on flows raw

sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
78.145.32.1
2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 
streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
  Download:  6.75 Mbps
    Upload:  0.59 Mbps
   Latency: (in msec, 59 pings, 0.00% packet loss)
       Min: 23.605
     10pct: 169.789
    Median: 282.155
       Avg: 267.099
     90pct: 333.283
       Max: 376.509

script configuring qdiscs and overhead 36 on

sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
80.44.96.1
2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 
streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
  Download:  6.56 Mbps
    Upload:  0.59 Mbps
   Latency: (in msec, 62 pings, 0.00% packet loss)
       Min: 22.975
     10pct: 195.473
    Median: 281.756
       Avg: 271.609
     90pct: 342.130
       Max: 398.573


On 10/07/15 16:19, Alan Jenkins wrote:
>
> I'm glad to hear there's a working version (even if it's not in the 
> current build :).
>
> Do you have measurable improvements with overhead configured (v.s. 
> unconfigured)?
>
> I've used netperfrunner from CeroWrtScripts, e.g.
>
> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>
> I believe accounting for overhead helps on this two-way test, because 
> a) it saturates the uplink b) about half that bandwidth is tiny ack 
> packets (depending on bandwidth asymmetry).  And small packets have 
> proportionally high overhead.
>
> (But it seems to only make a small difference for me, which always 
> surprises Seb).
>
> Alan
>
> On 10/07/15 15:52, Fred Stratton wrote:
>>
>> You are absolutely correct.
>>
>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>> undeclared version 2. Everything works as stated.
>>
>> On lupin undeclared version 4, the current release based on r46117, the
>> values were not recognised.
>>
>> Thank you.
>>
>> I had cake running on a Lantiq ADSL gateway running the same r46006
>> build. Unfortunately this was bricked by attempts to get homenet
>> working, so I have nothing to report about gateway usage at present.
>>
>>
>>
>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>
>>> You're already using correct syntax - I've written it to be quite
>>> lenient and use sensible defaults for missing information. There are
>>> several sets of keywords and parameters which are mutually orthogonal,
>>> and don't depend on each other, so "besteffort" has nothing to do with
>>> "overhead" or "atm".
>>>
>>> What's probably happening is that you're using a slightly old version
>>> of the cake kernel module which lacks the overhead parameter entirely,
>>> but a more up to date tc which does support it. We've seen this
>>> combination crop up ourselves recently.
>>>
>>> - Jonathan Morton
>>>
>>
>

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Sebastian Moeller]

Hi Fred,

your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
1) cat /etc/config/sqm
2) tc -d qdisc
3) tc -d class show dev pppoe-wan
4) tc -d class show dev ifb4pppoe-wqn
5) /etc/init.d/sqm stop
6) /etc/init.d/sqm start

hopefully these give some insight what might have happened.

And finally I would love to learn the output of:
sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4


Many Thanks & Best Regards
	Sebastian

On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:

> By your command
> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
> 
> script configuring qdiscs and overhead 40 on
> 
> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
> Download:  6.73 Mbps
>   Upload:  0.58 Mbps
>  Latency: (in msec, 62 pings, 0.00% packet loss)
>      Min: 24.094
>    10pct: 172.654
>   Median: 260.563
>      Avg: 253.580
>    90pct: 330.003
>      Max: 411.145
> 
> script configuring qdiscs on flows raw
> 
> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
> 78.145.32.1
> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
> Download:  6.75 Mbps
>   Upload:  0.59 Mbps
>  Latency: (in msec, 59 pings, 0.00% packet loss)
>      Min: 23.605
>    10pct: 169.789
>   Median: 282.155
>      Avg: 267.099
>    90pct: 333.283
>      Max: 376.509
> 
> script configuring qdiscs and overhead 36 on
> 
> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
> 80.44.96.1
> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
> Download:  6.56 Mbps
>   Upload:  0.59 Mbps
>  Latency: (in msec, 62 pings, 0.00% packet loss)
>      Min: 22.975
>    10pct: 195.473
>   Median: 281.756
>      Avg: 271.609
>    90pct: 342.130
>      Max: 398.573
> 
> 
> On 10/07/15 16:19, Alan Jenkins wrote:
>> 
>> I'm glad to hear there's a working version (even if it's not in the current build :).
>> 
>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>> 
>> I've used netperfrunner from CeroWrtScripts, e.g.
>> 
>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>> 
>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>> 
>> (But it seems to only make a small difference for me, which always surprises Seb).
>> 
>> Alan
>> 
>> On 10/07/15 15:52, Fred Stratton wrote:
>>> 
>>> You are absolutely correct.
>>> 
>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>> undeclared version 2. Everything works as stated.
>>> 
>>> On lupin undeclared version 4, the current release based on r46117, the
>>> values were not recognised.
>>> 
>>> Thank you.
>>> 
>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>> build. Unfortunately this was bricked by attempts to get homenet
>>> working, so I have nothing to report about gateway usage at present.
>>> 
>>> 
>>> 
>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>> 
>>>> You're already using correct syntax - I've written it to be quite
>>>> lenient and use sensible defaults for missing information. There are
>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>> "overhead" or "atm".
>>>> 
>>>> What's probably happening is that you're using a slightly old version
>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>> but a more up to date tc which does support it. We've seen this
>>>> combination crop up ourselves recently.
>>>> 
>>>> - Jonathan Morton
>>>> 
>>> 
>> 
> 
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

On 10/07/15 19:46, Sebastian Moeller wrote:
> Hi Fred,
>
> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
> 1) cat /etc/config/sqm
config queue 'eth1'
     option qdisc 'fq_codel'
     option script 'simple.qos'
     option qdisc_advanced '0'
     option linklayer 'none'
     option enabled '0'
     option interface 'eth1'
     option download '0'
     option upload '0'


> 2) tc -d qdisc
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 
300 target 5.0ms interval 100.0ms ecn
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 1024p flows 1024 quantum 
300 target 5.0ms interval 100.0ms ecn
qdisc mq 0: dev wlan1 root
qdisc fq_codel 0: dev wlan1 parent :1 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 0: dev wlan1 parent :2 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 0: dev wlan1 parent :3 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 0: dev wlan1 parent :4 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
qdisc mq 0: dev wlan0 root
qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
qdisc cake 8002: dev pppoe-wan root refcnt 2 bandwidth 850Kbit 
besteffort flows raw
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
qdisc cake 8001: dev ifb4pppoe-wan root refcnt 2 bandwidth 11500Kbit 
besteffort flows atm overhead 40

> 3) tc -d class show dev pppoe-wan
class cake 8002:2fa parent 8002:


> 4) tc -d class show dev ifb4pppoe-wqn

class cake 8001:6e parent 8001:
> 5) /etc/init.d/sqm stop
SQM: Trying to start/stop SQM on all interfaces.
SQM: run.sh stop
SQM: /usr/lib/sqm/run.sh SQM for interface eth1 is not enabled, skipping 
over...
> 6) /etc/init.d/sqm start
/etc/init.d/sqm start
SQM: Trying to start/stop SQM on all interfaces.
SQM: /usr/lib/sqm/run.sh SQM for interface eth1 is not enabled, skipping 
over...

>
> hopefully these give some insight what might have happened.
>
> And finally I would love to learn the output of:
> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4

betterspeedtest.sh not installed

sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.ne
t -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H 
netperf-
eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
sh: can't open 'betterspeedtest.sh'
2015-07-10 20:10:55 Testing netperf-eu.bufferbloat.net (ipv4) with 4 
streams down and up while pinging netperf-eu.bufferbloat.net. Takes 
about 150 seconds.
  Download:  6.8 Mbps
    Upload:  0.59 Mbps
   Latency: (in msec, 152 pings, 0.00% packet loss)
       Min: 73.911
     10pct: 232.211
    Median: 308.556
       Avg: 305.686
     90pct: 376.183
       Max: 412.553

>
>
> Many Thanks & Best Regards
> 	Sebastian
>
> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>
>> By your command
>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>
>> script configuring qdiscs and overhead 40 on
>>
>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>> Download:  6.73 Mbps
>>    Upload:  0.58 Mbps
>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>       Min: 24.094
>>     10pct: 172.654
>>    Median: 260.563
>>       Avg: 253.580
>>     90pct: 330.003
>>       Max: 411.145
>>
>> script configuring qdiscs on flows raw
>>
>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>> 78.145.32.1
>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>> Download:  6.75 Mbps
>>    Upload:  0.59 Mbps
>>   Latency: (in msec, 59 pings, 0.00% packet loss)
>>       Min: 23.605
>>     10pct: 169.789
>>    Median: 282.155
>>       Avg: 267.099
>>     90pct: 333.283
>>       Max: 376.509
>>
>> script configuring qdiscs and overhead 36 on
>>
>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>> 80.44.96.1
>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>> Download:  6.56 Mbps
>>    Upload:  0.59 Mbps
>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>       Min: 22.975
>>     10pct: 195.473
>>    Median: 281.756
>>       Avg: 271.609
>>     90pct: 342.130
>>       Max: 398.573
>>
>>
>> On 10/07/15 16:19, Alan Jenkins wrote:
>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>
>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>
>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>
>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>
>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>
>>> Alan
>>>
>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>> You are absolutely correct.
>>>>
>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>> undeclared version 2. Everything works as stated.
>>>>
>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>> values were not recognised.
>>>>
>>>> Thank you.
>>>>
>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>> working, so I have nothing to report about gateway usage at present.
>>>>
>>>>
>>>>
>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>> You're already using correct syntax - I've written it to be quite
>>>>> lenient and use sensible defaults for missing information. There are
>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>> "overhead" or "atm".
>>>>>
>>>>> What's probably happening is that you're using a slightly old version
>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>> but a more up to date tc which does support it. We've seen this
>>>>> combination crop up ourselves recently.
>>>>>
>>>>> - Jonathan Morton
>>>>>
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Dave Taht]

enabled 1

is needed

On Fri, Jul 10, 2015 at 12:14 PM, Fred Stratton <fredstratton@imap.cc> wrote:
>
>
> On 10/07/15 19:46, Sebastian Moeller wrote:
>>
>> Hi Fred,
>>
>> your results seem to indicate that cake is not active at all, as the
>> latency under load is abysmal (a quick check is to look at the median in
>> relation to the min and the 90% number, in your examples all of these are
>> terrible). Could you please post the result of the following commands on
>> your router:
>> 1) cat /etc/config/sqm
>
> config queue 'eth1'
>     option qdisc 'fq_codel'
>     option script 'simple.qos'
>     option qdisc_advanced '0'
>     option linklayer 'none'
>     option enabled '0'
>     option interface 'eth1'
>     option download '0'
>     option upload '0'
>
>
>> 2) tc -d qdisc
>
> qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth1 root refcnt 2 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc mq 0: dev wlan1 root
> qdisc fq_codel 0: dev wlan1 parent :1 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan1 parent :2 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan1 parent :3 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan1 parent :4 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc mq 0: dev wlan0 root
> qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300
> target 5.0ms interval 100.0ms ecn
> qdisc cake 8002: dev pppoe-wan root refcnt 2 bandwidth 850Kbit besteffort
> flows raw
> qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
> qdisc cake 8001: dev ifb4pppoe-wan root refcnt 2 bandwidth 11500Kbit
> besteffort flows atm overhead 40
>
>> 3) tc -d class show dev pppoe-wan
>
> class cake 8002:2fa parent 8002:
>
>
>> 4) tc -d class show dev ifb4pppoe-wqn
>
>
> class cake 8001:6e parent 8001:
>>
>> 5) /etc/init.d/sqm stop
>
> SQM: Trying to start/stop SQM on all interfaces.
> SQM: run.sh stop
> SQM: /usr/lib/sqm/run.sh SQM for interface eth1 is not enabled, skipping
> over...
>>
>> 6) /etc/init.d/sqm start
>
> /etc/init.d/sqm start
> SQM: Trying to start/stop SQM on all interfaces.
> SQM: /usr/lib/sqm/run.sh SQM for interface eth1 is not enabled, skipping
> over...
>
>>
>> hopefully these give some insight what might have happened.
>>
>> And finally I would love to learn the output of:
>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p
>> netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H
>> netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>
>
> betterspeedtest.sh not installed
>
> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.ne
> t -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H
> netperf-
> eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
> sh: can't open 'betterspeedtest.sh'
> 2015-07-10 20:10:55 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams
> down and up while pinging netperf-eu.bufferbloat.net. Takes about 150
> seconds.
>  Download:  6.8 Mbps
>    Upload:  0.59 Mbps
>   Latency: (in msec, 152 pings, 0.00% packet loss)
>       Min: 73.911
>     10pct: 232.211
>    Median: 308.556
>       Avg: 305.686
>     90pct: 376.183
>       Max: 412.553
>
>
>>
>>
>> Many Thanks & Best Regards
>>         Sebastian
>>
>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>>
>>> By your command
>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the
>>> command-line, so suboptimal process as end-point changed.
>>>
>>> script configuring qdiscs and overhead 40 on
>>>
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4
>>> streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>> Download:  6.73 Mbps
>>>    Upload:  0.58 Mbps
>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>       Min: 24.094
>>>     10pct: 172.654
>>>    Median: 260.563
>>>       Avg: 253.580
>>>     90pct: 330.003
>>>       Max: 411.145
>>>
>>> script configuring qdiscs on flows raw
>>>
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>> 78.145.32.1
>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4
>>> streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>> Download:  6.75 Mbps
>>>    Upload:  0.59 Mbps
>>>   Latency: (in msec, 59 pings, 0.00% packet loss)
>>>       Min: 23.605
>>>     10pct: 169.789
>>>    Median: 282.155
>>>       Avg: 267.099
>>>     90pct: 333.283
>>>       Max: 376.509
>>>
>>> script configuring qdiscs and overhead 36 on
>>>
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>> 80.44.96.1
>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4
>>> streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>> Download:  6.56 Mbps
>>>    Upload:  0.59 Mbps
>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>       Min: 22.975
>>>     10pct: 195.473
>>>    Median: 281.756
>>>       Avg: 271.609
>>>     90pct: 342.130
>>>       Max: 398.573
>>>
>>>
>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>
>>>> I'm glad to hear there's a working version (even if it's not in the
>>>> current build :).
>>>>
>>>> Do you have measurable improvements with overhead configured (v.s.
>>>> unconfigured)?
>>>>
>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>
>>>> I believe accounting for overhead helps on this two-way test, because a)
>>>> it saturates the uplink b) about half that bandwidth is tiny ack packets
>>>> (depending on bandwidth asymmetry).  And small packets have proportionally
>>>> high overhead.
>>>>
>>>> (But it seems to only make a small difference for me, which always
>>>> surprises Seb).
>>>>
>>>> Alan
>>>>
>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>
>>>>> You are absolutely correct.
>>>>>
>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>> undeclared version 2. Everything works as stated.
>>>>>
>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>> values were not recognised.
>>>>>
>>>>> Thank you.
>>>>>
>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>
>>>>>
>>>>>
>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>
>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>> "overhead" or "atm".
>>>>>>
>>>>>> What's probably happening is that you're using a slightly old version
>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>> combination crop up ourselves recently.
>>>>>>
>>>>>> - Jonathan Morton
>>>>>>
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel



-- 
Dave Täht
worldwide bufferbloat report:
http://www.dslreports.com/speedtest/results/bufferbloat
And:
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Jonathan Morton]

> qdisc cake 8002: dev pppoe-wan root refcnt 2 bandwidth 850Kbit besteffort flows raw

> qdisc cake 8001: dev ifb4pppoe-wan root refcnt 2 bandwidth 11500Kbit besteffort flows atm overhead 40

> Download:  6.8 Mbps
>   Upload:  0.59 Mbps

Does anyone else see the discrepancy here?

Simply put, if the shaper isn’t set to a lower bandwidth than the link rate, it won’t control the queue.

 - Jonathan Morton

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Sebastian Moeller]

Hi Jonathan, hi Fred,

On Jul 10, 2015, at 21:18 , Jonathan Morton <chromatix99@gmail.com> wrote:

>> qdisc cake 8002: dev pppoe-wan root refcnt 2 bandwidth 850Kbit besteffort flows raw
> 
>> qdisc cake 8001: dev ifb4pppoe-wan root refcnt 2 bandwidth 11500Kbit besteffort flows atm overhead 40
> 
>> Download:  6.8 Mbps
>>  Upload:  0.59 Mbps
> 
> Does anyone else see the discrepancy here?
> 
> Simply put, if the shaper isn’t set to a lower bandwidth than the link rate, it won’t control the queue.

	Good point. @Fred what are the current sync rates as reported  your modem?

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 3300 bytes --]

Hi Fred,

On Jul 10, 2015, at 21:14 , Fred Stratton <fredstratton@imap.cc> wrote:

> 
> 
> On 10/07/15 19:46, Sebastian Moeller wrote:
>> Hi Fred,
>> 
>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>> 1) cat /etc/config/sqm
> config queue 'eth1'
>    option qdisc 'fq_codel'
>    option script 'simple.qos'
>    option qdisc_advanced '0'
>    option linklayer ‘none'

	For an ADSL link I could swear link layer ‘none’ is not optimal ;)

>    option enabled '0'
>    option interface 'eth1'
>    option download '0'
>    option upload ‘0'

	Not that it matters as you do not use the sqm-scripts machinery.

> 
> 
>> 2) tc -d qdisc
> qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev eth1 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc mq 0: dev wlan1 root
> qdisc fq_codel 0: dev wlan1 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan1 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan1 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan1 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc mq 0: dev wlan0 root
> qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> qdisc cake 8002: dev pppoe-wan root refcnt 2 bandwidth 850Kbit besteffort flows raw

	You should enable the atm link layer accounting on both ingress (ifb4pppoe-wan) and egress (pppoe-wan).

> qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
> qdisc cake 8001: dev ifb4pppoe-wan root refcnt 2 bandwidth 11500Kbit besteffort flows atm overhead 40
> 
>> 3) tc -d class show dev pppoe-wan
> class cake 8002:2fa parent 8002:
> 
> 
>> 4) tc -d class show dev ifb4pppoe-wqn
> 
> class cake 8001:6e parent 8001:
>> 5) /etc/init.d/sqm stop
> SQM: Trying to start/stop SQM on all interfaces.
> SQM: run.sh stop
> SQM: /usr/lib/sqm/run.sh SQM for interface eth1 is not enabled, skipping over...
>> 6) /etc/init.d/sqm start
> /etc/init.d/sqm start
> SQM: Trying to start/stop SQM on all interfaces.
> SQM: /usr/lib/sqm/run.sh SQM for interface eth1 is not enabled, skipping over…

	Alas, not activated so this does not give any diagnostic output… Newer sqm-scripts should work with cake and luci-app-sqm also works well to set up cake. Please find the most recent files for sqm-scripts attached. The content of the sqm folder should replace the content of /usr/lib/sqm on your router.

[-- Attachment #2: sqm.zip --]
[-- Type: application/zip, Size: 21639 bytes --]

[-- Attachment #3: Type: text/plain, Size: 75 bytes --]

 The content of sqm.lua should replace /usr/lib/lua/luci/model/cbi/sqm.lua 

[-- Attachment #4: sqm.lua --]
[-- Type: application/octet-stream, Size: 9571 bytes --]

--[[
LuCI - Lua Configuration Interface

Copyright 2014 Steven Barth <steven@midlink.org>
Copyright 2014 Dave Taht <dave.taht@bufferbloat.net>

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

$Id$
]]--

local wa = require "luci.tools.webadmin"
local fs = require "nixio.fs"
local net = require "luci.model.network".init()
local sys = require "luci.sys"
--local ifaces = net:get_interfaces()
local ifaces = sys.net:devices()
local path = "/usr/lib/sqm"

m = Map("sqm", translate("Smart Queue Management"),
	translate("With <abbr title=\"Smart Queue Management\">SQM</abbr> you " ..
		"can enable traffic shaping, better mixing (Fair Queueing)," ..
		" active queue length management (AQM) " ..
		" and prioritisation on one " ..
		"network interface."))

s = m:section(TypedSection, "queue", translate("Queues"))
s:tab("tab_basic", translate("Basic Settings"))
s:tab("tab_qdisc", translate("Queue Discipline"))
s:tab("tab_linklayer", translate("Link Layer Adaptation"))
s.addremove = true -- set to true to allow adding SQM instances in the GUI
s.anonymous = true

-- BASIC
e = s:taboption("tab_basic", Flag, "enabled", translate("Enable this SQM instance."))
e.rmempty = false

-- sm: following jow's advise, be helpful to the user and enable
--     sqm's init script if even a single sm instance/interface
--     is enabled; this is unexpected in that the init script gets
--     enabled as soon as at least one sqm instance is enabled
--     and that state is saved, so it does not require "Save & Apply"
--     to effect the init scripts.
--     the implementation was inpired/lifted from 
--     https://github.com/openwrt/luci/blob/master/applications/luci-app-minidlna/luasrc/model/cbi/minidlna.lua
function e.write(self, section, value)
    if value == "1" then
	luci.sys.init.enable("sqm")
	m.message = translate("The SQM GUI has just enabled the sqm initscript on your behalf. Remember to disable the sqm initscript manually under System Startup menu in case this change was not wished for.")
--	luci.sys.call("/etc/init.d/sqm start >/dev/null")
--    else
--	luci.sys.call("/etc/init.d/sqm stop >/dev/null")
--	luci.sys.init.disable("sqm")
    end
    return Flag.write(self, section, value)
end
-- TODO: inform the user what we just did...

n = s:taboption("tab_basic", ListValue, "interface", translate("Interface name"))
-- sm lifted from luci-app-wol, the original implementation failed to show pppoe-ge00 type interface names
for _, iface in ipairs(ifaces) do
--     if iface:is_up() then
--	n:value(iface:name())
--     end
	if iface ~= "lo" then 
		n:value(iface) 
	end
end
n.rmempty = false


dl = s:taboption("tab_basic", Value, "download", translate("Download speed (kbit/s) (ingress) set to 0 to selectively disable ingress shaping:"))
dl.datatype = "and(uinteger,min(0))"
dl.rmempty = false

ul = s:taboption("tab_basic", Value, "upload", translate("Upload speed (kbit/s) (egress) set to 0 to selectively disable egress shaping:"))
ul.datatype = "and(uinteger,min(0))"
ul.rmempty = false

-- QDISC

c = s:taboption("tab_qdisc", ListValue, "qdisc", translate("Queueing discipline"))
c:value("fq_codel", "fq_codel ("..translate("default")..")")
c:value("efq_codel")
c:value("nfq_codel")
c:value("sfq")
c:value("codel")
c:value("ns2_codel")
c:value("pie")
c:value("sfq")
c:value("cake")
c.default = "fq_codel"
c.rmempty = false

local qos_desc = ""
sc = s:taboption("tab_qdisc", ListValue, "script", translate("Queue setup script"))
for file in fs.dir(path) do
  if string.find(file, ".qos$") then
    sc:value(file)
  end
  if string.find(file, ".qos.help$") then
    fh = io.open(path .. "/" .. file, "r")
    qos_desc = qos_desc .. "<p><b>" .. file:gsub(".help$", "") .. ":</b><br />" .. fh:read("*a") .. "</p>"
  end
end
sc.default = "simple.qos"
sc.rmempty = false
sc.description = qos_desc

ad = s:taboption("tab_qdisc", Flag, "qdisc_advanced", translate("Show and Use Advanced Configuration. Advanced options will only be used as long as this box is checked."))
ad.default = false
ad.rmempty = true

squash_dscp  = s:taboption("tab_qdisc", ListValue, "squash_dscp", translate("Squash DSCP on inbound packets (ingress):"))
squash_dscp:value("1", "SQUASH")
squash_dscp:value("0", "DO NOT SQUASH")
squash_dscp.default = "1"
squash_dscp.rmempty = true
squash_dscp:depends("qdisc_advanced", "1")

squash_ingress = s:taboption("tab_qdisc", ListValue, "squash_ingress", translate("Ignore DSCP on ingress:"))
squash_ingress:value("1", "Ignore")
squash_ingress:value("0", "Allow")
squash_ingress.default = "1"
squash_ingress.rmempty = true
squash_ingress:depends("qdisc_advanced", "1")

iecn = s:taboption("tab_qdisc", ListValue, "ingress_ecn", translate("Explicit congestion notification (ECN) status on inbound packets (ingress):"))
iecn:value("ECN", "ECN ("..translate("default")..")")
iecn:value("NOECN")
iecn.default = "ECN"
iecn.rmempty = true
iecn:depends("qdisc_advanced", "1")

eecn = s:taboption("tab_qdisc", ListValue, "egress_ecn", translate("Explicit congestion notification (ECN) status on outbound packets (egress)."))
eecn:value("NOECN", "NOECN ("..translate("default")..")")
eecn:value("ECN")
eecn.default = "NOECN"
eecn.rmempty = true
eecn:depends("qdisc_advanced", "1")

ad2 = s:taboption("tab_qdisc", Flag, "qdisc_really_really_advanced", translate("Show and Use Dangerous Configuration. Dangerous options will only be used as long as this box is checked."))
ad2.default = false
ad2.rmempty = true
ad2:depends("qdisc_advanced", "1")

ilim = s:taboption("tab_qdisc", Value, "ilimit", translate("Hard limit on ingress queues; leave empty for default."))
-- ilim.default = 1000
ilim.isnumber = true
ilim.datatype = "and(uinteger,min(0))"
ilim.rmempty = true
ilim:depends("qdisc_really_really_advanced", "1")

elim = s:taboption("tab_qdisc", Value, "elimit", translate("Hard limit on egress queues; leave empty for default."))
-- elim.default = 1000
elim.datatype = "and(uinteger,min(0))"
elim.rmempty = true
elim:depends("qdisc_really_really_advanced", "1")


itarg = s:taboption("tab_qdisc", Value, "itarget", translate("Latency target for ingress, e.g 5ms [units: s, ms, or  us]; leave empty for automatic selection, put in the word default for the qdisc's default."))
itarg.datatype = "string"
itarg.rmempty = true
itarg:depends("qdisc_really_really_advanced", "1")

etarg = s:taboption("tab_qdisc", Value, "etarget", translate("Latency target for egress, e.g. 5ms [units: s, ms, or  us]; leave empty for automatic selection, put in the word default for the qdisc's default."))
etarg.datatype = "string"
etarg.rmempty = true
etarg:depends("qdisc_really_really_advanced", "1")



iqdisc_opts = s:taboption("tab_qdisc", Value, "iqdisc_opts", translate("Advanced option string to pass to the ingress queueing disciplines; no error checking, use very carefully."))
iqdisc_opts.rmempty = true
iqdisc_opts:depends("qdisc_really_really_advanced", "1")

eqdisc_opts = s:taboption("tab_qdisc", Value, "eqdisc_opts", translate("Advanced option string to pass to the egress queueing disciplines; no error checking, use very carefully."))
eqdisc_opts.rmempty = true
eqdisc_opts:depends("qdisc_really_really_advanced", "1")

-- LINKLAYER
ll = s:taboption("tab_linklayer", ListValue, "linklayer", translate("Which link layer to account for:"))
ll:value("none", "none ("..translate("default")..")")
ll:value("ethernet", "Ethernet with overhead: select for e.g. VDSL2.")
ll:value("atm", "ATM: select for e.g. ADSL1, ADSL2, ADSL2+.")
-- ll:value("adsl")	-- reduce the options
ll.default = "none"

po = s:taboption("tab_linklayer", Value, "overhead", translate("Per Packet Overhead (byte):"))
po.datatype = "and(integer,min(-1500))"
po.default = 0
po.isnumber = true
po.rmempty = true
po:depends("linklayer", "ethernet")
-- po:depends("linklayer", "adsl")
po:depends("linklayer", "atm")


adll = s:taboption("tab_linklayer", Flag, "linklayer_advanced", translate("Show Advanced Linklayer Options, (only needed if MTU > 1500). Advanced options will only be used as long as this box is checked."))
adll.rmempty = true
adll:depends("linklayer", "ethernet")
-- adll:depends("linklayer", "adsl")
adll:depends("linklayer", "atm")

smtu = s:taboption("tab_linklayer", Value, "tcMTU", translate("Maximal Size for size and rate calculations, tcMTU (byte); needs to be >= interface MTU + overhead:"))
smtu.datatype = "and(uinteger,min(0))"
smtu.default = 2047
smtu.isnumber = true
smtu.rmempty = true
smtu:depends("linklayer_advanced", "1")

stsize = s:taboption("tab_linklayer", Value, "tcTSIZE", translate("Number of entries in size/rate tables, TSIZE; for ATM choose TSIZE = (tcMTU + 1) / 16:"))
stsize.datatype = "and(uinteger,min(0))"
stsize.default = 128
stsize.isnumber = true
stsize.rmempty = true
stsize:depends("linklayer_advanced", "1")

smpu = s:taboption("tab_linklayer", Value, "tcMPU", translate("Minimal packet size, MPU (byte); needs to be > 0 for ethernet size tables:"))
smpu.datatype = "and(uinteger,min(0))"
smpu.default = 0
smpu.isnumber = true
smpu.rmempty = true
smpu:depends("linklayer_advanced", "1")

lla = s:taboption("tab_linklayer", ListValue, "linklayer_adaptation_mechanism", translate("Which linklayer adaptation mechanism to use; for testing only"))
lla:value("cake")
lla:value("htb_private")
lla:value("tc_stab", "tc_stab ("..translate("default")..")")
lla.default = "tc_stab"
lla.rmempty = true
lla:depends("linklayer_advanced", "1")

-- PRORITIES?

return m

[-- Attachment #5: Type: text/plain, Size: 5696 bytes --]

. These should allow you to set up cake from inside the sqm gui (but it is only lightly tested).


> 
>> 
>> hopefully these give some insight what might have happened.
>> 
>> And finally I would love to learn the output of:
>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
> 
> betterspeedtest.sh not installed

	Too bad, this would be nice as it measures downlink and uplink sequentially instead of simultaneously so it can help figure out if only one direction is improperly shaped. Could I convince you to install Rich’s betterspeedtest.sh script as well, it should be in the same repository as netperfrunner.sh?

> 
> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.ne
> t -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-
> eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
> sh: can't open 'betterspeedtest.sh'
> 2015-07-10 20:10:55 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging netperf-eu.bufferbloat.net. Takes about 150 seconds.
> Download:  6.8 Mbps
>   Upload:  0.59 Mbps
>  Latency: (in msec, 152 pings, 0.00% packet loss)
>      Min: 73.911
>    10pct: 232.211
>   Median: 308.556
>      Avg: 305.686
>    90pct: 376.183
>      Max: 412.553

	This just shows that latency still is bounded badly...

> 
>> 
>> 
>> Many Thanks & Best Regards
>> 	Sebastian
>> 
>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>> 
>>> By your command
>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>> 
>>> script configuring qdiscs and overhead 40 on
>>> 
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>> Download:  6.73 Mbps
>>>   Upload:  0.58 Mbps
>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>      Min: 24.094
>>>    10pct: 172.654
>>>   Median: 260.563
>>>      Avg: 253.580
>>>    90pct: 330.003
>>>      Max: 411.145
>>> 
>>> script configuring qdiscs on flows raw
>>> 
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>> 78.145.32.1
>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>> Download:  6.75 Mbps
>>>   Upload:  0.59 Mbps
>>>  Latency: (in msec, 59 pings, 0.00% packet loss)
>>>      Min: 23.605
>>>    10pct: 169.789
>>>   Median: 282.155
>>>      Avg: 267.099
>>>    90pct: 333.283
>>>      Max: 376.509
>>> 
>>> script configuring qdiscs and overhead 36 on
>>> 
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>> 80.44.96.1
>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>> Download:  6.56 Mbps
>>>   Upload:  0.59 Mbps
>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>      Min: 22.975
>>>    10pct: 195.473
>>>   Median: 281.756
>>>      Avg: 271.609
>>>    90pct: 342.130
>>>      Max: 398.573
>>> 
>>> 
>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>> 
>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>> 
>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>> 
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>> 
>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>> 
>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>> 
>>>> Alan
>>>> 
>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>> You are absolutely correct.
>>>>> 
>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>> undeclared version 2. Everything works as stated.
>>>>> 
>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>> values were not recognised.
>>>>> 
>>>>> Thank you.
>>>>> 
>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>> working, so I have nothing to report about gateway usage at present.
>>>>> 
>>>>> 
>>>>> 
>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>> "overhead" or "atm".
>>>>>> 
>>>>>> What's probably happening is that you're using a slightly old version
>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>> combination crop up ourselves recently.
>>>>>> 
>>>>>> - Jonathan Morton
>>>>>> 
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

bridge sync is circa 10 000 kbit/s

with the cake option in sqm enabled

config queue 'eth1'
     option qdisc_advanced '0'
     option enabled '1'
     option interface 'pppoe-wan'
     option upload '850'
     option qdisc 'cake'
     option script 'simple_pppoe.qos'
     option linklayer 'atm'
     option overhead '40'
     option download '8500'

tc -s qdisc show dev pppoe-wan
qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 
direct_qlen 3
  Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
  backlog 0b 0p requeues 0
qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
  Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
            Class 0     Class 1     Class 2     Class 3
   rate          0bit        0bit        0bit        0bit
   target       5.0ms       5.0ms       5.0ms       5.0ms
interval     100.0ms     100.0ms     100.0ms     100.0ms
Pk delay         0us         0us         7us         2us
Av delay         0us         0us         0us         0us
Sp delay         0us         0us         0us         0us
   pkts             0           0          22           3
way inds           0           0           0           0
way miss           0           0          22           2
way cols           0           0           0           0
   bytes            0           0        3392        1007
   drops            0           0           0           0
   marks            0           0           0           0
qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
  Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
            Class 0     Class 1     Class 2     Class 3
   rate          0bit        0bit        0bit        0bit
   target       5.0ms       5.0ms       5.0ms       5.0ms
interval     100.0ms     100.0ms     100.0ms     100.0ms
Pk delay         0us      28.0ms         0us         0us
Av delay         0us       1.2ms         0us         0us
Sp delay         0us         4us         0us         0us
   pkts             0         417           0           0
way inds           0           0           0           0
way miss           0          23           0           0
way cols           0           0           0           0
   bytes            0       98951           0           0
   drops            0           2           0           0
   marks            0           0           0           0
qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
            Class 0     Class 1     Class 2     Class 3
   rate          0bit        0bit        0bit        0bit
   target       5.0ms       5.0ms       5.0ms       5.0ms
interval     100.0ms     100.0ms     100.0ms     100.0ms
Pk delay         0us         0us         0us         0us
Av delay         0us         0us         0us         0us
Sp delay         0us         0us         0us         0us
   pkts             0           0           0           0
way inds           0           0           0           0
way miss           0           0           0           0
way cols           0           0           0           0
   bytes            0           0           0           0
   drops            0           0           0           0
   marks            0           0           0           0
qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
            Class 0     Class 1     Class 2     Class 3
   rate          0bit        0bit        0bit        0bit
   target       5.0ms       5.0ms       5.0ms       5.0ms
interval     100.0ms     100.0ms     100.0ms     100.0ms
Pk delay         0us         0us         0us         0us
Av delay         0us         0us         0us         0us
Sp delay         0us         0us         0us         0us
   pkts             0           0           0           0
way inds           0           0           0           0
way miss           0           0           0           0
way cols           0           0           0           0
   bytes            0           0           0           0
   drops            0           0           0           0
   marks            0           0           0           0
qdisc ingress ffff: parent ffff:fff1 ----------------
  Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0




On 10/07/15 19:46, Sebastian Moeller wrote:
> Hi Fred,
>
> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
> 1) cat /etc/config/sqm
> 2) tc -d qdisc
> 3) tc -d class show dev pppoe-wan
> 4) tc -d class show dev ifb4pppoe-wqn
> 5) /etc/init.d/sqm stop
> 6) /etc/init.d/sqm start
>
> hopefully these give some insight what might have happened.
>
> And finally I would love to learn the output of:
> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>
>
> Many Thanks & Best Regards
> 	Sebastian
>
> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>
>> By your command
>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>
>> script configuring qdiscs and overhead 40 on
>>
>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>> Download:  6.73 Mbps
>>    Upload:  0.58 Mbps
>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>       Min: 24.094
>>     10pct: 172.654
>>    Median: 260.563
>>       Avg: 253.580
>>     90pct: 330.003
>>       Max: 411.145
>>
>> script configuring qdiscs on flows raw
>>
>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>> 78.145.32.1
>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>> Download:  6.75 Mbps
>>    Upload:  0.59 Mbps
>>   Latency: (in msec, 59 pings, 0.00% packet loss)
>>       Min: 23.605
>>     10pct: 169.789
>>    Median: 282.155
>>       Avg: 267.099
>>     90pct: 333.283
>>       Max: 376.509
>>
>> script configuring qdiscs and overhead 36 on
>>
>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>> 80.44.96.1
>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>> Download:  6.56 Mbps
>>    Upload:  0.59 Mbps
>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>       Min: 22.975
>>     10pct: 195.473
>>    Median: 281.756
>>       Avg: 271.609
>>     90pct: 342.130
>>       Max: 398.573
>>
>>
>> On 10/07/15 16:19, Alan Jenkins wrote:
>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>
>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>
>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>
>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>
>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>
>>> Alan
>>>
>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>> You are absolutely correct.
>>>>
>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>> undeclared version 2. Everything works as stated.
>>>>
>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>> values were not recognised.
>>>>
>>>> Thank you.
>>>>
>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>> working, so I have nothing to report about gateway usage at present.
>>>>
>>>>
>>>>
>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>> You're already using correct syntax - I've written it to be quite
>>>>> lenient and use sensible defaults for missing information. There are
>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>> "overhead" or "atm".
>>>>>
>>>>> What's probably happening is that you're using a slightly old version
>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>> but a more up to date tc which does support it. We've seen this
>>>>> combination crop up ourselves recently.
>>>>>
>>>>> - Jonathan Morton
>>>>>
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Sebastian Moeller]

Hi Fred,


On Jul 10, 2015, at 21:34 , Fred Stratton <fredstratton@imap.cc> wrote:

> bridge sync is circa 10 000 kbit/s
> 
> with the cake option in sqm enabled
> 
> config queue 'eth1'
>    option qdisc_advanced '0'
>    option enabled '1'
>    option interface 'pppoe-wan'
>    option upload '850'
>    option qdisc 'cake'
>    option script 'simple_pppoe.qos'
>    option linklayer 'atm'
>    option overhead '40'
>    option download ‘8500'

	So this looks reasonable. Then again, if the DSLAM is under provisioned/oversubscribed (= congested) shaping uypur DSL link might not fix all buffer bloat..

> 
> tc -s qdisc show dev pppoe-wan
> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 3
> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
> backlog 0b 0p requeues 0
> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>           Class 0     Class 1     Class 2     Class 3
>  rate          0bit        0bit        0bit        0bit
>  target       5.0ms       5.0ms       5.0ms       5.0ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk delay         0us         0us         7us         2us
> Av delay         0us         0us         0us         0us
> Sp delay         0us         0us         0us         0us
>  pkts             0           0          22           3
> way inds           0           0           0           0
> way miss           0           0          22           2
> way cols           0           0           0           0
>  bytes            0           0        3392        1007
>  drops            0           0           0           0
>  marks            0           0           0           0
> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>           Class 0     Class 1     Class 2     Class 3
>  rate          0bit        0bit        0bit        0bit
>  target       5.0ms       5.0ms       5.0ms       5.0ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk delay         0us      28.0ms         0us         0us
> Av delay         0us       1.2ms         0us         0us
> Sp delay         0us         4us         0us         0us
>  pkts             0         417           0           0
> way inds           0           0           0           0
> way miss           0          23           0           0
> way cols           0           0           0           0
>  bytes            0       98951           0           0
>  drops            0           2           0           0
>  marks            0           0           0           0
> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>           Class 0     Class 1     Class 2     Class 3
>  rate          0bit        0bit        0bit        0bit
>  target       5.0ms       5.0ms       5.0ms       5.0ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk delay         0us         0us         0us         0us
> Av delay         0us         0us         0us         0us
> Sp delay         0us         0us         0us         0us
>  pkts             0           0           0           0
> way inds           0           0           0           0
> way miss           0           0           0           0
> way cols           0           0           0           0
>  bytes            0           0           0           0
>  drops            0           0           0           0
>  marks            0           0           0           0
> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>           Class 0     Class 1     Class 2     Class 3
>  rate          0bit        0bit        0bit        0bit
>  target       5.0ms       5.0ms       5.0ms       5.0ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk delay         0us         0us         0us         0us
> Av delay         0us         0us         0us         0us
> Sp delay         0us         0us         0us         0us
>  pkts             0           0           0           0
> way inds           0           0           0           0
> way miss           0           0           0           0
> way cols           0           0           0           0
>  bytes            0           0           0           0
>  drops            0           0           0           0
>  marks            0           0           0           0
> qdisc ingress ffff: parent ffff:fff1 ----------------
> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0

	But this is the hallmark of out of date sqm-scripts, this just uses cake as leaf qdisc and keeps HTB as the main shaper; a configuration that is useful for testing. I assume this is the old set of sqm-scripts not the update I just sent as attachment? If so could you retry with the newer scripts, please?

Best Regards
	Sebastian

> 
> 
> 
> 
> On 10/07/15 19:46, Sebastian Moeller wrote:
>> Hi Fred,
>> 
>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>> 1) cat /etc/config/sqm
>> 2) tc -d qdisc
>> 3) tc -d class show dev pppoe-wan
>> 4) tc -d class show dev ifb4pppoe-wqn
>> 5) /etc/init.d/sqm stop
>> 6) /etc/init.d/sqm start
>> 
>> hopefully these give some insight what might have happened.
>> 
>> And finally I would love to learn the output of:
>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>> 
>> 
>> Many Thanks & Best Regards
>> 	Sebastian
>> 
>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>> 
>>> By your command
>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>> 
>>> script configuring qdiscs and overhead 40 on
>>> 
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>> Download:  6.73 Mbps
>>>   Upload:  0.58 Mbps
>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>      Min: 24.094
>>>    10pct: 172.654
>>>   Median: 260.563
>>>      Avg: 253.580
>>>    90pct: 330.003
>>>      Max: 411.145
>>> 
>>> script configuring qdiscs on flows raw
>>> 
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>> 78.145.32.1
>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>> Download:  6.75 Mbps
>>>   Upload:  0.59 Mbps
>>>  Latency: (in msec, 59 pings, 0.00% packet loss)
>>>      Min: 23.605
>>>    10pct: 169.789
>>>   Median: 282.155
>>>      Avg: 267.099
>>>    90pct: 333.283
>>>      Max: 376.509
>>> 
>>> script configuring qdiscs and overhead 36 on
>>> 
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>> 80.44.96.1
>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>> Download:  6.56 Mbps
>>>   Upload:  0.59 Mbps
>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>      Min: 22.975
>>>    10pct: 195.473
>>>   Median: 281.756
>>>      Avg: 271.609
>>>    90pct: 342.130
>>>      Max: 398.573
>>> 
>>> 
>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>> 
>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>> 
>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>> 
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>> 
>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>> 
>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>> 
>>>> Alan
>>>> 
>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>> You are absolutely correct.
>>>>> 
>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>> undeclared version 2. Everything works as stated.
>>>>> 
>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>> values were not recognised.
>>>>> 
>>>>> Thank you.
>>>>> 
>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>> working, so I have nothing to report about gateway usage at present.
>>>>> 
>>>>> 
>>>>> 
>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>> "overhead" or "atm".
>>>>>> 
>>>>>> What's probably happening is that you're using a slightly old version
>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>> combination crop up ourselves recently.
>>>>>> 
>>>>>> - Jonathan Morton
>>>>>> 
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

These are the latest scripts,  AFAIK

no overhead allowance. I note.



On 10/07/15 20:40, Sebastian Moeller wrote:
> Hi Fred,
>
>
> On Jul 10, 2015, at 21:34 , Fred Stratton <fredstratton@imap.cc> wrote:
>
>> bridge sync is circa 10 000 kbit/s
>>
>> with the cake option in sqm enabled
>>
>> config queue 'eth1'
>>     option qdisc_advanced '0'
>>     option enabled '1'
>>     option interface 'pppoe-wan'
>>     option upload '850'
>>     option qdisc 'cake'
>>     option script 'simple_pppoe.qos'
>>     option linklayer 'atm'
>>     option overhead '40'
>>     option download ‘8500'
> 	So this looks reasonable. Then again, if the DSLAM is under provisioned/oversubscribed (= congested) shaping uypur DSL link might not fix all buffer bloat..
>
>> tc -s qdisc show dev pppoe-wan
>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 3
>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>> backlog 0b 0p requeues 0
>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>            Class 0     Class 1     Class 2     Class 3
>>   rate          0bit        0bit        0bit        0bit
>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk delay         0us         0us         7us         2us
>> Av delay         0us         0us         0us         0us
>> Sp delay         0us         0us         0us         0us
>>   pkts             0           0          22           3
>> way inds           0           0           0           0
>> way miss           0           0          22           2
>> way cols           0           0           0           0
>>   bytes            0           0        3392        1007
>>   drops            0           0           0           0
>>   marks            0           0           0           0
>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>            Class 0     Class 1     Class 2     Class 3
>>   rate          0bit        0bit        0bit        0bit
>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk delay         0us      28.0ms         0us         0us
>> Av delay         0us       1.2ms         0us         0us
>> Sp delay         0us         4us         0us         0us
>>   pkts             0         417           0           0
>> way inds           0           0           0           0
>> way miss           0          23           0           0
>> way cols           0           0           0           0
>>   bytes            0       98951           0           0
>>   drops            0           2           0           0
>>   marks            0           0           0           0
>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>            Class 0     Class 1     Class 2     Class 3
>>   rate          0bit        0bit        0bit        0bit
>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk delay         0us         0us         0us         0us
>> Av delay         0us         0us         0us         0us
>> Sp delay         0us         0us         0us         0us
>>   pkts             0           0           0           0
>> way inds           0           0           0           0
>> way miss           0           0           0           0
>> way cols           0           0           0           0
>>   bytes            0           0           0           0
>>   drops            0           0           0           0
>>   marks            0           0           0           0
>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>            Class 0     Class 1     Class 2     Class 3
>>   rate          0bit        0bit        0bit        0bit
>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk delay         0us         0us         0us         0us
>> Av delay         0us         0us         0us         0us
>> Sp delay         0us         0us         0us         0us
>>   pkts             0           0           0           0
>> way inds           0           0           0           0
>> way miss           0           0           0           0
>> way cols           0           0           0           0
>>   bytes            0           0           0           0
>>   drops            0           0           0           0
>>   marks            0           0           0           0
>> qdisc ingress ffff: parent ffff:fff1 ----------------
>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
> 	But this is the hallmark of out of date sqm-scripts, this just uses cake as leaf qdisc and keeps HTB as the main shaper; a configuration that is useful for testing. I assume this is the old set of sqm-scripts not the update I just sent as attachment? If so could you retry with the newer scripts, please?
>
> Best Regards
> 	Sebastian
>
>>
>>
>>
>> On 10/07/15 19:46, Sebastian Moeller wrote:
>>> Hi Fred,
>>>
>>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>>> 1) cat /etc/config/sqm
>>> 2) tc -d qdisc
>>> 3) tc -d class show dev pppoe-wan
>>> 4) tc -d class show dev ifb4pppoe-wqn
>>> 5) /etc/init.d/sqm stop
>>> 6) /etc/init.d/sqm start
>>>
>>> hopefully these give some insight what might have happened.
>>>
>>> And finally I would love to learn the output of:
>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>>
>>>
>>> Many Thanks & Best Regards
>>> 	Sebastian
>>>
>>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>
>>>> By your command
>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>>>
>>>> script configuring qdiscs and overhead 40 on
>>>>
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>>> Download:  6.73 Mbps
>>>>    Upload:  0.58 Mbps
>>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>       Min: 24.094
>>>>     10pct: 172.654
>>>>    Median: 260.563
>>>>       Avg: 253.580
>>>>     90pct: 330.003
>>>>       Max: 411.145
>>>>
>>>> script configuring qdiscs on flows raw
>>>>
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>> 78.145.32.1
>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>>> Download:  6.75 Mbps
>>>>    Upload:  0.59 Mbps
>>>>   Latency: (in msec, 59 pings, 0.00% packet loss)
>>>>       Min: 23.605
>>>>     10pct: 169.789
>>>>    Median: 282.155
>>>>       Avg: 267.099
>>>>     90pct: 333.283
>>>>       Max: 376.509
>>>>
>>>> script configuring qdiscs and overhead 36 on
>>>>
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>> 80.44.96.1
>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>>> Download:  6.56 Mbps
>>>>    Upload:  0.59 Mbps
>>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>       Min: 22.975
>>>>     10pct: 195.473
>>>>    Median: 281.756
>>>>       Avg: 271.609
>>>>     90pct: 342.130
>>>>       Max: 398.573
>>>>
>>>>
>>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>>>
>>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>>>
>>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>>
>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>>
>>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>>>
>>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>>>
>>>>> Alan
>>>>>
>>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>> You are absolutely correct.
>>>>>>
>>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>>> undeclared version 2. Everything works as stated.
>>>>>>
>>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>>> values were not recognised.
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>>> "overhead" or "atm".
>>>>>>>
>>>>>>> What's probably happening is that you're using a slightly old version
>>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>>> combination crop up ourselves recently.
>>>>>>>
>>>>>>> - Jonathan Morton
>>>>>>>
>>>> _______________________________________________
>>>> Cerowrt-devel mailing list
>>>> Cerowrt-devel@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Alan Jenkins]

On 10/07/15 20:45, Fred Stratton wrote:
> These are the latest scripts,  AFAIK
>
> no overhead allowance. I note.
>

Excellent point.  There is, but SQM uses "tc stab" for it by default, 
and there's no way to read that out.

To test using the overhead code in cake, add

     option linklayer_adaptation_mechanism 'cake'

>
>
> On 10/07/15 20:40, Sebastian Moeller wrote:
>> Hi Fred,
>>
>>
>> On Jul 10, 2015, at 21:34 , Fred Stratton <fredstratton@imap.cc> wrote:
>>
>>> bridge sync is circa 10 000 kbit/s
>>>
>>> with the cake option in sqm enabled
>>>
>>> config queue 'eth1'
>>>     option qdisc_advanced '0'
>>>     option enabled '1'
>>>     option interface 'pppoe-wan'
>>>     option upload '850'
>>>     option qdisc 'cake'
>>>     option script 'simple_pppoe.qos'
>>>     option linklayer 'atm'
>>>     option overhead '40'
>>>     option download ‘8500'
>>     So this looks reasonable. Then again, if the DSLAM is under 
>> provisioned/oversubscribed (= congested) shaping uypur DSL link might 
>> not fix all buffer bloat..
>>
>>> tc -s qdisc show dev pppoe-wan
>>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 
>>> direct_qlen 3
>>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>>> backlog 0b 0p requeues 0
>>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>            Class 0     Class 1     Class 2     Class 3
>>>   rate          0bit        0bit        0bit        0bit
>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>> Pk delay         0us         0us         7us         2us
>>> Av delay         0us         0us         0us         0us
>>> Sp delay         0us         0us         0us         0us
>>>   pkts             0           0          22           3
>>> way inds           0           0           0           0
>>> way miss           0           0          22           2
>>> way cols           0           0           0           0
>>>   bytes            0           0        3392        1007
>>>   drops            0           0           0           0
>>>   marks            0           0           0           0
>>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>            Class 0     Class 1     Class 2     Class 3
>>>   rate          0bit        0bit        0bit        0bit
>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>> Pk delay         0us      28.0ms         0us         0us
>>> Av delay         0us       1.2ms         0us         0us
>>> Sp delay         0us         4us         0us         0us
>>>   pkts             0         417           0           0
>>> way inds           0           0           0           0
>>> way miss           0          23           0           0
>>> way cols           0           0           0           0
>>>   bytes            0       98951           0           0
>>>   drops            0           2           0           0
>>>   marks            0           0           0           0
>>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>            Class 0     Class 1     Class 2     Class 3
>>>   rate          0bit        0bit        0bit        0bit
>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>> Pk delay         0us         0us         0us         0us
>>> Av delay         0us         0us         0us         0us
>>> Sp delay         0us         0us         0us         0us
>>>   pkts             0           0           0           0
>>> way inds           0           0           0           0
>>> way miss           0           0           0           0
>>> way cols           0           0           0           0
>>>   bytes            0           0           0           0
>>>   drops            0           0           0           0
>>>   marks            0           0           0           0
>>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>            Class 0     Class 1     Class 2     Class 3
>>>   rate          0bit        0bit        0bit        0bit
>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>> Pk delay         0us         0us         0us         0us
>>> Av delay         0us         0us         0us         0us
>>> Sp delay         0us         0us         0us         0us
>>>   pkts             0           0           0           0
>>> way inds           0           0           0           0
>>> way miss           0           0           0           0
>>> way cols           0           0           0           0
>>>   bytes            0           0           0           0
>>>   drops            0           0           0           0
>>>   marks            0           0           0           0
>>> qdisc ingress ffff: parent ffff:fff1 ----------------
>>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>     But this is the hallmark of out of date sqm-scripts, this just 
>> uses cake as leaf qdisc and keeps HTB as the main shaper; a 
>> configuration that is useful for testing. I assume this is the old 
>> set of sqm-scripts not the update I just sent as attachment? If so 
>> could you retry with the newer scripts, please?
>>
>> Best Regards
>>     Sebastian
>>
>>>
>>>
>>>
>>> On 10/07/15 19:46, Sebastian Moeller wrote:
>>>> Hi Fred,
>>>>
>>>> your results seem to indicate that cake is not active at all, as 
>>>> the latency under load is abysmal (a quick check is to look at the 
>>>> median in relation to the min and the 90% number, in your examples 
>>>> all of these are terrible). Could you please post the result of the 
>>>> following commands on your router:
>>>> 1) cat /etc/config/sqm
>>>> 2) tc -d qdisc
>>>> 3) tc -d class show dev pppoe-wan
>>>> 4) tc -d class show dev ifb4pppoe-wqn
>>>> 5) /etc/init.d/sqm stop
>>>> 6) /etc/init.d/sqm start
>>>>
>>>> hopefully these give some insight what might have happened.
>>>>
>>>> And finally I would love to learn the output of:
>>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p 
>>>> netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H 
>>>> netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>>>
>>>>
>>>> Many Thanks & Best Regards
>>>>     Sebastian
>>>>
>>>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> 
>>>> wrote:
>>>>
>>>>> By your command
>>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from 
>>>>> the command-line, so suboptimal process as end-point changed.
>>>>>
>>>>> script configuring qdiscs and overhead 40 on
>>>>>
>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 
>>>>> 4 streams down and up while pinging 2.96.48.1. Takes about 60 
>>>>> seconds.
>>>>> Download:  6.73 Mbps
>>>>>    Upload:  0.58 Mbps
>>>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>       Min: 24.094
>>>>>     10pct: 172.654
>>>>>    Median: 260.563
>>>>>       Avg: 253.580
>>>>>     90pct: 330.003
>>>>>       Max: 411.145
>>>>>
>>>>> script configuring qdiscs on flows raw
>>>>>
>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>> 78.145.32.1
>>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 
>>>>> 4 streams down and up while pinging 78.145.32.1. Takes about 60 
>>>>> seconds.
>>>>> Download:  6.75 Mbps
>>>>>    Upload:  0.59 Mbps
>>>>>   Latency: (in msec, 59 pings, 0.00% packet loss)
>>>>>       Min: 23.605
>>>>>     10pct: 169.789
>>>>>    Median: 282.155
>>>>>       Avg: 267.099
>>>>>     90pct: 333.283
>>>>>       Max: 376.509
>>>>>
>>>>> script configuring qdiscs and overhead 36 on
>>>>>
>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>> 80.44.96.1
>>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 
>>>>> 4 streams down and up while pinging 80.44.96.1. Takes about 60 
>>>>> seconds.
>>>>> Download:  6.56 Mbps
>>>>>    Upload:  0.59 Mbps
>>>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>       Min: 22.975
>>>>>     10pct: 195.473
>>>>>    Median: 281.756
>>>>>       Avg: 271.609
>>>>>     90pct: 342.130
>>>>>       Max: 398.573
>>>>>
>>>>>
>>>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>>> I'm glad to hear there's a working version (even if it's not in 
>>>>>> the current build :).
>>>>>>
>>>>>> Do you have measurable improvements with overhead configured 
>>>>>> (v.s. unconfigured)?
>>>>>>
>>>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>>>
>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>>>
>>>>>> I believe accounting for overhead helps on this two-way test, 
>>>>>> because a) it saturates the uplink b) about half that bandwidth 
>>>>>> is tiny ack packets (depending on bandwidth asymmetry).  And 
>>>>>> small packets have proportionally high overhead.
>>>>>>
>>>>>> (But it seems to only make a small difference for me, which 
>>>>>> always surprises Seb).
>>>>>>
>>>>>> Alan
>>>>>>
>>>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>>> You are absolutely correct.
>>>>>>>
>>>>>>> I tried both a numeric overhead value, and alternatively 
>>>>>>> 'pppoe-vcmux'
>>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is 
>>>>>>> lupin
>>>>>>> undeclared version 2. Everything works as stated.
>>>>>>>
>>>>>>> On lupin undeclared version 4, the current release based on 
>>>>>>> r46117, the
>>>>>>> values were not recognised.
>>>>>>>
>>>>>>> Thank you.
>>>>>>>
>>>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>>>> working, so I have nothing to report about gateway usage at 
>>>>>>> present.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>>>> lenient and use sensible defaults for missing information. 
>>>>>>>> There are
>>>>>>>> several sets of keywords and parameters which are mutually 
>>>>>>>> orthogonal,
>>>>>>>> and don't depend on each other, so "besteffort" has nothing to 
>>>>>>>> do with
>>>>>>>> "overhead" or "atm".
>>>>>>>>
>>>>>>>> What's probably happening is that you're using a slightly old 
>>>>>>>> version
>>>>>>>> of the cake kernel module which lacks the overhead parameter 
>>>>>>>> entirely,
>>>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>>>> combination crop up ourselves recently.
>>>>>>>>
>>>>>>>> - Jonathan Morton
>>>>>>>>
>>>>> _______________________________________________
>>>>> Cerowrt-devel mailing list
>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Sebastian Moeller]

Hi Fred,

On Jul 10, 2015, at 21:45 , Fred Stratton <fredstratton@imap.cc> wrote:

> These are the latest scripts,  AFAIK

	Let me repeat my question: are these the scripts I attached in one of the last mails, or the most recent scripts from ceropackages-3.10? The version in the openwrt repository is NOT recent, yet. Pushing the latest verso into openwrt is on my todo list but that will need a bit more changes, so please try the files I attached which should work (unless I screwed up and attached the wrong version). Also I have no working cake on my router and hence require help for testing and that means there might be some undiscovered bugs in there.

> 
> no overhead allowance. I note.

	Well, that should work with the most recent version

Best Regards
	Sebastian

> 
> 
> 
> On 10/07/15 20:40, Sebastian Moeller wrote:
>> Hi Fred,
>> 
>> 
>> On Jul 10, 2015, at 21:34 , Fred Stratton <fredstratton@imap.cc> wrote:
>> 
>>> bridge sync is circa 10 000 kbit/s
>>> 
>>> with the cake option in sqm enabled
>>> 
>>> config queue 'eth1'
>>>    option qdisc_advanced '0'
>>>    option enabled '1'
>>>    option interface 'pppoe-wan'
>>>    option upload '850'
>>>    option qdisc 'cake'
>>>    option script 'simple_pppoe.qos'
>>>    option linklayer 'atm'
>>>    option overhead '40'
>>>    option download ‘8500'
>> 	So this looks reasonable. Then again, if the DSLAM is under provisioned/oversubscribed (= congested) shaping uypur DSL link might not fix all buffer bloat..
>> 
>>> tc -s qdisc show dev pppoe-wan
>>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 3
>>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>>> backlog 0b 0p requeues 0
>>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>           Class 0     Class 1     Class 2     Class 3
>>>  rate          0bit        0bit        0bit        0bit
>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>> Pk delay         0us         0us         7us         2us
>>> Av delay         0us         0us         0us         0us
>>> Sp delay         0us         0us         0us         0us
>>>  pkts             0           0          22           3
>>> way inds           0           0           0           0
>>> way miss           0           0          22           2
>>> way cols           0           0           0           0
>>>  bytes            0           0        3392        1007
>>>  drops            0           0           0           0
>>>  marks            0           0           0           0
>>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>           Class 0     Class 1     Class 2     Class 3
>>>  rate          0bit        0bit        0bit        0bit
>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>> Pk delay         0us      28.0ms         0us         0us
>>> Av delay         0us       1.2ms         0us         0us
>>> Sp delay         0us         4us         0us         0us
>>>  pkts             0         417           0           0
>>> way inds           0           0           0           0
>>> way miss           0          23           0           0
>>> way cols           0           0           0           0
>>>  bytes            0       98951           0           0
>>>  drops            0           2           0           0
>>>  marks            0           0           0           0
>>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>           Class 0     Class 1     Class 2     Class 3
>>>  rate          0bit        0bit        0bit        0bit
>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>> Pk delay         0us         0us         0us         0us
>>> Av delay         0us         0us         0us         0us
>>> Sp delay         0us         0us         0us         0us
>>>  pkts             0           0           0           0
>>> way inds           0           0           0           0
>>> way miss           0           0           0           0
>>> way cols           0           0           0           0
>>>  bytes            0           0           0           0
>>>  drops            0           0           0           0
>>>  marks            0           0           0           0
>>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>           Class 0     Class 1     Class 2     Class 3
>>>  rate          0bit        0bit        0bit        0bit
>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>> Pk delay         0us         0us         0us         0us
>>> Av delay         0us         0us         0us         0us
>>> Sp delay         0us         0us         0us         0us
>>>  pkts             0           0           0           0
>>> way inds           0           0           0           0
>>> way miss           0           0           0           0
>>> way cols           0           0           0           0
>>>  bytes            0           0           0           0
>>>  drops            0           0           0           0
>>>  marks            0           0           0           0
>>> qdisc ingress ffff: parent ffff:fff1 ----------------
>>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>> 	But this is the hallmark of out of date sqm-scripts, this just uses cake as leaf qdisc and keeps HTB as the main shaper; a configuration that is useful for testing. I assume this is the old set of sqm-scripts not the update I just sent as attachment? If so could you retry with the newer scripts, please?
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> 
>>> 
>>> 
>>> On 10/07/15 19:46, Sebastian Moeller wrote:
>>>> Hi Fred,
>>>> 
>>>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>>>> 1) cat /etc/config/sqm
>>>> 2) tc -d qdisc
>>>> 3) tc -d class show dev pppoe-wan
>>>> 4) tc -d class show dev ifb4pppoe-wqn
>>>> 5) /etc/init.d/sqm stop
>>>> 6) /etc/init.d/sqm start
>>>> 
>>>> hopefully these give some insight what might have happened.
>>>> 
>>>> And finally I would love to learn the output of:
>>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>>> 
>>>> 
>>>> Many Thanks & Best Regards
>>>> 	Sebastian
>>>> 
>>>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>> 
>>>>> By your command
>>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>>>> 
>>>>> script configuring qdiscs and overhead 40 on
>>>>> 
>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>>>> Download:  6.73 Mbps
>>>>>   Upload:  0.58 Mbps
>>>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>      Min: 24.094
>>>>>    10pct: 172.654
>>>>>   Median: 260.563
>>>>>      Avg: 253.580
>>>>>    90pct: 330.003
>>>>>      Max: 411.145
>>>>> 
>>>>> script configuring qdiscs on flows raw
>>>>> 
>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>> 78.145.32.1
>>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>>>> Download:  6.75 Mbps
>>>>>   Upload:  0.59 Mbps
>>>>>  Latency: (in msec, 59 pings, 0.00% packet loss)
>>>>>      Min: 23.605
>>>>>    10pct: 169.789
>>>>>   Median: 282.155
>>>>>      Avg: 267.099
>>>>>    90pct: 333.283
>>>>>      Max: 376.509
>>>>> 
>>>>> script configuring qdiscs and overhead 36 on
>>>>> 
>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>> 80.44.96.1
>>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>>>> Download:  6.56 Mbps
>>>>>   Upload:  0.59 Mbps
>>>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>      Min: 22.975
>>>>>    10pct: 195.473
>>>>>   Median: 281.756
>>>>>      Avg: 271.609
>>>>>    90pct: 342.130
>>>>>      Max: 398.573
>>>>> 
>>>>> 
>>>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>>>> 
>>>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>>>> 
>>>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>>> 
>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>>> 
>>>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>>>> 
>>>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>>>> 
>>>>>> Alan
>>>>>> 
>>>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>>> You are absolutely correct.
>>>>>>> 
>>>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>>>> undeclared version 2. Everything works as stated.
>>>>>>> 
>>>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>>>> values were not recognised.
>>>>>>> 
>>>>>>> Thank you.
>>>>>>> 
>>>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>>>> "overhead" or "atm".
>>>>>>>> 
>>>>>>>> What's probably happening is that you're using a slightly old version
>>>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>>>> combination crop up ourselves recently.
>>>>>>>> 
>>>>>>>> - Jonathan Morton
>>>>>>>> 
>>>>> _______________________________________________
>>>>> Cerowrt-devel mailing list
>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

replaced /usr/lib/sqm as ordered

cat /etc/config/sqm

config queue 'eth1'
     option qdisc_advanced '0'
     option enabled '1'
     option interface 'pppoe-wan'
     option upload '850'
     option qdisc 'cake'
     option linklayer 'atm'
     option overhead '40'
     option download '8500'
     option script 'simple.qos'
     option linklayer_advanced '1'
     option tcMTU '2047'
     option tcTSIZE '128'
     option tcMPU '0'
     option linklayer_adaptation_mechanism 'cake'

root@OpenWrt:~# tc -s qdisc show
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 
300 target 5.0ms interval 100.0ms ecn
  Sent 5904206 bytes 10771 pkt (dropped 0, overlimits 0 requeues 1)
  backlog 0b 0p requeues 1
   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 1024p flows 1024 quantum 
300 target 5.0ms interval 100.0ms ecn
  Sent 903154 bytes 7941 pkt (dropped 0, overlimits 0 requeues 2)
  backlog 0b 0p requeues 2
   maxpacket 1322 drop_overlimit 0 new_flow_count 1 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc mq 0: dev wlan0 root
  Sent 198495 bytes 816 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
  Sent 3709 bytes 19 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
  Sent 112 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
  Sent 194674 bytes 796 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc mq 0: dev wlan1 root
  Sent 53249 bytes 323 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
qdisc fq_codel 0: dev wlan1 parent :1 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
  Sent 1337 bytes 9 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan1 parent :2 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan1 parent :3 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
  Sent 51912 bytes 314 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan1 parent :4 limit 1024p flows 1024 quantum 300 
target 5.0ms interval 100.0ms ecn
  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
   new_flows_len 0 old_flows_len 0
qdisc cake 8009: dev pppoe-wan root refcnt 2 bandwidth 850Kbit diffserv4 
flows atm overhead 40
  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
            Class 0     Class 1     Class 2     Class 3
   rate       850Kbit   796880bit   637504bit   212496bit
   target      21.3ms      22.7ms      28.3ms      85.0ms
interval     170.1ms     181.4ms     226.8ms     680.4ms
Pk delay         0us         0us         0us         0us
Av delay         0us         0us         0us         0us
Sp delay         0us         0us         0us         0us
   pkts             0           0           0           0
way inds           0           0           0           0
way miss           0           0           0           0
way cols           0           0           0           0
   bytes            0           0           0           0
   drops            0           0           0           0
   marks            0           0           0           0
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
  Sent 68 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
qdisc cake 800a: dev ifb4pppoe-wan root refcnt 2 bandwidth 8500Kbit 
besteffort flows atm overhead 40
  Sent 90 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
  backlog 0b 0p requeues 0
            Class 0
   rate      8500Kbit
   target       5.0ms
interval     100.0ms
Pk delay         0us
Av delay         0us
Sp delay         0us
   pkts             1
way inds           0
way miss           1
way cols           0
   bytes           90
   drops            0
   marks            0



On 10/07/15 20:50, Sebastian Moeller wrote:
> Hi Fred,
>
> On Jul 10, 2015, at 21:45 , Fred Stratton <fredstratton@imap.cc> wrote:
>
>> These are the latest scripts,  AFAIK
> 	Let me repeat my question: are these the scripts I attached in one of the last mails, or the most recent scripts from ceropackages-3.10? The version in the openwrt repository is NOT recent, yet. Pushing the latest verso into openwrt is on my todo list but that will need a bit more changes, so please try the files I attached which should work (unless I screwed up and attached the wrong version). Also I have no working cake on my router and hence require help for testing and that means there might be some undiscovered bugs in there.
>
>> no overhead allowance. I note.
> 	Well, that should work with the most recent version
>
> Best Regards
> 	Sebastian
>
>>
>>
>> On 10/07/15 20:40, Sebastian Moeller wrote:
>>> Hi Fred,
>>>
>>>
>>> On Jul 10, 2015, at 21:34 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>
>>>> bridge sync is circa 10 000 kbit/s
>>>>
>>>> with the cake option in sqm enabled
>>>>
>>>> config queue 'eth1'
>>>>     option qdisc_advanced '0'
>>>>     option enabled '1'
>>>>     option interface 'pppoe-wan'
>>>>     option upload '850'
>>>>     option qdisc 'cake'
>>>>     option script 'simple_pppoe.qos'
>>>>     option linklayer 'atm'
>>>>     option overhead '40'
>>>>     option download ‘8500'
>>> 	So this looks reasonable. Then again, if the DSLAM is under provisioned/oversubscribed (= congested) shaping uypur DSL link might not fix all buffer bloat..
>>>
>>>> tc -s qdisc show dev pppoe-wan
>>>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 3
>>>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>>>> backlog 0b 0p requeues 0
>>>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>>>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>>>> backlog 0b 0p requeues 0
>>>>            Class 0     Class 1     Class 2     Class 3
>>>>   rate          0bit        0bit        0bit        0bit
>>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>> Pk delay         0us         0us         7us         2us
>>>> Av delay         0us         0us         0us         0us
>>>> Sp delay         0us         0us         0us         0us
>>>>   pkts             0           0          22           3
>>>> way inds           0           0           0           0
>>>> way miss           0           0          22           2
>>>> way cols           0           0           0           0
>>>>   bytes            0           0        3392        1007
>>>>   drops            0           0           0           0
>>>>   marks            0           0           0           0
>>>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>>>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>>>> backlog 0b 0p requeues 0
>>>>            Class 0     Class 1     Class 2     Class 3
>>>>   rate          0bit        0bit        0bit        0bit
>>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>> Pk delay         0us      28.0ms         0us         0us
>>>> Av delay         0us       1.2ms         0us         0us
>>>> Sp delay         0us         4us         0us         0us
>>>>   pkts             0         417           0           0
>>>> way inds           0           0           0           0
>>>> way miss           0          23           0           0
>>>> way cols           0           0           0           0
>>>>   bytes            0       98951           0           0
>>>>   drops            0           2           0           0
>>>>   marks            0           0           0           0
>>>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>> backlog 0b 0p requeues 0
>>>>            Class 0     Class 1     Class 2     Class 3
>>>>   rate          0bit        0bit        0bit        0bit
>>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>> Pk delay         0us         0us         0us         0us
>>>> Av delay         0us         0us         0us         0us
>>>> Sp delay         0us         0us         0us         0us
>>>>   pkts             0           0           0           0
>>>> way inds           0           0           0           0
>>>> way miss           0           0           0           0
>>>> way cols           0           0           0           0
>>>>   bytes            0           0           0           0
>>>>   drops            0           0           0           0
>>>>   marks            0           0           0           0
>>>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>> backlog 0b 0p requeues 0
>>>>            Class 0     Class 1     Class 2     Class 3
>>>>   rate          0bit        0bit        0bit        0bit
>>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>> Pk delay         0us         0us         0us         0us
>>>> Av delay         0us         0us         0us         0us
>>>> Sp delay         0us         0us         0us         0us
>>>>   pkts             0           0           0           0
>>>> way inds           0           0           0           0
>>>> way miss           0           0           0           0
>>>> way cols           0           0           0           0
>>>>   bytes            0           0           0           0
>>>>   drops            0           0           0           0
>>>>   marks            0           0           0           0
>>>> qdisc ingress ffff: parent ffff:fff1 ----------------
>>>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>>>> backlog 0b 0p requeues 0
>>> 	But this is the hallmark of out of date sqm-scripts, this just uses cake as leaf qdisc and keeps HTB as the main shaper; a configuration that is useful for testing. I assume this is the old set of sqm-scripts not the update I just sent as attachment? If so could you retry with the newer scripts, please?
>>>
>>> Best Regards
>>> 	Sebastian
>>>
>>>>
>>>>
>>>> On 10/07/15 19:46, Sebastian Moeller wrote:
>>>>> Hi Fred,
>>>>>
>>>>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>>>>> 1) cat /etc/config/sqm
>>>>> 2) tc -d qdisc
>>>>> 3) tc -d class show dev pppoe-wan
>>>>> 4) tc -d class show dev ifb4pppoe-wqn
>>>>> 5) /etc/init.d/sqm stop
>>>>> 6) /etc/init.d/sqm start
>>>>>
>>>>> hopefully these give some insight what might have happened.
>>>>>
>>>>> And finally I would love to learn the output of:
>>>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>>>>
>>>>>
>>>>> Many Thanks & Best Regards
>>>>> 	Sebastian
>>>>>
>>>>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>>>
>>>>>> By your command
>>>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>>>>>
>>>>>> script configuring qdiscs and overhead 40 on
>>>>>>
>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>>>>> Download:  6.73 Mbps
>>>>>>    Upload:  0.58 Mbps
>>>>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>       Min: 24.094
>>>>>>     10pct: 172.654
>>>>>>    Median: 260.563
>>>>>>       Avg: 253.580
>>>>>>     90pct: 330.003
>>>>>>       Max: 411.145
>>>>>>
>>>>>> script configuring qdiscs on flows raw
>>>>>>
>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>> 78.145.32.1
>>>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>>>>> Download:  6.75 Mbps
>>>>>>    Upload:  0.59 Mbps
>>>>>>   Latency: (in msec, 59 pings, 0.00% packet loss)
>>>>>>       Min: 23.605
>>>>>>     10pct: 169.789
>>>>>>    Median: 282.155
>>>>>>       Avg: 267.099
>>>>>>     90pct: 333.283
>>>>>>       Max: 376.509
>>>>>>
>>>>>> script configuring qdiscs and overhead 36 on
>>>>>>
>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>> 80.44.96.1
>>>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>>>>> Download:  6.56 Mbps
>>>>>>    Upload:  0.59 Mbps
>>>>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>       Min: 22.975
>>>>>>     10pct: 195.473
>>>>>>    Median: 281.756
>>>>>>       Avg: 271.609
>>>>>>     90pct: 342.130
>>>>>>       Max: 398.573
>>>>>>
>>>>>>
>>>>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>>>>>
>>>>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>>>>>
>>>>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>>>>
>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>>>>
>>>>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>>>>>
>>>>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>>>>>
>>>>>>> Alan
>>>>>>>
>>>>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>>>> You are absolutely correct.
>>>>>>>>
>>>>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>>>>> undeclared version 2. Everything works as stated.
>>>>>>>>
>>>>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>>>>> values were not recognised.
>>>>>>>>
>>>>>>>> Thank you.
>>>>>>>>
>>>>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>>>>> "overhead" or "atm".
>>>>>>>>>
>>>>>>>>> What's probably happening is that you're using a slightly old version
>>>>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>>>>> combination crop up ourselves recently.
>>>>>>>>>
>>>>>>>>> - Jonathan Morton
>>>>>>>>>
>>>>>> _______________________________________________
>>>>>> Cerowrt-devel mailing list
>>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Sebastian Moeller]

Hi Fred,

On Jul 10, 2015, at 22:07 , Fred Stratton <fredstratton@imap.cc> wrote:

> replaced /usr/lib/sqm as ordered

	Thanks.

> 
> cat /etc/config/sqm
> 
> config queue 'eth1'
>    option qdisc_advanced '0'
>    option enabled '1'
>    option interface 'pppoe-wan'
>    option upload '850'
>    option qdisc 'cake'
>    option linklayer 'atm'
>    option overhead '40'
>    option download '8500'
>    option script 'simple.qos'
>    option linklayer_advanced '1'
>    option tcMTU '2047'
>    option tcTSIZE '128'
>    option tcMPU '0'
>    option linklayer_adaptation_mechanism ‘cake'

	Looks reasonable.

> 
> root@OpenWrt:~# tc -s qdisc show
> qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 5904206 bytes 10771 pkt (dropped 0, overlimits 0 requeues 1)
> backlog 0b 0p requeues 1
>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc fq_codel 0: dev eth1 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 903154 bytes 7941 pkt (dropped 0, overlimits 0 requeues 2)
> backlog 0b 0p requeues 2
>  maxpacket 1322 drop_overlimit 0 new_flow_count 1 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc mq 0: dev wlan0 root
> Sent 198495 bytes 816 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
> qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 3709 bytes 19 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 112 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 194674 bytes 796 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc mq 0: dev wlan1 root
> Sent 53249 bytes 323 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
> qdisc fq_codel 0: dev wlan1 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 1337 bytes 9 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc fq_codel 0: dev wlan1 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc fq_codel 0: dev wlan1 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 51912 bytes 314 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc fq_codel 0: dev wlan1 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>  new_flows_len 0 old_flows_len 0
> qdisc cake 8009: dev pppoe-wan root refcnt 2 bandwidth 850Kbit diffserv4 flows atm overhead 40

	Good, egress accounts for the link layer adaptation.

> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>           Class 0     Class 1     Class 2     Class 3
>  rate       850Kbit   796880bit   637504bit   212496bit
>  target      21.3ms      22.7ms      28.3ms      85.0ms
> interval     170.1ms     181.4ms     226.8ms     680.4ms
> Pk delay         0us         0us         0us         0us
> Av delay         0us         0us         0us         0us
> Sp delay         0us         0us         0us         0us
>  pkts             0           0           0           0
> way inds           0           0           0           0
> way miss           0           0           0           0
> way cols           0           0           0           0
>  bytes            0           0           0           0
>  drops            0           0           0           0
>  marks            0           0           0           0
> qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
> Sent 68 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
> qdisc cake 800a: dev ifb4pppoe-wan root refcnt 2 bandwidth 8500Kbit besteffort flows atm overhead 40

	So, I would be interested to learn how this now performs with netperfrunner and/or betterspeedtest.sh


Best Regards
	Sebastian


> Sent 90 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>           Class 0
>  rate      8500Kbit
>  target       5.0ms
> interval     100.0ms
> Pk delay         0us
> Av delay         0us
> Sp delay         0us
>  pkts             1
> way inds           0
> way miss           1
> way cols           0
>  bytes           90
>  drops            0
>  marks            0
> 
> 
> 
> On 10/07/15 20:50, Sebastian Moeller wrote:
>> Hi Fred,
>> 
>> On Jul 10, 2015, at 21:45 , Fred Stratton <fredstratton@imap.cc> wrote:
>> 
>>> These are the latest scripts,  AFAIK
>> 	Let me repeat my question: are these the scripts I attached in one of the last mails, or the most recent scripts from ceropackages-3.10? The version in the openwrt repository is NOT recent, yet. Pushing the latest verso into openwrt is on my todo list but that will need a bit more changes, so please try the files I attached which should work (unless I screwed up and attached the wrong version). Also I have no working cake on my router and hence require help for testing and that means there might be some undiscovered bugs in there.
>> 
>>> no overhead allowance. I note.
>> 	Well, that should work with the most recent version
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> 
>>> 
>>> On 10/07/15 20:40, Sebastian Moeller wrote:
>>>> Hi Fred,
>>>> 
>>>> 
>>>> On Jul 10, 2015, at 21:34 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>> 
>>>>> bridge sync is circa 10 000 kbit/s
>>>>> 
>>>>> with the cake option in sqm enabled
>>>>> 
>>>>> config queue 'eth1'
>>>>>    option qdisc_advanced '0'
>>>>>    option enabled '1'
>>>>>    option interface 'pppoe-wan'
>>>>>    option upload '850'
>>>>>    option qdisc 'cake'
>>>>>    option script 'simple_pppoe.qos'
>>>>>    option linklayer 'atm'
>>>>>    option overhead '40'
>>>>>    option download ‘8500'
>>>> 	So this looks reasonable. Then again, if the DSLAM is under provisioned/oversubscribed (= congested) shaping uypur DSL link might not fix all buffer bloat..
>>>> 
>>>>> tc -s qdisc show dev pppoe-wan
>>>>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 3
>>>>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>>>>> backlog 0b 0p requeues 0
>>>>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>>>>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>>>>> backlog 0b 0p requeues 0
>>>>>           Class 0     Class 1     Class 2     Class 3
>>>>>  rate          0bit        0bit        0bit        0bit
>>>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>> Pk delay         0us         0us         7us         2us
>>>>> Av delay         0us         0us         0us         0us
>>>>> Sp delay         0us         0us         0us         0us
>>>>>  pkts             0           0          22           3
>>>>> way inds           0           0           0           0
>>>>> way miss           0           0          22           2
>>>>> way cols           0           0           0           0
>>>>>  bytes            0           0        3392        1007
>>>>>  drops            0           0           0           0
>>>>>  marks            0           0           0           0
>>>>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>>>>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>>>>> backlog 0b 0p requeues 0
>>>>>           Class 0     Class 1     Class 2     Class 3
>>>>>  rate          0bit        0bit        0bit        0bit
>>>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>> Pk delay         0us      28.0ms         0us         0us
>>>>> Av delay         0us       1.2ms         0us         0us
>>>>> Sp delay         0us         4us         0us         0us
>>>>>  pkts             0         417           0           0
>>>>> way inds           0           0           0           0
>>>>> way miss           0          23           0           0
>>>>> way cols           0           0           0           0
>>>>>  bytes            0       98951           0           0
>>>>>  drops            0           2           0           0
>>>>>  marks            0           0           0           0
>>>>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>>> backlog 0b 0p requeues 0
>>>>>           Class 0     Class 1     Class 2     Class 3
>>>>>  rate          0bit        0bit        0bit        0bit
>>>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>> Pk delay         0us         0us         0us         0us
>>>>> Av delay         0us         0us         0us         0us
>>>>> Sp delay         0us         0us         0us         0us
>>>>>  pkts             0           0           0           0
>>>>> way inds           0           0           0           0
>>>>> way miss           0           0           0           0
>>>>> way cols           0           0           0           0
>>>>>  bytes            0           0           0           0
>>>>>  drops            0           0           0           0
>>>>>  marks            0           0           0           0
>>>>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>>> backlog 0b 0p requeues 0
>>>>>           Class 0     Class 1     Class 2     Class 3
>>>>>  rate          0bit        0bit        0bit        0bit
>>>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>> Pk delay         0us         0us         0us         0us
>>>>> Av delay         0us         0us         0us         0us
>>>>> Sp delay         0us         0us         0us         0us
>>>>>  pkts             0           0           0           0
>>>>> way inds           0           0           0           0
>>>>> way miss           0           0           0           0
>>>>> way cols           0           0           0           0
>>>>>  bytes            0           0           0           0
>>>>>  drops            0           0           0           0
>>>>>  marks            0           0           0           0
>>>>> qdisc ingress ffff: parent ffff:fff1 ----------------
>>>>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>>>>> backlog 0b 0p requeues 0
>>>> 	But this is the hallmark of out of date sqm-scripts, this just uses cake as leaf qdisc and keeps HTB as the main shaper; a configuration that is useful for testing. I assume this is the old set of sqm-scripts not the update I just sent as attachment? If so could you retry with the newer scripts, please?
>>>> 
>>>> Best Regards
>>>> 	Sebastian
>>>> 
>>>>> 
>>>>> 
>>>>> On 10/07/15 19:46, Sebastian Moeller wrote:
>>>>>> Hi Fred,
>>>>>> 
>>>>>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>>>>>> 1) cat /etc/config/sqm
>>>>>> 2) tc -d qdisc
>>>>>> 3) tc -d class show dev pppoe-wan
>>>>>> 4) tc -d class show dev ifb4pppoe-wqn
>>>>>> 5) /etc/init.d/sqm stop
>>>>>> 6) /etc/init.d/sqm start
>>>>>> 
>>>>>> hopefully these give some insight what might have happened.
>>>>>> 
>>>>>> And finally I would love to learn the output of:
>>>>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>>>>> 
>>>>>> 
>>>>>> Many Thanks & Best Regards
>>>>>> 	Sebastian
>>>>>> 
>>>>>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>>>> 
>>>>>>> By your command
>>>>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>>>>>> 
>>>>>>> script configuring qdiscs and overhead 40 on
>>>>>>> 
>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>>>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>>>>>> Download:  6.73 Mbps
>>>>>>>   Upload:  0.58 Mbps
>>>>>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>>      Min: 24.094
>>>>>>>    10pct: 172.654
>>>>>>>   Median: 260.563
>>>>>>>      Avg: 253.580
>>>>>>>    90pct: 330.003
>>>>>>>      Max: 411.145
>>>>>>> 
>>>>>>> script configuring qdiscs on flows raw
>>>>>>> 
>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>>> 78.145.32.1
>>>>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>>>>>> Download:  6.75 Mbps
>>>>>>>   Upload:  0.59 Mbps
>>>>>>>  Latency: (in msec, 59 pings, 0.00% packet loss)
>>>>>>>      Min: 23.605
>>>>>>>    10pct: 169.789
>>>>>>>   Median: 282.155
>>>>>>>      Avg: 267.099
>>>>>>>    90pct: 333.283
>>>>>>>      Max: 376.509
>>>>>>> 
>>>>>>> script configuring qdiscs and overhead 36 on
>>>>>>> 
>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>>> 80.44.96.1
>>>>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>>>>>> Download:  6.56 Mbps
>>>>>>>   Upload:  0.59 Mbps
>>>>>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>>      Min: 22.975
>>>>>>>    10pct: 195.473
>>>>>>>   Median: 281.756
>>>>>>>      Avg: 271.609
>>>>>>>    90pct: 342.130
>>>>>>>      Max: 398.573
>>>>>>> 
>>>>>>> 
>>>>>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>>>>>> 
>>>>>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>>>>>> 
>>>>>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>>>>> 
>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>>>>> 
>>>>>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>>>>>> 
>>>>>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>>>>>> 
>>>>>>>> Alan
>>>>>>>> 
>>>>>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>>>>> You are absolutely correct.
>>>>>>>>> 
>>>>>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>>>>>> undeclared version 2. Everything works as stated.
>>>>>>>>> 
>>>>>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>>>>>> values were not recognised.
>>>>>>>>> 
>>>>>>>>> Thank you.
>>>>>>>>> 
>>>>>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>>>>>> "overhead" or "atm".
>>>>>>>>>> 
>>>>>>>>>> What's probably happening is that you're using a slightly old version
>>>>>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>>>>>> combination crop up ourselves recently.
>>>>>>>>>> 
>>>>>>>>>> - Jonathan Morton
>>>>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> Cerowrt-devel mailing list
>>>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Fred Stratton]

sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.ne
t -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H 
netperf-
eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
2015-07-10 21:15:19 Testing against netperf-eu.bufferbloat.net (ipv4) 
with 4 simultaneous sessions while pinging netperf-eu.bufferbloat.net 
(150 seconds in each direction)
......................................................................................................................................................
  Download:  6.8 Mbps
   Latency: (in msec, 151 pings, 0.00% packet loss)
       Min: 69.125
     10pct: 72.299
    Median: 75.077
       Avg: 75.613
     90pct: 79.014
       Max: 89.825
.......................................................................................................................................................
    Upload:  0.72 Mbps
   Latency: (in msec, 152 pings, 0.00% packet loss)
       Min: 68.691
     10pct: 72.506
    Median: 79.447
       Avg: 79.654
     90pct: 86.369
       Max: 95.037
2015-07-10 21:20:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 
streams down and up while pinging netperf-eu.bufferbloat.net. Takes 
about 150 seconds.
  Download:  5.92 Mbps
    Upload:  0.46 Mbps
   Latency: (in msec, 152 pings, 0.00% packet loss)
       Min: 71.877
     10pct: 76.197
    Median: 85.051
       Avg: 84.838
     90pct: 92.105
       Max: 109.600


On 10/07/15 21:12, Sebastian Moeller wrote:
> Hi Fred,
>
> On Jul 10, 2015, at 22:07 , Fred Stratton <fredstratton@imap.cc> wrote:
>
>> replaced /usr/lib/sqm as ordered
> 	Thanks.
>
>> cat /etc/config/sqm
>>
>> config queue 'eth1'
>>     option qdisc_advanced '0'
>>     option enabled '1'
>>     option interface 'pppoe-wan'
>>     option upload '850'
>>     option qdisc 'cake'
>>     option linklayer 'atm'
>>     option overhead '40'
>>     option download '8500'
>>     option script 'simple.qos'
>>     option linklayer_advanced '1'
>>     option tcMTU '2047'
>>     option tcTSIZE '128'
>>     option tcMPU '0'
>>     option linklayer_adaptation_mechanism ‘cake'
> 	Looks reasonable.
>
>> root@OpenWrt:~# tc -s qdisc show
>> qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 5904206 bytes 10771 pkt (dropped 0, overlimits 0 requeues 1)
>> backlog 0b 0p requeues 1
>>   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev eth1 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 903154 bytes 7941 pkt (dropped 0, overlimits 0 requeues 2)
>> backlog 0b 0p requeues 2
>>   maxpacket 1322 drop_overlimit 0 new_flow_count 1 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc mq 0: dev wlan0 root
>> Sent 198495 bytes 816 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 3709 bytes 19 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 112 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 194674 bytes 796 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc mq 0: dev wlan1 root
>> Sent 53249 bytes 323 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> qdisc fq_codel 0: dev wlan1 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 1337 bytes 9 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan1 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan1 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 51912 bytes 314 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan1 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>   maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>   new_flows_len 0 old_flows_len 0
>> qdisc cake 8009: dev pppoe-wan root refcnt 2 bandwidth 850Kbit diffserv4 flows atm overhead 40
> 	Good, egress accounts for the link layer adaptation.
>
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>            Class 0     Class 1     Class 2     Class 3
>>   rate       850Kbit   796880bit   637504bit   212496bit
>>   target      21.3ms      22.7ms      28.3ms      85.0ms
>> interval     170.1ms     181.4ms     226.8ms     680.4ms
>> Pk delay         0us         0us         0us         0us
>> Av delay         0us         0us         0us         0us
>> Sp delay         0us         0us         0us         0us
>>   pkts             0           0           0           0
>> way inds           0           0           0           0
>> way miss           0           0           0           0
>> way cols           0           0           0           0
>>   bytes            0           0           0           0
>>   drops            0           0           0           0
>>   marks            0           0           0           0
>> qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
>> Sent 68 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> qdisc cake 800a: dev ifb4pppoe-wan root refcnt 2 bandwidth 8500Kbit besteffort flows atm overhead 40
> 	So, I would be interested to learn how this now performs with netperfrunner and/or betterspeedtest.sh
>
>
> Best Regards
> 	Sebastian
>
>
>> Sent 90 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>            Class 0
>>   rate      8500Kbit
>>   target       5.0ms
>> interval     100.0ms
>> Pk delay         0us
>> Av delay         0us
>> Sp delay         0us
>>   pkts             1
>> way inds           0
>> way miss           1
>> way cols           0
>>   bytes           90
>>   drops            0
>>   marks            0
>>
>>
>>
>> On 10/07/15 20:50, Sebastian Moeller wrote:
>>> Hi Fred,
>>>
>>> On Jul 10, 2015, at 21:45 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>
>>>> These are the latest scripts,  AFAIK
>>> 	Let me repeat my question: are these the scripts I attached in one of the last mails, or the most recent scripts from ceropackages-3.10? The version in the openwrt repository is NOT recent, yet. Pushing the latest verso into openwrt is on my todo list but that will need a bit more changes, so please try the files I attached which should work (unless I screwed up and attached the wrong version). Also I have no working cake on my router and hence require help for testing and that means there might be some undiscovered bugs in there.
>>>
>>>> no overhead allowance. I note.
>>> 	Well, that should work with the most recent version
>>>
>>> Best Regards
>>> 	Sebastian
>>>
>>>>
>>>> On 10/07/15 20:40, Sebastian Moeller wrote:
>>>>> Hi Fred,
>>>>>
>>>>>
>>>>> On Jul 10, 2015, at 21:34 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>>>
>>>>>> bridge sync is circa 10 000 kbit/s
>>>>>>
>>>>>> with the cake option in sqm enabled
>>>>>>
>>>>>> config queue 'eth1'
>>>>>>     option qdisc_advanced '0'
>>>>>>     option enabled '1'
>>>>>>     option interface 'pppoe-wan'
>>>>>>     option upload '850'
>>>>>>     option qdisc 'cake'
>>>>>>     option script 'simple_pppoe.qos'
>>>>>>     option linklayer 'atm'
>>>>>>     option overhead '40'
>>>>>>     option download ‘8500'
>>>>> 	So this looks reasonable. Then again, if the DSLAM is under provisioned/oversubscribed (= congested) shaping uypur DSL link might not fix all buffer bloat..
>>>>>
>>>>>> tc -s qdisc show dev pppoe-wan
>>>>>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 3
>>>>>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>>>>>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>>            Class 0     Class 1     Class 2     Class 3
>>>>>>   rate          0bit        0bit        0bit        0bit
>>>>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>>> Pk delay         0us         0us         7us         2us
>>>>>> Av delay         0us         0us         0us         0us
>>>>>> Sp delay         0us         0us         0us         0us
>>>>>>   pkts             0           0          22           3
>>>>>> way inds           0           0           0           0
>>>>>> way miss           0           0          22           2
>>>>>> way cols           0           0           0           0
>>>>>>   bytes            0           0        3392        1007
>>>>>>   drops            0           0           0           0
>>>>>>   marks            0           0           0           0
>>>>>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>>>>>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>>            Class 0     Class 1     Class 2     Class 3
>>>>>>   rate          0bit        0bit        0bit        0bit
>>>>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>>> Pk delay         0us      28.0ms         0us         0us
>>>>>> Av delay         0us       1.2ms         0us         0us
>>>>>> Sp delay         0us         4us         0us         0us
>>>>>>   pkts             0         417           0           0
>>>>>> way inds           0           0           0           0
>>>>>> way miss           0          23           0           0
>>>>>> way cols           0           0           0           0
>>>>>>   bytes            0       98951           0           0
>>>>>>   drops            0           2           0           0
>>>>>>   marks            0           0           0           0
>>>>>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>>>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>>            Class 0     Class 1     Class 2     Class 3
>>>>>>   rate          0bit        0bit        0bit        0bit
>>>>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>>> Pk delay         0us         0us         0us         0us
>>>>>> Av delay         0us         0us         0us         0us
>>>>>> Sp delay         0us         0us         0us         0us
>>>>>>   pkts             0           0           0           0
>>>>>> way inds           0           0           0           0
>>>>>> way miss           0           0           0           0
>>>>>> way cols           0           0           0           0
>>>>>>   bytes            0           0           0           0
>>>>>>   drops            0           0           0           0
>>>>>>   marks            0           0           0           0
>>>>>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>>>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>>            Class 0     Class 1     Class 2     Class 3
>>>>>>   rate          0bit        0bit        0bit        0bit
>>>>>>   target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>>> Pk delay         0us         0us         0us         0us
>>>>>> Av delay         0us         0us         0us         0us
>>>>>> Sp delay         0us         0us         0us         0us
>>>>>>   pkts             0           0           0           0
>>>>>> way inds           0           0           0           0
>>>>>> way miss           0           0           0           0
>>>>>> way cols           0           0           0           0
>>>>>>   bytes            0           0           0           0
>>>>>>   drops            0           0           0           0
>>>>>>   marks            0           0           0           0
>>>>>> qdisc ingress ffff: parent ffff:fff1 ----------------
>>>>>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>> 	But this is the hallmark of out of date sqm-scripts, this just uses cake as leaf qdisc and keeps HTB as the main shaper; a configuration that is useful for testing. I assume this is the old set of sqm-scripts not the update I just sent as attachment? If so could you retry with the newer scripts, please?
>>>>>
>>>>> Best Regards
>>>>> 	Sebastian
>>>>>
>>>>>>
>>>>>> On 10/07/15 19:46, Sebastian Moeller wrote:
>>>>>>> Hi Fred,
>>>>>>>
>>>>>>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>>>>>>> 1) cat /etc/config/sqm
>>>>>>> 2) tc -d qdisc
>>>>>>> 3) tc -d class show dev pppoe-wan
>>>>>>> 4) tc -d class show dev ifb4pppoe-wqn
>>>>>>> 5) /etc/init.d/sqm stop
>>>>>>> 6) /etc/init.d/sqm start
>>>>>>>
>>>>>>> hopefully these give some insight what might have happened.
>>>>>>>
>>>>>>> And finally I would love to learn the output of:
>>>>>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>>>>>>
>>>>>>>
>>>>>>> Many Thanks & Best Regards
>>>>>>> 	Sebastian
>>>>>>>
>>>>>>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>>>>>
>>>>>>>> By your command
>>>>>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>>>>>>>
>>>>>>>> script configuring qdiscs and overhead 40 on
>>>>>>>>
>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>>>>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>>>>>>> Download:  6.73 Mbps
>>>>>>>>    Upload:  0.58 Mbps
>>>>>>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>>>       Min: 24.094
>>>>>>>>     10pct: 172.654
>>>>>>>>    Median: 260.563
>>>>>>>>       Avg: 253.580
>>>>>>>>     90pct: 330.003
>>>>>>>>       Max: 411.145
>>>>>>>>
>>>>>>>> script configuring qdiscs on flows raw
>>>>>>>>
>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>>>> 78.145.32.1
>>>>>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>>>>>>> Download:  6.75 Mbps
>>>>>>>>    Upload:  0.59 Mbps
>>>>>>>>   Latency: (in msec, 59 pings, 0.00% packet loss)
>>>>>>>>       Min: 23.605
>>>>>>>>     10pct: 169.789
>>>>>>>>    Median: 282.155
>>>>>>>>       Avg: 267.099
>>>>>>>>     90pct: 333.283
>>>>>>>>       Max: 376.509
>>>>>>>>
>>>>>>>> script configuring qdiscs and overhead 36 on
>>>>>>>>
>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>>>> 80.44.96.1
>>>>>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>>>>>>> Download:  6.56 Mbps
>>>>>>>>    Upload:  0.59 Mbps
>>>>>>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>>>       Min: 22.975
>>>>>>>>     10pct: 195.473
>>>>>>>>    Median: 281.756
>>>>>>>>       Avg: 271.609
>>>>>>>>     90pct: 342.130
>>>>>>>>       Max: 398.573
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>>>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>>>>>>>
>>>>>>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>>>>>>>
>>>>>>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>>>>>>
>>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>>>>>>
>>>>>>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>>>>>>>
>>>>>>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>>>>>>>
>>>>>>>>> Alan
>>>>>>>>>
>>>>>>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>>>>>> You are absolutely correct.
>>>>>>>>>>
>>>>>>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>>>>>>> undeclared version 2. Everything works as stated.
>>>>>>>>>>
>>>>>>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>>>>>>> values were not recognised.
>>>>>>>>>>
>>>>>>>>>> Thank you.
>>>>>>>>>>
>>>>>>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>>>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>>>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>>>>>>> "overhead" or "atm".
>>>>>>>>>>>
>>>>>>>>>>> What's probably happening is that you're using a slightly old version
>>>>>>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>>>>>>> combination crop up ourselves recently.
>>>>>>>>>>>
>>>>>>>>>>> - Jonathan Morton
>>>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Cerowrt-devel mailing list
>>>>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Sebastian Moeller]

Hi Fred,

and now the values look decent, the latency under load increase is bounded to 38ms worst case, not bad at all for am ADSL line.

Best Regards
	Sebastian


On Jul 10, 2015, at 22:24 , Fred Stratton <fredstratton@imap.cc> wrote:

> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.ne
> t -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-
> eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
> 2015-07-10 21:15:19 Testing against netperf-eu.bufferbloat.net (ipv4) with 4 simultaneous sessions while pinging netperf-eu.bufferbloat.net (150 seconds in each direction)
> ......................................................................................................................................................
> Download:  6.8 Mbps
>  Latency: (in msec, 151 pings, 0.00% packet loss)
>      Min: 69.125
>    10pct: 72.299
>   Median: 75.077
>      Avg: 75.613
>    90pct: 79.014
>      Max: 89.825
> .......................................................................................................................................................
>   Upload:  0.72 Mbps
>  Latency: (in msec, 152 pings, 0.00% packet loss)
>      Min: 68.691
>    10pct: 72.506
>   Median: 79.447
>      Avg: 79.654
>    90pct: 86.369
>      Max: 95.037
> 2015-07-10 21:20:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging netperf-eu.bufferbloat.net. Takes about 150 seconds.
> Download:  5.92 Mbps
>   Upload:  0.46 Mbps
>  Latency: (in msec, 152 pings, 0.00% packet loss)
>      Min: 71.877
>    10pct: 76.197
>   Median: 85.051
>      Avg: 84.838
>    90pct: 92.105
>      Max: 109.600
> 
> 
> On 10/07/15 21:12, Sebastian Moeller wrote:
>> Hi Fred,
>> 
>> On Jul 10, 2015, at 22:07 , Fred Stratton <fredstratton@imap.cc> wrote:
>> 
>>> replaced /usr/lib/sqm as ordered
>> 	Thanks.
>> 
>>> cat /etc/config/sqm
>>> 
>>> config queue 'eth1'
>>>    option qdisc_advanced '0'
>>>    option enabled '1'
>>>    option interface 'pppoe-wan'
>>>    option upload '850'
>>>    option qdisc 'cake'
>>>    option linklayer 'atm'
>>>    option overhead '40'
>>>    option download '8500'
>>>    option script 'simple.qos'
>>>    option linklayer_advanced '1'
>>>    option tcMTU '2047'
>>>    option tcTSIZE '128'
>>>    option tcMPU '0'
>>>    option linklayer_adaptation_mechanism ‘cake'
>> 	Looks reasonable.
>> 
>>> root@OpenWrt:~# tc -s qdisc show
>>> qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 5904206 bytes 10771 pkt (dropped 0, overlimits 0 requeues 1)
>>> backlog 0b 0p requeues 1
>>>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 0: dev eth1 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 903154 bytes 7941 pkt (dropped 0, overlimits 0 requeues 2)
>>> backlog 0b 0p requeues 2
>>>  maxpacket 1322 drop_overlimit 0 new_flow_count 1 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc mq 0: dev wlan0 root
>>> Sent 198495 bytes 816 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>> qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 3709 bytes 19 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 112 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 194674 bytes 796 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc mq 0: dev wlan1 root
>>> Sent 53249 bytes 323 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>> qdisc fq_codel 0: dev wlan1 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 1337 bytes 9 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 0: dev wlan1 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 0: dev wlan1 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 51912 bytes 314 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc fq_codel 0: dev wlan1 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>>>  new_flows_len 0 old_flows_len 0
>>> qdisc cake 8009: dev pppoe-wan root refcnt 2 bandwidth 850Kbit diffserv4 flows atm overhead 40
>> 	Good, egress accounts for the link layer adaptation.
>> 
>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>           Class 0     Class 1     Class 2     Class 3
>>>  rate       850Kbit   796880bit   637504bit   212496bit
>>>  target      21.3ms      22.7ms      28.3ms      85.0ms
>>> interval     170.1ms     181.4ms     226.8ms     680.4ms
>>> Pk delay         0us         0us         0us         0us
>>> Av delay         0us         0us         0us         0us
>>> Sp delay         0us         0us         0us         0us
>>>  pkts             0           0           0           0
>>> way inds           0           0           0           0
>>> way miss           0           0           0           0
>>> way cols           0           0           0           0
>>>  bytes            0           0           0           0
>>>  drops            0           0           0           0
>>>  marks            0           0           0           0
>>> qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
>>> Sent 68 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>> qdisc cake 800a: dev ifb4pppoe-wan root refcnt 2 bandwidth 8500Kbit besteffort flows atm overhead 40
>> 	So, I would be interested to learn how this now performs with netperfrunner and/or betterspeedtest.sh
>> 
>> 
>> Best Regards
>> 	Sebastian
>> 
>> 
>>> Sent 90 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
>>> backlog 0b 0p requeues 0
>>>           Class 0
>>>  rate      8500Kbit
>>>  target       5.0ms
>>> interval     100.0ms
>>> Pk delay         0us
>>> Av delay         0us
>>> Sp delay         0us
>>>  pkts             1
>>> way inds           0
>>> way miss           1
>>> way cols           0
>>>  bytes           90
>>>  drops            0
>>>  marks            0
>>> 
>>> 
>>> 
>>> On 10/07/15 20:50, Sebastian Moeller wrote:
>>>> Hi Fred,
>>>> 
>>>> On Jul 10, 2015, at 21:45 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>> 
>>>>> These are the latest scripts,  AFAIK
>>>> 	Let me repeat my question: are these the scripts I attached in one of the last mails, or the most recent scripts from ceropackages-3.10? The version in the openwrt repository is NOT recent, yet. Pushing the latest verso into openwrt is on my todo list but that will need a bit more changes, so please try the files I attached which should work (unless I screwed up and attached the wrong version). Also I have no working cake on my router and hence require help for testing and that means there might be some undiscovered bugs in there.
>>>> 
>>>>> no overhead allowance. I note.
>>>> 	Well, that should work with the most recent version
>>>> 
>>>> Best Regards
>>>> 	Sebastian
>>>> 
>>>>> 
>>>>> On 10/07/15 20:40, Sebastian Moeller wrote:
>>>>>> Hi Fred,
>>>>>> 
>>>>>> 
>>>>>> On Jul 10, 2015, at 21:34 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>>>> 
>>>>>>> bridge sync is circa 10 000 kbit/s
>>>>>>> 
>>>>>>> with the cake option in sqm enabled
>>>>>>> 
>>>>>>> config queue 'eth1'
>>>>>>>    option qdisc_advanced '0'
>>>>>>>    option enabled '1'
>>>>>>>    option interface 'pppoe-wan'
>>>>>>>    option upload '850'
>>>>>>>    option qdisc 'cake'
>>>>>>>    option script 'simple_pppoe.qos'
>>>>>>>    option linklayer 'atm'
>>>>>>>    option overhead '40'
>>>>>>>    option download ‘8500'
>>>>>> 	So this looks reasonable. Then again, if the DSLAM is under provisioned/oversubscribed (= congested) shaping uypur DSL link might not fix all buffer bloat..
>>>>>> 
>>>>>>> tc -s qdisc show dev pppoe-wan
>>>>>>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 3
>>>>>>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>>>>>>> backlog 0b 0p requeues 0
>>>>>>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>>>>>>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>>> backlog 0b 0p requeues 0
>>>>>>>           Class 0     Class 1     Class 2     Class 3
>>>>>>>  rate          0bit        0bit        0bit        0bit
>>>>>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>>>> Pk delay         0us         0us         7us         2us
>>>>>>> Av delay         0us         0us         0us         0us
>>>>>>> Sp delay         0us         0us         0us         0us
>>>>>>>  pkts             0           0          22           3
>>>>>>> way inds           0           0           0           0
>>>>>>> way miss           0           0          22           2
>>>>>>> way cols           0           0           0           0
>>>>>>>  bytes            0           0        3392        1007
>>>>>>>  drops            0           0           0           0
>>>>>>>  marks            0           0           0           0
>>>>>>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>>>>>>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>>>>>>> backlog 0b 0p requeues 0
>>>>>>>           Class 0     Class 1     Class 2     Class 3
>>>>>>>  rate          0bit        0bit        0bit        0bit
>>>>>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>>>> Pk delay         0us      28.0ms         0us         0us
>>>>>>> Av delay         0us       1.2ms         0us         0us
>>>>>>> Sp delay         0us         4us         0us         0us
>>>>>>>  pkts             0         417           0           0
>>>>>>> way inds           0           0           0           0
>>>>>>> way miss           0          23           0           0
>>>>>>> way cols           0           0           0           0
>>>>>>>  bytes            0       98951           0           0
>>>>>>>  drops            0           2           0           0
>>>>>>>  marks            0           0           0           0
>>>>>>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>>>>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>>> backlog 0b 0p requeues 0
>>>>>>>           Class 0     Class 1     Class 2     Class 3
>>>>>>>  rate          0bit        0bit        0bit        0bit
>>>>>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>>>> Pk delay         0us         0us         0us         0us
>>>>>>> Av delay         0us         0us         0us         0us
>>>>>>> Sp delay         0us         0us         0us         0us
>>>>>>>  pkts             0           0           0           0
>>>>>>> way inds           0           0           0           0
>>>>>>> way miss           0           0           0           0
>>>>>>> way cols           0           0           0           0
>>>>>>>  bytes            0           0           0           0
>>>>>>>  drops            0           0           0           0
>>>>>>>  marks            0           0           0           0
>>>>>>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>>>>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>>> backlog 0b 0p requeues 0
>>>>>>>           Class 0     Class 1     Class 2     Class 3
>>>>>>>  rate          0bit        0bit        0bit        0bit
>>>>>>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>>>>>>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>>>>>>> Pk delay         0us         0us         0us         0us
>>>>>>> Av delay         0us         0us         0us         0us
>>>>>>> Sp delay         0us         0us         0us         0us
>>>>>>>  pkts             0           0           0           0
>>>>>>> way inds           0           0           0           0
>>>>>>> way miss           0           0           0           0
>>>>>>> way cols           0           0           0           0
>>>>>>>  bytes            0           0           0           0
>>>>>>>  drops            0           0           0           0
>>>>>>>  marks            0           0           0           0
>>>>>>> qdisc ingress ffff: parent ffff:fff1 ----------------
>>>>>>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>>> backlog 0b 0p requeues 0
>>>>>> 	But this is the hallmark of out of date sqm-scripts, this just uses cake as leaf qdisc and keeps HTB as the main shaper; a configuration that is useful for testing. I assume this is the old set of sqm-scripts not the update I just sent as attachment? If so could you retry with the newer scripts, please?
>>>>>> 
>>>>>> Best Regards
>>>>>> 	Sebastian
>>>>>> 
>>>>>>> 
>>>>>>> On 10/07/15 19:46, Sebastian Moeller wrote:
>>>>>>>> Hi Fred,
>>>>>>>> 
>>>>>>>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>>>>>>>> 1) cat /etc/config/sqm
>>>>>>>> 2) tc -d qdisc
>>>>>>>> 3) tc -d class show dev pppoe-wan
>>>>>>>> 4) tc -d class show dev ifb4pppoe-wqn
>>>>>>>> 5) /etc/init.d/sqm stop
>>>>>>>> 6) /etc/init.d/sqm start
>>>>>>>> 
>>>>>>>> hopefully these give some insight what might have happened.
>>>>>>>> 
>>>>>>>> And finally I would love to learn the output of:
>>>>>>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Many Thanks & Best Regards
>>>>>>>> 	Sebastian
>>>>>>>> 
>>>>>>>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>>>>>>>> 
>>>>>>>>> By your command
>>>>>>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>>>>>>>> 
>>>>>>>>> script configuring qdiscs and overhead 40 on
>>>>>>>>> 
>>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>>>>>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>>>>>>>> Download:  6.73 Mbps
>>>>>>>>>   Upload:  0.58 Mbps
>>>>>>>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>>>>      Min: 24.094
>>>>>>>>>    10pct: 172.654
>>>>>>>>>   Median: 260.563
>>>>>>>>>      Avg: 253.580
>>>>>>>>>    90pct: 330.003
>>>>>>>>>      Max: 411.145
>>>>>>>>> 
>>>>>>>>> script configuring qdiscs on flows raw
>>>>>>>>> 
>>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>>>>> 78.145.32.1
>>>>>>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>>>>>>>> Download:  6.75 Mbps
>>>>>>>>>   Upload:  0.59 Mbps
>>>>>>>>>  Latency: (in msec, 59 pings, 0.00% packet loss)
>>>>>>>>>      Min: 23.605
>>>>>>>>>    10pct: 169.789
>>>>>>>>>   Median: 282.155
>>>>>>>>>      Avg: 267.099
>>>>>>>>>    90pct: 333.283
>>>>>>>>>      Max: 376.509
>>>>>>>>> 
>>>>>>>>> script configuring qdiscs and overhead 36 on
>>>>>>>>> 
>>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>>>>> 80.44.96.1
>>>>>>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>>>>>>>> Download:  6.56 Mbps
>>>>>>>>>   Upload:  0.59 Mbps
>>>>>>>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>>>>      Min: 22.975
>>>>>>>>>    10pct: 195.473
>>>>>>>>>   Median: 281.756
>>>>>>>>>      Avg: 271.609
>>>>>>>>>    90pct: 342.130
>>>>>>>>>      Max: 398.573
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>>>>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>>>>>>>> 
>>>>>>>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>>>>>>>> 
>>>>>>>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>>>>>>> 
>>>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>>>>>>> 
>>>>>>>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>>>>>>>> 
>>>>>>>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>>>>>>>> 
>>>>>>>>>> Alan
>>>>>>>>>> 
>>>>>>>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>>>>>>> You are absolutely correct.
>>>>>>>>>>> 
>>>>>>>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>>>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>>>>>>>> undeclared version 2. Everything works as stated.
>>>>>>>>>>> 
>>>>>>>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>>>>>>>> values were not recognised.
>>>>>>>>>>> 
>>>>>>>>>>> Thank you.
>>>>>>>>>>> 
>>>>>>>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>>>>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>>>>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>>>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>>>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>>>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>>>>>>>> "overhead" or "atm".
>>>>>>>>>>>> 
>>>>>>>>>>>> What's probably happening is that you're using a slightly old version
>>>>>>>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>>>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>>>>>>>> combination crop up ourselves recently.
>>>>>>>>>>>> 
>>>>>>>>>>>> - Jonathan Morton
>>>>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> Cerowrt-devel mailing list
>>>>>>>>> Cerowrt-devel@lists.bufferbloat.net
>>>>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Alan Jenkins]

On 10/07/15 20:34, Fred Stratton wrote:
> bridge sync is circa 10 000 kbit/s
>
> with the cake option in sqm enabled
>
> config queue 'eth1'
>     option qdisc_advanced '0'
>     option enabled '1'
>     option interface 'pppoe-wan'
>     option upload '850'
>     option qdisc 'cake'
>     option script 'simple_pppoe.qos'

Sorry, you want simple.qos.  simple_pppoe is for if you had set 
interface to the corresponding ethX.  I think Seb said there's not much 
reason to do that anymore, now we correctly handle interfaces going up 
and down.  ("hotplug").

> option linklayer 'atm'
>     option overhead '40'
>     option download '8500'
>
> tc -s qdisc show dev pppoe-wan
> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 
> direct_qlen 3
>  Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>  backlog 0b 0p requeues 0
> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>  Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>            Class 0     Class 1     Class 2     Class 3
>   rate          0bit        0bit        0bit        0bit
>   target       5.0ms       5.0ms       5.0ms       5.0ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk delay         0us         0us         7us         2us
> Av delay         0us         0us         0us         0us
> Sp delay         0us         0us         0us         0us
>   pkts             0           0          22           3
> way inds           0           0           0           0
> way miss           0           0          22           2
> way cols           0           0           0           0
>   bytes            0           0        3392        1007
>   drops            0           0           0           0
>   marks            0           0           0           0
> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>  Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>            Class 0     Class 1     Class 2     Class 3
>   rate          0bit        0bit        0bit        0bit
>   target       5.0ms       5.0ms       5.0ms       5.0ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk delay         0us      28.0ms         0us         0us
> Av delay         0us       1.2ms         0us         0us
> Sp delay         0us         4us         0us         0us
>   pkts             0         417           0           0
> way inds           0           0           0           0
> way miss           0          23           0           0
> way cols           0           0           0           0
>   bytes            0       98951           0           0
>   drops            0           2           0           0
>   marks            0           0           0           0
> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>            Class 0     Class 1     Class 2     Class 3
>   rate          0bit        0bit        0bit        0bit
>   target       5.0ms       5.0ms       5.0ms       5.0ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk delay         0us         0us         0us         0us
> Av delay         0us         0us         0us         0us
> Sp delay         0us         0us         0us         0us
>   pkts             0           0           0           0
> way inds           0           0           0           0
> way miss           0           0           0           0
> way cols           0           0           0           0
>   bytes            0           0           0           0
>   drops            0           0           0           0
>   marks            0           0           0           0
> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>  Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>            Class 0     Class 1     Class 2     Class 3
>   rate          0bit        0bit        0bit        0bit
>   target       5.0ms       5.0ms       5.0ms       5.0ms
> interval     100.0ms     100.0ms     100.0ms     100.0ms
> Pk delay         0us         0us         0us         0us
> Av delay         0us         0us         0us         0us
> Sp delay         0us         0us         0us         0us
>   pkts             0           0           0           0
> way inds           0           0           0           0
> way miss           0           0           0           0
> way cols           0           0           0           0
>   bytes            0           0           0           0
>   drops            0           0           0           0
>   marks            0           0           0           0
> qdisc ingress ffff: parent ffff:fff1 ----------------
>  Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>  backlog 0b 0p requeues 0
>
>
>
>
> On 10/07/15 19:46, Sebastian Moeller wrote:
>> Hi Fred,
>>
>> your results seem to indicate that cake is not active at all, as the 
>> latency under load is abysmal (a quick check is to look at the median 
>> in relation to the min and the 90% number, in your examples all of 
>> these are terrible). Could you please post the result of the 
>> following commands on your router:
>> 1) cat /etc/config/sqm
>> 2) tc -d qdisc
>> 3) tc -d class show dev pppoe-wan
>> 4) tc -d class show dev ifb4pppoe-wqn
>> 5) /etc/init.d/sqm stop
>> 6) /etc/init.d/sqm start
>>
>> hopefully these give some insight what might have happened.
>>
>> And finally I would love to learn the output of:
>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p 
>> netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H 
>> netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>
>>
>> Many Thanks & Best Regards
>>     Sebastian
>>
>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>>
>>> By your command
>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the 
>>> command-line, so suboptimal process as end-point changed.
>>>
>>> script configuring qdiscs and overhead 40 on
>>>
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 
>>> streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>> Download:  6.73 Mbps
>>>    Upload:  0.58 Mbps
>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>       Min: 24.094
>>>     10pct: 172.654
>>>    Median: 260.563
>>>       Avg: 253.580
>>>     90pct: 330.003
>>>       Max: 411.145
>>>
>>> script configuring qdiscs on flows raw
>>>
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>> 78.145.32.1
>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 
>>> streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>> Download:  6.75 Mbps
>>>    Upload:  0.59 Mbps
>>>   Latency: (in msec, 59 pings, 0.00% packet loss)
>>>       Min: 23.605
>>>     10pct: 169.789
>>>    Median: 282.155
>>>       Avg: 267.099
>>>     90pct: 333.283
>>>       Max: 376.509
>>>
>>> script configuring qdiscs and overhead 36 on
>>>
>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>> 80.44.96.1
>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 
>>> streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>> Download:  6.56 Mbps
>>>    Upload:  0.59 Mbps
>>>   Latency: (in msec, 62 pings, 0.00% packet loss)
>>>       Min: 22.975
>>>     10pct: 195.473
>>>    Median: 281.756
>>>       Avg: 271.609
>>>     90pct: 342.130
>>>       Max: 398.573
>>>
>>>
>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>> I'm glad to hear there's a working version (even if it's not in the 
>>>> current build :).
>>>>
>>>> Do you have measurable improvements with overhead configured (v.s. 
>>>> unconfigured)?
>>>>
>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>
>>>> I believe accounting for overhead helps on this two-way test, 
>>>> because a) it saturates the uplink b) about half that bandwidth is 
>>>> tiny ack packets (depending on bandwidth asymmetry).  And small 
>>>> packets have proportionally high overhead.
>>>>
>>>> (But it seems to only make a small difference for me, which always 
>>>> surprises Seb).
>>>>
>>>> Alan
>>>>
>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>> You are absolutely correct.
>>>>>
>>>>> I tried both a numeric overhead value, and alternatively 
>>>>> 'pppoe-vcmux'
>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is 
>>>>> lupin
>>>>> undeclared version 2. Everything works as stated.
>>>>>
>>>>> On lupin undeclared version 4, the current release based on 
>>>>> r46117, the
>>>>> values were not recognised.
>>>>>
>>>>> Thank you.
>>>>>
>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>
>>>>>
>>>>>
>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>> several sets of keywords and parameters which are mutually 
>>>>>> orthogonal,
>>>>>> and don't depend on each other, so "besteffort" has nothing to do 
>>>>>> with
>>>>>> "overhead" or "atm".
>>>>>>
>>>>>> What's probably happening is that you're using a slightly old 
>>>>>> version
>>>>>> of the cake kernel module which lacks the overhead parameter 
>>>>>> entirely,
>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>> combination crop up ourselves recently.
>>>>>>
>>>>>> - Jonathan Morton
>>>>>>
>>> _______________________________________________
>>> Cerowrt-devel mailing list
>>> Cerowrt-devel@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Sebastian Moeller]

Hi Alan,

On Jul 10, 2015, at 21:41 , Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:

> On 10/07/15 20:34, Fred Stratton wrote:
>> bridge sync is circa 10 000 kbit/s
>> 
>> with the cake option in sqm enabled
>> 
>> config queue 'eth1'
>>    option qdisc_advanced '0'
>>    option enabled '1'
>>    option interface 'pppoe-wan'
>>    option upload '850'
>>    option qdisc 'cake'
>>    option script 'simple_pppoe.qos'
> 
> Sorry, you want simple.qos.  simple_pppoe is for if you had set interface to the corresponding ethX.  

	This is correct.

> I think Seb said there's not much reason to do that anymore, now we correctly handle interfaces going up and down.  ("hotplug”).

	I should delete it one of those days, it is still in to remind me to finally switch the tc filters to hashes to see how this improves the speed…

Best Regards
	Sebastian

> 
>> option linklayer 'atm'
>>    option overhead '40'
>>    option download '8500'
>> 
>> tc -s qdisc show dev pppoe-wan
>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 3
>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>> backlog 0b 0p requeues 0
>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>           Class 0     Class 1     Class 2     Class 3
>>  rate          0bit        0bit        0bit        0bit
>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk delay         0us         0us         7us         2us
>> Av delay         0us         0us         0us         0us
>> Sp delay         0us         0us         0us         0us
>>  pkts             0           0          22           3
>> way inds           0           0           0           0
>> way miss           0           0          22           2
>> way cols           0           0           0           0
>>  bytes            0           0        3392        1007
>>  drops            0           0           0           0
>>  marks            0           0           0           0
>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>           Class 0     Class 1     Class 2     Class 3
>>  rate          0bit        0bit        0bit        0bit
>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk delay         0us      28.0ms         0us         0us
>> Av delay         0us       1.2ms         0us         0us
>> Sp delay         0us         4us         0us         0us
>>  pkts             0         417           0           0
>> way inds           0           0           0           0
>> way miss           0          23           0           0
>> way cols           0           0           0           0
>>  bytes            0       98951           0           0
>>  drops            0           2           0           0
>>  marks            0           0           0           0
>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>           Class 0     Class 1     Class 2     Class 3
>>  rate          0bit        0bit        0bit        0bit
>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk delay         0us         0us         0us         0us
>> Av delay         0us         0us         0us         0us
>> Sp delay         0us         0us         0us         0us
>>  pkts             0           0           0           0
>> way inds           0           0           0           0
>> way miss           0           0           0           0
>> way cols           0           0           0           0
>>  bytes            0           0           0           0
>>  drops            0           0           0           0
>>  marks            0           0           0           0
>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>>           Class 0     Class 1     Class 2     Class 3
>>  rate          0bit        0bit        0bit        0bit
>>  target       5.0ms       5.0ms       5.0ms       5.0ms
>> interval     100.0ms     100.0ms     100.0ms     100.0ms
>> Pk delay         0us         0us         0us         0us
>> Av delay         0us         0us         0us         0us
>> Sp delay         0us         0us         0us         0us
>>  pkts             0           0           0           0
>> way inds           0           0           0           0
>> way miss           0           0           0           0
>> way cols           0           0           0           0
>>  bytes            0           0           0           0
>>  drops            0           0           0           0
>>  marks            0           0           0           0
>> qdisc ingress ffff: parent ffff:fff1 ----------------
>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> 
>> 
>> 
>> 
>> On 10/07/15 19:46, Sebastian Moeller wrote:
>>> Hi Fred,
>>> 
>>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>>> 1) cat /etc/config/sqm
>>> 2) tc -d qdisc
>>> 3) tc -d class show dev pppoe-wan
>>> 4) tc -d class show dev ifb4pppoe-wqn
>>> 5) /etc/init.d/sqm stop
>>> 6) /etc/init.d/sqm start
>>> 
>>> hopefully these give some insight what might have happened.
>>> 
>>> And finally I would love to learn the output of:
>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>> 
>>> 
>>> Many Thanks & Best Regards
>>>    Sebastian
>>> 
>>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton@imap.cc> wrote:
>>> 
>>>> By your command
>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>>> 
>>>> script configuring qdiscs and overhead 40 on
>>>> 
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>>> Download:  6.73 Mbps
>>>>   Upload:  0.58 Mbps
>>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>      Min: 24.094
>>>>    10pct: 172.654
>>>>   Median: 260.563
>>>>      Avg: 253.580
>>>>    90pct: 330.003
>>>>      Max: 411.145
>>>> 
>>>> script configuring qdiscs on flows raw
>>>> 
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>> 78.145.32.1
>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>>> Download:  6.75 Mbps
>>>>   Upload:  0.59 Mbps
>>>>  Latency: (in msec, 59 pings, 0.00% packet loss)
>>>>      Min: 23.605
>>>>    10pct: 169.789
>>>>   Median: 282.155
>>>>      Avg: 267.099
>>>>    90pct: 333.283
>>>>      Max: 376.509
>>>> 
>>>> script configuring qdiscs and overhead 36 on
>>>> 
>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>> 80.44.96.1
>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>>> Download:  6.56 Mbps
>>>>   Upload:  0.59 Mbps
>>>>  Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>      Min: 22.975
>>>>    10pct: 195.473
>>>>   Median: 281.756
>>>>      Avg: 271.609
>>>>    90pct: 342.130
>>>>      Max: 398.573
>>>> 
>>>> 
>>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>>> 
>>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>>> 
>>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>> 
>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>> 
>>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry).  And small packets have proportionally high overhead.
>>>>> 
>>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>>> 
>>>>> Alan
>>>>> 
>>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>> You are absolutely correct.
>>>>>> 
>>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>>> undeclared version 2. Everything works as stated.
>>>>>> 
>>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>>> values were not recognised.
>>>>>> 
>>>>>> Thank you.
>>>>>> 
>>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>>> "overhead" or "atm".
>>>>>>> 
>>>>>>> What's probably happening is that you're using a slightly old version
>>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>>> combination crop up ourselves recently.
>>>>>>> 
>>>>>>> - Jonathan Morton
>>>>>>> 
>>>> _______________________________________________
>>>> Cerowrt-devel mailing list
>>>> Cerowrt-devel@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> 
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues.

[Alan Jenkins]

On 10/07/15 19:25, Fred Stratton wrote:
> By your command
> Rebooted to rerun qdisc script, rather than changing qdiscs from the 
> command-line, so suboptimal process as end-point changed.

Assuming you use sqm-scripts, you can restart SQM manually. 
"/etc/init.d/sqm restart".