[Cerowrt-devel] Correct syntax for cake commands and atm issues.
Fred Stratton
fredstratton at imap.cc
Fri Jul 10 16:24:08 EDT 2015
sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.ne
t -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H
netperf-
eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
2015-07-10 21:15:19 Testing against netperf-eu.bufferbloat.net (ipv4)
with 4 simultaneous sessions while pinging netperf-eu.bufferbloat.net
(150 seconds in each direction)
......................................................................................................................................................
Download: 6.8 Mbps
Latency: (in msec, 151 pings, 0.00% packet loss)
Min: 69.125
10pct: 72.299
Median: 75.077
Avg: 75.613
90pct: 79.014
Max: 89.825
.......................................................................................................................................................
Upload: 0.72 Mbps
Latency: (in msec, 152 pings, 0.00% packet loss)
Min: 68.691
10pct: 72.506
Median: 79.447
Avg: 79.654
90pct: 86.369
Max: 95.037
2015-07-10 21:20:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4
streams down and up while pinging netperf-eu.bufferbloat.net. Takes
about 150 seconds.
Download: 5.92 Mbps
Upload: 0.46 Mbps
Latency: (in msec, 152 pings, 0.00% packet loss)
Min: 71.877
10pct: 76.197
Median: 85.051
Avg: 84.838
90pct: 92.105
Max: 109.600
On 10/07/15 21:12, Sebastian Moeller wrote:
> Hi Fred,
>
> On Jul 10, 2015, at 22:07 , Fred Stratton <fredstratton at imap.cc> wrote:
>
>> replaced /usr/lib/sqm as ordered
> Thanks.
>
>> cat /etc/config/sqm
>>
>> config queue 'eth1'
>> option qdisc_advanced '0'
>> option enabled '1'
>> option interface 'pppoe-wan'
>> option upload '850'
>> option qdisc 'cake'
>> option linklayer 'atm'
>> option overhead '40'
>> option download '8500'
>> option script 'simple.qos'
>> option linklayer_advanced '1'
>> option tcMTU '2047'
>> option tcTSIZE '128'
>> option tcMPU '0'
>> option linklayer_adaptation_mechanism ‘cake'
> Looks reasonable.
>
>> root at OpenWrt:~# tc -s qdisc show
>> qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 5904206 bytes 10771 pkt (dropped 0, overlimits 0 requeues 1)
>> backlog 0b 0p requeues 1
>> maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev eth1 root refcnt 2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 903154 bytes 7941 pkt (dropped 0, overlimits 0 requeues 2)
>> backlog 0b 0p requeues 2
>> maxpacket 1322 drop_overlimit 0 new_flow_count 1 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc mq 0: dev wlan0 root
>> Sent 198495 bytes 816 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 3709 bytes 19 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 112 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 194674 bytes 796 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc mq 0: dev wlan1 root
>> Sent 53249 bytes 323 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> qdisc fq_codel 0: dev wlan1 parent :1 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 1337 bytes 9 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan1 parent :2 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan1 parent :3 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 51912 bytes 314 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc fq_codel 0: dev wlan1 parent :4 limit 1024p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
>> new_flows_len 0 old_flows_len 0
>> qdisc cake 8009: dev pppoe-wan root refcnt 2 bandwidth 850Kbit diffserv4 flows atm overhead 40
> Good, egress accounts for the link layer adaptation.
>
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> Class 0 Class 1 Class 2 Class 3
>> rate 850Kbit 796880bit 637504bit 212496bit
>> target 21.3ms 22.7ms 28.3ms 85.0ms
>> interval 170.1ms 181.4ms 226.8ms 680.4ms
>> Pk delay 0us 0us 0us 0us
>> Av delay 0us 0us 0us 0us
>> Sp delay 0us 0us 0us 0us
>> pkts 0 0 0 0
>> way inds 0 0 0 0
>> way miss 0 0 0 0
>> way cols 0 0 0 0
>> bytes 0 0 0 0
>> drops 0 0 0 0
>> marks 0 0 0 0
>> qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
>> Sent 68 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> qdisc cake 800a: dev ifb4pppoe-wan root refcnt 2 bandwidth 8500Kbit besteffort flows atm overhead 40
> So, I would be interested to learn how this now performs with netperfrunner and/or betterspeedtest.sh
>
>
> Best Regards
> Sebastian
>
>
>> Sent 90 bytes 1 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> Class 0
>> rate 8500Kbit
>> target 5.0ms
>> interval 100.0ms
>> Pk delay 0us
>> Av delay 0us
>> Sp delay 0us
>> pkts 1
>> way inds 0
>> way miss 1
>> way cols 0
>> bytes 90
>> drops 0
>> marks 0
>>
>>
>>
>> On 10/07/15 20:50, Sebastian Moeller wrote:
>>> Hi Fred,
>>>
>>> On Jul 10, 2015, at 21:45 , Fred Stratton <fredstratton at imap.cc> wrote:
>>>
>>>> These are the latest scripts, AFAIK
>>> Let me repeat my question: are these the scripts I attached in one of the last mails, or the most recent scripts from ceropackages-3.10? The version in the openwrt repository is NOT recent, yet. Pushing the latest verso into openwrt is on my todo list but that will need a bit more changes, so please try the files I attached which should work (unless I screwed up and attached the wrong version). Also I have no working cake on my router and hence require help for testing and that means there might be some undiscovered bugs in there.
>>>
>>>> no overhead allowance. I note.
>>> Well, that should work with the most recent version
>>>
>>> Best Regards
>>> Sebastian
>>>
>>>>
>>>> On 10/07/15 20:40, Sebastian Moeller wrote:
>>>>> Hi Fred,
>>>>>
>>>>>
>>>>> On Jul 10, 2015, at 21:34 , Fred Stratton <fredstratton at imap.cc> wrote:
>>>>>
>>>>>> bridge sync is circa 10 000 kbit/s
>>>>>>
>>>>>> with the cake option in sqm enabled
>>>>>>
>>>>>> config queue 'eth1'
>>>>>> option qdisc_advanced '0'
>>>>>> option enabled '1'
>>>>>> option interface 'pppoe-wan'
>>>>>> option upload '850'
>>>>>> option qdisc 'cake'
>>>>>> option script 'simple_pppoe.qos'
>>>>>> option linklayer 'atm'
>>>>>> option overhead '40'
>>>>>> option download ‘8500'
>>>>> So this looks reasonable. Then again, if the DSLAM is under provisioned/oversubscribed (= congested) shaping uypur DSL link might not fix all buffer bloat..
>>>>>
>>>>>> tc -s qdisc show dev pppoe-wan
>>>>>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 3
>>>>>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw
>>>>>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>> Class 0 Class 1 Class 2 Class 3
>>>>>> rate 0bit 0bit 0bit 0bit
>>>>>> target 5.0ms 5.0ms 5.0ms 5.0ms
>>>>>> interval 100.0ms 100.0ms 100.0ms 100.0ms
>>>>>> Pk delay 0us 0us 7us 2us
>>>>>> Av delay 0us 0us 0us 0us
>>>>>> Sp delay 0us 0us 0us 0us
>>>>>> pkts 0 0 22 3
>>>>>> way inds 0 0 0 0
>>>>>> way miss 0 0 22 2
>>>>>> way cols 0 0 0 0
>>>>>> bytes 0 0 3392 1007
>>>>>> drops 0 0 0 0
>>>>>> marks 0 0 0 0
>>>>>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw
>>>>>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>> Class 0 Class 1 Class 2 Class 3
>>>>>> rate 0bit 0bit 0bit 0bit
>>>>>> target 5.0ms 5.0ms 5.0ms 5.0ms
>>>>>> interval 100.0ms 100.0ms 100.0ms 100.0ms
>>>>>> Pk delay 0us 28.0ms 0us 0us
>>>>>> Av delay 0us 1.2ms 0us 0us
>>>>>> Sp delay 0us 4us 0us 0us
>>>>>> pkts 0 417 0 0
>>>>>> way inds 0 0 0 0
>>>>>> way miss 0 23 0 0
>>>>>> way cols 0 0 0 0
>>>>>> bytes 0 98951 0 0
>>>>>> drops 0 2 0 0
>>>>>> marks 0 0 0 0
>>>>>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw
>>>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>> Class 0 Class 1 Class 2 Class 3
>>>>>> rate 0bit 0bit 0bit 0bit
>>>>>> target 5.0ms 5.0ms 5.0ms 5.0ms
>>>>>> interval 100.0ms 100.0ms 100.0ms 100.0ms
>>>>>> Pk delay 0us 0us 0us 0us
>>>>>> Av delay 0us 0us 0us 0us
>>>>>> Sp delay 0us 0us 0us 0us
>>>>>> pkts 0 0 0 0
>>>>>> way inds 0 0 0 0
>>>>>> way miss 0 0 0 0
>>>>>> way cols 0 0 0 0
>>>>>> bytes 0 0 0 0
>>>>>> drops 0 0 0 0
>>>>>> marks 0 0 0 0
>>>>>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw
>>>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>>> Class 0 Class 1 Class 2 Class 3
>>>>>> rate 0bit 0bit 0bit 0bit
>>>>>> target 5.0ms 5.0ms 5.0ms 5.0ms
>>>>>> interval 100.0ms 100.0ms 100.0ms 100.0ms
>>>>>> Pk delay 0us 0us 0us 0us
>>>>>> Av delay 0us 0us 0us 0us
>>>>>> Sp delay 0us 0us 0us 0us
>>>>>> pkts 0 0 0 0
>>>>>> way inds 0 0 0 0
>>>>>> way miss 0 0 0 0
>>>>>> way cols 0 0 0 0
>>>>>> bytes 0 0 0 0
>>>>>> drops 0 0 0 0
>>>>>> marks 0 0 0 0
>>>>>> qdisc ingress ffff: parent ffff:fff1 ----------------
>>>>>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0)
>>>>>> backlog 0b 0p requeues 0
>>>>> But this is the hallmark of out of date sqm-scripts, this just uses cake as leaf qdisc and keeps HTB as the main shaper; a configuration that is useful for testing. I assume this is the old set of sqm-scripts not the update I just sent as attachment? If so could you retry with the newer scripts, please?
>>>>>
>>>>> Best Regards
>>>>> Sebastian
>>>>>
>>>>>>
>>>>>> On 10/07/15 19:46, Sebastian Moeller wrote:
>>>>>>> Hi Fred,
>>>>>>>
>>>>>>> your results seem to indicate that cake is not active at all, as the latency under load is abysmal (a quick check is to look at the median in relation to the min and the 90% number, in your examples all of these are terrible). Could you please post the result of the following commands on your router:
>>>>>>> 1) cat /etc/config/sqm
>>>>>>> 2) tc -d qdisc
>>>>>>> 3) tc -d class show dev pppoe-wan
>>>>>>> 4) tc -d class show dev ifb4pppoe-wqn
>>>>>>> 5) /etc/init.d/sqm stop
>>>>>>> 6) /etc/init.d/sqm start
>>>>>>>
>>>>>>> hopefully these give some insight what might have happened.
>>>>>>>
>>>>>>> And finally I would love to learn the output of:
>>>>>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4
>>>>>>>
>>>>>>>
>>>>>>> Many Thanks & Best Regards
>>>>>>> Sebastian
>>>>>>>
>>>>>>> On Jul 10, 2015, at 20:25 , Fred Stratton <fredstratton at imap.cc> wrote:
>>>>>>>
>>>>>>>> By your command
>>>>>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from the command-line, so suboptimal process as end-point changed.
>>>>>>>>
>>>>>>>> script configuring qdiscs and overhead 40 on
>>>>>>>>
>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1
>>>>>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 2.96.48.1. Takes about 60 seconds.
>>>>>>>> Download: 6.73 Mbps
>>>>>>>> Upload: 0.58 Mbps
>>>>>>>> Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>>> Min: 24.094
>>>>>>>> 10pct: 172.654
>>>>>>>> Median: 260.563
>>>>>>>> Avg: 253.580
>>>>>>>> 90pct: 330.003
>>>>>>>> Max: 411.145
>>>>>>>>
>>>>>>>> script configuring qdiscs on flows raw
>>>>>>>>
>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>>>> 78.145.32.1
>>>>>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 78.145.32.1. Takes about 60 seconds.
>>>>>>>> Download: 6.75 Mbps
>>>>>>>> Upload: 0.59 Mbps
>>>>>>>> Latency: (in msec, 59 pings, 0.00% packet loss)
>>>>>>>> Min: 23.605
>>>>>>>> 10pct: 169.789
>>>>>>>> Median: 282.155
>>>>>>>> Avg: 267.099
>>>>>>>> 90pct: 333.283
>>>>>>>> Max: 376.509
>>>>>>>>
>>>>>>>> script configuring qdiscs and overhead 36 on
>>>>>>>>
>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p
>>>>>>>> 80.44.96.1
>>>>>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) with 4 streams down and up while pinging 80.44.96.1. Takes about 60 seconds.
>>>>>>>> Download: 6.56 Mbps
>>>>>>>> Upload: 0.59 Mbps
>>>>>>>> Latency: (in msec, 62 pings, 0.00% packet loss)
>>>>>>>> Min: 22.975
>>>>>>>> 10pct: 195.473
>>>>>>>> Median: 281.756
>>>>>>>> Avg: 271.609
>>>>>>>> 90pct: 342.130
>>>>>>>> Max: 398.573
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/07/15 16:19, Alan Jenkins wrote:
>>>>>>>>> I'm glad to hear there's a working version (even if it's not in the current build :).
>>>>>>>>>
>>>>>>>>> Do you have measurable improvements with overhead configured (v.s. unconfigured)?
>>>>>>>>>
>>>>>>>>> I've used netperfrunner from CeroWrtScripts, e.g.
>>>>>>>>>
>>>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p $ISP_ROUTER
>>>>>>>>>
>>>>>>>>> I believe accounting for overhead helps on this two-way test, because a) it saturates the uplink b) about half that bandwidth is tiny ack packets (depending on bandwidth asymmetry). And small packets have proportionally high overhead.
>>>>>>>>>
>>>>>>>>> (But it seems to only make a small difference for me, which always surprises Seb).
>>>>>>>>>
>>>>>>>>> Alan
>>>>>>>>>
>>>>>>>>> On 10/07/15 15:52, Fred Stratton wrote:
>>>>>>>>>> You are absolutely correct.
>>>>>>>>>>
>>>>>>>>>> I tried both a numeric overhead value, and alternatively 'pppoe-vcmux'
>>>>>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which is lupin
>>>>>>>>>> undeclared version 2. Everything works as stated.
>>>>>>>>>>
>>>>>>>>>> On lupin undeclared version 4, the current release based on r46117, the
>>>>>>>>>> values were not recognised.
>>>>>>>>>>
>>>>>>>>>> Thank you.
>>>>>>>>>>
>>>>>>>>>> I had cake running on a Lantiq ADSL gateway running the same r46006
>>>>>>>>>> build. Unfortunately this was bricked by attempts to get homenet
>>>>>>>>>> working, so I have nothing to report about gateway usage at present.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 10/07/15 13:57, Jonathan Morton wrote:
>>>>>>>>>>> You're already using correct syntax - I've written it to be quite
>>>>>>>>>>> lenient and use sensible defaults for missing information. There are
>>>>>>>>>>> several sets of keywords and parameters which are mutually orthogonal,
>>>>>>>>>>> and don't depend on each other, so "besteffort" has nothing to do with
>>>>>>>>>>> "overhead" or "atm".
>>>>>>>>>>>
>>>>>>>>>>> What's probably happening is that you're using a slightly old version
>>>>>>>>>>> of the cake kernel module which lacks the overhead parameter entirely,
>>>>>>>>>>> but a more up to date tc which does support it. We've seen this
>>>>>>>>>>> combination crop up ourselves recently.
>>>>>>>>>>>
>>>>>>>>>>> - Jonathan Morton
>>>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Cerowrt-devel mailing list
>>>>>>>> Cerowrt-devel at lists.bufferbloat.net
>>>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
More information about the Cerowrt-devel
mailing list