From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 182E421FF49 for ; Fri, 10 Jul 2015 13:12:44 -0700 (PDT) Received: from hms-beagle-7.home.lan ([217.237.70.193]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0MOfQw-1ZJiOt3iti-0064DD; Fri, 10 Jul 2015 22:12:41 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: <55A025F7.3000602@imap.cc> Date: Fri, 10 Jul 2015 22:12:39 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <60C15149-4219-4AED-A7AB-DEB90CAC8603@gmx.de> References: <559FA8D4.2030305@imap.cc> <559FDC38.9010400@imap.cc> <559FE294.60801@gmail.com> <55A00E06.3000000@imap.cc> <55A01E2A.4050508@imap.cc> <55A020CE.2060700@imap.cc> <55A025F7.3000602@imap.cc> To: Fred Stratton X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:1FzoCUaCKoivAQ43xq+XhAdRIoulS86D8hEtKV+gC3MMtueNKeZ sSSCMoCVhT9XaY8dLgiHfSF8jWvCsAjOy7ZbMyQERIVlacIAHwzKg+tOMmFaUtozjNpCuCu CZxRTsxGSN5RIshkqW8IDISk8F4gGgRMp75XGgaqLcsdGq9jpQBdteLOvMos2gNq8fYTXZo Dm83p/7WSV5Os/7sPfJQA== X-UI-Out-Filterresults: notjunk:1;V01:K0:lHzwK3BGavw=:S6D1rfc3qFKXmbWQX0hbQU D3EXTaf5quH8V3t6Q6um600wbgREe4UPCCigKwzrXFw9DR4dbLNclXVZF8HGIfALHDry84WO8 tE6y1qwmbpo2sKWCasdR3J2fIgOf5V3AHsNlCfm9N958mMmlcgJchwKLN7mLpJRklfvxMrRiP gJcXx+EN1JJZdhNGVqnfTeSUefdVbNvq9m67dj6i8H6JjcQ/dtYNllUXZzOmojA6ihFJKl+yc 0E9/IOkfNwvr4enUFPZLJvG51aIzPvj1mvUvHZH507bPP5IFOZqkjN3wx0lzaqNrmVqCuyflO qu2eY4BlVeg4HTJ+KJRXx2QRrhlUcRmjpAEYjJylkYRPt/yhdX06QdhRdnO7VMDPxu4fcV04r SmtPeqq2IyZNQ8KzuLorYO9NEsbvS1YB2h1ZM9m9/SRTj/yu/kmI20FUPOmfQRDGJBR2aaEGn HaE3TWKF+qy/FjRsYWlsO45CMSyVb1DS+mA0gz6dgSt2CP6lZMdMnyGrEf39mLALgHjvKT7uq VTTuCwZ3XDrSxy1sBwYuUbwQzNHAIx0WnCQrdpT+SYDkdXyuPl+7SumIHYrOGQxV6jxrvSgvW 4fQg5UiKfmtQiIlADUAMlV5b9FU93nC/16aDgK/thxa63Rc5ZFtTE66kSusKdfMYGEQXVbrGx nCfSr9pR9DYdrMpcgb+5Ng5CmfW4HnYDDusTb9CoPlouT3mo3ChAqNBSXwz1u57wf0aU= Cc: cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] Correct syntax for cake commands and atm issues. X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jul 2015 20:13:13 -0000 Hi Fred, On Jul 10, 2015, at 22:07 , Fred Stratton wrote: > replaced /usr/lib/sqm as ordered Thanks. >=20 > cat /etc/config/sqm >=20 > config queue 'eth1' > option qdisc_advanced '0' > option enabled '1' > option interface 'pppoe-wan' > option upload '850' > option qdisc 'cake' > option linklayer 'atm' > option overhead '40' > option download '8500' > option script 'simple.qos' > option linklayer_advanced '1' > option tcMTU '2047' > option tcTSIZE '128' > option tcMPU '0' > option linklayer_adaptation_mechanism =91cake' Looks reasonable. >=20 > root@OpenWrt:~# tc -s qdisc show > qdisc fq_codel 0: dev eth0 root refcnt 2 limit 1024p flows 1024 = quantum 300 target 5.0ms interval 100.0ms ecn > Sent 5904206 bytes 10771 pkt (dropped 0, overlimits 0 requeues 1) > backlog 0b 0p requeues 1 > maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc fq_codel 0: dev eth1 root refcnt 2 limit 1024p flows 1024 = quantum 300 target 5.0ms interval 100.0ms ecn > Sent 903154 bytes 7941 pkt (dropped 0, overlimits 0 requeues 2) > backlog 0b 0p requeues 2 > maxpacket 1322 drop_overlimit 0 new_flow_count 1 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc mq 0: dev wlan0 root > Sent 198495 bytes 816 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc fq_codel 0: dev wlan0 parent :1 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > Sent 3709 bytes 19 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc fq_codel 0: dev wlan0 parent :2 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > Sent 112 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc fq_codel 0: dev wlan0 parent :3 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > Sent 194674 bytes 796 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc fq_codel 0: dev wlan0 parent :4 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc mq 0: dev wlan1 root > Sent 53249 bytes 323 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc fq_codel 0: dev wlan1 parent :1 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > Sent 1337 bytes 9 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc fq_codel 0: dev wlan1 parent :2 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc fq_codel 0: dev wlan1 parent :3 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > Sent 51912 bytes 314 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc fq_codel 0: dev wlan1 parent :4 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc cake 8009: dev pppoe-wan root refcnt 2 bandwidth 850Kbit = diffserv4 flows atm overhead 40 Good, egress accounts for the link layer adaptation. > Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > Class 0 Class 1 Class 2 Class 3 > rate 850Kbit 796880bit 637504bit 212496bit > target 21.3ms 22.7ms 28.3ms 85.0ms > interval 170.1ms 181.4ms 226.8ms 680.4ms > Pk delay 0us 0us 0us 0us > Av delay 0us 0us 0us 0us > Sp delay 0us 0us 0us 0us > pkts 0 0 0 0 > way inds 0 0 0 0 > way miss 0 0 0 0 > way cols 0 0 0 0 > bytes 0 0 0 0 > drops 0 0 0 0 > marks 0 0 0 0 > qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ---------------- > Sent 68 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc cake 800a: dev ifb4pppoe-wan root refcnt 2 bandwidth 8500Kbit = besteffort flows atm overhead 40 So, I would be interested to learn how this now performs with = netperfrunner and/or betterspeedtest.sh Best Regards Sebastian > Sent 90 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > Class 0 > rate 8500Kbit > target 5.0ms > interval 100.0ms > Pk delay 0us > Av delay 0us > Sp delay 0us > pkts 1 > way inds 0 > way miss 1 > way cols 0 > bytes 90 > drops 0 > marks 0 >=20 >=20 >=20 > On 10/07/15 20:50, Sebastian Moeller wrote: >> Hi Fred, >>=20 >> On Jul 10, 2015, at 21:45 , Fred Stratton = wrote: >>=20 >>> These are the latest scripts, AFAIK >> Let me repeat my question: are these the scripts I attached in = one of the last mails, or the most recent scripts from = ceropackages-3.10? The version in the openwrt repository is NOT recent, = yet. Pushing the latest verso into openwrt is on my todo list but that = will need a bit more changes, so please try the files I attached which = should work (unless I screwed up and attached the wrong version). Also I = have no working cake on my router and hence require help for testing and = that means there might be some undiscovered bugs in there. >>=20 >>> no overhead allowance. I note. >> Well, that should work with the most recent version >>=20 >> Best Regards >> Sebastian >>=20 >>>=20 >>>=20 >>> On 10/07/15 20:40, Sebastian Moeller wrote: >>>> Hi Fred, >>>>=20 >>>>=20 >>>> On Jul 10, 2015, at 21:34 , Fred Stratton = wrote: >>>>=20 >>>>> bridge sync is circa 10 000 kbit/s >>>>>=20 >>>>> with the cake option in sqm enabled >>>>>=20 >>>>> config queue 'eth1' >>>>> option qdisc_advanced '0' >>>>> option enabled '1' >>>>> option interface 'pppoe-wan' >>>>> option upload '850' >>>>> option qdisc 'cake' >>>>> option script 'simple_pppoe.qos' >>>>> option linklayer 'atm' >>>>> option overhead '40' >>>>> option download =918500' >>>> So this looks reasonable. Then again, if the DSLAM is under = provisioned/oversubscribed (=3D congested) shaping uypur DSL link might = not fix all buffer bloat.. >>>>=20 >>>>> tc -s qdisc show dev pppoe-wan >>>>> qdisc htb 1: root refcnt 2 r2q 10 default 12 direct_packets_stat 0 = direct_qlen 3 >>>>> Sent 101336 bytes 440 pkt (dropped 2, overlimits 66 requeues 0) >>>>> backlog 0b 0p requeues 0 >>>>> qdisc cake 110: parent 1:11 unlimited diffserv4 flows raw >>>>> Sent 4399 bytes 25 pkt (dropped 0, overlimits 0 requeues 0) >>>>> backlog 0b 0p requeues 0 >>>>> Class 0 Class 1 Class 2 Class 3 >>>>> rate 0bit 0bit 0bit 0bit >>>>> target 5.0ms 5.0ms 5.0ms 5.0ms >>>>> interval 100.0ms 100.0ms 100.0ms 100.0ms >>>>> Pk delay 0us 0us 7us 2us >>>>> Av delay 0us 0us 0us 0us >>>>> Sp delay 0us 0us 0us 0us >>>>> pkts 0 0 22 3 >>>>> way inds 0 0 0 0 >>>>> way miss 0 0 22 2 >>>>> way cols 0 0 0 0 >>>>> bytes 0 0 3392 1007 >>>>> drops 0 0 0 0 >>>>> marks 0 0 0 0 >>>>> qdisc cake 120: parent 1:12 unlimited diffserv4 flows raw >>>>> Sent 96937 bytes 415 pkt (dropped 2, overlimits 0 requeues 0) >>>>> backlog 0b 0p requeues 0 >>>>> Class 0 Class 1 Class 2 Class 3 >>>>> rate 0bit 0bit 0bit 0bit >>>>> target 5.0ms 5.0ms 5.0ms 5.0ms >>>>> interval 100.0ms 100.0ms 100.0ms 100.0ms >>>>> Pk delay 0us 28.0ms 0us 0us >>>>> Av delay 0us 1.2ms 0us 0us >>>>> Sp delay 0us 4us 0us 0us >>>>> pkts 0 417 0 0 >>>>> way inds 0 0 0 0 >>>>> way miss 0 23 0 0 >>>>> way cols 0 0 0 0 >>>>> bytes 0 98951 0 0 >>>>> drops 0 2 0 0 >>>>> marks 0 0 0 0 >>>>> qdisc cake 130: parent 1:13 unlimited diffserv4 flows raw >>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) >>>>> backlog 0b 0p requeues 0 >>>>> Class 0 Class 1 Class 2 Class 3 >>>>> rate 0bit 0bit 0bit 0bit >>>>> target 5.0ms 5.0ms 5.0ms 5.0ms >>>>> interval 100.0ms 100.0ms 100.0ms 100.0ms >>>>> Pk delay 0us 0us 0us 0us >>>>> Av delay 0us 0us 0us 0us >>>>> Sp delay 0us 0us 0us 0us >>>>> pkts 0 0 0 0 >>>>> way inds 0 0 0 0 >>>>> way miss 0 0 0 0 >>>>> way cols 0 0 0 0 >>>>> bytes 0 0 0 0 >>>>> drops 0 0 0 0 >>>>> marks 0 0 0 0 >>>>> qdisc cake 140: parent 1:14 unlimited diffserv4 flows raw >>>>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) >>>>> backlog 0b 0p requeues 0 >>>>> Class 0 Class 1 Class 2 Class 3 >>>>> rate 0bit 0bit 0bit 0bit >>>>> target 5.0ms 5.0ms 5.0ms 5.0ms >>>>> interval 100.0ms 100.0ms 100.0ms 100.0ms >>>>> Pk delay 0us 0us 0us 0us >>>>> Av delay 0us 0us 0us 0us >>>>> Sp delay 0us 0us 0us 0us >>>>> pkts 0 0 0 0 >>>>> way inds 0 0 0 0 >>>>> way miss 0 0 0 0 >>>>> way cols 0 0 0 0 >>>>> bytes 0 0 0 0 >>>>> drops 0 0 0 0 >>>>> marks 0 0 0 0 >>>>> qdisc ingress ffff: parent ffff:fff1 ---------------- >>>>> Sent 273341 bytes 435 pkt (dropped 0, overlimits 0 requeues 0) >>>>> backlog 0b 0p requeues 0 >>>> But this is the hallmark of out of date sqm-scripts, this just = uses cake as leaf qdisc and keeps HTB as the main shaper; a = configuration that is useful for testing. I assume this is the old set = of sqm-scripts not the update I just sent as attachment? If so could you = retry with the newer scripts, please? >>>>=20 >>>> Best Regards >>>> Sebastian >>>>=20 >>>>>=20 >>>>>=20 >>>>> On 10/07/15 19:46, Sebastian Moeller wrote: >>>>>> Hi Fred, >>>>>>=20 >>>>>> your results seem to indicate that cake is not active at all, as = the latency under load is abysmal (a quick check is to look at the = median in relation to the min and the 90% number, in your examples all = of these are terrible). Could you please post the result of the = following commands on your router: >>>>>> 1) cat /etc/config/sqm >>>>>> 2) tc -d qdisc >>>>>> 3) tc -d class show dev pppoe-wan >>>>>> 4) tc -d class show dev ifb4pppoe-wqn >>>>>> 5) /etc/init.d/sqm stop >>>>>> 6) /etc/init.d/sqm start >>>>>>=20 >>>>>> hopefully these give some insight what might have happened. >>>>>>=20 >>>>>> And finally I would love to learn the output of: >>>>>> sh betterspeedtest.sh -4 -H netperf-eu.bufferbloat.net -t 150 -p = netperf-eu.bufferbloat.net -n 4 ; sh netperfrunner.sh -4 -H = netperf-eu.bufferbloat.net -t 150 -p netperf-eu.bufferbloat.net -n 4 >>>>>>=20 >>>>>>=20 >>>>>> Many Thanks & Best Regards >>>>>> Sebastian >>>>>>=20 >>>>>> On Jul 10, 2015, at 20:25 , Fred Stratton = wrote: >>>>>>=20 >>>>>>> By your command >>>>>>> Rebooted to rerun qdisc script, rather than changing qdiscs from = the command-line, so suboptimal process as end-point changed. >>>>>>>=20 >>>>>>> script configuring qdiscs and overhead 40 on >>>>>>>=20 >>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p 2.96.48.1 >>>>>>> 2015-07-10 18:22:08 Testing netperf-eu.bufferbloat.net (ipv4) = with 4 streams down and up while pinging 2.96.48.1. Takes about 60 = seconds. >>>>>>> Download: 6.73 Mbps >>>>>>> Upload: 0.58 Mbps >>>>>>> Latency: (in msec, 62 pings, 0.00% packet loss) >>>>>>> Min: 24.094 >>>>>>> 10pct: 172.654 >>>>>>> Median: 260.563 >>>>>>> Avg: 253.580 >>>>>>> 90pct: 330.003 >>>>>>> Max: 411.145 >>>>>>>=20 >>>>>>> script configuring qdiscs on flows raw >>>>>>>=20 >>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p >>>>>>> 78.145.32.1 >>>>>>> 2015-07-10 18:49:21 Testing netperf-eu.bufferbloat.net (ipv4) = with 4 streams down and up while pinging 78.145.32.1. Takes about 60 = seconds. >>>>>>> Download: 6.75 Mbps >>>>>>> Upload: 0.59 Mbps >>>>>>> Latency: (in msec, 59 pings, 0.00% packet loss) >>>>>>> Min: 23.605 >>>>>>> 10pct: 169.789 >>>>>>> Median: 282.155 >>>>>>> Avg: 267.099 >>>>>>> 90pct: 333.283 >>>>>>> Max: 376.509 >>>>>>>=20 >>>>>>> script configuring qdiscs and overhead 36 on >>>>>>>=20 >>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p >>>>>>> 80.44.96.1 >>>>>>> 2015-07-10 19:20:18 Testing netperf-eu.bufferbloat.net (ipv4) = with 4 streams down and up while pinging 80.44.96.1. Takes about 60 = seconds. >>>>>>> Download: 6.56 Mbps >>>>>>> Upload: 0.59 Mbps >>>>>>> Latency: (in msec, 62 pings, 0.00% packet loss) >>>>>>> Min: 22.975 >>>>>>> 10pct: 195.473 >>>>>>> Median: 281.756 >>>>>>> Avg: 271.609 >>>>>>> 90pct: 342.130 >>>>>>> Max: 398.573 >>>>>>>=20 >>>>>>>=20 >>>>>>> On 10/07/15 16:19, Alan Jenkins wrote: >>>>>>>> I'm glad to hear there's a working version (even if it's not in = the current build :). >>>>>>>>=20 >>>>>>>> Do you have measurable improvements with overhead configured = (v.s. unconfigured)? >>>>>>>>=20 >>>>>>>> I've used netperfrunner from CeroWrtScripts, e.g. >>>>>>>>=20 >>>>>>>> sh netperfrunner.sh -H netperf-eu.bufferbloat.net -p = $ISP_ROUTER >>>>>>>>=20 >>>>>>>> I believe accounting for overhead helps on this two-way test, = because a) it saturates the uplink b) about half that bandwidth is tiny = ack packets (depending on bandwidth asymmetry). And small packets have = proportionally high overhead. >>>>>>>>=20 >>>>>>>> (But it seems to only make a small difference for me, which = always surprises Seb). >>>>>>>>=20 >>>>>>>> Alan >>>>>>>>=20 >>>>>>>> On 10/07/15 15:52, Fred Stratton wrote: >>>>>>>>> You are absolutely correct. >>>>>>>>>=20 >>>>>>>>> I tried both a numeric overhead value, and alternatively = 'pppoe-vcmux' >>>>>>>>> and 'ether-fcs' in the build I crafted based on r46006, which = is lupin >>>>>>>>> undeclared version 2. Everything works as stated. >>>>>>>>>=20 >>>>>>>>> On lupin undeclared version 4, the current release based on = r46117, the >>>>>>>>> values were not recognised. >>>>>>>>>=20 >>>>>>>>> Thank you. >>>>>>>>>=20 >>>>>>>>> I had cake running on a Lantiq ADSL gateway running the same = r46006 >>>>>>>>> build. Unfortunately this was bricked by attempts to get = homenet >>>>>>>>> working, so I have nothing to report about gateway usage at = present. >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> On 10/07/15 13:57, Jonathan Morton wrote: >>>>>>>>>> You're already using correct syntax - I've written it to be = quite >>>>>>>>>> lenient and use sensible defaults for missing information. = There are >>>>>>>>>> several sets of keywords and parameters which are mutually = orthogonal, >>>>>>>>>> and don't depend on each other, so "besteffort" has nothing = to do with >>>>>>>>>> "overhead" or "atm". >>>>>>>>>>=20 >>>>>>>>>> What's probably happening is that you're using a slightly old = version >>>>>>>>>> of the cake kernel module which lacks the overhead parameter = entirely, >>>>>>>>>> but a more up to date tc which does support it. We've seen = this >>>>>>>>>> combination crop up ourselves recently. >>>>>>>>>>=20 >>>>>>>>>> - Jonathan Morton >>>>>>>>>>=20 >>>>>>> _______________________________________________ >>>>>>> Cerowrt-devel mailing list >>>>>>> Cerowrt-devel@lists.bufferbloat.net >>>>>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel >=20