From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 26A1921FC70 for ; Mon, 29 Jun 2015 01:33:59 -0700 (PDT) Received: from u-089-d065.biologie.uni-tuebingen.de ([134.2.89.65]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0LgptO-1YgX8U0QQ2-00oIf5; Mon, 29 Jun 2015 10:33:56 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: Date: Mon, 29 Jun 2015 10:34:05 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <8B853F1C-DE5D-4F3D-88CC-CB8DA2D3E8B1@gmx.de> <04331509-F163-4184-90B4-8589073AFD62@gmx.de> <09BA156C-460D-4794-A082-33E805F3D6FD@gmx.de> <5436B48C-0803-46DA-B355-14E917A5BB37@gmx.de> To: Mikael Abrahamsson X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:7r9NppEgoAA5F9EBToZJmp1m9Qd9UpnSrLiDOebuzpwL7aqsowO Frh5Lz/CL+0AIjd1JvZyQh3lutTJJZQj7bUE/YM54tDWPKMdF5aVZsvwPRk/KYJJVCc9deu IS0sAW51cpX+zUlaj3LUexuOOGP9V1devLuNn/ElMosbZo+sjhQuLQUa0e67OFAX1WPFWMP 091Gqeuzbn7LKo49OQBSA== X-UI-Out-Filterresults: notjunk:1;V01:K0:+F1n2NijBYQ=:ZC8pXFiYMeT41oE2bBAbpT DOit/GFTqfmnIAkrHRunWFdy98DGPKc1daSk0gx82I3Md4xwHSU1BXjnFKpiSEhpdBmY60+g/ N3Wd94nW7f+pUZXc/SpcjUMfHPYljNGpdg9FD8FnGsytnjEcpc2+8GS61uqs22VrxhJv0TbNP RimzDoSG4aDmBCk+YBJ+PQT4x2YQSgVbXC3vD0qFsLklqYMmIdjOKCLZsTJYzv4stX3kaXdHc L47WzKUAqE5Zb65P2xzNEnul/SYBV3JkardiJfkBPNnC6fjJfr8fIXlCmSUua9Lt/WDIeOwmG oxNnGt/5/wwpk6cQ1MO/0ytaaSUzg23v/TE/OUgJk9FetCAlhldYR7zz/FiaUcKl64ZvQcbLK aJghFGDKOsFzT+vMp4kdDguI8zsI0MdZoz5NMpswHEZEfCJYOF04Lx2sSBLTeX9rnb72xg4Uf KddZbLadN79RKwppOxUzbs/RXIjh8OJEt9+iUN6PUfFRddw5cgqlrFOgofo+N22b2yq0mgNmg cKGJkP+wFlHJbFQXIWG+cZ5bm2Kwh7HaaU8hsnkNuwq7NkZ36bXB7TQU2mt6NlVe0aBhMLBgB 4tt6Q5r1xXsJmcDs47ORBgzhYBtLCMwDQlakqV/Jao6g+J+0xl3biSne4DIQC1e79OXEfeTCc YFbFgA+chs5wMf6ZUHTIbGwuRahFPWyBZ7zeMzeWzFekqIFEoEWwhKvAL183anmxnvOY= Cc: "cerowrt-devel@lists.bufferbloat.net" Subject: Re: [Cerowrt-devel] performance numbers from WRT1200AC (Re: Latest build test - new sqm-scripts seem to work; "cake overhead 40" didn't) X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2015 08:34:28 -0000 HI Mikael, On Jun 29, 2015, at 10:09 , Mikael Abrahamsson wrote: > On Mon, 29 Jun 2015, Sebastian Moeller wrote: >=20 >> Ah, I see, you are still using tc=92s stab mechanism for account = of per packet overhead and link layer adjustments instead of cake=92s = (you need to check the advanced options check box in the link layer = adjustments tab, and then select =93cake=94 as the lnk layer adjustment = mechanism). >=20 > Ok, so when I changed "stab" to cake, I get the following, but it = still only shapes one way. If I change to discipline to "fq_codel" or = "pie" I get bidirectonal shaping with otherwise same settings. Probabkr breakage I introduced... >=20 > root@OpenWrt:~# cat /etc/config/sqm >=20 > config queue 'eth1' > option interface 'eth0' > option qdisc_advanced '1' > option squash_dscp '0' > option squash_ingress '0' > option ingress_ecn 'ECN' > option egress_ecn 'ECN' > option qdisc_really_really_advanced '0' > option download '50000' > option upload '50000' > option linklayer 'ethernet' > option overhead '42' > option linklayer_advanced '1' > option tcMTU '2047' > option tcTSIZE '128' > option tcMPU =910' we can ignore the last 3 savely. > option linklayer_adaptation_mechanism 'cake' > option enabled '1' > option qdisc 'cake' > option script 'simple.qos=92 Okay, it really is trying cake=92s LLA now. >=20 > root@OpenWrt:~# tc -d qdisc > qdisc cake 8002: dev eth0 root refcnt 9 bandwidth 50Mbit diffserv4 = flows noatm overhead 42 And that works on egress, excellent so bot sch_cake and tc are = recent enough. > qdisc ingress ffff: dev eth0 parent ffff:fff1 ---------------- > qdisc mq 0: dev eth1 root > qdisc fq_codel 0: dev eth1 parent :1 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :2 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :3 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :4 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :5 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :6 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :7 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev eth1 parent :8 limit 1024p flows 1024 quantum = 300 target 5.0ms interval 100.0ms ecn > qdisc fq_codel 0: dev ifb4eth0 root refcnt 2 limit 1024p flows 1024 = quantum 300 target 5.0ms interval 100.0ms ecn Oops, ingress is not right, yet. >=20 > root@OpenWrt:~# tc -d class show dev eth0 > class cake 8002:4e1 parent 8002: > class cake 8002:b89 parent 8002: >=20 > root@OpenWrt:~# tc -d class show dev ifb4eth0 > class fq_codel :12c parent none > class fq_codel :222 parent none >=20 > root@OpenWrt:~# /etc/init.d/sqm restart > SQM: Trying to start/stop SQM on all interfaces. > SQM: /usr/lib/sqm/run.sh Stopping SQM on interface: eth0 > SQM: ifb associated with interface eth0: > SQM: trying to create new IFB: ifb4eth0 > RTNETLINK answers: File exists > SQM: /usr/lib/sqm/stop.sh: Stopping eth0 > SQM: ifb associated with interface eth0: > SQM: /usr/lib/sqm/run.sh Queue Setup Script: /usr/lib/sqm/simple.qos > + . /usr/lib/sqm/functions.sh > + [ -z 50000 ] > + [ -z 50000 ] > + [ -z eth0 ] > + [ -z cake ] > + [ -z cake ] > + [ -z ethernet ] > + [ -z 42 ] > + [ -z 2047 ] > + [ -z 0 ] > + [ -z 128 ] > + [ -z ] > + AUTOFLOW=3D0 > + [ -z ] > + LIMIT=3D1001 > + [ -z ] > + ILIMIT=3D > + [ -z ] > + ELIMIT=3D > + [ -z ] > + ITARGET=3D > + [ -z ] > + ETARGET=3D > + [ -z ECN ] > + [ -z ECN ] > + [ -z 0 ] > + [ -z 0 ] > + [ -z ] > + IQDISC_OPTS=3D > + [ -z ] > + EQDISC_OPTS=3D > + [ -z ] > + which tc > + TC=3D/usr/sbin/tc > + [ -z ] > + which ip > + IP=3D/usr/sbin/ip > + [ -z ] > + which insmod > + INSMOD=3D/usr/sbin/insmod > + [ -z ] > + TARGET=3D5ms > + [ -z ] > + IPT_MASK=3D0xff > + [ -z ] > + IPT_MASK_STRING=3D/0xff > + [ -z ] > + get_ifb_for_if eth0 > + CUR_IF=3Deth0 > + get_ifb_associated_with_if eth0 > + CUR_IF=3Deth0 > + tc+ -p filtergrep -o -e ifb[^)]\+ > show parent ffff: dev eth0 > + CUR_IFB=3D > + sqm_logger ifb associated with interface eth0: > + logger -t SQM -s ifb associated with interface eth0: > SQM: ifb associated with interface eth0: > + echo > + CUR_IFB=3D > + [ -z ] > + create_new_ifb_for_if eth0 > + CUR_IF=3Deth0 > + MAX_IF_NAME_LENGTH=3D15 > + IFB_PREFIX=3Difb4 > + NEW_IFB=3Difb4eth0 > + IFB_NAME_LENGTH=3D8 > + [ 8 -gt 15 ] > + sqm_logger trying to create new IFB: ifb4eth0 > + logger -t SQM -s trying to create new IFB: ifb4eth0 > SQM: trying to create new IFB: ifb4eth0 > + /usr/sbin/ip link add name ifb4eth0 type ifb > RTNETLINK answers: File exists > + echo ifb4eth0 > + CUR_IFB=3Difb4eth0 > + [ -z ifb4eth0 ] > + echo ifb4eth0 > + DEV=3Difb4eth0 > + do_modules > + insmod act_ipt > + lsmod+ grep > -q ^act_ipt > + insmod sch_cake > + + grep -q ^sch_cake > lsmod > + insmod sch_ingress > + + greplsmod > -q ^sch_ingress > + insmod act_mirred > + + greplsmod -q ^act_mirred >=20 > + insmod cls_fw > + + grep -q ^cls_fw > lsmod > + insmod sch_htb > + + lsmodgrep -q ^sch_htb >=20 > + ipt_setup > + ipt -t mangle -N QOS_MARK_eth0 > + echo -t mangle -N QOS_MARK_eth0 > + sed s/-A/-D/g > + d=3D-t mangle -N QOS_MARK_eth0 > + [ -t mangle -N QOS_MARK_eth0 !=3D -t mangle -N QOS_MARK_eth0 ] > + echo -t mangle -N QOS_MARK_eth0 > + sed s/-I/-D/g > + d=3D-t mangle -N QOS_MARK_eth0 > + [ -t mangle -N QOS_MARK_eth0 !=3D -t mangle -N QOS_MARK_eth0 ] > + iptables -t mangle -N QOS_MARK_eth0 > + ip6tables -t mangle -N QOS_MARK_eth0 > + sqm_logger cake does all the diffserv work - no need for iptables = rules > + logger -t SQM -s cake > SQM: cake > + [ 0 =3D 1 ] > + sqm_logger Keeping differentiated services code points (DSCP) from = ingress. > + logger -t SQM -s Keeping differentiated services code points (DSCP) = from ingress. > SQM: Keeping differentiated services code points (DSCP) from ingress. > + CAKE_OPTS=3D > + ipt -t mangle -A PREROUTING -i eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + echo+ sed s/-A/-D/g > -t mangle -A PREROUTING -i eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + d=3D-t mangle -D PREROUTING -i eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + [ -t mangle -D PREROUTING -i eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 !=3D -t mangle -A PREROUTING -i eth0 -m mark --mark = 0x00/0xff -g QOS_MARK_eth0 ] > + iptables -t mangle -D PREROUTING -i eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + ip6tables -t mangle -D PREROUTING -i eth0 -m mark --mark 0x00/0xff = -g QOS_MARK_eth0 > + echo+ sed s/-I/-D/g > -t mangle -A PREROUTING -i eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + d=3D-t mangle -A PREROUTING -i eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + [ -t mangle -A PREROUTING -i eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 !=3D -t mangle -A PREROUTING -i eth0 -m mark --mark = 0x00/0xff -g QOS_MARK_eth0 ] > + iptables -t mangle -A PREROUTING -i eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + ip6tables -t mangle -A PREROUTING -i eth0 -m mark --mark 0x00/0xff = -g QOS_MARK_eth0 > + ipt -t mangle -A POSTROUTING -o eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + echo+ sed s/-A/-D/g > -t mangle -A POSTROUTING -o eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + d=3D-t mangle -D POSTROUTING -o eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + [ -t mangle -D POSTROUTING -o eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 !=3D -t mangle -A POSTROUTING -o eth0 -m mark --mark = 0x00/0xff -g QOS_MARK_eth0 ] > + iptables -t mangle -D POSTROUTING -o eth0 -m mark --mark 0x00/0xff = -g QOS_MARK_eth0 > + ip6tables -t mangle -D POSTROUTING -o eth0 -m mark --mark 0x00/0xff = -g QOS_MARK_eth0 > + echo+ sed s/-I/-D/g > -t mangle -A POSTROUTING -o eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + d=3D-t mangle -A POSTROUTING -o eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 > + [ -t mangle -A POSTROUTING -o eth0 -m mark --mark 0x00/0xff -g = QOS_MARK_eth0 !=3D -t mangle -A POSTROUTING -o eth0 -m mark --mark = 0x00/0xff -g QOS_MARK_eth0 ] > + iptables -t mangle -A POSTROUTING -o eth0 -m mark --mark 0x00/0xff = -g QOS_MARK_eth0 > + ip6tables -t mangle -A POSTROUTING -o eth0 -m mark --mark 0x00/0xff = -g QOS_MARK_eth0 > + ipt -t mangle -I PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff > + echo+ sed s/-A/-D/g > -t mangle -I PREROUTING -i vtun+ -p tcp -j MARK --set-mark 0x2/0xff > + d=3D-t mangle -I PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff > + [ -t mangle -I PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff !=3D -t mangle -I PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff ] > + echo+ sed s/-I/-D/g > -t mangle -I PREROUTING -i vtun+ -p tcp -j MARK --set-mark 0x2/0xff > + d=3D-t mangle -D PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff > + [ -t mangle -D PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff !=3D -t mangle -I PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff ] > + iptables -t mangle -D PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff > + ip6tables -t mangle -D PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff > + iptables -t mangle -I PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff > + ip6tables -t mangle -I PREROUTING -i vtun+ -p tcp -j MARK --set-mark = 0x2/0xff > + ipt -t mangle -A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP = --set-dscp-class AF42 > + echo+ sed s/-A/-D/g > -t mangle -A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP = --set-dscp-class AF42 > + d=3D-t mangle -D OUTPUT -p udp -m multiport --ports 123,53 -j DSCP = --set-dscp-class AF42 > + [ -t mangle -D OUTPUT -p udp -m multiport --ports 123,53 -j DSCP = --set-dscp-class AF42 !=3D -t mangle -A OUTPUT -p udp -m multiport = --ports 123,53 -j DSCP --set-dscp-class AF42 ] > + iptables -t mangle -D OUTPUT -p udp -m multiport --ports 123,53 -j = DSCP --set-dscp-class AF42 > + ip6tables -t mangle -D OUTPUT -p udp -m multiport --ports 123,53 -j = DSCP --set-dscp-class AF42 > + echo+ sed s/-I/-D/g > -t mangle -A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP = --set-dscp-class AF42 > + d=3D-t mangle -A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP = --set-dscp-class AF42 > + [ -t mangle -A OUTPUT -p udp -m multiport --ports 123,53 -j DSCP = --set-dscp-class AF42 !=3D -t mangle -A OUTPUT -p udp -m multiport = --ports 123,53 -j DSCP --set-dscp-class AF42 ] > + iptables -t mangle -A OUTPUT -p udp -m multiport --ports 123,53 -j = DSCP --set-dscp-class AF42 > + ip6tables -t mangle -A OUTPUT -p udp -m multiport --ports 123,53 -j = DSCP --set-dscp-class AF42 > + [ 50000 -ne 0 ] > + egress > + CEIL=3D50000 > + expr 50000 / 3 > + PRIO_RATE=3D16666 > + expr 50000 / 6 > + BE_RATE=3D8333 > + expr 50000 / 6 > + BK_RATE=3D8333 > + expr 50000 - 16 > + BE_CEIL=3D49984 > + get_mtu eth0 50000 > + BW=3D50000 > + cat /sys/class/net/eth0/mtu > + F=3D1500 > + [ -z 1500 ] > + [ 50000 -gt 20000 ] > + F=3D3000 > + [ 50000 -gt 30000 ] > + F=3D6000 > + [ 50000 -gt 40000 ] > + F=3D12000 > + [ 50000 -gt 50000 ] > + [ 50000 -gt 60000 ] > + [ 50000 -gt 80000 ] > + echo 12000 > + LQ=3Dquantum 12000 > + /usr/sbin/tc qdisc del dev eth0 root > + get_stab_string > + STABSTRING=3D > + [ cake =3D tc_stab -a ethernet !=3D none ] > + echo > + get_cake_lla_string > + STABSTRING=3D > + [ cake =3D cake -a ethernet !=3D none ] > + [ ethernet =3D atm ] > + STABSTRING=3D overhead 42 > + sqm_logger cake link layer adjustments: overhead 42 > + logger -t SQM -s cake link layer adjustments: overhead 42 > SQM: cake link layer adjustments: overhead 42 > + sqm_logger SQM: overhead 42 > + logger -t SQM -s SQM: overhead 42 > SQM: SQM: overhead 42 > + echo overhead 42 > + /usr/sbin/tc qdisc add dev eth0 root cake bandwidth 50000kbit = overhead 42 As seen above egress works. > + sqm_logger egress shaping activated > + logger -t SQM -s egress shaping activated > SQM: egress shaping activated > + [ 50000 -ne 0 ] > + ingress > + CEIL=3D50000 > + expr 50000 / 3 > + PRIO_RATE=3D16666 > + expr 50000 / 6 > + BE_RATE=3D8333 > + expr 50000 / 6 > + BK_RATE=3D8333 > + expr 50000 - 16 > + BE_CEIL=3D49984 > + get_mtu eth0 50000 > + BW=3D50000 > + cat /sys/class/net/eth0/mtu > + F=3D1500 > + [ -z 1500 ] > + [ 50000 -gt 20000 ] > + F=3D3000 > + [ 50000 -gt 30000 ] > + F=3D6000 > + [ 50000 -gt 40000 ] > + F=3D12000 > + e 50000 -gt 50000 ] > + [ 50000 -gt 60000 ] > + [ 50000 -gt 80000 ] > + echo 12000 > + LQ=3Dquantum 12000 > + /usr/sbin/tc qdisc del dev eth0 handle ffff: ingress > + /usr/sbin/tc qdisc add dev eth0 handle ffff: ingress > + /usr/sbin/tc qdisc del dev ifb4eth0 root > + [ 0 =3D 1 ] > + sqm_logger Perform DSCP based filtering on ingress. (3-tier = classification) > + logger -t SQM -s Perform DSCP based filtering on ingress. (3-tier = classification) > SQM: Perform DSCP based filtering on ingress. (3-tier classification) > + get_stab_string > + STABSTRING=3D > + [ cake =3D tc_stab -a ethernet !=3D none ] > + echo > + get_cake_lla_string > + STABSTRING=3D > + [ cake =3D cake -a ethernet !=3D none ] > + [ ethernet =3D atm ] > + STABSTRING=3D overhead 42 > + sqm_logger cake link layer adjustments: overhead 42 > + logger -t SQM -s cake link layer adjustments: overhead 42 > SQM: cake link layer adjustments: overhead 42 > + sqm_logger SQM: overhead 42 > + logger -t SQM -s SQM: overhead 42 > SQM: SQM: overhead 42 > + echo overhead 42 > + /usr/sbin/tc qdisc add dev eth0 root cake bandwidth 50000kbit = overhead 42 As seen above egress works. What the F. There is an embarrassing = copy and paste error in simple.qosline 180, instead of: $TC qdisc add dev $IFACE root `get_stab_string` $QDISC bandwidth = ${DOWNLINK}kbit `get_cake_lla_string` $CAKE_OPTS ${IQDISC_OPTS} this line should obviously read: $TC qdisc add dev $DEV root `get_stab_string` $QDISC bandwidth = ${DOWNLINK}kbit `get_cake_lla_string` $CAKE_OPTS ${IQDISC_OPTS} so simple.qos set-up egress twice, once with the egress parameters and = once with the ingress parameters, I guess I did not test 3-tier ingress = classification on Dave=92s test machine. Also I should rename a few variables. $DEV and $IFACE are not the best = names. (Also cake needs its own .qos file(s) instead of simple and = simplest becoming rather =93complex=94=85) Thanks for keeping testing = this. Best Regards Sebastian > RTNETLINK answers: File exists > + ifconfig ifb4eth0 up > + /usr/sbin/tc filter add dev eth0 parent ffff: protocol all prio 10 = u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb4eth0 > + sqm_logger ingress shaping activated > + logger -t SQM -s ingress shaping activated > SQM: ingress shaping activated >=20 > --=20 > Mikael Abrahamsson email: swmike@swm.pp.se