From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 008EB201240 for ; Tue, 23 Sep 2014 15:16:30 -0700 (PDT) Received: from hms-beagle.home.lan ([91.50.102.81]) by mail.gmx.com (mrgmx101) with ESMTPSA (Nemesis) id 0LgdBZ-1Y1zVV18uD-00nymG; Wed, 24 Sep 2014 00:16:24 +0200 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Sebastian Moeller In-Reply-To: <5421C490.3020104@gmail.com> Date: Wed, 24 Sep 2014 00:16:20 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <541C9527.1070105@yescomputersolutions.com> <541DA8B5.70701@gmail.com> <6DF5DFA0-D88E-470E-ACB6-37703EA964E7@gmx.de> <54201F8B.70408@yescomputersolutions.com> <3136C05A-8069-4F04-B352-69A2BF3578FC@gmx.de> <5420AAA1.70203@yescomputersolutions.com> <54218D48.1020906@gmail.com> <5421C490.3020104@gmail.com> To: Andy Furniss X-Mailer: Apple Mail (2.1878.6) X-Provags-ID: V03:K0:5CfEDN1cQU8qfGNVgIzycJjyBdK1UwqpM3LxSIfG3mLCti4Y3eq 7AeDx1QVzePzxfWHg7hnUNp62aZ0qnQYLc0WexE6I3h66SVZvoVArJV0XpINvXGZJSPUZhE YEjB28Aa40nh3UI5h6BahdHdGDJL3oaaYkp/q9E+wkcA6ETFws4fPhqcGk4rIAf+YkIeeiM ZkQFvRxKZtj/W6vUhpXPA== X-UI-Out-Filterresults: notjunk:1; Cc: "lartc@vger.kernel.org" , Alan Goodman , "cerowrt-devel@lists.bufferbloat.net" Subject: Re: [Cerowrt-devel] Correctly calculating overheads on unknown connections X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Sep 2014 22:16:59 -0000 Hi Andy, On Sep 23, 2014, at 21:05 , Andy Furniss wrote: > Sebastian Moeller wrote: >> Hi Andy, >>=20 >> On Sep 23, 2014, at 17:10 , Andy Furniss >> wrote: >=20 >> BUT if you look at the kernel code, stab does not automatically >> include the ethernet overhead, so the subtract 14 in the above is >> actually wrong. See >> http://lxr.free-electrons.com/source/net/sched/sch_api.c#L538 where >> =93pkt_len =3D skb->len + stab->szopts.overhead; is used instead of = using >> =93qdisc_skb_cb(skb)->pkt_len=94 that as filled properly in >> http://lxr.free-electrons.com/source/net/core/dev.c#L2705 . At least >> to me this clearly looks like the ethernet overhead is not pre-added >> when using stab, but I could be wrong. And on an ADSL link you can >> see this quite well, with the proper overhead values sqm-scripts >> still controls the latency under netperf-wrapper=92s RRUL test nicely >> even if the shaping rate equals the line rate, with the overhead to >> small latency goes down the drain ;) >=20 > I guess skb->len varies depending on the interface. >=20 > Anyway here's a quick test on my desktop PC running a git kernel and = tc. >=20 > I used to shape remotely pppoa/vc mux dsl so know that for me >=20 > ping -s 10 .... =3D one cell and -s 11 =3D 2 cells - overhead on IP = was 10. >=20 > Paste time - >=20 > ph4[/mnt/sda8/Qos/stab-tests]# cat stab-hfsc > #set -x > TC=3D/sbin/tc >=20 > $TC qdisc del dev eth0 root &>/dev/null >=20 > if [ "$1" =3D "stop" ] > then > exit > fi >=20 > $TC qdisc add dev eth0 root handle 1: stab overhead -4 linklayer atm = hfsc default ffff > $TC class add dev eth0 parent 1: classid 1:1 hfsc sc rate 1kbit ul = rate 1kbit > $TC qdisc add dev eth0 parent 1:1 pfifo limit 200 > $TC class add dev eth0 parent 1:0 classid 1:ffff hfsc sc rate 80mbit = ul rate 80mbit >=20 > $TC filter add dev eth0 parent 1: protocol ip prio 1 \ > u32 match ip protocol 1 0xff classid 1:1 >=20 > ph4[/mnt/sda8/Qos/stab-tests]# ./stab-hfsc > ph4[/mnt/sda8/Qos/stab-tests]# ping -s 10 -c 1 noki > PING noki.andys.lan (192.168.0.1) 10(38) bytes of data. > 18 bytes from noki.andys.lan (192.168.0.1): icmp_req=3D1 ttl=3D64 >=20 > --- noki.andys.lan ping statistics --- > 1 packets transmitted, 1 received, 0% packet loss, time 0ms >=20 > ph4[/mnt/sda8/Qos/stab-tests]# tc -s qdisc ls dev eth0 > qdisc hfsc 1: root refcnt 2 default ffff > Sent 106 bytes 2 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc pfifo 8005: parent 1:1 limit 200p > Sent 53 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > ph4[/mnt/sda8/Qos/stab-tests]# ./stab-hfsc > ph4[/mnt/sda8/Qos/stab-tests]# ping -s 11 -c 1 noki > PING noki.andys.lan (192.168.0.1) 11(39) bytes of data. > 19 bytes from noki.andys.lan (192.168.0.1): icmp_req=3D1 ttl=3D64 >=20 > --- noki.andys.lan ping statistics --- > 1 packets transmitted, 1 received, 0% packet loss, time 0ms >=20 > ph4[/mnt/sda8/Qos/stab-tests]# tc -s qdisc ls dev eth0 > qdisc hfsc 1: root refcnt 2 default ffff > Sent 106 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > qdisc pfifo 8006: parent 1:1 limit 200p > Sent 106 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > ph4[/mnt/sda8/Qos/stab-tests]# Thanks for sharing your test case; I can repeat these results = exactly on my machines (I also tried htb instead hfsc for fun: same = result as to be expected see below). Looking back at http://lxr.free-electrons.com/ident?i=3Dqdisc_pkt_len_init= (line 2731): qdisc_skb_cb(skb)->pkt_len +=3D (gso_segs - 1) * hdr_len ; I begin to realize this function is not responsible for adding single = wire packet=92s ethernet header, but for figuring out in how many = on-the-wire packets to chop down a GSO packet , and add the header = overhead for the additional wire packets, I had completely looked over = the (gso-segs - 1) part, oops.=20 @cerowrt-devel: everyone using link layer ATM you might want to = try to reduce the the per packet overhead by 14=85 (but please test) So I stand corrected, you are right, tic=92s stab automatically adds the = ethernet header. So I am off to repeat my netperf-wrapper tests right = now again with overhead of 30 instead of 44, again these tests confirm = your observation. Interestingly, it seems netperf-wrapper=92s RRUL test = really is suited to figure out the overhead: while shaping to 100% of = line rate (on ADSL2+ where line rate rate is the net line rate (after = FEC)) specifying too small an overhead the ICMP latency plot shows = larger deviations from the expected unload RTT plus 10ms. Too large an = overhead however just decreases the good put bait while leaving the = latency well under control. >=20 > So it seems that overhead -4 is the correct thing to do. And thanks to your help I fully agree. >=20 > I also tested backlogged (-i 0.2) with -s 10 and 11 and tcpdump showed = the correct deltas - >=20 > ph4[/mnt/sda8/Qos/stab-tests]# tcpdump -nnttti eth0 icmp and dst host = noki >=20 > snip >=20 > 00:00:00.424000 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1345, seq 92, length 18 > 00:00:00.424000 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1345, seq 93, length 18 > 00:00:00.424000 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1345, seq 94, length 18 > 00:00:00.424000 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1345, seq 95, length 18 > 00:00:00.424000 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1345, seq 96, length 18 > 00:00:00.424001 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1345, seq 97, length 18 > 00:00:00.423999 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1345, seq 98, length 18 > 00:00:00.424000 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1347, seq 1, length 19 > 00:00:00.848000 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1347, seq 2, length 19 > 00:00:00.848001 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1347, seq 3, length 19 > 00:00:00.847999 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1347, seq 4, length 19 > 00:00:00.848000 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1347, seq 5, length 19 > 00:00:00.847999 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1347, seq 6, length 19 > 00:00:00.848002 IP 192.168.0.3 > 192.168.0.1: ICMP echo request, id = 1347, seq 7, length 19 >=20 >=20 I really appreciate your test script, thanks for taking the = time. Best Regards Sebastian =09