From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 1F5293BA8E for ; Sun, 17 Dec 2017 10:23:35 -0500 (EST) Received: from hms-beagle2.lan ([79.210.203.204]) by mail.gmx.com (mrgmx001 [212.227.17.190]) with ESMTPSA (Nemesis) id 0LtZcC-1f7SUa463o-010wKW; Sun, 17 Dec 2017 16:23:33 +0100 Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) From: Sebastian Moeller In-Reply-To: Date: Sun, 17 Dec 2017 16:23:31 +0100 Cc: cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <73C84EA7-2ADD-4914-BBDE-92E8408C106F@gmx.de> References: To: Mark Captur X-Mailer: Apple Mail (2.3273) X-Provags-ID: V03:K0:5gZHIXRwV29obT92DPF1Fdik3X9SSe0KMhNiOG10zzsoXxBvx57 r7+XqQgN5z2B9ikhvqI++Jz7MXPXloawFXm0fdW0Plp03JZpmho31nQEcVazTSHVj47afVd 9VoAIv4UePHTyoIlGIaETp7e+Z89mCenjJk2vqMjuhHVet8j+DCPO5UgRV5S9dKnjYRi07u qf1p0Hv27jZ4nOTZxsb/w== X-UI-Out-Filterresults: notjunk:1;V01:K0:Pur1ZhIRGcE=:3qnVGC4GYwfF6WrfJfeb7e 7tOPv+hthrUA+MNuCeOQvm1BK/J8h7SSOD9Ac/bcYsS4hjAPC1D4XJXprBh3Z15ppMOpuEj/x CEr2+20rHuuh/biOd74VRidAbW8prq80unYWUZwDlFqxmcjWBbtXXAQ5hA+N30uJdo4ivUZ00 SV4tcnj946bccvvcsYCSs2uejb/dcSru9/NSrMA51AnUrUvzfyk9uoct1dF1DqKVRZpZy0a3D dw5yp4+nlwV/ogBrqh2RtGOX3yO8YoN4KKwCEk8bY6cmWi+Vm0VRU7jptGK0CDKwvT4GULlBr DC3RTQHzW8GLpBQwJaO1+im63scDgaowRfMec7Zk+4wxh+biz3l6eaEGyY1J6GL+/Wy/eq8MJ 8hD/JKVsIQiTqDfR8yL8Gk9z7x1VZXCBGO4Ojmu3zog5xSpCk3APtPs15ORISXmV62AYWFIXi 2R+jIDto+Cg7du1UaDDEgrujxNu1SWpyNVe7XzT3n4Z9mWFaSUjvN7CUo5dCDbHzFwiPpz6Xj XfWz2Km7tfYe39TzBDr2xvnkRm6rmbiyZKyB08/cQxtpqNkR/YWp8+09Uz1xyKCI1pDiRsNdq dPYNToL0V49NQ7dNDviNSMCxrObMZV8e1nZhauiXNTj/YomYbAWEdebsqnFm+iI6ScKdwKPi2 CVG61H5tGWJgQgWviFuwJYYThWEAUUbAdPhhKrFtmkkJGmwrgw2Bsn6gimMd+ayb8wLlEwVdY dK2oYNTqTsJmdds65FujZSaw4L2QkKTw/jMFNP22IY/kaTHD5M8/EjQCP6wwXvFuiceQvfPX1 o4xR6/htizASxzrrlpDxMOLx9QotmE8Ng9G7YQ8LI6V0bvs9WU= Subject: Re: [Cake] overhead for double nat VDSL2 connection X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Dec 2017 15:23:35 -0000 Hi Mark, > On Dec 17, 2017, at 11:45, Mark Captur wrote: >=20 > My setup is as follows >=20 > vdsl2 modem doing pppoe itself and nat to 10.x.x.x -> lede master = eth0.2 (wan static ip in modem's DMZ) eth0.1 (lan) doing nat to = 192.168.1.x >=20 > Here is my current SQM config > config queue 'eth1' > option debug_logging '0' > option verbosity '5' > option qdisc 'cake' > option qdisc_advanced '1' > option ingress_ecn 'ECN' > option egress_ecn 'NOECN' > option qdisc_really_really_advanced '1' > option script 'layer_cake.qos' > option interface 'eth0.2' > option enabled '1' > option eqdisc_opts 'nat rtt 50000 bridged-ptm dual-srchost = diffserv4' > option upload '2400' > option linklayer 'ethernet' > option overhead '8' > option squash_dscp '1' > option squash_ingress '1' > option iqdisc_opts 'nat rtt 50000 bridged-ptm dual-dsthost' > option download '0' >=20 > config queue > option debug_logging '0' > option verbosity '5' > option download '0' > option qdisc 'cake' > option script 'layer_cake.qos' > option qdisc_advanced '1' > option squash_dscp '0' > option squash_ingress '0' > option ingress_ecn 'ECN' > option qdisc_really_really_advanced '1' > option egress_ecn 'ECN' > option interface 'eth0.1' > option enabled '1' > option eqdisc_opts 'nat rtt 50000 bridged-ptm dual-dsthost = diffserv4' > option upload '30000' > option linklayer 'ethernet' > option overhead '8' >=20 > Is the overhead correct? should i use the bridged-ptm keyword (or = should i use pppoe-ptm). The overhead certainly seems confusing. Personally, I dislike = the overhead related compound keywords like *-ptm and would recommend = the following: 1) remove the bridged-ptm from the eqdisc/iqdisc fields 2) add "mpu 64" to the eqdisc/iqdisc fields 3) set overhad to 8+18+4 =3D 30 bytes (you will need to account for = everything added on the bottleneck, so your packets will be MTU 1492, to = leave room for the PPPoE header that the modem adds). 4) DO not set the ptm keyword at all, instead make sure to set the = shaper bandwidth to <=3D sync bandwidth * 64/65 =3D sync bandwidth * = 0.984615384615 (to account for ptm's 64/65 encoding _without_ incurring = needless operations per packet). 4) tell us about your ISP and plan ;) > I tried using ATM_overhead_detector and get the following output Well, yes sorry, this will not work at all for links using PTM; = in theory VDSL links could be using ATM and still be in compliance with = the ITU specs (at lest that is my reading) but fortunately nobody seems = to be doing that. (Also more unfortunately no ISP sseems to be using PTM = encoding on ADSL links, but I digress). >=20 > 869001 lines parsed... > Found 144921 ping packets in = /home/mark/src/ATM_overhead_detector/ping_sweep__20171217_075202.txt > Elapsed time is 972.739 seconds. > Minimum size of ping payload used: 16 bytes. > warning: division by zero > warning: called from > ATM_overhead_detector at line 201 column 15 > warning: legend: ignoring extra labels > Unknown or ambiguous terminal name 'wxt' > Unknown or ambiguous terminal name 'wxt' > Saved figure (5) to: = /home/mark/src/ATM_overhead_detector/ping_sweep__20171217_075202_data.png > lower bound estimate for one ATM cell RTT based of specified up and = downlink is 0.052419 ms. > estimate for one ATM cell RTT based on linear fit of the ping sweep = data is 0.052419 ms. > Starting brute-force search for optimal stair fit, might take a = while... > Unknown or ambiguous terminal name 'wxt' > Unknown or ambiguous terminal name 'wxt' > Best staircase fit cumulative difference is: 2.6952 > Best linear fit cumulative difference is: 2.7314 > Quantized ATM carrier LIKELY (cummulative residual: stair fit 2.6952 = linear fit 2.7314 Yes, that difference is simply too small to be meaningful. I am = currently collecting reference values for these for different = technologies and it seems on real ATM/AAL5 links linear residuals are >=3D= 2 * stair residuals, but curently the code simply reports the smaller = value as more likely... > remaining ATM cell length after ICMP header is 5 bytes. > ICMP RTT of a single ATM cell is 0.05871 ms. >=20 > Estimated overhead preceding the IP header: 48 bytes > Saved figure (6) to: = /home/mark/src/ATM_overhead_detector/ping_sweep__20171217_075202_results.p= ng >=20 > According to http://ace-host.stuart.id.au/russell/files/tc/tc-atm/ > 48 bytes overhead indicate > Overhead of 0 bytes is not possible so assume 1 full packet (48 bytes) = overhead... > Connection: Bridged, LLC/SNAP+FCS RFC-1483/2684 + VLAN tag terminated = at modem > Protocol (bytes): Ethernet Header (14), VLAN tag (4), Ethernet PAD [8] = (0), Ethernet Checksum (4), ATM LLC (3), ATM SNAP (5), ATM pad (2), ATM = AAL5 SAR (8) : Total 36 >=20 > Add the following to both the egress root qdisc: > A) Assuming the router connects over ethernet to the DSL-modem: > stab mtu 2048 tsize 128 overhead 48 linklayer atm >=20 > Add the following to both the ingress root qdisc: >=20 > A) Assuming the router connects over ethernet to the DSL-modem: > stab mtu 2048 tsize 128 overhead 48 linklayer atm >=20 > Elapsed time is 1042.06 seconds. > Done... > ans =3D [](0x0) >=20 >=20 > Could my VDSL2 be using ATM? so i have the wrong overhead set? Yes in theory it could, but I am almost 100% certain it is not = doing that (AFAICT ISPs try to get rid of all ATM equipment ASAP). But I = also believe you have the wrong overhead specified... (if you insist on = the keywords try pppoe-ptm). Now, please keep in mind that you properly = need to "model" the encapsulation on the bottleneck link, if your ISP = uses a traffic shaper on the otherside of your link you need to actually = model that shapers settings. As far as I can see, ISPs will model the = per packet overhead roughly correctly, but will reduce the bandwidth = from the sync values reported by the modem, this requires a few = iterations of trial and error to get right. Also some ISPs also use VLAN = tags on the bottleneck link... Hope this helps... Best Regards Sebastian >=20 > here is my tc -s qdisc output >=20 > root@OpenWrt-Mi3G:~# tc -s qdisc > qdisc noqueue 0: dev lo root refcnt 2 > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 = quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn > Sent 3460986864 bytes 3651814 pkts (dropped 0, overlimits 0) > maxpacket 17032 drop_overlimit 0 new_flow_count 29673 ecn_mark 0 > new_flows_len 0 old_flows_len 0 > qdisc noqueue 0: dev br-lan root refcnt 2 > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > qdisc cake 8055: dev eth0.1 root refcnt 2 bandwidth 30Mbit diffserv4 = dual-dsthost nat rtt 50.0ms ptm overhead 22 via-ethernet total_overhead = 22 hard_header_len 14 > Sent 20012069 bytes 35472 pkt (dropped 10, overlimits 7587 requeues = 0) > backlog 0b 0p requeues 0 > memory used: 36684b of 4Mb > capacity estimate: 30Mbit > Bulk Best Effort Video Voice > thresh 1875Kbit 30Mbit 15Mbit 7500Kbit > target 9.7ms 2.5ms 2.5ms 2.5ms > interval 57.2ms 50.0ms 50.0ms 50.0ms > pk_delay 0us 5.6ms 855us 15us > av_delay 0us 436us 28us 10us > sp_delay 0us 27us 8us 6us > pkts 0 33679 457 1346 > bytes 0 19869595 80517 69940 > way_inds 0 249 0 0 > way_miss 0 3099 19 7 > way_cols 0 0 0 0 > drops 0 10 0 0 > marks 0 0 0 0 > ack_drop 0 0 0 0 > sp_flows 0 1 1 0 > bk_flows 0 1 0 0 > un_flows 0 0 0 0 > max_len 0 7064 2982 350 >=20 > qdisc cake 8053: dev eth0.2 root refcnt 2 bandwidth 2400Kbit diffserv4 = dual-srchost nat rtt 50.0ms ptm overhead 22 via-ethernet total_overhead = 22 hard_header_len 14 > Sent 23218772 bytes 35904 pkt (dropped 205, overlimits 27550 requeues = 0) > backlog 0b 0p requeues 0 > memory used: 94752b of 4Mb > capacity estimate: 2400Kbit > Bulk Best Effort Video Voice > thresh 150Kbit 2400Kbit 1200Kbit 600Kbit > target 121.1ms 7.6ms 15.1ms 30.3ms > interval 242.2ms 55.1ms 62.6ms 77.8ms > pk_delay 0us 29.5ms 19us 4.4ms > av_delay 0us 3.8ms 1us 2.3ms > sp_delay 0us 50us 1us 9us > pkts 0 34979 31 1099 > bytes 0 23342354 2930 162457 > way_inds 0 177 0 0 > way_miss 0 2003 27 16 > way_cols 0 0 0 0 > drops 0 205 0 0 > marks 0 0 0 0 > ack_drop 0 0 0 0 > sp_flows 0 1 0 0 > bk_flows 0 1 0 0 > un_flows 0 0 0 0 > max_len 0 8342 98 598 >=20 > qdisc noqueue 0: dev wlan1 root refcnt 2 > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > qdisc noqueue 0: dev wlan0 root refcnt 2 > Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 = quantum 1500 target 5.0ms interval 100.0ms memory_limit 4Mb ecn > Sent 304 bytes 4 pkts (dropped 0, overlimits 0) > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 > new_flows_len 0 old_flows_len 0 >=20 > Thanks, > Mark > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake