[Cake] overhead for double nat VDSL2 connection
Mark Captur
mark.captur at gmail.com
Sun Dec 17 11:00:38 EST 2017
Thanks for the comprehensive and quick reply :) below are some further
answers
On 17 December 2017 at 16:23, Sebastian Moeller <moeller0 at gmx.de> wrote:
> Hi Mark,
>
> > On Dec 17, 2017, at 11:45, Mark Captur <mark.captur at gmail.com> wrote:
> >
> > My setup is as follows
> >
> > vdsl2 modem doing pppoe itself and nat to 10.x.x.x -> lede master eth0.2
> (wan static ip in modem's DMZ) eth0.1 (lan) doing nat to 192.168.1.x
> >
> > Here is my current SQM config
> > config queue 'eth1'
> > option debug_logging '0'
> > option verbosity '5'
> > option qdisc 'cake'
> > option qdisc_advanced '1'
> > option ingress_ecn 'ECN'
> > option egress_ecn 'NOECN'
> > option qdisc_really_really_advanced '1'
> > option script 'layer_cake.qos'
> > option interface 'eth0.2'
> > option enabled '1'
> > option eqdisc_opts 'nat rtt 50000 bridged-ptm dual-srchost
> diffserv4'
> > option upload '2400'
> > option linklayer 'ethernet'
> > option overhead '8'
> > option squash_dscp '1'
> > option squash_ingress '1'
> > option iqdisc_opts 'nat rtt 50000 bridged-ptm dual-dsthost'
> > option download '0'
> >
> > config queue
> > option debug_logging '0'
> > option verbosity '5'
> > option download '0'
> > option qdisc 'cake'
> > option script 'layer_cake.qos'
> > option qdisc_advanced '1'
> > option squash_dscp '0'
> > option squash_ingress '0'
> > option ingress_ecn 'ECN'
> > option qdisc_really_really_advanced '1'
> > option egress_ecn 'ECN'
> > option interface 'eth0.1'
> > option enabled '1'
> > option eqdisc_opts 'nat rtt 50000 bridged-ptm dual-dsthost
> diffserv4'
> > option upload '30000'
> > option linklayer 'ethernet'
> > option overhead '8'
> >
> > Is the overhead correct? should i use the bridged-ptm keyword (or should
> i use pppoe-ptm).
>
> The overhead certainly seems confusing. Personally, I dislike the
> overhead related compound keywords like *-ptm and would recommend the
> following:
> 1) remove the bridged-ptm from the eqdisc/iqdisc fields
>
Done
> 2) add "mpu 64" to the eqdisc/iqdisc fields
>
Done
> 3) set overhad to 8+18+4 = 30 bytes (you will need to account for
> everything added on the bottleneck, so your packets will be MTU 1492, to
> leave room for the PPPoE header that the modem adds).
>
Dne
> 4) DO not set the ptm keyword at all, instead make sure to set the shaper
> bandwidth to <= sync bandwidth * 64/65 = sync bandwidth * 0.984615384615
> (to account for ptm's 64/65 encoding _without_ incurring needless
> operations per packet).
>
Done 0.98 (2440 is .98 of 2480) for upstream but had to reduce to .95 for
downstream as other wise ping go very high when line saturated
root at OpenWrt-Mi3G:~# cat /etc/config/sqm
config queue 'eth1'
option debug_logging '0'
option verbosity '5'
option qdisc 'cake'
option qdisc_advanced '1'
option ingress_ecn 'ECN'
option egress_ecn 'NOECN'
option qdisc_really_really_advanced '1'
option script 'layer_cake.qos'
option interface 'eth0.2'
option enabled '1'
option linklayer 'ethernet'
option squash_dscp '1'
option squash_ingress '1'
option download '0'
option upload '2440'
option overhead '30'
option iqdisc_opts 'nat rtt 50000 mpu 64 dual-dsthost'
option eqdisc_opts 'nat rtt 50000 mpu 64 dual-srchost diffserv4'
config queue
option debug_logging '0'
option verbosity '5'
option download '0'
option qdisc 'cake'
option script 'layer_cake.qos'
option qdisc_advanced '1'
option squash_dscp '0'
option squash_ingress '0'
option ingress_ecn 'ECN'
option qdisc_really_really_advanced '1'
option egress_ecn 'ECN'
option interface 'eth0.1'
option enabled '1'
option linklayer 'ethernet'
option overhead '30'
option eqdisc_opts 'nat rtt 50000 mpu 64 dual-dsthost diffserv4'
option upload '30500'
4) tell us about your ISP and plan ;)
ISP is Go Malta VDSL2 70/10, however i'm very far from cabinet so i sync at
32/2.48
>
>
>
> > I tried using ATM_overhead_detector and get the following output
>
> Well, yes sorry, this will not work at all for links using PTM; in
> theory VDSL links could be using ATM and still be in compliance with the
> ITU specs (at lest that is my reading) but fortunately nobody seems to be
> doing that. (Also more unfortunately no ISP sseems to be using PTM encoding
> on ADSL links, but I digress).
>
> >
> > 869001 lines parsed...
> > Found 144921 ping packets in /home/mark/src/ATM_overhead_
> detector/ping_sweep__20171217_075202.txt
> > Elapsed time is 972.739 seconds.
> > Minimum size of ping payload used: 16 bytes.
> > warning: division by zero
> > warning: called from
> > ATM_overhead_detector at line 201 column 15
> > warning: legend: ignoring extra labels
> > Unknown or ambiguous terminal name 'wxt'
> > Unknown or ambiguous terminal name 'wxt'
> > Saved figure (5) to: /home/mark/src/ATM_overhead_
> detector/ping_sweep__20171217_075202_data.png
> > lower bound estimate for one ATM cell RTT based of specified up and
> downlink is 0.052419 ms.
> > estimate for one ATM cell RTT based on linear fit of the ping sweep data
> is 0.052419 ms.
> > Starting brute-force search for optimal stair fit, might take a while...
> > Unknown or ambiguous terminal name 'wxt'
> > Unknown or ambiguous terminal name 'wxt'
> > Best staircase fit cumulative difference is: 2.6952
> > Best linear fit cumulative difference is: 2.7314
> > Quantized ATM carrier LIKELY (cummulative residual: stair fit 2.6952
> linear fit 2.7314
>
> Yes, that difference is simply too small to be meaningful. I am
> currently collecting reference values for these for different technologies
> and it seems on real ATM/AAL5 links linear residuals are >= 2 * stair
> residuals, but curently the code simply reports the smaller value as more
> likely...
>
>
> > remaining ATM cell length after ICMP header is 5 bytes.
> > ICMP RTT of a single ATM cell is 0.05871 ms.
> >
> > Estimated overhead preceding the IP header: 48 bytes
> > Saved figure (6) to: /home/mark/src/ATM_overhead_detector/ping_sweep__
> 20171217_075202_results.png
> >
> > According to http://ace-host.stuart.id.au/russell/files/tc/tc-atm/
> > 48 bytes overhead indicate
> > Overhead of 0 bytes is not possible so assume 1 full packet (48 bytes)
> overhead...
> > Connection: Bridged, LLC/SNAP+FCS RFC-1483/2684 + VLAN tag terminated at
> modem
> > Protocol (bytes): Ethernet Header (14), VLAN tag (4), Ethernet PAD [8]
> (0), Ethernet Checksum (4), ATM LLC (3), ATM SNAP (5), ATM pad (2), ATM
> AAL5 SAR (8) : Total 36
> >
> > Add the following to both the egress root qdisc:
> > A) Assuming the router connects over ethernet to the DSL-modem:
> > stab mtu 2048 tsize 128 overhead 48 linklayer atm
> >
> > Add the following to both the ingress root qdisc:
> >
> > A) Assuming the router connects over ethernet to the DSL-modem:
> > stab mtu 2048 tsize 128 overhead 48 linklayer atm
> >
> > Elapsed time is 1042.06 seconds.
> > Done...
> > ans = [](0x0)
> >
> >
> > Could my VDSL2 be using ATM? so i have the wrong overhead set?
>
> Yes in theory it could, but I am almost 100% certain it is not
> doing that (AFAICT ISPs try to get rid of all ATM equipment ASAP). But I
> also believe you have the wrong overhead specified... (if you insist on the
> keywords try pppoe-ptm). Now, please keep in mind that you properly need to
> "model" the encapsulation on the bottleneck link, if your ISP uses a
> traffic shaper on the otherside of your link you need to actually model
> that shapers settings. As far as I can see, ISPs will model the per packet
> overhead roughly correctly, but will reduce the bandwidth from the sync
> values reported by the modem, this requires a few iterations of trial and
> error to get right. Also some ISPs also use VLAN tags on the bottleneck
> link...
>
> Hope this helps...
>
> Best Regards
> Sebastian
>
> Here is tc output
>
root at OpenWrt-Mi3G:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum
1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 13974072314 bytes 14402910 pkts (dropped 0, overlimits 0)
maxpacket 17032 drop_overlimit 0 new_flow_count 132368 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
qdisc cake 8069: dev eth0.1 root refcnt 2 bandwidth 31500Kbit diffserv4
dual-dsthost nat rtt 50.0ms raw total_overhead 14 hard_header_len 14 mpu 64
Sent 89833020 bytes 59632 pkt (dropped 5, overlimits 44617 requeues 0)
backlog 0b 0p requeues 0
memory used: 14112b of 4Mb
capacity estimate: 31500Kbit
Bulk Best Effort Video Voice
thresh 1968Kbit 31500Kbit 15750Kbit 7875Kbit
target 9.2ms 2.5ms 2.5ms 2.5ms
interval 56.7ms 50.0ms 50.0ms 50.0ms
pk_delay 7us 4.3ms 3.5ms 3.2ms
av_delay 0us 183us 173us 271us
sp_delay 0us 7us 8us 7us
pkts 4 408 20384 38841
bytes 717 64851 30954992 58820110
way_inds 0 0 0 0
way_miss 4 64 4 11
way_cols 0 0 0 0
drops 0 0 5 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 0 0 0
bk_flows 0 0 1 0
un_flows 0 0 0 0
max_len 189 1536 2988 2988
qdisc cake 8067: dev eth0.2 root refcnt 2 bandwidth 2440Kbit diffserv4
dual-srchost nat rtt 50.0ms raw total_overhead 14 hard_header_len 14 mpu 64
Sent 3780967 bytes 33806 pkt (dropped 130, overlimits 12207 requeues 0)
backlog 10860b 9p requeues 0
memory used: 116928b of 4Mb
capacity estimate: 2440Kbit
Bulk Best Effort Video Voice
thresh 152496bit 2440Kbit 1220Kbit 610Kbit
target 119.1ms 7.4ms 14.9ms 29.8ms
interval 238.3ms 54.9ms 62.4ms 77.3ms
pk_delay 0us 101.5ms 8.2ms 7us
av_delay 0us 49.7ms 2.6ms 0us
sp_delay 0us 6.8ms 245us 0us
pkts 0 22757 11178 10
bytes 0 3029896 951692 1096
way_inds 0 0 0 0
way_miss 0 61 3 3
way_cols 0 0 0 0
drops 0 130 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 1 1 0
bk_flows 0 1 0 0
un_flows 0 0 0 0
max_len 0 11700 120 179
qdisc noqueue 0: dev wlan1 root refcnt 2
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
qdisc noqueue 0: dev wlan0 root refcnt 2
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum
1500 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 304 bytes 4 pkts (dropped 0, overlimits 0)
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
Best regards,
Mark
> >
> > here is my tc -s qdisc output
> >
> > root at OpenWrt-Mi3G:~# tc -s qdisc
> > qdisc noqueue 0: dev lo root refcnt 2
> > Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> > qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum
> 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
> > Sent 3460986864 bytes 3651814 pkts (dropped 0, overlimits 0)
> > maxpacket 17032 drop_overlimit 0 new_flow_count 29673 ecn_mark 0
> > new_flows_len 0 old_flows_len 0
> > qdisc noqueue 0: dev br-lan root refcnt 2
> > Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> > qdisc cake 8055: dev eth0.1 root refcnt 2 bandwidth 30Mbit diffserv4
> dual-dsthost nat rtt 50.0ms ptm overhead 22 via-ethernet total_overhead 22
> hard_header_len 14
> > Sent 20012069 bytes 35472 pkt (dropped 10, overlimits 7587 requeues 0)
> > backlog 0b 0p requeues 0
> > memory used: 36684b of 4Mb
> > capacity estimate: 30Mbit
> > Bulk Best Effort Video Voice
> > thresh 1875Kbit 30Mbit 15Mbit 7500Kbit
> > target 9.7ms 2.5ms 2.5ms 2.5ms
> > interval 57.2ms 50.0ms 50.0ms 50.0ms
> > pk_delay 0us 5.6ms 855us 15us
> > av_delay 0us 436us 28us 10us
> > sp_delay 0us 27us 8us 6us
> > pkts 0 33679 457 1346
> > bytes 0 19869595 80517 69940
> > way_inds 0 249 0 0
> > way_miss 0 3099 19 7
> > way_cols 0 0 0 0
> > drops 0 10 0 0
> > marks 0 0 0 0
> > ack_drop 0 0 0 0
> > sp_flows 0 1 1 0
> > bk_flows 0 1 0 0
> > un_flows 0 0 0 0
> > max_len 0 7064 2982 350
> >
> > qdisc cake 8053: dev eth0.2 root refcnt 2 bandwidth 2400Kbit diffserv4
> dual-srchost nat rtt 50.0ms ptm overhead 22 via-ethernet total_overhead 22
> hard_header_len 14
> > Sent 23218772 bytes 35904 pkt (dropped 205, overlimits 27550 requeues 0)
> > backlog 0b 0p requeues 0
> > memory used: 94752b of 4Mb
> > capacity estimate: 2400Kbit
> > Bulk Best Effort Video Voice
> > thresh 150Kbit 2400Kbit 1200Kbit 600Kbit
> > target 121.1ms 7.6ms 15.1ms 30.3ms
> > interval 242.2ms 55.1ms 62.6ms 77.8ms
> > pk_delay 0us 29.5ms 19us 4.4ms
> > av_delay 0us 3.8ms 1us 2.3ms
> > sp_delay 0us 50us 1us 9us
> > pkts 0 34979 31 1099
> > bytes 0 23342354 2930 162457
> > way_inds 0 177 0 0
> > way_miss 0 2003 27 16
> > way_cols 0 0 0 0
> > drops 0 205 0 0
> > marks 0 0 0 0
> > ack_drop 0 0 0 0
> > sp_flows 0 1 0 0
> > bk_flows 0 1 0 0
> > un_flows 0 0 0 0
> > max_len 0 8342 98 598
> >
> > qdisc noqueue 0: dev wlan1 root refcnt 2
> > Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> > qdisc noqueue 0: dev wlan0 root refcnt 2
> > Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> > qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum
> 1500 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
> > Sent 304 bytes 4 pkts (dropped 0, overlimits 0)
> > maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
> > new_flows_len 0 old_flows_len 0
> >
> > Thanks,
> > Mark
> > _______________________________________________
> > Cake mailing list
> > Cake at lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20171217/72138652/attachment-0001.html>
More information about the Cake
mailing list