[Cerowrt-devel] trivial 6in4 fix(?)

[Cerowrt-devel] trivial 6in4 fix(?)

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 1488 bytes --]

The 3.8.13-6 dev non-release has a trivial patch in it that I hoped would
improve 6in4 and 6to4 tunneling performance. I'm not in a position to test
at the moment.

I am curious: Is there a current user of cero using simplest.qos, tunneling
ipv6 via hurricane or 6to4, that can hammer it with multiple ipv6 streams
(as in the rrul test)? (If you have rrul and ipv6, you can force it to do
pure ipv6 testing with the -6 option.)

During that test, the output, on 3.8.13-5 and earlier, of

tc -s class show dev ge00

I *think*, will show only a few fq_codel classes in use, rather than one
per stream. (you may need to run the command a few times to identify all
the streams in use)

It looked to me that the following patch would de-encapsulate the 6in4
stuff for purposes of the fq_codel flow hash calculation, and fix it... (I
put this patch into -6 but haven't tested)

---
 net/core/flow_dissector.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c
index 00ee068..25ebce7 100644
--- a/net/core/flow_dissector.c
+++ b/net/core/flow_dissector.c
@@ -138,6 +138,8 @@ ipv6:
                }
                break;
        }
+       case IPPROTO_IPV6:
+               proto = __constant_htons(ETH_P_IPV6);
        case IPPROTO_IPIP:
                goto again;
        default:
--
1.8.1.2

-- 
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html

[-- Attachment #2: Type: text/html, Size: 1660 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Toke Høiland-Jørgensen]

[-- Attachment #1.1: Type: text/plain, Size: 575 bytes --]

Dave Taht <dave.taht@gmail.com> writes:

> I am curious: Is there a current user of cero using simplest.qos,
> tunneling ipv6 via hurricane or 6to4, that can hammer it with multiple
> ipv6 streams (as in the rrul test)? (If you have rrul and ipv6, you
> can force it to do pure ipv6 testing with the -6 option.)

Attaching a plot of rrul over ipv6 while running cerowrt-3.8.13-17 (or
my own build based on it that, is), and some output of tc -s class show
dev ge00 while the test was running.

IPv4 pings stay flat at 45ms to the same host while the test is running.

-Toke


[-- Attachment #1.2: output of tc -s class who dev ge00 --]
[-- Type: text/plain, Size: 25194 bytes --]

root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 6535 bytes 58 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 2056bit 2pps backlog 0b 0p requeues 0 
 lended: 58 borrowed: 0 giants: 0
 tokens: 92375000 ctokens: 99695000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 12676445 bytes 23128 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 4647Kbit 1073pps backlog 0b 0p requeues 0 
 lended: 18428 borrowed: 0 giants: 0
 tokens: -529585 ctokens: -529585

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 54 bytes 1 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 1 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 12669856 bytes 23069 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 4645Kbit 1070pps backlog 0b 6p requeues 0 
 lended: 4641 borrowed: 18428 giants: 0
 tokens: -319859 ctokens: 6118529

class fq_codel 110:20e parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 5us
class fq_codel 120:283 parent 120: 
 (dropped 22, overlimits 0 requeues 0) 
 backlog 6550b 5p requeues 0 
  deficit -1070 count 1 lastcount 1 ldelay 6.8ms
class fq_codel 130:194 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 5us
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 7057 bytes 63 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 2056bit 2pps backlog 0b 0p requeues 0 
 lended: 63 borrowed: 0 giants: 0
 tokens: 93750000 ctokens: 99750000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 13368617 bytes 24145 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 4647Kbit 1073pps backlog 0b 0p requeues 0 
 lended: 19261 borrowed: 0 giants: 0
 tokens: -1067498 ctokens: -1067498

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 162 bytes 3 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 3 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 13361398 bytes 24079 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 4645Kbit 1070pps backlog 0b 12p requeues 0 
 lended: 4818 borrowed: 19261 giants: 0
 tokens: -3726660 ctokens: 4597200

class fq_codel 110:18a parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 200 count 0 lastcount 0 ldelay 7us
class fq_codel 120:283 parent 120: 
 (dropped 23, overlimits 0 requeues 0) 
 backlog 15720b 12p requeues 0 
  deficit -1160 count 1 lastcount 1 ldelay 7.6ms
class fq_codel 130:194 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 138 count 0 lastcount 0 ldelay 5us
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 7394 bytes 66 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 2056bit 2pps backlog 0b 0p requeues 0 
 lended: 66 borrowed: 0 giants: 0
 tokens: 93750000 ctokens: 99750000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 13994629 bytes 25215 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 4647Kbit 1073pps backlog 0b 0p requeues 0 
 lended: 20110 borrowed: 0 giants: 0
 tokens: -2087546 ctokens: -2087546

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 162 bytes 3 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 3 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 13987073 bytes 25146 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 4645Kbit 1070pps backlog 0b 8p requeues 0 
 lended: 5036 borrowed: 20110 giants: 0
 tokens: -6542029 ctokens: 3625742

class fq_codel 110:215 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 200 count 0 lastcount 0 ldelay 7us
class fq_codel 120:100 parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 1725b 2p requeues 0 
  deficit -1012 count 0 lastcount 0 ldelay 17.6ms
class fq_codel 120:283 parent 120: 
 (dropped 24, overlimits 0 requeues 0) 
 backlog 7860b 6p requeues 0 
  deficit -1270 count 1 lastcount 1 ldelay 15.5ms
class fq_codel 130:194 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 138 count 0 lastcount 0 ldelay 5us
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 9289 bytes 83 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 3160bit 4pps backlog 0b 0p requeues 0 
 lended: 83 borrowed: 0 giants: 0
 tokens: 92375000 ctokens: 99695000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 17012065 bytes 30290 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 5701Kbit 1244pps backlog 0b 0p requeues 0 
 lended: 24212 borrowed: 0 giants: 0
 tokens: -60856 ctokens: -60856

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 216 bytes 4 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 104bit 0pps backlog 0b 0p requeues 0 
 lended: 4 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 17002560 bytes 30203 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 5697Kbit 1241pps backlog 0b 0p requeues 0 
 lended: 5991 borrowed: 24212 giants: 0
 tokens: -879783 ctokens: 6588768

class fq_codel 110:5b parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 5us
class fq_codel 120:243 parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit -848 count 0 lastcount 0 ldelay 6us
class fq_codel 120:283 parent 120: 
 (dropped 31, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit -1110 count 1 lastcount 1 ldelay 12us
class fq_codel 130:3fc parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 6us
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 12204 bytes 109 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 3472bit 4pps backlog 0b 0p requeues 0 
 lended: 109 borrowed: 0 giants: 0
 tokens: 93750000 ctokens: 99750000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 22723747 bytes 40432 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 6398Kbit 1415pps backlog 0b 0p requeues 0 
 lended: 32342 borrowed: 0 giants: 0
 tokens: 1272159 ctokens: 1272159

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 270 bytes 5 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 80bit 0pps backlog 0b 0p requeues 0 
 lended: 5 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 22711273 bytes 40318 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 6394Kbit 1410pps backlog 0b 0p requeues 0 
 lended: 7976 borrowed: 32342 giants: 0
 tokens: -159693 ctokens: 7938759

class fq_codel 110:305 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 200 count 0 lastcount 0 ldelay 5us
class fq_codel 120:3a parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit -339 count 0 lastcount 0 ldelay 6us
class fq_codel 120:240 parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 88 count 0 lastcount 0 ldelay 11us
class fq_codel 120:247 parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 227 count 0 lastcount 0 ldelay 11us
class fq_codel 120:283 parent 120: 
 (dropped 44, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit -1160 count 1 lastcount 1 ldelay 49us
class fq_codel 130:186 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 7us
root@guardian:/etc# uname -a
Linux guardian 3.8.13 #2 Sat Jun 15 21:51:23 CEST 2013 mips GNU/Linux
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 66385 bytes 609 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 6720bit 8pps backlog 0b 0p requeues 0 
 lended: 609 borrowed: 0 giants: 0
 tokens: 92375000 ctokens: 99695000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 79121329 bytes 134071 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8419Kbit 1755pps backlog 0b 0p requeues 0 
 lended: 106748 borrowed: 0 giants: 0
 tokens: -1180986 ctokens: -1180986

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 2916 bytes 54 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 304bit 1pps backlog 0b 0p requeues 0 
 lended: 54 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 79052028 bytes 133408 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8412Kbit 1747pps backlog 0b 84p requeues 0 
 lended: 26660 borrowed: 106748 giants: 0
 tokens: -6540153 ctokens: 1627521

class fq_codel 110:d3 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 7us
class fq_codel 120:187 parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 54b 1p requeues 0 
  deficit 300 count 0 lastcount 0 ldelay 10us
class fq_codel 120:240 parent 120: 
 (dropped 10, overlimits 0 requeues 0) 
 backlog 57450b 75p requeues 0 
  deficit -39 count 10 lastcount 1 ldelay 131.8ms dropping drop_next 16.6ms
class fq_codel 120:243 parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 52 count 0 lastcount 0 ldelay 1.8ms
class fq_codel 120:283 parent 120: 
 (dropped 141, overlimits 0 requeues 0) 
 backlog 11790b 9p requeues 0 
  deficit -1190 count 2 lastcount 1 ldelay 23.1ms
class fq_codel 130:388 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 6us
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 71348 bytes 655 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 6424bit 7pps backlog 0b 0p requeues 0 
 lended: 655 borrowed: 0 giants: 0
 tokens: 92375000 ctokens: 99695000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 86609356 bytes 148889 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8638Kbit 1872pps backlog 0b 0p requeues 0 
 lended: 118659 borrowed: 0 giants: 0
 tokens: -77459 ctokens: -77459

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 3024 bytes 56 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 256bit 1pps backlog 0b 0p requeues 0 
 lended: 56 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 86534984 bytes 148178 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8632Kbit 1864pps backlog 0b 0p requeues 0 
 lended: 29519 borrowed: 118659 giants: 0
 tokens: -4255928 ctokens: 6565085

class fq_codel 110:2a1 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 9us
class fq_codel 120:240 parent 120: 
 (dropped 74, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit -64 count 14 lastcount 2 ldelay 300us
class fq_codel 130:388 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 6us
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 73491 bytes 674 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 6304bit 7pps backlog 0b 0p requeues 0 
 lended: 674 borrowed: 0 giants: 0
 tokens: 92375000 ctokens: 99695000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 90161832 bytes 155542 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8523Kbit 1914pps backlog 0b 0p requeues 0 
 lended: 124264 borrowed: 0 giants: 0
 tokens: -2101817 ctokens: -2101817

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 3024 bytes 56 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 216bit 1pps backlog 0b 0p requeues 0 
 lended: 56 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 90085317 bytes 154812 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8517Kbit 1906pps backlog 0b 21p requeues 0 
 lended: 30548 borrowed: 124264 giants: 0
 tokens: -7358286 ctokens: 3454779

class fq_codel 110:5b parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 5us
class fq_codel 120:1ee parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 10Kb 7p requeues 0 
  deficit 258 count 0 lastcount 0 ldelay 11.2ms
class fq_codel 120:243 parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 8844b 6p requeues 0 
  deficit -1396 count 0 lastcount 0 ldelay 12.4ms
class fq_codel 120:283 parent 120: 
 (dropped 167, overlimits 0 requeues 0) 
 backlog 10480b 8p requeues 0 
  deficit -520 count 1 lastcount 1 ldelay 3.5ms
class fq_codel 130:388 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 6us
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 78774 bytes 723 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 5728bit 7pps backlog 0b 0p requeues 0 
 lended: 723 borrowed: 0 giants: 0
 tokens: 92375000 ctokens: 99695000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 99552075 bytes 173612 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8876Kbit 2044pps backlog 0b 0p requeues 0 
 lended: 139481 borrowed: 0 giants: 0
 tokens: -1060463 ctokens: -1060463

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 3456 bytes 64 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 312bit 1pps backlog 0b 0p requeues 0 
 lended: 64 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 99469845 bytes 172825 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8870Kbit 2037pps backlog 0b 50p requeues 0 
 lended: 33344 borrowed: 139481 giants: 0
 tokens: -482095 ctokens: 3747882

class fq_codel 110:157 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 5us
class fq_codel 120:23b parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 413us
class fq_codel 120:240 parent 120: 
 (dropped 126, overlimits 0 requeues 0) 
 backlog 33949b 49p requeues 0 
  deficit -263 count 3 lastcount 1 ldelay 85.5ms dropping drop_next 14.5ms
class fq_codel 120:283 parent 120: 
 (dropped 190, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit -1270 count 1 lastcount 1 ldelay 16.0ms
class fq_codel 120:3d6 parent 120: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 117us
class fq_codel 130:3c7 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 5us
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 93507 bytes 859 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 6328bit 7pps backlog 0b 0p requeues 0 
 lended: 859 borrowed: 0 giants: 0
 tokens: 95375000 ctokens: 99815000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 120117860 bytes 212556 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 9114Kbit 2142pps backlog 0b 0p requeues 0 
 lended: 171646 borrowed: 0 giants: 0
 tokens: -1070617 ctokens: -1070617

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 4050 bytes 75 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 304bit 1pps backlog 0b 0p requeues 0 
 lended: 75 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 120020303 bytes 211622 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 9107Kbit 2134pps backlog 0b 71p requeues 0 
 lended: 39976 borrowed: 171646 giants: 0
 tokens: -563372 ctokens: 785092

class fq_codel 110:2a6 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 226 count 0 lastcount 0 ldelay 5us
class fq_codel 120:240 parent 120: 
 (dropped 256, overlimits 0 requeues 0) 
 backlog 25943b 64p requeues 0 
  deficit -353 count 7 lastcount 1 ldelay 85.0ms dropping drop_next 5.7ms
class fq_codel 120:283 parent 120: 
 (dropped 244, overlimits 0 requeues 0) 
 backlog 9170b 7p requeues 0 
  deficit -1040 count 1 lastcount 1 ldelay 9.1ms dropping drop_next 73.4ms
class fq_codel 130:235 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 138 count 0 lastcount 0 ldelay 6us
root@guardian:/etc# tc -s class show dev ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 103574 bytes 960 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 7760bit 9pps backlog 0b 0p requeues 0 
 lended: 960 borrowed: 0 giants: 0
 tokens: 92375000 ctokens: 99695000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 129654777 bytes 230791 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8936Kbit 2121pps backlog 0b 0p requeues 0 
 lended: 186547 borrowed: 0 giants: 0
 tokens: -1085285 ctokens: -1085285

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 4482 bytes 83 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 336bit 1pps backlog 0b 0p requeues 0 
 lended: 83 borrowed: 0 giants: 0
 tokens: 7730000 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 129546721 bytes 229748 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8928Kbit 2111pps backlog 0b 13p requeues 0 
 lended: 43201 borrowed: 186547 giants: 0
 tokens: -3496636 ctokens: 5153405

class fq_codel 110:b parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 200 count 0 lastcount 0 ldelay 6us
class fq_codel 110:5b parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 7us
class fq_codel 110:d8 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 200 count 0 lastcount 0 ldelay 5us
class fq_codel 110:12b parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 6us
class fq_codel 110:157 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 6us
class fq_codel 110:185 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 200 count 0 lastcount 0 ldelay 6us
class fq_codel 110:1b1 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 220 count 0 lastcount 0 ldelay 5us
class fq_codel 110:1d6 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 200 count 0 lastcount 0 ldelay 5us
class fq_codel 110:20e parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 98 count 0 lastcount 0 ldelay 7us
class fq_codel 110:252 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 163 count 0 lastcount 0 ldelay 6us
class fq_codel 110:2a1 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 5us
class fq_codel 110:2ac parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 5us
class fq_codel 110:2f8 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 6us
class fq_codel 120:240 parent 120: 
 (dropped 292, overlimits 0 requeues 0) 
 backlog 1691b 3p requeues 0 
  deficit 63 count 8 lastcount 1 ldelay 3.6ms
class fq_codel 120:283 parent 120: 
 (dropped 266, overlimits 0 requeues 0) 
 backlog 10480b 8p requeues 0 
  deficit -1270 count 1 lastcount 1 ldelay 15.6ms
class fq_codel 130:b7 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 246 count 0 lastcount 0 ldelay 7us

[-- Attachment #1.3: 6in4-encap-test.pdf --]
[-- Type: application/pdf, Size: 58286 bytes --]

[-- Attachment #1.4: rrul-2013-06-16T205145.652568.6in4_encapsulation_test_cerowrt_3_8_13_7.json.gz --]
[-- Type: application/octet-stream, Size: 42471 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Dave Taht]

I don't think I pushed out the 6in4 patch into the build. Hell, I
forgot to tag it too. Remind me to take a vacation next vacation?

If these patches aren't in your build, it looks like we are indeed
only using one class for fq_codel 6in4 traffic and thus are reverting
to nearly pure codel behavior rather than fq_codel.

I might be getting good at reading the patterns here - it looks like
this is fq_codel rather than nfq_codel?

#       target/linux/generic/patches-3.8/677-flow_dissector-Add-6in4-support-to-hash-keys.patch
#       target/linux/generic/patches-3.8/678-Handle-encapsulated-ipv6-better.patch
#       target/linux/generic/patches-3.8/678-remove-bad-timeout-logic-in-fast-recovery.patch

as

tc -s class show dev ge00 output during the test seems to show only one or two
tc classes in use.

But a better test if you are using simplest.qos, rather than
simplest.qos, would be the 4BE test.

On Sun, Jun 16, 2013 at 11:56 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Dave Taht <dave.taht@gmail.com> writes:
>
>> I am curious: Is there a current user of cero using simplest.qos,
>> tunneling ipv6 via hurricane or 6to4, that can hammer it with multiple
>> ipv6 streams (as in the rrul test)? (If you have rrul and ipv6, you
>> can force it to do pure ipv6 testing with the -6 option.)
>
> Attaching a plot of rrul over ipv6 while running cerowrt-3.8.13-17 (or
> my own build based on it that, is), and some output of tc -s class show
> dev ge00 while the test was running.
>
> IPv4 pings stay flat at 45ms to the same host while the test is running.
>
> -Toke
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Dave Taht]

ok, 3.8.13-7 committed, tagged, and pushed. Sorry about that...

If you could redo that test with simplest.qos or with the
rrul_noclassification test, it would be interesting, then with the
patches, again...

TIA.

On Sun, Jun 16, 2013 at 12:13 PM, Dave Taht <dave.taht@gmail.com> wrote:
> I don't think I pushed out the 6in4 patch into the build. Hell, I
> forgot to tag it too. Remind me to take a vacation next vacation?
>
> If these patches aren't in your build, it looks like we are indeed
> only using one class for fq_codel 6in4 traffic and thus are reverting
> to nearly pure codel behavior rather than fq_codel.
>
> I might be getting good at reading the patterns here - it looks like
> this is fq_codel rather than nfq_codel?
>
> #       target/linux/generic/patches-3.8/677-flow_dissector-Add-6in4-support-to-hash-keys.patch
> #       target/linux/generic/patches-3.8/678-Handle-encapsulated-ipv6-better.patch
> #       target/linux/generic/patches-3.8/678-remove-bad-timeout-logic-in-fast-recovery.patch
>
> as
>
> tc -s class show dev ge00 output during the test seems to show only one or two
> tc classes in use.
>
> But a better test if you are using simplest.qos, rather than
> simplest.qos, would be the 4BE test.
>
> On Sun, Jun 16, 2013 at 11:56 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> Dave Taht <dave.taht@gmail.com> writes:
>>
>>> I am curious: Is there a current user of cero using simplest.qos,
>>> tunneling ipv6 via hurricane or 6to4, that can hammer it with multiple
>>> ipv6 streams (as in the rrul test)? (If you have rrul and ipv6, you
>>> can force it to do pure ipv6 testing with the -6 option.)
>>
>> Attaching a plot of rrul over ipv6 while running cerowrt-3.8.13-17 (or
>> my own build based on it that, is), and some output of tc -s class show
>> dev ge00 while the test was running.
>>
>> IPv4 pings stay flat at 45ms to the same host while the test is running.
>>
>> -Toke
>>
>
>
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Toke Høiland-Jørgensen]

[-- Attachment #1.1: Type: text/plain, Size: 629 bytes --]

Dave Taht <dave.taht@gmail.com> writes:

> I might be getting good at reading the patterns here - it looks like
> this is fq_codel rather than nfq_codel?

It is.

> #       target/linux/generic/patches-3.8/677-flow_dissector-Add-6in4-support-to-hash-keys.patch
> #       target/linux/generic/patches-3.8/678-Handle-encapsulated-ipv6-better.patch
> #       target/linux/generic/patches-3.8/678-remove-bad-timeout-logic-in-fast-recovery.patch

Nope, those are not there.

> But a better test if you are using simplest.qos, rather than
> simplest.qos, would be the 4BE test.

Right, attached rerun of rrul_noclassification.

-Toke


[-- Attachment #1.2: tc-class-noclassification.txt --]
[-- Type: text/plain, Size: 8347 bytes --]

root@guardian:/sys/devices# tc -s class show dev  ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 1079959 bytes 9835 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 5248bit 6pps backlog 0b 0p requeues 0 
 lended: 9835 borrowed: 0 giants: 0
 tokens: 93750000 ctokens: 99750000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 1775603579 bytes 2669117 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 9283Kbit 1597pps backlog 0b 0p requeues 0 
 lended: 2173810 borrowed: 0 giants: 0
 tokens: -1229568 ctokens: -1229568

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 50394 bytes 933 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 288bit 1pps backlog 0b 0p requeues 0 
 lended: 933 borrowed: 0 giants: 0
 tokens: 7534736 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 1774473226 bytes 2658349 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 9277Kbit 1590pps backlog 0b 19p requeues 0 
 lended: 484539 borrowed: 2173810 giants: 0
 tokens: -316518 ctokens: 1454963

class fq_codel 110:1d7 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 104 count 0 lastcount 0 ldelay 5us
class fq_codel 110:335 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 200 count 0 lastcount 0 ldelay 5us
class fq_codel 110:3b3 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 6us
class fq_codel 110:3fd parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 5us
class fq_codel 120:145 parent 120: 
 (dropped 44, overlimits 0 requeues 0) 
 backlog 1584b 2p requeues 0 
  deficit -912 count 1 lastcount 1 ldelay 8.7ms
class fq_codel 120:16c parent 120: 
 (dropped 1802, overlimits 0 requeues 0) 
 backlog 3Kb 15p requeues 0 
  deficit -1193 count 12 lastcount 6 ldelay 29.9ms dropping drop_next 16.6ms
class fq_codel 120:1f5 parent 120: 
 (dropped 5283, overlimits 0 requeues 0) 
 backlog 1310b 1p requeues 0 
  deficit -520 count 2 lastcount 2 ldelay 5.3ms
class fq_codel 120:399 parent 120: 
 (dropped 51, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit -912 count 1 lastcount 1 ldelay 7.7ms
class fq_codel 120:3d9 parent 120: 
 (dropped 77, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit -834 count 1 lastcount 1 ldelay 6.8ms
class fq_codel 130:2e parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 138 count 0 lastcount 0 ldelay 3us
class fq_codel 130:b7 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 192 count 0 lastcount 0 ldelay 4us
class fq_codel 130:125 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 222 count 0 lastcount 0 ldelay 9us
root@guardian:/sys/devices# tc -s class show dev  ge00
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 3200Kbit burst 1600b cburst 40000b 
 Sent 1086584 bytes 9896 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 5520bit 6pps backlog 0b 0p requeues 0 
 lended: 9896 borrowed: 0 giants: 0
 tokens: 93750000 ctokens: 99750000

class htb 1:1 root rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 1786678063 bytes 2684508 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 9251Kbit 1615pps backlog 0b 0p requeues 0 
 lended: 2186458 borrowed: 0 giants: 0
 tokens: 36760 ctokens: 36760

class htb 1:10 parent 1:1 prio 0 rate 9600Kbit ceil 9600Kbit burst 1598b cburst 1598b 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0 
 lended: 0 borrowed: 0 giants: 0
 tokens: 20828 ctokens: 20828

class htb 1:13 parent 1:1 leaf 130: prio 3 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 50394 bytes 933 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 224bit 1pps backlog 0b 0p requeues 0 
 lended: 933 borrowed: 0 giants: 0
 tokens: 7534736 ctokens: 7954698

class htb 1:12 parent 1:1 leaf 120: prio 2 rate 1600Kbit ceil 9536Kbit burst 1600b cburst 9536b 
 Sent 1785541085 bytes 2673679 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 9245Kbit 1608pps backlog 0b 16p requeues 0 
 lended: 487221 borrowed: 2186458 giants: 0
 tokens: -4446710 ctokens: -1236283

class fq_codel 110:60 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 6us
class fq_codel 110:6e parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 231 count 0 lastcount 0 ldelay 11us
class fq_codel 110:fd parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 52 count 0 lastcount 0 ldelay 4us
class fq_codel 110:125 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 234 count 0 lastcount 0 ldelay 5us
class fq_codel 110:13e parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 6us
class fq_codel 110:15e parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 225 count 0 lastcount 0 ldelay 10us
class fq_codel 110:177 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 200 count 0 lastcount 0 ldelay 8us
class fq_codel 110:1d2 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 225 count 0 lastcount 0 ldelay 6us
class fq_codel 110:1d7 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 52 count 0 lastcount 0 ldelay 4us
class fq_codel 110:20e parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 6us
class fq_codel 110:28e parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 100 count 0 lastcount 0 ldelay 6us
class fq_codel 110:316 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 225 count 0 lastcount 0 ldelay 10us
class fq_codel 110:34c parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 231 count 0 lastcount 0 ldelay 9us
class fq_codel 110:368 parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 225 count 0 lastcount 0 ldelay 11us
class fq_codel 110:37c parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 6us
class fq_codel 110:39c parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 10us
class fq_codel 110:3fd parent 110: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 178 count 0 lastcount 0 ldelay 5us
class fq_codel 120:10a parent 120: 
 (dropped 19, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 286 count 1 lastcount 1 ldelay 186us
class fq_codel 120:16c parent 120: 
 (dropped 1972, overlimits 0 requeues 0) 
 backlog 1169b 11p requeues 0 
  deficit -1324 count 3 lastcount 3 ldelay 37.8ms dropping drop_next 34.6ms
class fq_codel 120:1f5 parent 120: 
 (dropped 5333, overlimits 0 requeues 0) 
 backlog 4150b 5p requeues 0 
  deficit -840 count 2 lastcount 2 ldelay 12.3ms dropping drop_next 38.2ms
class fq_codel 130:2e parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 138 count 0 lastcount 0 ldelay 3us
class fq_codel 130:b7 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 192 count 0 lastcount 0 ldelay 4us
class fq_codel 130:125 parent 130: 
 (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
  deficit 222 count 0 lastcount 0 ldelay 9us

[-- Attachment #1.3: rrul_noclassification-2013-06-16T211921.304317.6in4_encapsulation_test_cerowrt_3_8_13_7.json.gz --]
[-- Type: application/octet-stream, Size: 42213 bytes --]

[-- Attachment #1.4: 6in4-encap-test-noclassification.pdf --]
[-- Type: application/pdf, Size: 55707 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Dave Taht]

You have this huge latency spike late in your test... (?)

On Sun, Jun 16, 2013 at 11:56 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Dave Taht <dave.taht@gmail.com> writes:
>
>> I am curious: Is there a current user of cero using simplest.qos,
>> tunneling ipv6 via hurricane or 6to4, that can hammer it with multiple
>> ipv6 streams (as in the rrul test)? (If you have rrul and ipv6, you
>> can force it to do pure ipv6 testing with the -6 option.)
>
> Attaching a plot of rrul over ipv6 while running cerowrt-3.8.13-17 (or
> my own build based on it that, is), and some output of tc -s class show
> dev ge00 while the test was running.
>
> IPv4 pings stay flat at 45ms to the same host while the test is running.
>
> -Toke
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Dave Taht]

simplest.qos would be less noisy. ;)

On Sun, Jun 16, 2013 at 12:21 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Dave Taht <dave.taht@gmail.com> writes:
>
>> I might be getting good at reading the patterns here - it looks like
>> this is fq_codel rather than nfq_codel?
>
> It is.

ns2_codel is "tighter"

>
>> #       target/linux/generic/patches-3.8/677-flow_dissector-Add-6in4-support-to-hash-keys.patch
>> #       target/linux/generic/patches-3.8/678-Handle-encapsulated-ipv6-better.patch
>> #       target/linux/generic/patches-3.8/678-remove-bad-timeout-logic-in-fast-recovery.patch
>
> Nope, those are not there.

Committed, pushed.

>> But a better test if you are using simplest.qos, rather than
>> simplest.qos, would be the 4BE test.

Well, that result is mildly puzzling. netperf-wrapper -6 throughout? no ipv4?

You are on a dsl line, too? There has been some fixes to the overhead
issue that have landed but encapsulation atm is still borked (you
using atm?)

>
> Right, attached rerun of rrul_noclassification.
>
> -Toke
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 207 bytes --]

Dave Taht <dave.taht@gmail.com> writes:

> You have this huge latency spike late in your test... (?)

Yeah, not sure why. I must add that this is with other traffic running
in the background, though.

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 474 bytes --]

Dave Taht <dave.taht@gmail.com> writes:

> Well, that result is mildly puzzling. netperf-wrapper -6 throughout?
> no ipv4?

There's some ipv4 traffic in the background. Dunno exactly how much.

> You are on a dsl line, too? There has been some fixes to the overhead
> issue that have landed but encapsulation atm is still borked (you
> using atm?)

Yeah. I'm on VDSL. No idea what encapsulation (and can't access my
isp-provided router since that is in bridge mode).

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 305 bytes --]

Toke Høiland-Jørgensen <toke@toke.dk> writes:

> There's some ipv4 traffic in the background. Dunno exactly how much.

Seems there's some torrenting going on. Will re-run the tests at some
time where the network is quiet (or where I won't have to disturb my
roommates to make it so). :)

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Sebastian Moeller]

Hi Toke,

On Jun 16, 2013, at 21:36 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Dave Taht <dave.taht@gmail.com> writes:
> 
>> Well, that result is mildly puzzling. netperf-wrapper -6 throughout?
>> no ipv4?
> 
> There's some ipv4 traffic in the background. Dunno exactly how much.
> 
>> You are on a dsl line, too? There has been some fixes to the overhead
>> issue that have landed but encapsulation atm is still borked (you
>> using atm?)
> 
> Yeah. I'm on VDSL. No idea what encapsulation (and can't access my
> isp-provided router since that is in bridge mode).

	As far as I can tell at least VDSL typically means VDSL2 and that probably means PTM instead of ATM. In essence this means you do not have to deal with ATMs 48 payload bytes per 53 byte cell transport inefficiencies. So all you need to deal with is per packet overhead. Then again I am sure you probably know that already. (Sidenote, as far as I understand (so not very far) using ATM for DSL connections with POTS service in the lower frequency range never made much sense at all, the 5 byte ATM header typically was constant and by that just ballast and the 48 byte quantization on the last mile never came with any benefits, but I digress)

Best
	Sebastian

> 
> -Toke
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 794 bytes --]

Sebastian Moeller <moeller0@gmx.de> writes:

> As far as I can tell at least VDSL typically means VDSL2 and that
> probably means PTM instead of ATM. In essence this means you do not
> have to deal with ATMs 48 payload bytes per 53 byte cell transport
> inefficiencies. So all you need to deal with is per packet overhead.
> Then again I am sure you probably know that already. (Sidenote, as far
> as I understand (so not very far) using ATM for DSL connections with
> POTS service in the lower frequency range never made much sense at
> all, the 5 byte ATM header typically was constant and by that just
> ballast and the 48 byte quantization on the last mile never came with
> any benefits, but I digress)

Right, thanks. So that means the overhead is constant per (ethernet)
package?

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Dave Taht]

On Sun, Jun 16, 2013 at 1:55 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> Sebastian Moeller <moeller0@gmx.de> writes:
>
>> As far as I can tell at least VDSL typically means VDSL2 and that
>> probably means PTM instead of ATM. In essence this means you do not
>> have to deal with ATMs 48 payload bytes per 53 byte cell transport
>> inefficiencies. So all you need to deal with is per packet overhead.
>> Then again I am sure you probably know that already. (Sidenote, as far
>> as I understand (so not very far) using ATM for DSL connections with
>> POTS service in the lower frequency range never made much sense at
>> all, the 5 byte ATM header typically was constant and by that just
>> ballast and the 48 byte quantization on the last mile never came with
>> any benefits, but I digress)
>
> Right, thanks. So that means the overhead is constant per (ethernet)
> package?

So what's the MTU on VSDL2? Is PPPOE used?

> -Toke



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 224 bytes --]

Dave Taht <dave.taht@gmail.com> writes:

> So what's the MTU on VSDL2? Is PPPOE used?

No idea what the ISP-provided modem is doing; I just get and ethernet
plug that gives me DHCP. So no PPPoE on the cerowrt device.

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Sebastian Moeller]

Hi Toke,


On Jun 16, 2013, at 22:55 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Sebastian Moeller <moeller0@gmx.de> writes:
> 
>> As far as I can tell at least VDSL typically means VDSL2 and that
>> probably means PTM instead of ATM. In essence this means you do not
>> have to deal with ATMs 48 payload bytes per 53 byte cell transport
>> inefficiencies. So all you need to deal with is per packet overhead.
>> Then again I am sure you probably know that already. (Sidenote, as far
>> as I understand (so not very far) using ATM for DSL connections with
>> POTS service in the lower frequency range never made much sense at
>> all, the 5 byte ATM header typically was constant and by that just
>> ballast and the 48 byte quantization on the last mile never came with
>> any benefits, but I digress)
> 
> Right, thanks. So that means the overhead is constant per (ethernet)
> package?

	That is my interpretation, I am still waiting for vdxl deployment in my area so I have no actual hands-on experience yet. Honestly, I think the best thing to do is not so much assume ATM or lack of ATM, but simply measure it :) (while VDSL offers PTM, it can also operate over ATM if the telco wishes, so vdsl is technically not guaranteed to be free of ATM). If you collect a large quantity of pings to the nearest IP address ouside of your control for 16 to 113 byte ping sizes (say 100 packets at each size) you should be able to see a step profile in the RTTs for an ATM carrier (with two steps) and no steps (but rather a ramp) for no PTM. 

Best
	Sebastian

> 
> -Toke

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Sebastian Moeller]

Hi Dave, hi Toke,


On Jun 16, 2013, at 22:57 , Dave Taht <dave.taht@gmail.com> wrote:

> On Sun, Jun 16, 2013 at 1:55 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> Sebastian Moeller <moeller0@gmx.de> writes:
>> 
>>> As far as I can tell at least VDSL typically means VDSL2 and that
>>> probably means PTM instead of ATM. In essence this means you do not
>>> have to deal with ATMs 48 payload bytes per 53 byte cell transport
>>> inefficiencies. So all you need to deal with is per packet overhead.
>>> Then again I am sure you probably know that already. (Sidenote, as far
>>> as I understand (so not very far) using ATM for DSL connections with
>>> POTS service in the lower frequency range never made much sense at
>>> all, the 5 byte ATM header typically was constant and by that just
>>> ballast and the 48 byte quantization on the last mile never came with
>>> any benefits, but I digress)
>> 
>> Right, thanks. So that means the overhead is constant per (ethernet)
>> package?
> 
> So what's the MTU on VSDL2? Is PPPOE used?

	Easy to figure out empirically by hand, by finding the largest ping packet size that still passes without fragmentation (see http://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_finding_optimal_mtu)

Best
	Sebastian

> 
>> -Toke
> 
> 
> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Toke Høiland-Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 1331 bytes --]

Sebastian Moeller <moeller0@gmx.de> writes:

> Honestly, I think the best thing to do is not so much assume ATM or
> lack of ATM, but simply measure it :)

Right, doing the ping test with payload sizes from 16 to 113 packets
gives me an almost completely flat ping time distribution ranging from
20.3 to 21.3 ms (see attached graphic). So probably I'm on PTM...

> Easy to figure out empirically by hand, by finding the largest ping
> packet size that still passes without fragmentation (see
> http://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_finding_optimal_mtu)

$ ping -c 1 -s $((1500-28)) -M do www.debian.org
PING www.debian.org (128.31.0.51) 1472(1500) bytes of data.
1480 bytes from senfl.debian.org (128.31.0.51): icmp_seq=1 ttl=45 time=114 ms

--- www.debian.org ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 114.522/114.522/114.522/0.000 ms

$ ping -c 1 -s $((1500-27)) -M do www.debian.org
PING www.debian.org (128.31.0.51) 1473(1501) bytes of data.
From 10.42.3.5 icmp_seq=1 Frag needed and DF set (mtu = 1500)

--- www.debian.org ping statistics ---
0 packets transmitted, 0 received, +1 errors

So the MTU seems to be 1500 bytes.

Now, how do I figure out what the PTM overhead is and feed it to HTB? :)

-Toke

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Sebastian Moeller]

Hi Toke,

On Jun 17, 2013, at 11:44 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Sebastian Moeller <moeller0@gmx.de> writes:
> 
>> Honestly, I think the best thing to do is not so much assume ATM or
>> lack of ATM, but simply measure it :)
> 
> Right, doing the ping test with payload sizes from 16 to 113 packets
> gives me an almost completely flat ping time distribution ranging from
> 20.3 to 21.3 ms (see attached graphic). So probably I'm on PTM…

	I fully believe you that it is flat (graph did not make it into my inbox…) So that looks like PTM. Good! But beware the expected step size depends on your down and uplink speeds, at VDSL I would only expect a very tiny increase (basically the time it takes to see an additional ATM cell back and forth, (RTT step per ATM cell in milliseconds = (53*8 / line.down.bit + 53*8 / line.up.bit ) * 1000); this means that potentially a large sample size per ping packet size is required to be reasonably sure that there is no step....

> 
>> Easy to figure out empirically by hand, by finding the largest ping
>> packet size that still passes without fragmentation (see
>> http://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_finding_optimal_mtu)
> 
> $ ping -c 1 -s $((1500-28)) -M do www.debian.org
> PING www.debian.org (128.31.0.51) 1472(1500) bytes of data.
> 1480 bytes from senfl.debian.org (128.31.0.51): icmp_seq=1 ttl=45 time=114 ms
> 
> --- www.debian.org ping statistics ---
> 1 packets transmitted, 1 received, 0% packet loss, time 0ms
> rtt min/avg/max/mdev = 114.522/114.522/114.522/0.000 ms
> 
> $ ping -c 1 -s $((1500-27)) -M do www.debian.org
> PING www.debian.org (128.31.0.51) 1473(1501) bytes of data.
> From 10.42.3.5 icmp_seq=1 Frag needed and DF set (mtu = 1500)
> 
> --- www.debian.org ping statistics ---
> 0 packets transmitted, 0 received, +1 errors
> 
> So the MTU seems to be 1500 bytes.

	That is great!

> 
> Now, how do I figure out what the PTM overhead is and feed it to HTB? :)

	All I know so far is that PTM will not drag in the quite baroque ATM encapsulation options. Googling for vdsl2 makes me hope that maybe there is no additional user visible overhead; so if you have PPP that would still need handling. It would be quite interesting to determine the overhead empirically. ATM's quantization makes overhead detection in atm based del lines conceptually easy; but for VDSL I am not so sure. In principle we expect to see "buffer bloat" and its signature increase of latency on saturated links if we shape with too high rates. So too small an overhead should fill the modems buffers and might increase latency (depending on the modems configuration, but assuming pfifo the buffer should just fill up slowly until latencies should be noticeably affected, or?). Hence in theory using a saturating load and measuring the latencies for different overhead values should still work. I wonder whether rrul might just be the right probe? If you go that route I would be delighted to learn the outcome :). Sorry to be of no more help here. 

Best
	Sebastian


> 
> -Toke

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Toke Høiland-Jørgensen]

[-- Attachment #1.1: Type: text/plain, Size: 1705 bytes --]

Sebastian Moeller <moeller0@gmx.de> writes:

> I fully believe you that it is flat (graph did not make it into my
> inbox…)

Heh. May have forgotten to attach it... Should be there now...

> So that looks like PTM. Good! But beware the expected step size
> depends on your down and uplink speeds, at VDSL I would only expect a
> very tiny increase (basically the time it takes to see an additional
> ATM cell back and forth, (RTT step per ATM cell in milliseconds =
> (53*8 / line.down.bit + 53*8 / line.up.bit ) * 1000); this means that
> potentially a large sample size per ping packet size is required to be
> reasonably sure that there is no step....

Right, well in my case that comes out as something like 0.05 ms, which
is way below the measuring accuracy of my ping test (lowest mdev as
reported by ping is 0.7ms; highest is 3.3). So I guess testing is not
really going to be viable in this case. But then perhaps it's not going
to make much of a difference either way in this case?

> Hence in theory using a saturating load and measuring the latencies
> for different overhead values should still work. I wonder whether rrul
> might just be the right probe? If you go that route I would be
> delighted to learn the outcome :). Sorry to be of no more help here.

Right. That seems reasonable. However, it also seems to require a bit
more testing than I really have the time to spare right now, so I think
I'll defer it for the time being. I wonder if it would be possible to
persuade my ISP to set up a netperf server to test against...

Either way, thanks for your insight; I'll be sure to ping you if I come
up with something more conclusive... :)

-Toke


[-- Attachment #1.2: pings.png --]
[-- Type: image/png, Size: 10733 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 489 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 3644 bytes --]

Hi Toke,


On Jun 17, 2013, at 12:50 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Sebastian Moeller <moeller0@gmx.de> writes:
> 
>> I fully believe you that it is flat (graph did not make it into my
>> inbox…)
> 
> Heh. May have forgotten to attach it... Should be there now…

	It is… I agree no noticeable stepping (btw. in theory taking the minimum at each ping size would be more precise, in practice for ATM the median seems to work best). The min because that is limited by wire speed and optimal processing, the median as it turned out to be less noisy...

> 
>> So that looks like PTM. Good! But beware the expected step size
>> depends on your down and uplink speeds, at VDSL I would only expect a
>> very tiny increase (basically the time it takes to see an additional
>> ATM cell back and forth, (RTT step per ATM cell in milliseconds =
>> (53*8 / line.down.bit + 53*8 / line.up.bit ) * 1000); this means that
>> potentially a large sample size per ping packet size is required to be
>> reasonably sure that there is no step....
> 
> Right, well in my case that comes out as something like 0.05 ms, which
> is way below the measuring accuracy of my ping test (lowest mdev as
> reported by ping is 0.7ms; highest is 3.3).

	I tend to agree, if interpreted as a statistical power problem at stddev 0.7, around 1000 samples should do fine (power of 72% for detecting an effect of 0.05), but at 3.3 even 20500 samples per size will only give you 70% power. To save some time the ATM quantization test can be reduced to 3 sizes: say 16 (seems to be the smallest ping size( on 64 bit systems?) that allows timestamps), 16+24 and 16+48; out of these 3 two are guaranteed to require the same number of ATM cells the other one will be either one small or one bigger. But at n ~20k this still takes too long :)

> So I guess testing is not
> really going to be viable in this case. But then perhaps it's not going
> to make much of a difference either way in this case?

	Nah it still is, as ATM will pad the unused reminder of the last cell of each packet, for small packets that will mean a considerable amount of your bandwidth is wasted on padding. On average you are going to waste half a cell, naively speaking; and for small packets that can easily be above 33%. And if you do not regard that into your shaping you will run into issues for a flood of small packets, namely you are not shaping enough and your modem will fill its buffers… Thinking of this, another war to test for ATM carrier is to shape to the nominal link speeds and see whether flooding some upstream with minimally small packets affects latency. But unlike the step method this will not give you the satisfaction of seeing the quantization in a decent plot...

> 
>> Hence in theory using a saturating load and measuring the latencies
>> for different overhead values should still work. I wonder whether rrul
>> might just be the right probe? If you go that route I would be
>> delighted to learn the outcome :). Sorry to be of no more help here.
> 
> Right. That seems reasonable. However, it also seems to require a bit
> more testing than I really have the time to spare right now, so I think
> I'll defer it for the time being. I wonder if it would be possible to
> persuade my ISP to set up a netperf server to test against…

	Well, if you talk to your ISP you could also ask them for ATM or PTM and any potential overhead :)

> 
> Either way, thanks for your insight; I'll be sure to ping you if I come
> up with something more conclusive... :)

	Great and thanks
Best
	Sebastian

> 
> -Toke
> 


[-- Attachment #2.1: Type: text/html, Size: 5015 bytes --]

[-- Attachment #2.2: pings.png --]
[-- Type: image/png, Size: 10733 bytes --]

Re: [Cerowrt-devel] trivial 6in4 fix(?)

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 5142 bytes --]

Hi Toke, hi list

Here is more on the topic of (potential) ATM quantization on a DSL link. (Note ATM typically is used for all/most ADSL1/DASL2/ADSL2+ links and might also be used with VDSL links, even though for VDSL non-cell quantized packet transfer mode (PTM) hopefully is more likely).
Anyway, so here is what I use to collect ping times to test for ATM quantization. Just replace TARGET with the nearest IP on the other side of your DSL link that returns ping packets with low variation. The only potentially clever twist here is to call ping for each packet independently and sleep for a tiny bit in-between to allow non-root to ping rates > 1Hz (based on sleep accepting non-integer inputs). (It is worth mentioning that in my tests to high frequencies to the same host led to very long ping RTTs, as if the host was putting my requests into a slow path, so for each host it might be required to titrate the lowest period to still get typical ping responses...)
I typically would let this run overnight, at 0.01 seconds PINGPERIOD period this will take 10100seconds or ~168 minutes. For my ADSL2+ link at 2558kbit uplink and 16402kbit downlink the numbers below give a very noticeable quantization step, for higher link speeds one might need to increase PINGSPERSIZE… Now, I have some reworked matlab code to parse and display the data from the ping log file, that also will attempt to estimate the ATM encapsulation related overhead on an ATM link; let me know whether there is any interest for that…

best
	sebastian



#! /bin/bash
# TODO use seq or bash to generate a list of the requested sizes (to alow for non-equdistantly spaced sizes)

# Telekom Tuebingen Moltkestrasse 6
TECH=ADSL2
# finding a proper target IP is somewhat of an art, just traceroute a remote site 
# and find the nearest host reliably responding to pings showing the smallet variation of pingtimes
TARGET=87.186.197.70		# T
DATESTR=`date +%Y%m%d_%H%M%S`	# to allow multiple sequential records
LOG=ping_sweep_${TECH}_${DATESTR}.txt


# by default non-root ping will only end one packet per second, so work around that by calling ping independently for each package
# empirically figure out the shortest period still giving the standard ping time (to avoid being slow-pathed by our host)
PINGPERIOD=0.01		# in seconds
PINGSPERSIZE=10000

# Start, needed to find the per packet overhead dependent on the ATM encapsulation
# to reiably show ATM quantization one would like to see at least two steps, so cover a range > 2 ATM cells (so > 96 bytes)
SWEEPMINSIZE=16		# 64bit systems seem to require 16 bytes of payload to include a timestamp...
SWEEPMAXSIZE=116
    

n_SWEEPS=`expr ${SWEEPMAXSIZE} - ${SWEEPMINSIZE}`


i_sweep=0
i_size=0

while [ ${i_sweep} -lt ${PINGSPERSIZE} ]
do
    (( i_sweep++ ))
    echo "Current iteration: ${i_sweep}"
    # now loop from sweepmin to sweepmax
    i_size=${SWEEPMINSIZE}
    while [ ${i_size} -le ${SWEEPMAXSIZE} ]
    do
	echo "${i_sweep}. repetition of ping size ${i_size}"
	ping -c 1 -s ${i_size} ${TARGET} >> ${LOG} &
	(( i_size++ ))
	# we need a sleep binary that allows non integer times (GNU sleep is fine as is sleep of macosx 10.8.4)
	sleep ${PINGPERIOD}
    done
done

#tail -f ${LOG}

echo "Done... ($0)




On Jun 17, 2013, at 12:50 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:

> Sebastian Moeller <moeller0@gmx.de> writes:
> 
>> I fully believe you that it is flat (graph did not make it into my
>> inbox…)
> 
> Heh. May have forgotten to attach it... Should be there now...
> 
>> So that looks like PTM. Good! But beware the expected step size
>> depends on your down and uplink speeds, at VDSL I would only expect a
>> very tiny increase (basically the time it takes to see an additional
>> ATM cell back and forth, (RTT step per ATM cell in milliseconds =
>> (53*8 / line.down.bit + 53*8 / line.up.bit ) * 1000); this means that
>> potentially a large sample size per ping packet size is required to be
>> reasonably sure that there is no step....
> 
> Right, well in my case that comes out as something like 0.05 ms, which
> is way below the measuring accuracy of my ping test (lowest mdev as
> reported by ping is 0.7ms; highest is 3.3). So I guess testing is not
> really going to be viable in this case. But then perhaps it's not going
> to make much of a difference either way in this case?
> 
>> Hence in theory using a saturating load and measuring the latencies
>> for different overhead values should still work. I wonder whether rrul
>> might just be the right probe? If you go that route I would be
>> delighted to learn the outcome :). Sorry to be of no more help here.
> 
> Right. That seems reasonable. However, it also seems to require a bit
> more testing than I really have the time to spare right now, so I think
> I'll defer it for the time being. I wonder if it would be possible to
> persuade my ISP to set up a netperf server to test against...
> 
> Either way, thanks for your insight; I'll be sure to ping you if I come
> up with something more conclusive... :)
> 
> -Toke
> 


[-- Attachment #2.1: Type: text/html, Size: 6787 bytes --]

[-- Attachment #2.2: pings.png --]
[-- Type: image/png, Size: 10733 bytes --]