[Bloat] 450 flow tests
Dave Taht
dave.taht at gmail.com
Wed Apr 22 08:59:52 PDT 2015
I wanted to exercise cake's 8 way set associative hash, so I tried to
get to where I had 450 flows going at gigE and could see collisions.
Then out of perverse curiosity, I went and looked at some other qdiscs
like pie, sch_fq, codel, ns2_codel etc. I will not claim to have been
terribly scientific...
simple netperf-wrapper script and some data against multiple qdiscs here:
http://snapon.lab.bufferbloat.net/~d/cake_gige.tgz
(what I ended up doing was comparing in netperf-wrapper against the
*_9.json.gz files)
script: t.sh
topology:
client<->cheap switch<->server (client changing qdisc), gigE ethernet
between the machines. Client running 3.18, server running 4.0. Client
is a nuc i3, server a rangeley.
9 instances of netperf-wrapper each running rrul_50_up
CPU running hard to accomplish stuff at this rate without offloads
(like 50% softirq on a dual core)
I am rather puzzled as to why cake did so well (on queue length) when
the others did not. Bytelimit? (there is also some data on trying to
shape at 920Mbit in there, with very limited success).
* Pie never gets the queue under control - the default packet limit of
1000 is hit. When upped to 10000....
root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc pie 800a: root refcnt 2 limit 10000p target 20.0ms tupdate
30.0ms alpha 2 beta 20
Sent 4064425384 bytes 3978959 pkt (dropped 33601, overlimits 0 requeues 92211)
backlog 10317916b 9869p requeues 92211
prob 0.000000 delay 1430us avg_dq_rate 3186249490
pkts_in 3988828 overlimit 33601 dropped 33601 maxq 9999 ecn_mark 0
And it returns what seems to be a bogus delay
* sch_fq does not either - that flow_limit 100p is not helpful methinks
root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc fq 800b: root refcnt 2 limit 10000p flow_limit 100p buckets 1024
quantum 3028 initial_quantum 15140
Sent 2211625843 bytes 2263178 pkt (dropped 93506, overlimits 0 requeues 45394)
backlog 14190248b 9807p requeues 45394
987 flows (518 inactive, 0 throttled)
0 gc, 0 highprio, 5407 throttled, 85636 flows_plimit
* fq_codel bounces around between about 1400-2400 packets
root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc fq_codel 800c: root refcnt 2 limit 10000p flows 1024 quantum
1514 target 5.0ms interval 100.0ms ecn
Sent 1501918165 bytes 1589097 pkt (dropped 49749, overlimits 0 requeues 29086)
backlog 1734550b 1468p requeues 29086
maxpacket 1514 drop_overlimit 0 new_flow_count 64971 ecn_mark 0
new_flows_len 0 old_flows_len 382
* Codel slowly gets things under control
peaking at 10000 packets, then gradually getting below 4000
root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc codel 800d: root refcnt 2 limit 10000p target 5.0ms interval 100.0ms
Sent 3056093465 bytes 3025051 pkt (dropped 21719, overlimits 0 requeues 67529)
backlog 4270321b 3955p requeues 67529
count 15938 lastcount 1 ldelay 34.8ms dropping drop_next 274us
maxpacket 1514 ecn_mark 0 drop_overlimit 5781
* ns2_codel fails to get things under control
root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc ns2_codel 800e: root refcnt 2 limit 10000p target 5.0ms interval 100.0ms
Sent 2634556896 bytes 2582228 pkt (dropped 17787, overlimits 0 requeues 57907)
backlog 10148425b 9861p requeues 57907
count 3029 lastcount 0 ldelay 83.2ms dropping drop_next 420us
maxpacket 0 ecn_mark 0 drop_overlimit 14758
but after a minute gets to where it is better than horrible
root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc ns2_codel 800e: root refcnt 2 limit 10000p target 5.0ms interval 100.0ms
Sent 7272860233 bytes 7068770 pkt (dropped 38109, overlimits 0 requeues 162324)
backlog 5953303b 5721p requeues 162324
count 22845 lastcount 0 ldelay 48.7ms dropping drop_next 804us
maxpacket 0 ecn_mark 0 drop_overlimit 15263
* Cake totally controls things
root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc cake 800f: root refcnt 2 unlimited diffserv4 flows
Sent 1599700032 bytes 1600134 pkt (dropped 158320, overlimits 0 requeues 26621)
backlog 116908b 82p requeues 26621
Class 0 Class 1 Class 2 Class 3
rate 0bit 0bit 0bit 0bit
target 5.0ms 5.0ms 5.0ms 5.0ms
interval 105.0ms 105.0ms 105.0ms 105.0ms
Pk delay 0us 4.4ms 0us 67us
Av delay 0us 1.8ms 0us 1us
Sp delay 0us 397us 0us 1us
pkts 0 1758527 0 9
way inds 0 57354 0 0
way miss 0 142860 0 1
way cols 0 124 0 0
bytes 0 1834269446 0 1042
drops 0 158320 0 0
marks 0 0 0 0
* As does cake flowblind, but less so
root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc cake 8010: root refcnt 2 unlimited diffserv4 flowblind
Sent 1601618284 bytes 1667391 pkt (dropped 96598, overlimits 0 requeues 30514)
backlog 260226b 258p requeues 30514
Class 0 Class 1 Class 2 Class 3
rate 0bit 0bit 0bit 0bit
target 5.0ms 5.0ms 5.0ms 5.0ms
interval 105.0ms 105.0ms 105.0ms 105.0ms
Pk delay 0us 2.4ms 0us 283us
Av delay 0us 2.1ms 0us 5us
Sp delay 0us 1.5ms 0us 5us
pkts 0 1764239 0 7
way inds 0 0 0 0
way miss 0 0 0 0
way cols 0 0 0 0
bytes 0 1698245972 0 742
drops 0 96598 0 0
marks 0 0 0 0
--
Dave Täht
Open Networking needs **Open Source Hardware**
https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67
More information about the Bloat
mailing list