[Cake] 450 flow tests

Dave Taht dave.taht at gmail.com
Wed Apr 22 11:59:52 EDT 2015


I wanted to exercise cake's 8 way set associative hash, so I tried to
get to where I had 450 flows going at gigE and could see collisions.
Then out of perverse curiosity, I went and looked at some other qdiscs
like pie, sch_fq, codel, ns2_codel etc. I will not claim to have been
terribly scientific...

simple netperf-wrapper script and some data against multiple qdiscs here:

http://snapon.lab.bufferbloat.net/~d/cake_gige.tgz

(what I ended up doing was comparing in netperf-wrapper against the
*_9.json.gz files)

script: t.sh

topology:

client<->cheap switch<->server (client changing qdisc), gigE ethernet
between the machines. Client running 3.18, server running 4.0. Client
is a nuc i3, server a rangeley.

9 instances of netperf-wrapper each running rrul_50_up

CPU running hard to accomplish stuff at this rate without offloads
(like 50% softirq on a dual core)

I am rather puzzled as to why cake did so well (on queue length) when
the others did not. Bytelimit? (there is also some data on trying to
shape at 920Mbit in there, with very limited success).

* Pie never gets the queue under control - the default packet limit of
1000 is hit. When upped to 10000....

root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc pie 800a: root refcnt 2 limit 10000p target 20.0ms tupdate
30.0ms alpha 2 beta 20
 Sent 4064425384 bytes 3978959 pkt (dropped 33601, overlimits 0 requeues 92211)
 backlog 10317916b 9869p requeues 92211
prob 0.000000 delay 1430us avg_dq_rate 3186249490
pkts_in 3988828 overlimit 33601 dropped 33601 maxq 9999 ecn_mark 0

And it returns what seems to be a bogus delay

* sch_fq does not either - that flow_limit 100p is not helpful methinks

root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc fq 800b: root refcnt 2 limit 10000p flow_limit 100p buckets 1024
quantum 3028 initial_quantum 15140
 Sent 2211625843 bytes 2263178 pkt (dropped 93506, overlimits 0 requeues 45394)
 backlog 14190248b 9807p requeues 45394
  987 flows (518 inactive, 0 throttled)
  0 gc, 0 highprio, 5407 throttled, 85636 flows_plimit

* fq_codel bounces around between about 1400-2400 packets

root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc fq_codel 800c: root refcnt 2 limit 10000p flows 1024 quantum
1514 target 5.0ms interval 100.0ms ecn
 Sent 1501918165 bytes 1589097 pkt (dropped 49749, overlimits 0 requeues 29086)
 backlog 1734550b 1468p requeues 29086
  maxpacket 1514 drop_overlimit 0 new_flow_count 64971 ecn_mark 0
  new_flows_len 0 old_flows_len 382

* Codel slowly gets things under control
peaking at 10000 packets, then gradually getting below 4000

root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc codel 800d: root refcnt 2 limit 10000p target 5.0ms interval 100.0ms
 Sent 3056093465 bytes 3025051 pkt (dropped 21719, overlimits 0 requeues 67529)
 backlog 4270321b 3955p requeues 67529
  count 15938 lastcount 1 ldelay 34.8ms dropping drop_next 274us
  maxpacket 1514 ecn_mark 0 drop_overlimit 5781

* ns2_codel fails to get things under control

root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc ns2_codel 800e: root refcnt 2 limit 10000p target 5.0ms interval 100.0ms
 Sent 2634556896 bytes 2582228 pkt (dropped 17787, overlimits 0 requeues 57907)
 backlog 10148425b 9861p requeues 57907
  count 3029 lastcount 0 ldelay 83.2ms dropping drop_next 420us
  maxpacket 0 ecn_mark 0 drop_overlimit 14758

but after a minute gets to where it is better than horrible

root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc ns2_codel 800e: root refcnt 2 limit 10000p target 5.0ms interval 100.0ms
 Sent 7272860233 bytes 7068770 pkt (dropped 38109, overlimits 0 requeues 162324)
 backlog 5953303b 5721p requeues 162324
  count 22845 lastcount 0 ldelay 48.7ms dropping drop_next 804us
  maxpacket 0 ecn_mark 0 drop_overlimit 15263

* Cake totally controls things

root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc cake 800f: root refcnt 2 unlimited diffserv4 flows
 Sent 1599700032 bytes 1600134 pkt (dropped 158320, overlimits 0 requeues 26621)
 backlog 116908b 82p requeues 26621
           Class 0     Class 1     Class 2     Class 3
  rate          0bit        0bit        0bit        0bit
  target       5.0ms       5.0ms       5.0ms       5.0ms
interval     105.0ms     105.0ms     105.0ms     105.0ms
Pk delay         0us       4.4ms         0us        67us
Av delay         0us       1.8ms         0us         1us
Sp delay         0us       397us         0us         1us
  pkts             0     1758527           0           9
way inds           0       57354           0           0
way miss           0      142860           0           1
way cols           0         124           0           0
  bytes            0  1834269446           0        1042
  drops            0      158320           0           0
  marks            0           0           0           0

* As does cake flowblind, but less so

root at nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0
qdisc cake 8010: root refcnt 2 unlimited diffserv4 flowblind
 Sent 1601618284 bytes 1667391 pkt (dropped 96598, overlimits 0 requeues 30514)
 backlog 260226b 258p requeues 30514
           Class 0     Class 1     Class 2     Class 3
  rate          0bit        0bit        0bit        0bit
  target       5.0ms       5.0ms       5.0ms       5.0ms
interval     105.0ms     105.0ms     105.0ms     105.0ms
Pk delay         0us       2.4ms         0us       283us
Av delay         0us       2.1ms         0us         5us
Sp delay         0us       1.5ms         0us         5us
  pkts             0     1764239           0           7
way inds           0           0           0           0
way miss           0           0           0           0
way cols           0           0           0           0
  bytes            0  1698245972           0         742
  drops            0       96598           0           0
  marks            0           0           0           0

-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67



More information about the Cake mailing list