From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-x236.google.com (mail-ob0-x236.google.com [IPv6:2607:f8b0:4003:c01::236]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 6C20F21F2D2; Wed, 22 Apr 2015 08:59:53 -0700 (PDT) Received: by obcux3 with SMTP id ux3so69147237obc.2; Wed, 22 Apr 2015 08:59:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=41e4m9TbCny8lW9qRoYe2oPZQZMh9h83eI2yiSy7oEI=; b=MTDiUGc02QJlVqoFLWONjoULf6+NKjxbWoGuCCryjcSYcgfU02YnvgumXQmlrBDe7c GmZRkCIu8dJx8imTaTkwZt2F7kKr5zh2lRRU9ZySm9R7GnowXbVNDDxH4tiWDeWAWnNF k4y8XBkMDxZAoQZUvukhWaIf+Qr2yMS7kkUKQfOXlCxq+qLz8jOer8S36LQvDRoL0kWB PbAzlMUQVNGDnjNRoD3CLqjC3zTa5nZsv5jOvv6qt5MxAAcJ5IMmnlf/pnGfQe2RlptU oOfQ4dBMwkE0WX4lcwSZvkNAx6IV6NLkfnk/TiddvuaQr8lQhH5LOC0cfZ1eQZZSHbmI shgw== MIME-Version: 1.0 X-Received: by 10.60.223.228 with SMTP id qx4mr24449850oec.24.1429718392342; Wed, 22 Apr 2015 08:59:52 -0700 (PDT) Received: by 10.202.71.139 with HTTP; Wed, 22 Apr 2015 08:59:52 -0700 (PDT) Date: Wed, 22 Apr 2015 08:59:52 -0700 Message-ID: From: Dave Taht To: cake@lists.bufferbloat.net, bloat Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: [Cake] 450 flow tests X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Apr 2015 16:00:21 -0000 I wanted to exercise cake's 8 way set associative hash, so I tried to get to where I had 450 flows going at gigE and could see collisions. Then out of perverse curiosity, I went and looked at some other qdiscs like pie, sch_fq, codel, ns2_codel etc. I will not claim to have been terribly scientific... simple netperf-wrapper script and some data against multiple qdiscs here: http://snapon.lab.bufferbloat.net/~d/cake_gige.tgz (what I ended up doing was comparing in netperf-wrapper against the *_9.json.gz files) script: t.sh topology: client<->cheap switch<->server (client changing qdisc), gigE ethernet between the machines. Client running 3.18, server running 4.0. Client is a nuc i3, server a rangeley. 9 instances of netperf-wrapper each running rrul_50_up CPU running hard to accomplish stuff at this rate without offloads (like 50% softirq on a dual core) I am rather puzzled as to why cake did so well (on queue length) when the others did not. Bytelimit? (there is also some data on trying to shape at 920Mbit in there, with very limited success). * Pie never gets the queue under control - the default packet limit of 1000 is hit. When upped to 10000.... root@nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0 qdisc pie 800a: root refcnt 2 limit 10000p target 20.0ms tupdate 30.0ms alpha 2 beta 20 Sent 4064425384 bytes 3978959 pkt (dropped 33601, overlimits 0 requeues 92= 211) backlog 10317916b 9869p requeues 92211 prob 0.000000 delay 1430us avg_dq_rate 3186249490 pkts_in 3988828 overlimit 33601 dropped 33601 maxq 9999 ecn_mark 0 And it returns what seems to be a bogus delay * sch_fq does not either - that flow_limit 100p is not helpful methinks root@nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0 qdisc fq 800b: root refcnt 2 limit 10000p flow_limit 100p buckets 1024 quantum 3028 initial_quantum 15140 Sent 2211625843 bytes 2263178 pkt (dropped 93506, overlimits 0 requeues 45= 394) backlog 14190248b 9807p requeues 45394 987 flows (518 inactive, 0 throttled) 0 gc, 0 highprio, 5407 throttled, 85636 flows_plimit * fq_codel bounces around between about 1400-2400 packets root@nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0 qdisc fq_codel 800c: root refcnt 2 limit 10000p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn Sent 1501918165 bytes 1589097 pkt (dropped 49749, overlimits 0 requeues 29= 086) backlog 1734550b 1468p requeues 29086 maxpacket 1514 drop_overlimit 0 new_flow_count 64971 ecn_mark 0 new_flows_len 0 old_flows_len 382 * Codel slowly gets things under control peaking at 10000 packets, then gradually getting below 4000 root@nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0 qdisc codel 800d: root refcnt 2 limit 10000p target 5.0ms interval 100.0ms Sent 3056093465 bytes 3025051 pkt (dropped 21719, overlimits 0 requeues 67= 529) backlog 4270321b 3955p requeues 67529 count 15938 lastcount 1 ldelay 34.8ms dropping drop_next 274us maxpacket 1514 ecn_mark 0 drop_overlimit 5781 * ns2_codel fails to get things under control root@nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0 qdisc ns2_codel 800e: root refcnt 2 limit 10000p target 5.0ms interval 100.= 0ms Sent 2634556896 bytes 2582228 pkt (dropped 17787, overlimits 0 requeues 57= 907) backlog 10148425b 9861p requeues 57907 count 3029 lastcount 0 ldelay 83.2ms dropping drop_next 420us maxpacket 0 ecn_mark 0 drop_overlimit 14758 but after a minute gets to where it is better than horrible root@nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0 qdisc ns2_codel 800e: root refcnt 2 limit 10000p target 5.0ms interval 100.= 0ms Sent 7272860233 bytes 7068770 pkt (dropped 38109, overlimits 0 requeues 16= 2324) backlog 5953303b 5721p requeues 162324 count 22845 lastcount 0 ldelay 48.7ms dropping drop_next 804us maxpacket 0 ecn_mark 0 drop_overlimit 15263 * Cake totally controls things root@nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0 qdisc cake 800f: root refcnt 2 unlimited diffserv4 flows Sent 1599700032 bytes 1600134 pkt (dropped 158320, overlimits 0 requeues 2= 6621) backlog 116908b 82p requeues 26621 Class 0 Class 1 Class 2 Class 3 rate 0bit 0bit 0bit 0bit target 5.0ms 5.0ms 5.0ms 5.0ms interval 105.0ms 105.0ms 105.0ms 105.0ms Pk delay 0us 4.4ms 0us 67us Av delay 0us 1.8ms 0us 1us Sp delay 0us 397us 0us 1us pkts 0 1758527 0 9 way inds 0 57354 0 0 way miss 0 142860 0 1 way cols 0 124 0 0 bytes 0 1834269446 0 1042 drops 0 158320 0 0 marks 0 0 0 0 * As does cake flowblind, but less so root@nuc-client:~/git/iproute2-cake# tc -s qdisc show dev eth0 qdisc cake 8010: root refcnt 2 unlimited diffserv4 flowblind Sent 1601618284 bytes 1667391 pkt (dropped 96598, overlimits 0 requeues 30= 514) backlog 260226b 258p requeues 30514 Class 0 Class 1 Class 2 Class 3 rate 0bit 0bit 0bit 0bit target 5.0ms 5.0ms 5.0ms 5.0ms interval 105.0ms 105.0ms 105.0ms 105.0ms Pk delay 0us 2.4ms 0us 283us Av delay 0us 2.1ms 0us 5us Sp delay 0us 1.5ms 0us 5us pkts 0 1764239 0 7 way inds 0 0 0 0 way miss 0 0 0 0 way cols 0 0 0 0 bytes 0 1698245972 0 742 drops 0 96598 0 0 marks 0 0 0 0 --=20 Dave T=C3=A4ht Open Networking needs **Open Source Hardware** https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67