
<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">I have a script (called flenter) which can run flent with different parameter variations and produce an html report. I’m very sorry again that this doesn’t yet use flent’s batch feature- it was started before I knew about that.</div><div class=""><br class=""></div><div class="">I’ll use it to produce some “rounds” of cake tests, with notes/analysis of each round and plans for future rounds. If there’s any feedback on anything, results or what to change, let me know, otherwise I’ll just take it where I’m able to.</div><div class=""><br class=""></div><div class="">I’m calling this round 0, because these tests weren’t designed originally for cake at all, but it’s a vastly stripped down version of tests I was in the middle of doing for point-to-point WiFi. The tests need a lot of changes for the next round to focus more on cake and those are noted at the end.</div><div class=""><br class=""></div><div class="">Round 0 index of all tests:</div><div class=""><br class=""></div><div class=""><a href="http://www.drhleny" class="">http://www.drhleny</a>.cz/bufferbloat/cake/round0/</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">*** Notes/Analysis ***</div><div class=""><br class=""></div><div class=""><div class="">* When “over-limited” to 200mbit, pfifo bloats everything, sfq improves the UDP flows but bloats TCP rtt, and cake clearly holds the lowest rtt for the ping/udp flows as well as TCP rtt, even when compared to fq_codel:</div><div class=""><br class=""></div><div class=""><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_200mbit/index.html" class="">http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_200mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_200mbit/index.html" class="">http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_200mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/eg_csrt_rrulbe_eg_fq_codel_900mbit/index.html" class="">eg_csrt_rrulbe_eg_fq_codel_200mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/eg_csrt_rrulbe_eg_cake_900mbit/index.html" class="">eg_csrt_rrulbe_eg_cake_200mbit/index.html</a></div></div><div class=""><br class=""></div></div><div class=""><br class=""></div><div class="">* At 950 mbit rate limiting, fq_codel holds latency of the ping and udp flows a bit lower than cake, whereas cake holds tcp rtt a bit lower. sfq does pretty well actually and pfifo is, well, pfifo:</div><div class=""><br class=""></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_950mbit/index.html" class="">http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_950mbit/index.html" class="">http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/eg_csrt_rrulbe_eg_fq_codel_900mbit/index.html" class="">eg_csrt_rrulbe_eg_fq_codel_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/eg_csrt_rrulbe_eg_cake_900mbit/index.html" class="">eg_csrt_rrulbe_eg_cake_950mbit/index.html</a></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">* This is a little surprising, at 950mbit rate limiting with nflows 8/8, 16/16 and 32/32, fq_codel seems to outperform cake both in TCP rtt and latency of the UDP flows. Is cake’s “ethernet” keyword is really crucial here, or is there a difference between Cake and fq_codel at these high bandwidths? I’ll add it in my later tests. Also I’m losing the queue with 64/64 flows so will lower the bandwidth limit.</div><div class=""><br class=""></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/nflows_32_32_eg_pfifo_950mbit/index.html" class="">nflows_32_32_eg_pfifo_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/nflows_32_32_eg_sfq_950mbit/index.html" class="">nflows_32_32_eg_sfq_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/nflows_32_32_eg_fq_codel_950mbit/index.html" class="">nflows_32_32_eg_fq_codel_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/nflows_32_32_eg_cake_950mbit/index.html" class="">nflows_32_32_eg_cake_950mbit/index.html</a></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">* Obviously cake (with diffserv3 and not besteffort), roundly defeats everything else in the torrent test because of the de-prioritization of the background flows:</div><div class=""><br class=""></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_pfifo_950mbit/index.html" class="">tor_rrultor_eg_pfifo_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_sfq_950mbit/index.html" class="">tor_rrultor_eg_sfq_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_fq_codel_950mbit/index.html" class="">tor_rrultor_eg_fq_codel_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_cakebe_950mbit/index.html" class="">tor_rrultor_eg_cakebe_950mbit/index.html</a></div><div class=""><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_cake_950mbit/index.html" class="">tor_rrultor_eg_cake_950mbit/index.html</a></div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">* The benefit to lowering cake’s rtt parameter in an Ethernet environment can be seen on the TCP rtt:</div><div class=""><br class=""></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/cake_rtt_100ms_rrulbe_eg_cake_950mbit/index.html" class="">cake_rtt_100ms_rrulbe_eg_cake_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/cake_rtt_60ms_rrulbe_eg_cake_950mbit/index.html" class="">cake_rtt_60ms_rrulbe_eg_cake_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/cake_rtt_20ms_rrulbe_eg_cake_950mbit/index.html" class="">cake_rtt_20ms_rrulbe_eg_cake_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_10ms_rrulbe_eg_cake_950mbit/index.html" class="">http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_10ms_rrulbe_eg_cake_950mbit/index.html</a></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">* I’m a little surprised that fq_codel holds UDP flow latency a little lower at "target 1ms interval 10ms" than cake’s "rtt 10ms”. It almost seems like a trend that Cake outperforms at lower bandwidths and fq_codel at higher bandwidths.</div><div class=""><br class=""></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/fq_codel_ti_t5ms_i100ms_rrulbe_eg_fq_codel_950mbit/index.html" class="">fq_codel_ti_t5ms_i100ms_rrulbe_eg_fq_codel_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/fq_codel_ti_t3ms_i60ms_rrulbe_eg_fq_codel_950mbit/index.html" class="">fq_codel_ti_t3ms_i60ms_rrulbe_eg_fq_codel_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/f" class="">http://www.drhleny.cz/bufferbloat/cake/round0/f</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/fq_codel_ti_t2ms_i20ms_rrulbe_eg_fq_codel_950mbit/index.html" class="">q_codel_ti_t2ms_i20ms_rrulbe_eg_fq_codel_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/fq_codel_ti_t1ms_i10ms_rrulbe_eg_fq_codel_950mbit/index.html" class="">fq_codel_ti_t1ms_i10ms_rrulbe_eg_fq_codel_950mbit/index.html</a></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">* ECN helps marginally with UDP flow rtt, but I’ve never seen ECN do very much. When does it help the most?</div><div class=""><br class=""></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/ecn_off_rrulbe_eg_cake_950mbit/index.html" class="">ecn_off_rrulbe_eg_cake_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/ecn_on_rrulbe_eg_cake_950mbit/index.html" class="">ecn_on_rrulbe_eg_cake_950mbit/index.html</a></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">* Cake’s “ethernet” parameter helps a bit, I’ll add it to all other rate limited tests in my next round:</div><div class=""><br class=""></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/cake_overhead_rrulbe_eg_cake_950mbit/index.html" class="">cake_overhead_rrulbe_eg_cake_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/apu2-eth/cake_overhead_rrulbe_eg_cakeeth_950mbit/index.html" class="">http://www.drhleny.cz/bufferbloat/cake/round0/apu2-eth/cake_overhead_rrulbe_eg_cakeeth_950mbit/index.html</a></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">* Cake’s host isolation clearly works, but I’m a little surprised that “srchost/dsthost” is more fair on a host level than “dual-srchost/dual-dsthost” (I usually find it easiest to just scroll to the bottom of the page and look at the numbers):</div><div class=""><br class=""></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/hostiso_eg_fq_codel_950mbit/index.html" class="">hostiso_eg_fq_codel_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/hostiso_eg_cake_src_cake_dst_950mbit/index.html" class="">hostiso_eg_cake_src_cake_dst_950mbit/index.html</a></div><div class=""><a href="http://www.drhleny.cz/bufferbloat/cake/round0/" class="">http://www.drhleny.cz/bufferbloat/cake/round0/</a><a href="file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/hostiso_eg_cake_dsrc_cake_ddst_950mbit/index.html" class="">hostiso_eg_cake_dsrc_cake_ddst_950mbit/index.html</a></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">* Anyone see anything in my “Flow Isolation Mix” tests? Those are a little hard to read. :) They used to be combined with a VoIP test but I don’t have a d-itg setup now.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">*** Round 1 Plans ***</div><div class=""><br class=""></div><div class="">- Update cake to latest (will do this with every round)</div><div class="">- Remove all “full-duplex limiting” (egress and ingress) tests as I don’t see the use here</div><div class="">- Add cake’s “ethernet” keyword to all rate limited tests</div><div class="">- Lower standard rate limit to 900mbit ensure no queue loss (particularly for nflows tests)</div><div class="">- Take standard rrulbe tests to even lower bandwidths: 1mbit, 10mbit, 50mbit, 100mbit</div><div class="">- Add bql tests (no rate limiting)</div><div class="">- Add 100us, 1ms, 2ms, 3ms, 5ms, 8ms to Cake RTT tests</div><div class="">- Add nflows tests at lower bandwidths</div><div class="">- Fix UDP flood tests (no suitable iperf binary found)</div><div class="">- Remove or improve flow isolation tests</div><div class=""><div class="">- Add ethtool output to host info</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">*** Plans for Future Rounds ***</div><div class=""><br class=""></div><div class=""><div class="">- Add flow isolation tests with rtt variation (to look again at problem I reported in an earlier thread)</div><div class="">- Use netem to make a spread of rtts and bandwidths (maybe the most useful of all?)</div><div class="">- Add ack filtering tests</div><div class="">- Add VoIP tests (I hope to do this with irtt)</div></div><div class="">- Test BBR</div><div class=""><div class="">- Use qemu to test other archs (I may never get to this, honestly)</div></div><div class=""><br class=""></div></body></html>