[Cake] cake flenter results round 0

[Cake] cake flenter results round 0

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 9902 bytes --]

I have a script (called flenter) which can run flent with different parameter variations and produce an html report. I’m very sorry again that this doesn’t yet use flent’s batch feature- it was started before I knew about that.

I’ll use it to produce some “rounds” of cake tests, with notes/analysis of each round and plans for future rounds. If there’s any feedback on anything, results or what to change, let me know, otherwise I’ll just take it where I’m able to.

I’m calling this round 0, because these tests weren’t designed originally for cake at all, but it’s a vastly stripped down version of tests I was in the middle of doing for point-to-point WiFi. The tests need a lot of changes for the next round to focus more on cake and those are noted at the end.

Round 0 index of all tests:

http://www.drhleny <http://www.drhleny/>.cz/bufferbloat/cake/round0/


*** Notes/Analysis ***

* When “over-limited” to 200mbit, pfifo bloats everything, sfq improves the UDP flows but bloats TCP rtt, and cake clearly holds the lowest rtt for the ping/udp flows as well as TCP rtt, even when compared to fq_codel:

http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_200mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_200mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_200mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/eg_csrt_rrulbe_eg_fq_codel_900mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_200mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/eg_csrt_rrulbe_eg_cake_900mbit/index.html>


* At 950 mbit rate limiting, fq_codel holds latency of the ping and udp flows a bit lower than cake, whereas cake holds tcp rtt a bit lower. sfq does pretty well actually and pfifo is, well, pfifo:

http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/eg_csrt_rrulbe_eg_fq_codel_900mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/eg_csrt_rrulbe_eg_cake_900mbit/index.html>


* This is a little surprising, at 950mbit rate limiting with nflows 8/8, 16/16 and 32/32, fq_codel seems to outperform cake both in TCP rtt and latency of the UDP flows. Is cake’s “ethernet” keyword is really crucial here, or is there a difference between Cake and fq_codel at these high bandwidths? I’ll add it in my later tests. Also I’m losing the queue with 64/64 flows so will lower the bandwidth limit.

http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_pfifo_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/nflows_32_32_eg_pfifo_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_sfq_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/nflows_32_32_eg_sfq_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_fq_codel_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/nflows_32_32_eg_fq_codel_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_cake_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/nflows_32_32_eg_cake_950mbit/index.html>


* Obviously cake (with diffserv3 and not besteffort), roundly defeats everything else in the torrent test because of the de-prioritization of the background flows:

http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_pfifo_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_pfifo_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_sfq_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_sfq_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_fq_codel_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_fq_codel_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cakebe_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_cakebe_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cake_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/tor_rrultor_eg_cake_950mbit/index.html>


* The benefit to lowering cake’s rtt parameter in an Ethernet environment can be seen on the TCP rtt:

http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_100ms_rrulbe_eg_cake_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/cake_rtt_100ms_rrulbe_eg_cake_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_60ms_rrulbe_eg_cake_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/cake_rtt_60ms_rrulbe_eg_cake_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_20ms_rrulbe_eg_cake_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/cake_rtt_20ms_rrulbe_eg_cake_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_10ms_rrulbe_eg_cake_950mbit/index.html


* I’m a little surprised that fq_codel holds UDP flow latency a little lower at "target 1ms interval 10ms" than cake’s "rtt 10ms”. It almost seems like a trend that Cake outperforms at lower bandwidths and fq_codel at higher bandwidths.

http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t5ms_i100ms_rrulbe_eg_fq_codel_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/fq_codel_ti_t5ms_i100ms_rrulbe_eg_fq_codel_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t3ms_i60ms_rrulbe_eg_fq_codel_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/fq_codel_ti_t3ms_i60ms_rrulbe_eg_fq_codel_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t2ms_i20ms_rrulbe_eg_fq_codel_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/fq_codel_ti_t2ms_i20ms_rrulbe_eg_fq_codel_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t1ms_i10ms_rrulbe_eg_fq_codel_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/fq_codel_ti_t1ms_i10ms_rrulbe_eg_fq_codel_950mbit/index.html>


* ECN helps marginally with UDP flow rtt, but I’ve never seen ECN do very much. When does it help the most?

http://www.drhleny.cz/bufferbloat/cake/round0/ecn_off_rrulbe_eg_cake_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/ecn_off_rrulbe_eg_cake_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/ecn_on_rrulbe_eg_cake_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/ecn_on_rrulbe_eg_cake_950mbit/index.html>


* Cake’s “ethernet” parameter helps a bit, I’ll add it to all other rate limited tests in my next round:

http://www.drhleny.cz/bufferbloat/cake/round0/cake_overhead_rrulbe_eg_cake_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/cake_overhead_rrulbe_eg_cake_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/apu2-eth/cake_overhead_rrulbe_eg_cakeeth_950mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round0/apu2-eth/cake_overhead_rrulbe_eg_cakeeth_950mbit/index.html>


* Cake’s host isolation clearly works, but I’m a little surprised that “srchost/dsthost” is more fair on a host level than “dual-srchost/dual-dsthost” (I usually find it easiest to just scroll to the bottom of the page and look at the numbers):

http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_fq_codel_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/hostiso_eg_fq_codel_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_src_cake_dst_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/hostiso_eg_cake_src_cake_dst_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_dsrc_cake_ddst_950mbit/index.html <file:///Users/peteheist/Sites/apu2a/src/flenter/apu2-eth/hostiso_eg_cake_dsrc_cake_ddst_950mbit/index.html>


* Anyone see anything in my “Flow Isolation Mix” tests? Those are a little hard to read. :) They used to be combined with a VoIP test but I don’t have a d-itg setup now.


*** Round 1 Plans ***

- Update cake to latest (will do this with every round)
- Remove all “full-duplex limiting” (egress and ingress) tests as I don’t see the use here
- Add cake’s “ethernet” keyword to all rate limited tests
- Lower standard rate limit to 900mbit ensure no queue loss (particularly for nflows tests)
- Take standard rrulbe tests to even lower bandwidths: 1mbit, 10mbit, 50mbit, 100mbit
- Add bql tests (no rate limiting)
- Add 100us, 1ms, 2ms, 3ms, 5ms, 8ms to Cake RTT tests
- Add nflows tests at lower bandwidths
- Fix UDP flood tests (no suitable iperf binary found)
- Remove or improve flow isolation tests
- Add ethtool output to host info


*** Plans for Future Rounds ***

- Add flow isolation tests with rtt variation (to look again at problem I reported in an earlier thread)
- Use netem to make a spread of rtts and bandwidths (maybe the most useful of all?)
- Add ack filtering tests
- Add VoIP tests (I hope to do this with irtt)
- Test BBR
- Use qemu to test other archs (I may never get to this, honestly)


[-- Attachment #2: Type: text/html, Size: 15678 bytes --]

Re: [Cake] cake flenter results round 0

[Dave Taht]

That is a really, really long, and extremely pleasant, way of saying:

 "OK, it doesn't crash".

:)

can flenter work with the veth stuff and namespaces?





On Sun, Nov 26, 2017 at 12:12 AM, Pete Heist <peteheist@gmail.com> wrote:
> I have a script (called flenter) which can run flent with different
> parameter variations and produce an html report. I’m very sorry again that
> this doesn’t yet use flent’s batch feature- it was started before I knew
> about that.
>
> I’ll use it to produce some “rounds” of cake tests, with notes/analysis of
> each round and plans for future rounds. If there’s any feedback on anything,
> results or what to change, let me know, otherwise I’ll just take it where
> I’m able to.
>
> I’m calling this round 0, because these tests weren’t designed originally
> for cake at all, but it’s a vastly stripped down version of tests I was in
> the middle of doing for point-to-point WiFi. The tests need a lot of changes
> for the next round to focus more on cake and those are noted at the end.
>
> Round 0 index of all tests:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/
>
>
> *** Notes/Analysis ***
>
> * When “over-limited” to 200mbit, pfifo bloats everything, sfq improves the
> UDP flows but bloats TCP rtt, and cake clearly holds the lowest rtt for the
> ping/udp flows as well as TCP rtt, even when compared to fq_codel:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_200mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_200mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_200mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_200mbit/index.html
>
>
> * At 950 mbit rate limiting, fq_codel holds latency of the ping and udp
> flows a bit lower than cake, whereas cake holds tcp rtt a bit lower. sfq
> does pretty well actually and pfifo is, well, pfifo:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_950mbit/index.html
>
>
> * This is a little surprising, at 950mbit rate limiting with nflows 8/8,
> 16/16 and 32/32, fq_codel seems to outperform cake both in TCP rtt and
> latency of the UDP flows. Is cake’s “ethernet” keyword is really crucial
> here, or is there a difference between Cake and fq_codel at these high
> bandwidths? I’ll add it in my later tests. Also I’m losing the queue with
> 64/64 flows so will lower the bandwidth limit.
>
> http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_pfifo_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_sfq_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_cake_950mbit/index.html
>
>
> * Obviously cake (with diffserv3 and not besteffort), roundly defeats
> everything else in the torrent test because of the de-prioritization of the
> background flows:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_pfifo_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_sfq_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cakebe_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cake_950mbit/index.html
>
>
> * The benefit to lowering cake’s rtt parameter in an Ethernet environment
> can be seen on the TCP rtt:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_100ms_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_60ms_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_20ms_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_10ms_rrulbe_eg_cake_950mbit/index.html
>
>
> * I’m a little surprised that fq_codel holds UDP flow latency a little lower
> at "target 1ms interval 10ms" than cake’s "rtt 10ms”. It almost seems like a
> trend that Cake outperforms at lower bandwidths and fq_codel at higher
> bandwidths.
>
> http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t5ms_i100ms_rrulbe_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t3ms_i60ms_rrulbe_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t2ms_i20ms_rrulbe_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t1ms_i10ms_rrulbe_eg_fq_codel_950mbit/index.html
>
>
> * ECN helps marginally with UDP flow rtt, but I’ve never seen ECN do very
> much. When does it help the most?
>
> http://www.drhleny.cz/bufferbloat/cake/round0/ecn_off_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/ecn_on_rrulbe_eg_cake_950mbit/index.html
>
>
> * Cake’s “ethernet” parameter helps a bit, I’ll add it to all other rate
> limited tests in my next round:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_overhead_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/apu2-eth/cake_overhead_rrulbe_eg_cakeeth_950mbit/index.html
>
>
> * Cake’s host isolation clearly works, but I’m a little surprised that
> “srchost/dsthost” is more fair on a host level than
> “dual-srchost/dual-dsthost” (I usually find it easiest to just scroll to the
> bottom of the page and look at the numbers):
>
> http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_src_cake_dst_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_dsrc_cake_ddst_950mbit/index.html
>
>
> * Anyone see anything in my “Flow Isolation Mix” tests? Those are a little
> hard to read. :) They used to be combined with a VoIP test but I don’t have
> a d-itg setup now.
>
>
> *** Round 1 Plans ***
>
> - Update cake to latest (will do this with every round)
> - Remove all “full-duplex limiting” (egress and ingress) tests as I don’t
> see the use here
> - Add cake’s “ethernet” keyword to all rate limited tests
> - Lower standard rate limit to 900mbit ensure no queue loss (particularly
> for nflows tests)
> - Take standard rrulbe tests to even lower bandwidths: 1mbit, 10mbit,
> 50mbit, 100mbit
> - Add bql tests (no rate limiting)
> - Add 100us, 1ms, 2ms, 3ms, 5ms, 8ms to Cake RTT tests
> - Add nflows tests at lower bandwidths
> - Fix UDP flood tests (no suitable iperf binary found)
> - Remove or improve flow isolation tests
> - Add ethtool output to host info
>
>
> *** Plans for Future Rounds ***
>
> - Add flow isolation tests with rtt variation (to look again at problem I
> reported in an earlier thread)
> - Use netem to make a spread of rtts and bandwidths (maybe the most useful
> of all?)
> - Add ack filtering tests
> - Add VoIP tests (I hope to do this with irtt)
> - Test BBR
> - Use qemu to test other archs (I may never get to this, honestly)
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 0

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1527 bytes --]


> On Nov 26, 2017, at 9:42 AM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> That is a really, really long, and extremely pleasant, way of saying:
> 
> "OK, it doesn't crash".
> 
> :)
> 
> can flenter work with the veth stuff and namespaces?

Maybe a little more than that, but not much more. :) I put it on git for reference: https://github.com/peteheist/flenter <https://github.com/peteheist/flenter>

So far no to the veth and namespace support, but I don’t see why it couldn’t. A flenter “rig" includes up to six roles. In this test I’m only using four since it’s not over p2p wifi: client, client router, server router and server. The script doesn’t care where those are as long as it can ssh to them without a password so it can run flenter_shell.sh to do its setup and run flent on the client. My dream was that you could just define your rig with a few parameters, run it, and enjoy all the results. That dream is still more of a waking nightmare as you have to modify the tests based on what your rig can do.

I think the most important thing to making the tests more relevant for cake after "round 1” changes is using different bandwidths and rtts, right? I could use netem for this. The use of veth and namespaces is mainly for getting it to run on one box, right? Where’s the latest code for that, btw? I’m a little lost as to whether or if I need to be using net-next, if I can just use the current netem I have, where to get the latest netem and veth / namespaces stuff, etc...


[-- Attachment #2: Type: text/html, Size: 2199 bytes --]

Re: [Cake] cake flenter results round 0

[Dave Taht]

On Sun, Nov 26, 2017 at 1:47 AM, Pete Heist <peteheist@gmail.com> wrote:
>
> On Nov 26, 2017, at 9:42 AM, Dave Taht <dave.taht@gmail.com> wrote:
>
> That is a really, really long, and extremely pleasant, way of saying:
>
> "OK, it doesn't crash".
>
> :)
>
> can flenter work with the veth stuff and namespaces?
>
>
> Maybe a little more than that, but not much more. :) I put it on git for
> reference: https://github.com/peteheist/flenter
>
> So far no to the veth and namespace support, but I don’t see why it
> couldn’t. A flenter “rig" includes up to six roles. In this test I’m only
> using four since it’s not over p2p wifi: client, client router, server
> router and server. The script doesn’t care where those are as long as it can
> ssh to them without a password so it can run flenter_shell.sh to do its
> setup and run flent on the client. My dream was that you could just define
> your rig with a few parameters, run it, and enjoy all the results. That
> dream is still more of a waking nightmare as you have to modify the tests
> based on what your rig can do.
>
> I think the most important thing to making the tests more relevant for cake
> after "round 1” changes is using different bandwidths and rtts, right? I
> could use netem for this.

the real test of a aqm is how it handles a range of rtts in real traffic.

While a simple, useful test is merely to insert delays of say, 20, 40, 80, 160ms
inline with

a real test would have multiple tcp targets of those delays.

The main trick with using netem properly is to not have it be the
bottleneck link, and have a huge packet limit, so it doesn't turn into
your tail loss queue.

>The use of veth and namespaces is mainly for
> getting it to run on one box, right? Where’s the latest code for that, btw?

It's not up anywhere yet, I was heads down on cleaning up cake.

> I’m a little lost as to whether or if I need to be using net-next, if I can
> just use the current netem I have, where to get the latest netem and veth /
> namespaces stuff, etc...

A whole lotta namespace stuff landed in net-next, some of which I was
probably leveraging.

I kind of need to go heads down this week on finishing up the slotting
feature in iproute2. And looking over the cake codebase there.



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 0

[Dave Taht]

Pete Heist <peteheist@gmail.com> writes:

> I have a script (called flenter) which can run flent with different parameter
> variations and produce an html report. I’m very sorry again that this doesn’t
> yet use flent’s batch feature- it was started before I knew about that.
>
> I’ll use it to produce some “rounds” of cake tests, with notes/analysis of each
> round and plans for future rounds. If there’s any feedback on anything, results
> or what to change, let me know, otherwise I’ll just take it where I’m able to.
>
> I’m calling this round 0, because these tests weren’t designed originally for
> cake at all, but it’s a vastly stripped down version of tests I was in the
> middle of doing for point-to-point WiFi. The tests need a lot of changes for the
> next round to focus more on cake and those are noted at the end.
>
> Round 0 index of all tests:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/
>
> *** Notes/Analysis ***
>
> * When “over-limited” to 200mbit, pfifo bloats everything, sfq improves the UDP
> flows but bloats TCP rtt, and cake clearly holds the lowest rtt for the ping/udp
> flows as well as TCP rtt, even when compared to fq_codel:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_200mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_200mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_200mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_200mbit/index.html
>
> * At 950 mbit rate limiting, fq_codel holds latency of the ping and udp flows a
> bit lower than cake, whereas cake holds tcp rtt a bit lower. sfq does pretty
> well actually and pfifo is, well, pfifo:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_950mbit/index.html
>
> * This is a little surprising, at 950mbit rate limiting with nflows 8/8, 16/16
> and 32/32, fq_codel seems to outperform cake both in TCP rtt and latency of the
> UDP flows. Is cake’s “ethernet” keyword is really crucial here, or is there a
> difference between Cake and fq_codel at these high bandwidths? I’ll add it in my
> later tests. Also I’m losing the queue with 64/64 flows so will lower the
> bandwidth limit.
>
> http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_pfifo_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_sfq_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_cake_950mbit/index.html
>
> * Obviously cake (with diffserv3 and not besteffort), roundly defeats everything
> else in the torrent test because of the de-prioritization of the background
> flows:

I think you can safely drop pfifo from future tests.

I'd rather like combined charts so it is possible to eyeball these
differences directly. Just between fq_codel and cake. Or, is there a
tarball available to browse this stuff?

Another nice thing to try capturing is queue depth/loss/marks/etc, which
is --test-parameter qdisc_stats_hosts=X,y,z and qdisc_stats_interfaces=

There's also capturing the tcp statistics on the server that is
possible.

>
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_pfifo_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_sfq_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cakebe_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cake_950mbit/index.html
>
> * The benefit to lowering cake’s rtt parameter in an Ethernet environment can be
> seen on the TCP rtt:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_100ms_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_60ms_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_20ms_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_10ms_rrulbe_eg_cake_950mbit/index.html
>
> * I’m a little surprised that fq_codel holds UDP flow latency a little lower at
> "target 1ms interval 10ms" than cake’s "rtt 10ms”. It almost seems like a trend
> that Cake outperforms at lower bandwidths and fq_codel at higher bandwidths.

Since fq_codel supports superpackets and cake peels them, we have a cpu
and latency hit that originates from that. Also the codel derived
algorithm in cake differs quite significantly from mainline codel, and
my principal gripe about it has been that it has not been extensively
tested against higher delays.

>
> http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t5ms_i100ms_rrulbe_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t3ms_i60ms_rrulbe_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t2ms_i20ms_rrulbe_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t1ms_i10ms_rrulbe_eg_fq_codel_950mbit/index.html
>
> * ECN helps marginally with UDP flow rtt, but I’ve never seen ECN do very much.
> When does it help the most?
>
> http://www.drhleny.cz/bufferbloat/cake/round0/ecn_off_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/ecn_on_rrulbe_eg_cake_950mbit/index.html
>
> * Cake’s “ethernet” parameter helps a bit, I’ll add it to all other rate limited
> tests in my next round:
>
> http://www.drhleny.cz/bufferbloat/cake/round0/cake_overhead_rrulbe_eg_cake_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/apu2-eth/cake_overhead_rrulbe_eg_cakeeth_950mbit/index.html
>
> * Cake’s host isolation clearly works, but I’m a little surprised that
> “srchost/dsthost” is more fair on a host level than “dual-srchost/dual-dsthost”
> (I usually find it easiest to just scroll to the bottom of the page and look at
> the numbers):
>
> http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_fq_codel_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_src_cake_dst_950mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_dsrc_cake_ddst_950mbit/index.html
>
> * Anyone see anything in my “Flow Isolation Mix” tests? Those are a little hard
> to read. :) They used to be combined with a VoIP test but I don’t have a d-itg
> setup now.

I look forward to you adding OWD irtt based tests.

> *** Round 1 Plans ***
>
> - Update cake to latest (will do this with every round)
> - Remove all “full-duplex limiting” (egress and ingress) tests as I don’t see
> the use here
> - Add cake’s “ethernet” keyword to all rate limited tests
> - Lower standard rate limit to 900mbit ensure no queue loss (particularly for
> nflows tests)
> - Take standard rrulbe tests to even lower bandwidths: 1mbit, 10mbit, 50mbit,
> 100mbit
> - Add bql tests (no rate limiting)
> - Add 100us, 1ms, 2ms, 3ms, 5ms, 8ms to Cake RTT tests
> - Add nflows tests at lower bandwidths
> - Fix UDP flood tests (no suitable iperf binary found)
> - Remove or improve flow isolation tests
> - Add ethtool output to host info
>
> *** Plans for Future Rounds ***
>
> - Add flow isolation tests with rtt variation (to look again at problem I
> reported in an earlier thread)
> - Use netem to make a spread of rtts and bandwidths (maybe the most useful of
> all?)

Yes.

> - Add ack filtering tests
> - Add VoIP tests (I hope to do this with irtt)
> - Test BBR
> - Use qemu to test other archs (I may never get to this, honestly)
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake flenter results round 0

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1384 bytes --]


> On Nov 26, 2017, at 6:45 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> the real test of a aqm is how it handles a range of rtts in real traffic.
> 
> While a simple, useful test is merely to insert delays of say, 20, 40, 80, 160ms
> inline with
> 
> a real test would have multiple tcp targets of those delays.
> 
> The main trick with using netem properly is to not have it be the
> bottleneck link, and have a huge packet limit, so it doesn't turn into
> your tail loss queue.

Ok, I’ll try to keep this in mind when I get to that.

>> I’m a little lost as to whether or if I need to be using net-next, if I can
>> just use the current netem I have, where to get the latest netem and veth /
>> namespaces stuff, etc...
> 
> A whole lotta namespace stuff landed in net-next, some of which I was
> probably leveraging.
> 
> I kind of need to go heads down this week on finishing up the slotting
> feature in iproute2. And looking over the cake codebase there.

Got it, I’ll gradually figure this out over time. I’ve been enjoying the “simplicity” of RTT, OWD and related stats for a while.

Testing queueing can feel like directing runs at the LHC (not that I’d know). After this is done I think I want to learn some more theory. Sometimes theoretical discoveries can make years of experimental work irrelevant (and vice-versa, I guess).

[-- Attachment #2: Type: text/html, Size: 11587 bytes --]

Re: [Cake] cake flenter results round 0

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 2745 bytes --]


> On Nov 26, 2017, at 7:19 PM, Dave Taht <dave@taht.net> wrote:
> 
> Pete Heist <peteheist@gmail.com <mailto:peteheist@gmail.com>> writes:
> 
> I think you can safely drop pfifo from future tests.

Yeah, done, chuckled at that actually. I was using pfifo as the “cold pool”.

> I'd rather like combined charts so it is possible to eyeball these
> differences directly. Just between fq_codel and cake. Or, is there a
> tarball available to browse this stuff?

Oh I know, I wanted to make the script do “combination plots” at the end but never got to it. I’ll try to make an archive available next time. Meanwhile I just open up run pages in multiple browser tabs and keyboard shortcut between them. Tests the visual memory.

BTW each run links to the .flent.gz file, but if you want to review more runs at once it's a pain to download each of them...

> Another nice thing to try capturing is queue depth/loss/marks/etc, which
> is --test-parameter qdisc_stats_hosts=X,y,z and qdisc_stats_interfaces=

You probably saw the `tc -s` output on teardown right? But can check out the test param...

> There's also capturing the tcp statistics on the server that is
> possible.

I do —socket-stats and have TCP RTT plots, but you mean something beyond that?

>> * I’m a little surprised that fq_codel holds UDP flow latency a little lower at
>> "target 1ms interval 10ms" than cake’s "rtt 10ms”. It almost seems like a trend
>> that Cake outperforms at lower bandwidths and fq_codel at higher bandwidths.
> 
> Since fq_codel supports superpackets and cake peels them, we have a cpu
> and latency hit that originates from that. Also the codel derived
> algorithm in cake differs quite significantly from mainline codel, and
> my principal gripe about it has been that it has not been extensively
> tested against higher delays.

Ok, next run is going to show lower bandwidths with nflows=32/32 and I think cake is really going to shine there.

>> * Anyone see anything in my “Flow Isolation Mix” tests? Those are a little hard
>> to read. :) They used to be combined with a VoIP test but I don’t have a d-itg
>> setup now.
> 
> I look forward to you adding OWD irtt based tests.

Me too, going to try to get that in soon...

>> *** Plans for Future Rounds ***
>> 
>> - Add flow isolation tests with rtt variation (to look again at problem I
>> reported in an earlier thread)
>> - Use netem to make a spread of rtts and bandwidths (maybe the most useful of
>> all?)
> 
> Yes.

Thought so, that didn’t make it into round 1 but asap after that. Round 1 just kicked off for the night. I got most of the changes I wanted in but only had an hour on it today so nothing more...

[-- Attachment #2: Type: text/html, Size: 17984 bytes --]