From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B54503B29E for ; Sun, 26 Nov 2017 03:12:50 -0500 (EST) Received: by mail-wm0-x231.google.com with SMTP id 124so11906973wmv.1 for ; Sun, 26 Nov 2017 00:12:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:subject:message-id:date:to:mime-version; bh=D+237ZtbOKnr5sI3hqeRyQsUufbmNRBDR2RIO1eiLCk=; b=LBTaBsdE6TXBW2GKLy2kCOu2wtrmSEuISAa3hkVStepFvg0FdEWGgAh53dt1peKY9a jydNrh2U47fNzuMY/bHbyW1hKL7P3TJX57aTsnaDztS4iW/CHCtL7OSmUMxl3b/5YKr6 Q3/j3+pGFu/vnQm2OcRBE+7uODS4Ggm+wRf4jMxu1SJuJ9r86YAC7NNr9oGZf5YprJua BdtxpXn0CaPr/O9+aOhTc/foGrPwM3zYISJC69Su0C7j171rg8k7wFnMY5D6AcCVIhLX 1HgXLX5Q4vJAMxXFrZDK5hLK/Kw0165G4UWnvlGfbnZ3ZOtvgTctMxKFah+unT2LkuGb 0NBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:message-id:date:to:mime-version; bh=D+237ZtbOKnr5sI3hqeRyQsUufbmNRBDR2RIO1eiLCk=; b=gm+2ZhE2e35DvmajxQH/eh3hHboCRXOEBw9zOvphzg/v+n4zSVwU0sjLH9dMZGYdZ4 hhB6/RMfC7qlVU0Js5gzafpUxjvk5QX6jjzkn9/emqoMCGljtGCNRS0WE/e7tp9sALBU Axnsc61nj0y89KxdB0jUI0bgNmAUkwrOOeyGUD+dq/td1U5LiscepPtrYHWPC3NzH9aE 8dKwwOTn6RUGQblHtz0JBKTyNlVtkFDmh39US+gDvj35eeJcruLSLnyhBaA8wyPsLt0b YsIiXN5gJ5dMvhdqtqG+Arv1Agg4FXHsri66FFBlv8MS3a4f2PAEEoVazgA0QmtZpHAJ NX3w== X-Gm-Message-State: AJaThX6oP/jC5W9G8Is+Jyv7kwmhd5/yKzE2Vll2HkToR12y2IebewG1 5GYJrbQ83Fis6XRjC0ipXgI9wXVs X-Google-Smtp-Source: AGs4zMahoLQyykiU6WPpqF7dsTI8IoHgiQfAODDhyvzaJkAJhQUrOXJmCfydzDuLYsKwW/rG41SE/Q== X-Received: by 10.28.50.70 with SMTP id y67mr14267528wmy.159.1511683969456; Sun, 26 Nov 2017 00:12:49 -0800 (PST) Received: from [10.72.0.130] (h-1169.lbcfree.net. [185.99.119.68]) by smtp.gmail.com with ESMTPSA id l140sm22696333wma.5.2017.11.26.00.12.48 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 26 Nov 2017 00:12:48 -0800 (PST) From: Pete Heist Content-Type: multipart/alternative; boundary="Apple-Mail=_5D0AAE19-98BB-4086-8D76-EA8A0C7C0704" Message-Id: <6BE29FE9-6E32-4324-8B56-6BB3B6E5F033@gmail.com> Date: Sun, 26 Nov 2017 09:12:46 +0100 To: Cake List Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) Subject: [Cake] cake flenter results round 0 X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 08:12:51 -0000 --Apple-Mail=_5D0AAE19-98BB-4086-8D76-EA8A0C7C0704 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 I have a script (called flenter) which can run flent with different = parameter variations and produce an html report. I=E2=80=99m very sorry = again that this doesn=E2=80=99t yet use flent=E2=80=99s batch feature- = it was started before I knew about that. I=E2=80=99ll use it to produce some =E2=80=9Crounds=E2=80=9D of cake = tests, with notes/analysis of each round and plans for future rounds. If = there=E2=80=99s any feedback on anything, results or what to change, let = me know, otherwise I=E2=80=99ll just take it where I=E2=80=99m able to. I=E2=80=99m calling this round 0, because these tests weren=E2=80=99t = designed originally for cake at all, but it=E2=80=99s a vastly stripped = down version of tests I was in the middle of doing for point-to-point = WiFi. The tests need a lot of changes for the next round to focus more = on cake and those are noted at the end. Round 0 index of all tests: http://www.drhleny .cz/bufferbloat/cake/round0/ *** Notes/Analysis *** * When =E2=80=9Cover-limited=E2=80=9D to 200mbit, pfifo bloats = everything, sfq improves the UDP flows but bloats TCP rtt, and cake = clearly holds the lowest rtt for the ping/udp flows as well as TCP rtt, = even when compared to fq_codel: = http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_200m= bit/index.html = http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_200mbi= t/index.html = http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_2= 00mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_200mb= it/index.html = * At 950 mbit rate limiting, fq_codel holds latency of the ping and udp = flows a bit lower than cake, whereas cake holds tcp rtt a bit lower. sfq = does pretty well actually and pfifo is, well, pfifo: = http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_950m= bit/index.html = http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_950mbi= t/index.html = http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_9= 50mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_950mb= it/index.html = * This is a little surprising, at 950mbit rate limiting with nflows 8/8, = 16/16 and 32/32, fq_codel seems to outperform cake both in TCP rtt and = latency of the UDP flows. Is cake=E2=80=99s =E2=80=9Cethernet=E2=80=9D = keyword is really crucial here, or is there a difference between Cake = and fq_codel at these high bandwidths? I=E2=80=99ll add it in my later = tests. Also I=E2=80=99m losing the queue with 64/64 flows so will lower = the bandwidth limit. = http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_pfifo_950mbi= t/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_sfq_950mbit/= index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_fq_codel_950= mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_cake_950mbit= /index.html = * Obviously cake (with diffserv3 and not besteffort), roundly defeats = everything else in the torrent test because of the de-prioritization of = the background flows: = http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_pfifo_950mbit= /index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_sfq_950mbit/i= ndex.html = = http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_fq_codel_950m= bit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cakebe_950mbi= t/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cake_950mbit/= index.html = * The benefit to lowering cake=E2=80=99s rtt parameter in an Ethernet = environment can be seen on the TCP rtt: = http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_100ms_rrulbe_eg_cak= e_950mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_60ms_rrulbe_eg_cake= _950mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_20ms_rrulbe_eg_cake= _950mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_10ms_rrulbe_eg_cake= _950mbit/index.html * I=E2=80=99m a little surprised that fq_codel holds UDP flow latency a = little lower at "target 1ms interval 10ms" than cake=E2=80=99s "rtt = 10ms=E2=80=9D. It almost seems like a trend that Cake outperforms at = lower bandwidths and fq_codel at higher bandwidths. = http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t5ms_i100ms_rrul= be_eg_fq_codel_950mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t3ms_i60ms_rrulb= e_eg_fq_codel_950mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t2ms_i20ms_rrulb= e_eg_fq_codel_950mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t1ms_i10ms_rrulb= e_eg_fq_codel_950mbit/index.html = * ECN helps marginally with UDP flow rtt, but I=E2=80=99ve never seen = ECN do very much. When does it help the most? = http://www.drhleny.cz/bufferbloat/cake/round0/ecn_off_rrulbe_eg_cake_950mb= it/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/ecn_on_rrulbe_eg_cake_950mbi= t/index.html = * Cake=E2=80=99s =E2=80=9Cethernet=E2=80=9D parameter helps a bit, = I=E2=80=99ll add it to all other rate limited tests in my next round: = http://www.drhleny.cz/bufferbloat/cake/round0/cake_overhead_rrulbe_eg_cake= _950mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/apu2-eth/cake_overhead_rrulb= e_eg_cakeeth_950mbit/index.html = * Cake=E2=80=99s host isolation clearly works, but I=E2=80=99m a little = surprised that =E2=80=9Csrchost/dsthost=E2=80=9D is more fair on a host = level than =E2=80=9Cdual-srchost/dual-dsthost=E2=80=9D (I usually find = it easiest to just scroll to the bottom of the page and look at the = numbers): = http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_fq_codel_950mbit/= index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_src_cake_dst= _950mbit/index.html = = http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_dsrc_cake_dd= st_950mbit/index.html = * Anyone see anything in my =E2=80=9CFlow Isolation Mix=E2=80=9D tests? = Those are a little hard to read. :) They used to be combined with a VoIP = test but I don=E2=80=99t have a d-itg setup now. *** Round 1 Plans *** - Update cake to latest (will do this with every round) - Remove all =E2=80=9Cfull-duplex limiting=E2=80=9D (egress and ingress) = tests as I don=E2=80=99t see the use here - Add cake=E2=80=99s =E2=80=9Cethernet=E2=80=9D keyword to all rate = limited tests - Lower standard rate limit to 900mbit ensure no queue loss = (particularly for nflows tests) - Take standard rrulbe tests to even lower bandwidths: 1mbit, 10mbit, = 50mbit, 100mbit - Add bql tests (no rate limiting) - Add 100us, 1ms, 2ms, 3ms, 5ms, 8ms to Cake RTT tests - Add nflows tests at lower bandwidths - Fix UDP flood tests (no suitable iperf binary found) - Remove or improve flow isolation tests - Add ethtool output to host info *** Plans for Future Rounds *** - Add flow isolation tests with rtt variation (to look again at problem = I reported in an earlier thread) - Use netem to make a spread of rtts and bandwidths (maybe the most = useful of all?) - Add ack filtering tests - Add VoIP tests (I hope to do this with irtt) - Test BBR - Use qemu to test other archs (I may never get to this, honestly) --Apple-Mail=_5D0AAE19-98BB-4086-8D76-EA8A0C7C0704 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
I have a script (called flenter) which can = run flent with different parameter variations and produce an html = report. I=E2=80=99m very sorry again that this doesn=E2=80=99t yet use = flent=E2=80=99s batch feature- it was started before I knew about = that.

I=E2=80=99= ll use it to produce some =E2=80=9Crounds=E2=80=9D of cake tests, with = notes/analysis of each round and plans for future rounds. If there=E2=80=99= s any feedback on anything, results or what to change, let me know, = otherwise I=E2=80=99ll just take it where I=E2=80=99m able to.

I=E2=80=99m calling this = round 0, because these tests weren=E2=80=99t designed originally for = cake at all, but it=E2=80=99s a vastly stripped down version of tests I = was in the middle of doing for point-to-point WiFi. The tests need a lot = of changes for the next round to focus more on cake and those are noted = at the end.

Round 0 index of all tests:

http://www.drhleny.cz/bufferbloat/cake/round0/


*** Notes/Analysis ***

* When = =E2=80=9Cover-limited=E2=80=9D to 200mbit, pfifo bloats everything, sfq = improves the UDP flows but bloats TCP rtt, and cake clearly holds the = lowest rtt for the ping/udp flows as well as TCP rtt, even when compared = to fq_codel:

http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg= _pfifo_200mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg= _sfq_200mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_200mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_200mbit/index.html


* At 950 mbit rate limiting, fq_codel = holds latency of the ping and udp flows a bit lower than cake, whereas = cake holds tcp rtt a bit lower. sfq does pretty well actually and pfifo = is, well, pfifo:

http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg= _pfifo_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg= _sfq_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_950mbit/index.html


* This is a little surprising, at 950mbit rate limiting with = nflows 8/8, 16/16 and 32/32, fq_codel seems to outperform cake both in = TCP rtt and latency of the UDP flows. Is cake=E2=80=99s =E2=80=9Cethernet=E2= =80=9D keyword is really crucial here, or is there a difference between = Cake and fq_codel at these high bandwidths? I=E2=80=99ll add it in my = later tests. Also I=E2=80=99m losing the queue with 64/64 flows so will = lower the bandwidth limit.

http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_pfifo_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_sfq_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_fq_codel_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_cake_950mbit/index.html


* Obviously cake (with diffserv3 and not besteffort), roundly = defeats everything else in the torrent test because of the = de-prioritization of the background flows:

http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_pfifo_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_sfq_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_fq_codel_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cakebe_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cake_950mbit/index.html


* The benefit to lowering cake=E2=80=99s rtt parameter in an = Ethernet environment can be seen on the TCP rtt:

http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_100ms_rrulbe_eg_cake_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_60ms_rrulbe_eg_cake_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_20ms_rrulbe_eg_cake_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_10ms_rru= lbe_eg_cake_950mbit/index.html


* = I=E2=80=99m a little surprised that fq_codel holds UDP flow latency a = little lower at "target 1ms interval 10ms" than cake=E2=80=99s "rtt = 10ms=E2=80=9D. It almost seems like a trend that Cake outperforms at = lower bandwidths and fq_codel at higher bandwidths.

http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t5ms_i100ms_rrulbe_eg_fq_codel_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t3ms_i60ms_rrulbe_eg_fq_codel_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t2ms_i20ms_rrulbe_eg_fq_codel_950mbit/index.html=
http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t1ms_i10ms_rrulbe_eg_fq_codel_950mbit/index.html


* ECN helps marginally with UDP flow = rtt, but I=E2=80=99ve never seen ECN do very much. When does it help the = most?

http://www.drhleny.cz/bufferbloat/cake/round0/ecn_off_rrulbe_eg_cake_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/ecn_on_rrulbe_eg_cake_950mbit/index.html


* Cake=E2=80=99s =E2=80=9Cethernet=E2=80=9D parameter helps a = bit, I=E2=80=99ll add it to all other rate limited tests in my next = round:

http://www.drhleny.cz/bufferbloat/cake/round0/cake_overhead_rrulbe_eg_cake_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/apu2-eth/cake_ove= rhead_rrulbe_eg_cakeeth_950mbit/index.html


* = Cake=E2=80=99s host isolation clearly works, but I=E2=80=99m a little = surprised that =E2=80=9Csrchost/dsthost=E2=80=9D is more fair on a host = level than =E2=80=9Cdual-srchost/dual-dsthost=E2=80=9D (I usually find = it easiest to just scroll to the bottom of the page and look at the = numbers):

http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_fq_codel_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_src_cake_dst_950mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_dsrc_cake_ddst_950mbit/index.html


* Anyone see anything in my =E2=80=9CFlow Isolation Mix=E2=80=9D= tests? Those are a little hard to read. :) They used to be combined = with a VoIP test but I don=E2=80=99t have a d-itg setup now.


*** Round 1 Plans ***

- Update cake to latest (will do this = with every round)
- Remove all =E2=80=9Cfull-duplex = limiting=E2=80=9D (egress and ingress) tests as I don=E2=80=99t see the = use here
- Add cake=E2=80=99s =E2=80=9Cethernet=E2=80= =9D keyword to all rate limited tests
- Lower = standard rate limit to 900mbit ensure no queue loss (particularly for = nflows tests)
- Take standard rrulbe tests to even = lower bandwidths: 1mbit, 10mbit, 50mbit, 100mbit
- = Add bql tests (no rate limiting)
- Add 100us, 1ms, = 2ms, 3ms, 5ms, 8ms to Cake RTT tests
- Add nflows = tests at lower bandwidths
- Fix UDP flood tests (no = suitable iperf binary found)
- Remove or improve = flow isolation tests
- Add ethtool = output to host info


*** Plans for Future = Rounds ***

- Add flow isolation tests with rtt variation (to look again = at problem I reported in an earlier thread)
- Use = netem to make a spread of rtts and bandwidths (maybe the most useful of = all?)
- Add ack filtering tests
- Add VoIP tests (I hope to do this with = irtt)
- Test BBR
- Use qemu to test other archs (I may never get to this, = honestly)

= --Apple-Mail=_5D0AAE19-98BB-4086-8D76-EA8A0C7C0704--