From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.taht.net (mail.taht.net [IPv6:2a01:7e00::f03c:91ff:feae:7028]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 055003BA8E for ; Sun, 26 Nov 2017 13:19:08 -0500 (EST) Received: from nemesis.taht.net (unknown [IPv6:2603:3024:1536:86f0:2e0:4cff:fec1:1206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.taht.net (Postfix) with ESMTPSA id 48BCC21473; Sun, 26 Nov 2017 18:19:07 +0000 (UTC) From: Dave Taht To: Pete Heist Cc: Cake List References: <6BE29FE9-6E32-4324-8B56-6BB3B6E5F033@gmail.com> Date: Sun, 26 Nov 2017 10:19:05 -0800 In-Reply-To: <6BE29FE9-6E32-4324-8B56-6BB3B6E5F033@gmail.com> (Pete Heist's message of "Sun, 26 Nov 2017 09:12:46 +0100") Message-ID: <877eucx5ra.fsf@nemesis.taht.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] cake flenter results round 0 X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Nov 2017 18:19:09 -0000 Pete Heist writes: > I have a script (called flenter) which can run flent with different param= eter > variations and produce an html report. I=E2=80=99m very sorry again that = this doesn=E2=80=99t > yet use flent=E2=80=99s batch feature- it was started before I knew about= that. > > I=E2=80=99ll use it to produce some =E2=80=9Crounds=E2=80=9D of cake test= s, with notes/analysis of each > round and plans for future rounds. If there=E2=80=99s any feedback on any= thing, results > or what to change, let me know, otherwise I=E2=80=99ll just take it where= I=E2=80=99m able to. > > I=E2=80=99m calling this round 0, because these tests weren=E2=80=99t des= igned originally for > cake at all, but it=E2=80=99s a vastly stripped down version of tests I w= as in the > middle of doing for point-to-point WiFi. The tests need a lot of changes = for the > next round to focus more on cake and those are noted at the end. > > Round 0 index of all tests: > > http://www.drhleny.cz/bufferbloat/cake/round0/ > > *** Notes/Analysis *** > > * When =E2=80=9Cover-limited=E2=80=9D to 200mbit, pfifo bloats everything= , sfq improves the UDP > flows but bloats TCP rtt, and cake clearly holds the lowest rtt for the p= ing/udp > flows as well as TCP rtt, even when compared to fq_codel: > > http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_200= mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_200mb= it/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_= 200mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_200m= bit/index.html > > * At 950 mbit rate limiting, fq_codel holds latency of the ping and udp f= lows a > bit lower than cake, whereas cake holds tcp rtt a bit lower. sfq does pre= tty > well actually and pfifo is, well, pfifo: > > http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_pfifo_950= mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_sfq_950mb= it/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_fq_codel_= 950mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/eg_csrt_rrulbe_eg_cake_950m= bit/index.html > > * This is a little surprising, at 950mbit rate limiting with nflows 8/8, = 16/16 > and 32/32, fq_codel seems to outperform cake both in TCP rtt and latency = of the > UDP flows. Is cake=E2=80=99s =E2=80=9Cethernet=E2=80=9D keyword is really= crucial here, or is there a > difference between Cake and fq_codel at these high bandwidths? I=E2=80=99= ll add it in my > later tests. Also I=E2=80=99m losing the queue with 64/64 flows so will l= ower the > bandwidth limit. > > http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_pfifo_950mb= it/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_sfq_950mbit= /index.html > http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_fq_codel_95= 0mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/nflows_32_32_eg_cake_950mbi= t/index.html > > * Obviously cake (with diffserv3 and not besteffort), roundly defeats eve= rything > else in the torrent test because of the de-prioritization of the backgrou= nd > flows: I think you can safely drop pfifo from future tests. I'd rather like combined charts so it is possible to eyeball these differences directly. Just between fq_codel and cake. Or, is there a tarball available to browse this stuff? Another nice thing to try capturing is queue depth/loss/marks/etc, which is --test-parameter qdisc_stats_hosts=3DX,y,z and qdisc_stats_interfaces=3D There's also capturing the tcp statistics on the server that is possible. > > http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_pfifo_950mbi= t/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_sfq_950mbit/= index.html > http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_fq_codel_950= mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cakebe_950mb= it/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/tor_rrultor_eg_cake_950mbit= /index.html > > * The benefit to lowering cake=E2=80=99s rtt parameter in an Ethernet env= ironment can be > seen on the TCP rtt: > > http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_100ms_rrulbe_eg_ca= ke_950mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_60ms_rrulbe_eg_cak= e_950mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_20ms_rrulbe_eg_cak= e_950mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/cake_rtt_10ms_rrulbe_eg_cak= e_950mbit/index.html > > * I=E2=80=99m a little surprised that fq_codel holds UDP flow latency a l= ittle lower at > "target 1ms interval 10ms" than cake=E2=80=99s "rtt 10ms=E2=80=9D. It alm= ost seems like a trend > that Cake outperforms at lower bandwidths and fq_codel at higher bandwidt= hs. Since fq_codel supports superpackets and cake peels them, we have a cpu and latency hit that originates from that. Also the codel derived algorithm in cake differs quite significantly from mainline codel, and my principal gripe about it has been that it has not been extensively tested against higher delays. > > http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t5ms_i100ms_rru= lbe_eg_fq_codel_950mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t3ms_i60ms_rrul= be_eg_fq_codel_950mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t2ms_i20ms_rrul= be_eg_fq_codel_950mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/fq_codel_ti_t1ms_i10ms_rrul= be_eg_fq_codel_950mbit/index.html > > * ECN helps marginally with UDP flow rtt, but I=E2=80=99ve never seen ECN= do very much. > When does it help the most? > > http://www.drhleny.cz/bufferbloat/cake/round0/ecn_off_rrulbe_eg_cake_950m= bit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/ecn_on_rrulbe_eg_cake_950mb= it/index.html > > * Cake=E2=80=99s =E2=80=9Cethernet=E2=80=9D parameter helps a bit, I=E2= =80=99ll add it to all other rate limited > tests in my next round: > > http://www.drhleny.cz/bufferbloat/cake/round0/cake_overhead_rrulbe_eg_cak= e_950mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/apu2-eth/cake_overhead_rrul= be_eg_cakeeth_950mbit/index.html > > * Cake=E2=80=99s host isolation clearly works, but I=E2=80=99m a little s= urprised that > =E2=80=9Csrchost/dsthost=E2=80=9D is more fair on a host level than =E2= =80=9Cdual-srchost/dual-dsthost=E2=80=9D > (I usually find it easiest to just scroll to the bottom of the page and l= ook at > the numbers): > > http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_fq_codel_950mbit= /index.html > http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_src_cake_ds= t_950mbit/index.html > http://www.drhleny.cz/bufferbloat/cake/round0/hostiso_eg_cake_dsrc_cake_d= dst_950mbit/index.html > > * Anyone see anything in my =E2=80=9CFlow Isolation Mix=E2=80=9D tests? T= hose are a little hard > to read. :) They used to be combined with a VoIP test but I don=E2=80=99t= have a d-itg > setup now. I look forward to you adding OWD irtt based tests. > *** Round 1 Plans *** > > - Update cake to latest (will do this with every round) > - Remove all =E2=80=9Cfull-duplex limiting=E2=80=9D (egress and ingress) = tests as I don=E2=80=99t see > the use here > - Add cake=E2=80=99s =E2=80=9Cethernet=E2=80=9D keyword to all rate limit= ed tests > - Lower standard rate limit to 900mbit ensure no queue loss (particularly= for > nflows tests) > - Take standard rrulbe tests to even lower bandwidths: 1mbit, 10mbit, 50m= bit, > 100mbit > - Add bql tests (no rate limiting) > - Add 100us, 1ms, 2ms, 3ms, 5ms, 8ms to Cake RTT tests > - Add nflows tests at lower bandwidths > - Fix UDP flood tests (no suitable iperf binary found) > - Remove or improve flow isolation tests > - Add ethtool output to host info > > *** Plans for Future Rounds *** > > - Add flow isolation tests with rtt variation (to look again at problem I > reported in an earlier thread) > - Use netem to make a spread of rtts and bandwidths (maybe the most usefu= l of > all?) Yes. > - Add ack filtering tests > - Add VoIP tests (I hope to do this with irtt) > - Test BBR > - Use qemu to test other archs (I may never get to this, honestly) > > > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake