Cake - FQ_codel the next generation
 help / color / mirror / Atom feed
From: Dave Taht <dave.taht@gmail.com>
To: Cake List <cake@lists.bufferbloat.net>
Subject: [Cake] a bunch of sch_cake vs everything tests at 100mbit on net-next
Date: Tue, 31 Jul 2018 14:12:58 -0700	[thread overview]
Message-ID: <CAA93jw5mEWTYzwmOmTPBwds7cKUt+Dm4oh3sbBXgu7EvoGc-YA@mail.gmail.com> (raw)

[-- Attachment #1: Type: text/plain, Size: 2631 bytes --]

I wanted to look at a few things - cpu usage, 4 different tcps,
different server schedulers, ecn vs non-ecn, sqm fq_codel simplest.qos
vs cake, etc, etc. I just did tcp_ndown tests of 128 flows to see what
happened for starters. I also tried to capture tcp_cwnd, etc, stats,
but that seems not to be plotting in flent....

My first objective, of course, was to make sure upstream cake didn't
crash. It didn't. No real surprises, cake ecn causes some more
collateral inter-packet latency damage, cake (aside from general cpu
over-usage) is a mild win across the board... including, surprisingly,
tail drop queue depth. (see below)

metric ton of flent files: http://www.taht.net/~d/cake_128flows.tgz
(script therein)
topology: server -> apu2 -> switch -> client (no switch between the
apu2 and server)

A couple notes:

1) ecn_vs_bbr_ineffective.png:

vs bbr in ecn mode, (thus rendering codel or cake ineffective and
reverting to tail drop) "sqm simplest.qos" at this speed (100mbit)
uses a too large packet limit vs what cake uses. I will argue in favor
of using the new "memlimit" parameter to fq_codel in the sqm scripts
to better limit the buffer size.

bbr is remarkably good for taildrop especially given how gnarly and
unreasonable this set of tests is. I was tempted to tweak cake's
memlimit value in half... but certainly also tweak sqm.

see assumptiontaildrop.png

also, cake oscillates interestingly compared against itself in ecn vs
noecn on a couple tests and tcps.

bbr noecn wins in terms of queue depth when cake's aqm dropping it...

2) cpuwise, on egress cake besteffort flows bandwidth 100mbit is
~5-20% slower than the equivalent htb+fq_codel.

gso-splitting or not does not appear to change much cpu at these
speeds, but I imagine it will
start to matter as we approach a gbit.

3) cake starts marking ecn sooner than fq_codel does (gso? cobalt? no
idea). This doesn't do
any good on this 128 flows through 100mbit test (at least on
everything but bbr) as we are bound by capping cwnd reductions I
think. bbr uses gain to control the pacing rate and somewhat ignores
cwnd.

I'm thinking that getting more than one CE per observed RTT should
lead to even more rate reductions in cubic, etc. More CEs per RTT
means your RTT estimate is way out of wack.

Too bad we can't have fractional cwnds!

4) You can clearly see the effect of the giant GSO burp from starting
128 flows at the same time, not that anybody sane would do that....

anyway, back to the day job

-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

[-- Attachment #2: assumptiontaildrop.png --]
[-- Type: image/png, Size: 107071 bytes --]

[-- Attachment #3: cake_besteffort_flows_slower.png --]
[-- Type: image/png, Size: 187387 bytes --]

[-- Attachment #4: cantcomparedropsbecauseofgso.png --]
[-- Type: image/png, Size: 146374 bytes --]

[-- Attachment #5: cakelosinginterpacketlatency.png --]
[-- Type: image/png, Size: 76098 bytes --]

[-- Attachment #6: cakeeatsmorecpu.png --]
[-- Type: image/png, Size: 214338 bytes --]

[-- Attachment #7: ecncakefaster.png --]
[-- Type: image/png, Size: 163966 bytes --]

[-- Attachment #8: gso-splitmaybe.png --]
[-- Type: image/png, Size: 100694 bytes --]

[-- Attachment #9: ecn_vs_bbr_ineffective.png --]
[-- Type: image/png, Size: 235188 bytes --]

             reply	other threads:[~2018-07-31 21:13 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-07-31 21:12 Dave Taht [this message]
2018-08-01  9:52 ` Toke Høiland-Jørgensen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.bufferbloat.net/postorius/lists/cake.lists.bufferbloat.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAA93jw5mEWTYzwmOmTPBwds7cKUt+Dm4oh3sbBXgu7EvoGc-YA@mail.gmail.com \
    --to=dave.taht@gmail.com \
    --cc=cake@lists.bufferbloat.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox