[Cake] cake flenter results round 1

[Cake] cake flenter results round 1

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 8452 bytes --]

http://www.drhleny.cz/bufferbloat/cake/round1/ <http://www.drhleny.cz/bufferbloat/cake/round1/>

Round 1 Tarball: http://www.drhleny.cz/bufferbloat/cake/round1.tgz <http://www.drhleny.cz/bufferbloat/cake/round1.tgz>

Round 0 Tarball (previous run): http://www.drhleny.cz/bufferbloat/cake/round0.tgz <http://www.drhleny.cz/bufferbloat/cake/round0.tgz>

*** Notes/Analysis ***

* New bql tests show the effectiveness of cake’s TSO/GSO/GRO “peeling” vs fq_codel? Or am I seeing an mq artifact on my 4-queue device?

http://www.drhleny.cz/bufferbloat/cake/round1/bql_csrt_rrulbe_eg_fq_codel_nolimit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/bql_csrt_rrulbe_eg_fq_codel_nolimit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/bql_csrt_rrulbe_eg_cakeeth_nolimit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/bql_csrt_rrulbe_eg_cakeeth_nolimit/index.html>

* Cake holds TCP RTT to half that of fq_codel at 10mbit bandwidth. I like to call this technique of rate limiting well below the interface’s maximum "over-limiting”, which seems to work well with stable point-to-point WiFi connections. (Of course, point-to-multipoint or unstable rates requires the new ath9k/10k driver changes as limiting in this way would not be effective, well explained here- https://www.youtube.com/watch?v=Rb-UnHDw02o <https://www.youtube.com/watch?v=Rb-UnHDw02o>):

http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_sfq_10.0mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_sfq_10.0mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_fq_codel_10.0mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_fq_codel_10.0mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_cakeeth_10.0mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_cakeeth_10.0mbit/index.html>

* Cake at 950mbit performed just as well as fq_codel, vs the round0 runs where fq_codel had a bit of an advantage. Perhaps the addition of the “ethernet” keyword did this?

http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_fq_codel_950mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_fq_codel_950mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_cakeeth_950mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/eg_csrt_rrulbe_eg_cakeeth_950mbit/index.html>

** I’m finding the "32 Flows, RRUL Best-Effort” tests fascinating to look at. It might be possible to spot implementation differences between fq_codel and cake from these.

* At 10mbit, cake and fq_codel are better at most things than sfq by an order of magnitude or more. But interestingly, at this bandwidth fq_codel’s results look a bit better than cake, where total bandwidth for fq_codel is higher (4.78/9.12mbit for fq_codel and 3.91/8.63mbit for cake) and ping latency a bit lower (1.79ms vs 1.92ms), and TCP RTT significantly better (~30ms vs ~45 ms). Maybe cake's “ethernet” keyword at these low bandwidths affects a test like this disproportionally?

http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_sfq_10.0mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_sfq_10.0mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_fq_codel_10.0mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_fq_codel_10.0mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_cakeeth_10.0mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_cakeeth_10.0mbit/index.html>

* At 100mbit, the situation reverses, with fq_codel TCP RTT above 10ms and cake around 4.75ms.

http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_fq_codel_100mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_fq_codel_100mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_cakeeth_100mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_cakeeth_100mbit/index.html>

* And then above 200mbit, fq_codel performs considerably better than cake at the 32/32 flow tests. At 900mbit, UDP/ping is 1.1ms for fq_codel and 10ms for cake. TCP RTT is ~6.5ms for fq_codel and ~12ms for cake. Dave’s earlier explanation probably applies here: "Since fq_codel supports superpackets and cake peels them, we have a cpu and latency hit that originates from that. Also the code derived algorithm in cake differs quite significantly from mainline codel, and my principal gripe about it has been that it has not been extensively tested against higher delays."

http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_fq_codel_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_fq_codel_900mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_cakeeth_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_cakeeth_900mbit/index.html>

* On the Cake RTT tests, we take about a 15% hit in total TCP throughput at rtt 1ms vs rtt 10ms (1454mbit vs 1700mbit), and a 55% hit at rtt 100us (which is why you’d probably only consider using that on 10gbit links). If we don’t remove the ‘ethernet’ keyword altogether, I guess I’d like to see it at least be 10ms, as TCP RTT only goes from around 0.8ms to 1.8ms, which I don’t think makes a huge latency difference in real world terms. Or it might be another argument for removing datacentre, ethernet and metro altogether, because there are tradeoffs to decide about.

http://www.drhleny.cz/bufferbloat/cake/round1/cake_rtt_10ms_rrulbe_eg_cake_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/cake_rtt_10ms_rrulbe_eg_cake_900mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/cake_rtt_1ms_rrulbe_eg_cake_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/cake_rtt_1ms_rrulbe_eg_cake_900mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/cake_rtt_100us_rrulbe_eg_cake_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/cake_rtt_100us_rrulbe_eg_cake_900mbit/index.html>

* I wonder if the UDP flood tests really work at 900mbit:

http://www.drhleny.cz/bufferbloat/cake/round1/udpflood_eg_fq_codel_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/udpflood_eg_fq_codel_900mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/udpflood_eg_cakeeth_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/udpflood_eg_cakeeth_900mbit/index.html>

* Again as before, I’m surprised that srchost/dsthost is much more fair. Numbers that follow are 1-flow/12-flow throughput. For srchost/dsthost, it’s 413/439mbit up, 413/447 down and for dual-srchost/dual-dsthost it’s 126/647mbit up, 77/749mbit down. Rampant speculation: does this have to do with the “peeling”? And should we / do we even do peeling with soft rate limiting? I think I saw it help with bql(?), but I’m not sure I’ve seen it help when rate limited below the interface’s rate.

http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_src_cake_dst_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_src_cake_dst_900mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_dsrc_cake_ddst_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_dsrc_cake_ddst_900mbit/index.html>

* I still need a better understanding of what triple-isolate does. It isn’t clear to me from the man page. Results here are similar to dual-srchost/dual-dsthost:

http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_dsrc_cake_ddst_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_dsrc_cake_ddst_900mbit/index.html>


*** Round 2 Plans ***

- Add bql tests to anywhere rate limiting is used
- Add ethernet keyword to host isolation tests
- Add ethtool output to host info
- Remove or improve flow isolation tests
- Add host isolation tests with rtt variation (to look again at problem I reported in an earlier thread)

*** Future Plans ***

- Use netem to make a spread of rtts and bandwidths
- Add VoIP tests (I hope to do this with irtt)
- Add ack filtering tests
- Test BBR
- Use qemu to test other archs (I may never get to this, honestly)


[-- Attachment #2: Type: text/html, Size: 11128 bytes --]

Re: [Cake] cake flenter results round 1

[Toke Høiland-Jørgensen]

Pete Heist <peteheist@gmail.com> writes:

> * I wonder if the UDP flood tests really work at 900mbit:

Did you set the UDP bandwidth? --test-parameter udp_bandwidth=1000M for
instance

-Toke

Re: [Cake] cake flenter results round 1

[Pete Heist]

> On Nov 27, 2017, at 12:10 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Pete Heist <peteheist@gmail.com> writes:
> 
>> * I wonder if the UDP flood tests really work at 900mbit:
> 
> Did you set the UDP bandwidth? --test-parameter udp_bandwidth=1000M for
> instance

Aha, that’s undoubtedly the problem. Will do that next time, thanks!

Re: [Cake] cake flenter results round 1

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 665 bytes --]

Here's the difference between "srchost" and "dual-srchost": the latter
imposes per-flow fairness on traffic to each host, with a separate
queue/AQM per flow like with "flows".  The former only has one queue/AQM
per host.

Analogously for dsthost.

Then "hosts" mode allocates a separate queue for each host-pair encountered.

But "triple-isolate" isn't quite analogous to "hosts".  Instead it tries to
heuristically behave like either of the dual modes, depending on which one
is likely to be on the LAN side of the link.  This allows it to be a
reasonable default setting, though the "dual" modes will perform more
reliably if chosen correctly.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 839 bytes --]

Re: [Cake] cake flenter results round 1

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1811 bytes --]


> On Nov 27, 2017, at 1:18 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
> Here's the difference between "srchost" and "dual-srchost": the latter imposes per-flow fairness on traffic to each host, with a separate queue/AQM per flow like with "flows".  The former only has one queue/AQM per host.
> 
> Analogously for dsthost.
> 
> Then "hosts" mode allocates a separate queue for each host-pair encountered.
> 
> But "triple-isolate" isn't quite analogous to "hosts".  Instead it tries to heuristically behave like either of the dual modes, depending on which one is likely to be on the LAN side of the link.  This allows it to be a reasonable default setting, though the "dual" modes will perform more reliably if chosen correctly.
> 

Thanks, so as I understand it, fairness at the host level for srchost and dual-srchost should be the same, only with dual-srchost there should additionally be fairness among each host’s flows.

If that’s right, I’m confused by this result where there are four clients, each with a separate source IP connecting to the same server IP:

1: tcp_up with 1 flow
2: tcp_up with 12 flows
3: tcp_down with 1 flow
4: tcp_down with 12 flows

With srchost/dsthost it’s fair at the host level (easiest to look at TCP upload/TCP upload sum and TCP download/TCP download sum numbers at the bottom of the page):

http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_src_cake_dst_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_src_cake_dst_900mbit/index.html>

and with dual-srchost/dual-dsthost it’s not:

http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_dsrc_cake_ddst_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_dsrc_cake_ddst_900mbit/index.html>


[-- Attachment #2: Type: text/html, Size: 2707 bytes --]

Re: [Cake] cake flenter results round 1

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 215 bytes --]

Looking at the Cake stats for that run, it doesn't seem to have been
signalling congestion at all, when you'd expect it to with 13 bulk flows
running through it.  Something odd is going on there.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 266 bytes --]

Re: [Cake] cake flenter results round 1

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 448 bytes --]


> On Nov 27, 2017, at 3:01 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
> Looking at the Cake stats for that run, it doesn't seem to have been signalling congestion at all, when you'd expect it to with 13 bulk flows running through it.  Something odd is going on there.
> 
Ok, I’ll re-run this at a lower rate to make sure I’m not running out of CPU, which I suppose would be more likely with the dual-whatever keywords than not.


[-- Attachment #2: Type: text/html, Size: 872 bytes --]

Re: [Cake] cake flenter results round 1

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1295 bytes --]


> On Nov 27, 2017, at 3:07 PM, Pete Heist <peteheist@gmail.com> wrote:
> 
>> On Nov 27, 2017, at 3:01 PM, Jonathan Morton <chromatix99@gmail.com <mailto:chromatix99@gmail.com>> wrote:
>> Looking at the Cake stats for that run, it doesn't seem to have been signalling congestion at all, when you'd expect it to with 13 bulk flows running through it.  Something odd is going on there.
>> 
> Ok, I’ll re-run this at a lower rate to make sure I’m not running out of CPU, which I suppose would be more likely with the dual-whatever keywords than not.

That’s almost for sure the problem, as fairness works as expected at 500mbit instead of 950mbit:

http://drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_dsrc_cake_ddst_500mbit/index.html <http://drhleny.cz/bufferbloat/cake/round1/hostiso_eg_cake_dsrc_cake_ddst_500mbit/index.html>

Either I have to test at this reduced rate or use fewer flows for this test. The netperf instances were probably not able to output in total at line rate, and going from srchost/dsthost to dual-srchost/dual-dsthost was probably enough to make the difference in CPU consumption.

Follow-up question, in theory, would it be possible for cake to know that it doesn’t have enough CPU to operate properly so it can emit a warning every so often?

[-- Attachment #2: Type: text/html, Size: 2134 bytes --]

Re: [Cake] cake flenter results round 1

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 201 bytes --]

It's not at all obvious how we'd detect that.  Packets are staying in the
queue for less time than the codel target, which is exactly what you'd get
if you weren't saturated at all.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 260 bytes --]

Re: [Cake] cake flenter results round 1

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]


> On Nov 27, 2017, at 3:48 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
> It's not at all obvious how we'd detect that.  Packets are staying in the queue for less time than the codel target, which is exactly what you'd get if you weren't saturated at all.
> 

That makes complete sense when you put it that way. Cake has no way of knowing why the input rate is lower than expected, even if it’s part of the cause.

I don’t think flent can know this either. It can’t easily know the cause for its total output to be lower than expected.

All I know is, this is a common problem in deployments, particularly on low-end hardware like ER-Xs, that can be tricky for users to figure out.

I don’t even think monitoring CPU in general would work. The CPU could be high because it’s doing other calculations, but there’s still enough for cake at a low rate, and there’s no need to warn in that case. I’d be interested in any ideas on how to know this is happening in the system as a whole. So far, there are just various clues that one needs to piece together (no or few drops or marks, less total throughput that expected, high cpu without other external usage, etc). Then it needs to be proven with a test.

Anyway thanks, your clue was what I needed! I need to remember to review the qdisc stats when something unexpected happens.

[-- Attachment #2: Type: text/html, Size: 1970 bytes --]

Re: [Cake] cake flenter results round 1

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 5268 bytes --]

Dear Pete,

I am trying to replicate the unfair behaviour you are seeing with
dual-{src,dst}host, albeit on different hardware and I am getting a fair
distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients). All
running Archlinux, latest cake and patched iproute2-4.14.1, connected with
Gbit ethernet, TSO/GSO/GRO enabled.

Qdisc setup:
----------------
Router:
qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3
dual-dsthost rtt 100.0ms raw

Client A(kernel default):
qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantum
1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn

Client B (kernel default):
qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quantum
1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
----------------


Cli:
----------------
Router:
netserver &

Client A:
flent tcp_1down -H router

Client B:
flent tcp_12down -H router
----------------


Results:
----------------
Router:
qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt
100.0ms raw
 Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 requeues
0)
 backlog 0b 0p requeues 0
 memory used: 1224872b of 15140Kb
 capacity estimate: 900Mbit
                 Bulk   Best Effort      Voice
  thresh     56250Kbit     900Mbit     225Mbit
  target         5.0ms       5.0ms       5.0ms
  interval     100.0ms     100.0ms     100.0ms
  pk_delay        14us       751us         7us
  av_delay         2us       642us         1us
  sp_delay         1us         1us         1us
  pkts          109948     4601651       14315
  bytes      160183242  6964893773     1618242
  way_inds           0       21009           0
  way_miss         160         188           5
  way_cols           0           0           0
  drops              0          10           0
  marks              0           0           0
  ack_drop           0           0           0
  sp_flows           0           1           1
  bk_flows           1           0           0
  un_flows           0           0           0
  max_len         7570       68130        1022


Client A:
                           avg       median          # data pts
 Ping (ms) ICMP :         0.11         0.08 ms              350
 TCP download   :       443.65       430.38 Mbits/s         301


Client B:
                             avg       median          # data pts
 Ping (ms) ICMP   :         0.09         0.06 ms              350
 TCP download avg :        37.03        35.87 Mbits/s         301
 TCP download sum :       444.35       430.40 Mbits/s         301
 TCP download::1  :        37.00        35.87 Mbits/s         301
 TCP download::10 :        37.01        35.87 Mbits/s         301
 TCP download::11 :        37.02        35.87 Mbits/s         301
 TCP download::12 :        37.00        35.87 Mbits/s         301
 TCP download::2  :        37.03        35.87 Mbits/s         301
 TCP download::3  :        36.99        35.87 Mbits/s         301
 TCP download::4  :        37.03        35.87 Mbits/s         301
 TCP download::5  :        37.07        35.87 Mbits/s         301
 TCP download::6  :        37.00        35.87 Mbits/s         301
 TCP download::7  :        37.12        35.87 Mbits/s         301
 TCP download::8  :        37.05        35.87 Mbits/s         301
 TCP download::9  :        37.03        35.87 Mbits/s         301
----------------

Does this suggest that it is indeed a problem of an underpowered CPU in
your case?

George


On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist <peteheist@gmail.com> wrote:

>
> On Nov 27, 2017, at 3:48 PM, Jonathan Morton <chromatix99@gmail.com>
> wrote:
>
> It's not at all obvious how we'd detect that.  Packets are staying in the
> queue for less time than the codel target, which is exactly what you'd get
> if you weren't saturated at all.
>
> That makes complete sense when you put it that way. Cake has no way of
> knowing why the input rate is lower than expected, even if it’s part of the
> cause.
>
> I don’t think flent can know this either. It can’t easily know the cause
> for its total output to be lower than expected.
>
> All I know is, this is a common problem in deployments, particularly on
> low-end hardware like ER-Xs, that can be tricky for users to figure out.
>
> I don’t even think monitoring CPU in general would work. The CPU could be
> high because it’s doing other calculations, but there’s still enough for
> cake at a low rate, and there’s no need to warn in that case. I’d be
> interested in any ideas on how to know this is happening in the system as a
> whole. So far, there are just various clues that one needs to piece
> together (no or few drops or marks, less total throughput that expected,
> high cpu without other external usage, etc). Then it needs to be proven
> with a test.
>
> Anyway thanks, your clue was what I needed! I need to remember to review
> the qdisc stats when something unexpected happens.
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>
>

[-- Attachment #2: Type: text/html, Size: 7550 bytes --]

Re: [Cake] cake flenter results round 1

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 6302 bytes --]

Yes, especially since you’ve got higher-end hardware than I. 443.65mbit vs 444.35mbit looks pretty fair. :)

Thanks for reproducing it. I’m going to have to review some of my flenter tests in light of this. I’m getting a handle on the limitations of the APU2 hardware. It’s quite good especially for the price, but has limits on what it can do at Gbit rates. It can actually be useful to test what happens when the CPU is overburdened, only I need to avoid being fooled by it.

You could also add the ‘ethernet’ keyword, which I’m going to add for this test in my next round, although that wouldn’t have fixed what I was seeing anyway...

Pete

> On Nov 27, 2017, at 5:17 PM, Georgios Amanakis <gamanakis@gmail.com> wrote:
> 
> Dear Pete,
> 
> I am trying to replicate the unfair behaviour you are seeing with dual-{src,dst}host, albeit on different hardware and I am getting a fair distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients). All running Archlinux, latest cake and patched iproute2-4.14.1, connected with Gbit ethernet, TSO/GSO/GRO enabled.
> 
> Qdisc setup:
> ----------------
> Router: 
> qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt 100.0ms raw
> 
> Client A(kernel default):
> qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
> 
> Client B (kernel default): 
> qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
> ----------------
> 
> 
> Cli:
> ----------------
> Router:
> netserver &
> 
> Client A:
> flent tcp_1down -H router
> 
> Client B:
> flent tcp_12down -H router
> ----------------
> 
> 
> Results:
> ----------------
> Router:
> qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt 100.0ms raw
>  Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 requeues 0)
>  backlog 0b 0p requeues 0
>  memory used: 1224872b of 15140Kb
>  capacity estimate: 900Mbit
>                  Bulk   Best Effort      Voice
>   thresh     56250Kbit     900Mbit     225Mbit
>   target         5.0ms       5.0ms       5.0ms
>   interval     100.0ms     100.0ms     100.0ms
>   pk_delay        14us       751us         7us
>   av_delay         2us       642us         1us
>   sp_delay         1us         1us         1us
>   pkts          109948     4601651       14315
>   bytes      160183242  6964893773     1618242
>   way_inds           0       21009           0
>   way_miss         160         188           5
>   way_cols           0           0           0
>   drops              0          10           0
>   marks              0           0           0
>   ack_drop           0           0           0
>   sp_flows           0           1           1
>   bk_flows           1           0           0
>   un_flows           0           0           0
>   max_len         7570       68130        1022
> 
> 
> Client A:
>                            avg       median          # data pts
>  Ping (ms) ICMP :         0.11         0.08 ms              350
>  TCP download   :       443.65       430.38 Mbits/s         301
> 
> 
> Client B:
>                              avg       median          # data pts
>  Ping (ms) ICMP   :         0.09         0.06 ms              350
>  TCP download avg :        37.03        35.87 Mbits/s         301
>  TCP download sum :       444.35       430.40 Mbits/s         301
>  TCP download::1  :        37.00        35.87 Mbits/s         301
>  TCP download::10 :        37.01        35.87 Mbits/s         301
>  TCP download::11 :        37.02        35.87 Mbits/s         301
>  TCP download::12 :        37.00        35.87 Mbits/s         301
>  TCP download::2  :        37.03        35.87 Mbits/s         301
>  TCP download::3  :        36.99        35.87 Mbits/s         301
>  TCP download::4  :        37.03        35.87 Mbits/s         301
>  TCP download::5  :        37.07        35.87 Mbits/s         301
>  TCP download::6  :        37.00        35.87 Mbits/s         301
>  TCP download::7  :        37.12        35.87 Mbits/s         301
>  TCP download::8  :        37.05        35.87 Mbits/s         301
>  TCP download::9  :        37.03        35.87 Mbits/s         301
> ----------------
> 
> Does this suggest that it is indeed a problem of an underpowered CPU in your case?
> 
> George
> 
> 
> On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist <peteheist@gmail.com <mailto:peteheist@gmail.com>> wrote:
> 
>> On Nov 27, 2017, at 3:48 PM, Jonathan Morton <chromatix99@gmail.com <mailto:chromatix99@gmail.com>> wrote:
>> It's not at all obvious how we'd detect that.  Packets are staying in the queue for less time than the codel target, which is exactly what you'd get if you weren't saturated at all.
>> 
> 
> That makes complete sense when you put it that way. Cake has no way of knowing why the input rate is lower than expected, even if it’s part of the cause.
> 
> I don’t think flent can know this either. It can’t easily know the cause for its total output to be lower than expected.
> 
> All I know is, this is a common problem in deployments, particularly on low-end hardware like ER-Xs, that can be tricky for users to figure out.
> 
> I don’t even think monitoring CPU in general would work. The CPU could be high because it’s doing other calculations, but there’s still enough for cake at a low rate, and there’s no need to warn in that case. I’d be interested in any ideas on how to know this is happening in the system as a whole. So far, there are just various clues that one needs to piece together (no or few drops or marks, less total throughput that expected, high cpu without other external usage, etc). Then it needs to be proven with a test.
> 
> Anyway thanks, your clue was what I needed! I need to remember to review the qdisc stats when something unexpected happens.
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
> https://lists.bufferbloat.net/listinfo/cake <https://lists.bufferbloat.net/listinfo/cake>
> 
> 


[-- Attachment #2: Type: text/html, Size: 14151 bytes --]

Re: [Cake] cake flenter results round 1

[Dave Taht]

georgios

the result you got was "fair", but you shoul have seen something
closer to 900mbit than 400.

On Mon, Nov 27, 2017 at 8:17 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
> Dear Pete,
>
> I am trying to replicate the unfair behaviour you are seeing with
> dual-{src,dst}host, albeit on different hardware and I am getting a fair
> distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients). All
> running Archlinux, latest cake and patched iproute2-4.14.1, connected with
> Gbit ethernet, TSO/GSO/GRO enabled.
>
> Qdisc setup:
> ----------------
> Router:
> qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3
> dual-dsthost rtt 100.0ms raw
>
> Client A(kernel default):
> qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantum
> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
>
> Client B (kernel default):
> qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quantum
> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
> ----------------
>
>
> Cli:
> ----------------
> Router:
> netserver &
>
> Client A:
> flent tcp_1down -H router
>
> Client B:
> flent tcp_12down -H router
> ----------------
>
>
> Results:
> ----------------
> Router:
> qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt
> 100.0ms raw
>  Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 requeues
> 0)
>  backlog 0b 0p requeues 0
>  memory used: 1224872b of 15140Kb
>  capacity estimate: 900Mbit
>                  Bulk   Best Effort      Voice
>   thresh     56250Kbit     900Mbit     225Mbit
>   target         5.0ms       5.0ms       5.0ms
>   interval     100.0ms     100.0ms     100.0ms
>   pk_delay        14us       751us         7us
>   av_delay         2us       642us         1us
>   sp_delay         1us         1us         1us
>   pkts          109948     4601651       14315
>   bytes      160183242  6964893773     1618242
>   way_inds           0       21009           0
>   way_miss         160         188           5
>   way_cols           0           0           0
>   drops              0          10           0
>   marks              0           0           0
>   ack_drop           0           0           0
>   sp_flows           0           1           1
>   bk_flows           1           0           0
>   un_flows           0           0           0
>   max_len         7570       68130        1022
>
>
> Client A:
>                            avg       median          # data pts
>  Ping (ms) ICMP :         0.11         0.08 ms              350
>  TCP download   :       443.65       430.38 Mbits/s         301
>
>
> Client B:
>                              avg       median          # data pts
>  Ping (ms) ICMP   :         0.09         0.06 ms              350
>  TCP download avg :        37.03        35.87 Mbits/s         301
>  TCP download sum :       444.35       430.40 Mbits/s         301
>  TCP download::1  :        37.00        35.87 Mbits/s         301
>  TCP download::10 :        37.01        35.87 Mbits/s         301
>  TCP download::11 :        37.02        35.87 Mbits/s         301
>  TCP download::12 :        37.00        35.87 Mbits/s         301
>  TCP download::2  :        37.03        35.87 Mbits/s         301
>  TCP download::3  :        36.99        35.87 Mbits/s         301
>  TCP download::4  :        37.03        35.87 Mbits/s         301
>  TCP download::5  :        37.07        35.87 Mbits/s         301
>  TCP download::6  :        37.00        35.87 Mbits/s         301
>  TCP download::7  :        37.12        35.87 Mbits/s         301
>  TCP download::8  :        37.05        35.87 Mbits/s         301
>  TCP download::9  :        37.03        35.87 Mbits/s         301
> ----------------
>
> Does this suggest that it is indeed a problem of an underpowered CPU in your
> case?
>
> George
>
>
> On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist <peteheist@gmail.com> wrote:
>>
>>
>> On Nov 27, 2017, at 3:48 PM, Jonathan Morton <chromatix99@gmail.com>
>> wrote:
>>
>> It's not at all obvious how we'd detect that.  Packets are staying in the
>> queue for less time than the codel target, which is exactly what you'd get
>> if you weren't saturated at all.
>>
>> That makes complete sense when you put it that way. Cake has no way of
>> knowing why the input rate is lower than expected, even if it’s part of the
>> cause.
>>
>> I don’t think flent can know this either. It can’t easily know the cause
>> for its total output to be lower than expected.
>>
>> All I know is, this is a common problem in deployments, particularly on
>> low-end hardware like ER-Xs, that can be tricky for users to figure out.
>>
>> I don’t even think monitoring CPU in general would work. The CPU could be
>> high because it’s doing other calculations, but there’s still enough for
>> cake at a low rate, and there’s no need to warn in that case. I’d be
>> interested in any ideas on how to know this is happening in the system as a
>> whole. So far, there are just various clues that one needs to piece together
>> (no or few drops or marks, less total throughput that expected, high cpu
>> without other external usage, etc). Then it needs to be proven with a test.
>>
>> Anyway thanks, your clue was what I needed! I need to remember to review
>> the qdisc stats when something unexpected happens.
>>
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
>>
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 1

[Sebastian Moeller]

But 444.35 + 443.65 = 888, no?

> On Nov 27, 2017, at 18:33, Dave Taht <dave.taht@gmail.com> wrote:
> 
> georgios
> 
> the result you got was "fair", but you shoul have seen something
> closer to 900mbit than 400.
> 
> On Mon, Nov 27, 2017 at 8:17 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
>> Dear Pete,
>> 
>> I am trying to replicate the unfair behaviour you are seeing with
>> dual-{src,dst}host, albeit on different hardware and I am getting a fair
>> distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients). All
>> running Archlinux, latest cake and patched iproute2-4.14.1, connected with
>> Gbit ethernet, TSO/GSO/GRO enabled.
>> 
>> Qdisc setup:
>> ----------------
>> Router:
>> qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3
>> dual-dsthost rtt 100.0ms raw
>> 
>> Client A(kernel default):
>> qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantum
>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
>> 
>> Client B (kernel default):
>> qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quantum
>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
>> ----------------
>> 
>> 
>> Cli:
>> ----------------
>> Router:
>> netserver &
>> 
>> Client A:
>> flent tcp_1down -H router
>> 
>> Client B:
>> flent tcp_12down -H router
>> ----------------
>> 
>> 
>> Results:
>> ----------------
>> Router:
>> qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt
>> 100.0ms raw
>> Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 requeues
>> 0)
>> backlog 0b 0p requeues 0
>> memory used: 1224872b of 15140Kb
>> capacity estimate: 900Mbit
>>                 Bulk   Best Effort      Voice
>>  thresh     56250Kbit     900Mbit     225Mbit
>>  target         5.0ms       5.0ms       5.0ms
>>  interval     100.0ms     100.0ms     100.0ms
>>  pk_delay        14us       751us         7us
>>  av_delay         2us       642us         1us
>>  sp_delay         1us         1us         1us
>>  pkts          109948     4601651       14315
>>  bytes      160183242  6964893773     1618242
>>  way_inds           0       21009           0
>>  way_miss         160         188           5
>>  way_cols           0           0           0
>>  drops              0          10           0
>>  marks              0           0           0
>>  ack_drop           0           0           0
>>  sp_flows           0           1           1
>>  bk_flows           1           0           0
>>  un_flows           0           0           0
>>  max_len         7570       68130        1022
>> 
>> 
>> Client A:
>>                           avg       median          # data pts
>> Ping (ms) ICMP :         0.11         0.08 ms              350
>> TCP download   :       443.65       430.38 Mbits/s         301
>> 
>> 
>> Client B:
>>                             avg       median          # data pts
>> Ping (ms) ICMP   :         0.09         0.06 ms              350
>> TCP download avg :        37.03        35.87 Mbits/s         301
>> TCP download sum :       444.35       430.40 Mbits/s         301
>> TCP download::1  :        37.00        35.87 Mbits/s         301
>> TCP download::10 :        37.01        35.87 Mbits/s         301
>> TCP download::11 :        37.02        35.87 Mbits/s         301
>> TCP download::12 :        37.00        35.87 Mbits/s         301
>> TCP download::2  :        37.03        35.87 Mbits/s         301
>> TCP download::3  :        36.99        35.87 Mbits/s         301
>> TCP download::4  :        37.03        35.87 Mbits/s         301
>> TCP download::5  :        37.07        35.87 Mbits/s         301
>> TCP download::6  :        37.00        35.87 Mbits/s         301
>> TCP download::7  :        37.12        35.87 Mbits/s         301
>> TCP download::8  :        37.05        35.87 Mbits/s         301
>> TCP download::9  :        37.03        35.87 Mbits/s         301
>> ----------------
>> 
>> Does this suggest that it is indeed a problem of an underpowered CPU in your
>> case?
>> 
>> George
>> 
>> 
>> On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist <peteheist@gmail.com> wrote:
>>> 
>>> 
>>> On Nov 27, 2017, at 3:48 PM, Jonathan Morton <chromatix99@gmail.com>
>>> wrote:
>>> 
>>> It's not at all obvious how we'd detect that.  Packets are staying in the
>>> queue for less time than the codel target, which is exactly what you'd get
>>> if you weren't saturated at all.
>>> 
>>> That makes complete sense when you put it that way. Cake has no way of
>>> knowing why the input rate is lower than expected, even if it’s part of the
>>> cause.
>>> 
>>> I don’t think flent can know this either. It can’t easily know the cause
>>> for its total output to be lower than expected.
>>> 
>>> All I know is, this is a common problem in deployments, particularly on
>>> low-end hardware like ER-Xs, that can be tricky for users to figure out.
>>> 
>>> I don’t even think monitoring CPU in general would work. The CPU could be
>>> high because it’s doing other calculations, but there’s still enough for
>>> cake at a low rate, and there’s no need to warn in that case. I’d be
>>> interested in any ideas on how to know this is happening in the system as a
>>> whole. So far, there are just various clues that one needs to piece together
>>> (no or few drops or marks, less total throughput that expected, high cpu
>>> without other external usage, etc). Then it needs to be proven with a test.
>>> 
>>> Anyway thanks, your clue was what I needed! I need to remember to review
>>> the qdisc stats when something unexpected happens.
>>> 
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>>> 
>> 
>> 
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
>> 
> 
> 
> 
> -- 
> 
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake flenter results round 1

[Dave Taht]

On Mon, Nov 27, 2017 at 9:34 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
> But 444.35 + 443.65 = 888, no?

My bad. I miss-read the test setup. Pre-coffee here, though, that
caused an adrenalin spike.

Yea! per host fairness 1v12! and correct bandwidth on this cpu. :whew:

>
>> On Nov 27, 2017, at 18:33, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> georgios
>>
>> the result you got was "fair", but you shoul have seen something
>> closer to 900mbit than 400.
>>
>> On Mon, Nov 27, 2017 at 8:17 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
>>> Dear Pete,
>>>
>>> I am trying to replicate the unfair behaviour you are seeing with
>>> dual-{src,dst}host, albeit on different hardware and I am getting a fair
>>> distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients). All
>>> running Archlinux, latest cake and patched iproute2-4.14.1, connected with
>>> Gbit ethernet, TSO/GSO/GRO enabled.
>>>
>>> Qdisc setup:
>>> ----------------
>>> Router:
>>> qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3
>>> dual-dsthost rtt 100.0ms raw
>>>
>>> Client A(kernel default):
>>> qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantum
>>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
>>>
>>> Client B (kernel default):
>>> qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quantum
>>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
>>> ----------------
>>>
>>>
>>> Cli:
>>> ----------------
>>> Router:
>>> netserver &
>>>
>>> Client A:
>>> flent tcp_1down -H router
>>>
>>> Client B:
>>> flent tcp_12down -H router
>>> ----------------
>>>
>>>
>>> Results:
>>> ----------------
>>> Router:
>>> qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt
>>> 100.0ms raw
>>> Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 requeues
>>> 0)
>>> backlog 0b 0p requeues 0
>>> memory used: 1224872b of 15140Kb
>>> capacity estimate: 900Mbit
>>>                 Bulk   Best Effort      Voice
>>>  thresh     56250Kbit     900Mbit     225Mbit
>>>  target         5.0ms       5.0ms       5.0ms
>>>  interval     100.0ms     100.0ms     100.0ms
>>>  pk_delay        14us       751us         7us
>>>  av_delay         2us       642us         1us
>>>  sp_delay         1us         1us         1us
>>>  pkts          109948     4601651       14315
>>>  bytes      160183242  6964893773     1618242
>>>  way_inds           0       21009           0
>>>  way_miss         160         188           5
>>>  way_cols           0           0           0
>>>  drops              0          10           0
>>>  marks              0           0           0
>>>  ack_drop           0           0           0
>>>  sp_flows           0           1           1
>>>  bk_flows           1           0           0
>>>  un_flows           0           0           0
>>>  max_len         7570       68130        1022
>>>
>>>
>>> Client A:
>>>                           avg       median          # data pts
>>> Ping (ms) ICMP :         0.11         0.08 ms              350
>>> TCP download   :       443.65       430.38 Mbits/s         301
>>>
>>>
>>> Client B:
>>>                             avg       median          # data pts
>>> Ping (ms) ICMP   :         0.09         0.06 ms              350
>>> TCP download avg :        37.03        35.87 Mbits/s         301
>>> TCP download sum :       444.35       430.40 Mbits/s         301
>>> TCP download::1  :        37.00        35.87 Mbits/s         301
>>> TCP download::10 :        37.01        35.87 Mbits/s         301
>>> TCP download::11 :        37.02        35.87 Mbits/s         301
>>> TCP download::12 :        37.00        35.87 Mbits/s         301
>>> TCP download::2  :        37.03        35.87 Mbits/s         301
>>> TCP download::3  :        36.99        35.87 Mbits/s         301
>>> TCP download::4  :        37.03        35.87 Mbits/s         301
>>> TCP download::5  :        37.07        35.87 Mbits/s         301
>>> TCP download::6  :        37.00        35.87 Mbits/s         301
>>> TCP download::7  :        37.12        35.87 Mbits/s         301
>>> TCP download::8  :        37.05        35.87 Mbits/s         301
>>> TCP download::9  :        37.03        35.87 Mbits/s         301
>>> ----------------
>>>
>>> Does this suggest that it is indeed a problem of an underpowered CPU in your
>>> case?
>>>
>>> George
>>>
>>>
>>> On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist <peteheist@gmail.com> wrote:
>>>>
>>>>
>>>> On Nov 27, 2017, at 3:48 PM, Jonathan Morton <chromatix99@gmail.com>
>>>> wrote:
>>>>
>>>> It's not at all obvious how we'd detect that.  Packets are staying in the
>>>> queue for less time than the codel target, which is exactly what you'd get
>>>> if you weren't saturated at all.
>>>>
>>>> That makes complete sense when you put it that way. Cake has no way of
>>>> knowing why the input rate is lower than expected, even if it’s part of the
>>>> cause.
>>>>
>>>> I don’t think flent can know this either. It can’t easily know the cause
>>>> for its total output to be lower than expected.
>>>>
>>>> All I know is, this is a common problem in deployments, particularly on
>>>> low-end hardware like ER-Xs, that can be tricky for users to figure out.
>>>>
>>>> I don’t even think monitoring CPU in general would work. The CPU could be
>>>> high because it’s doing other calculations, but there’s still enough for
>>>> cake at a low rate, and there’s no need to warn in that case. I’d be
>>>> interested in any ideas on how to know this is happening in the system as a
>>>> whole. So far, there are just various clues that one needs to piece together
>>>> (no or few drops or marks, less total throughput that expected, high cpu
>>>> without other external usage, etc). Then it needs to be proven with a test.
>>>>
>>>> Anyway thanks, your clue was what I needed! I need to remember to review
>>>> the qdisc stats when something unexpected happens.
>>>>
>>>> _______________________________________________
>>>> Cake mailing list
>>>> Cake@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>
>>>
>>>
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>>>
>>
>>
>>
>> --
>>
>> Dave Täht
>> CEO, TekLibre, LLC
>> http://www.teklibre.com
>> Tel: 1-669-226-2619
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 1

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 7553 bytes --]

I’m also finding out how simple it is to miss one little thing when looking at hundreds of test runs. Finding the “god metric” for rrul would make life easier...

> On Nov 27, 2017, at 6:38 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> On Mon, Nov 27, 2017 at 9:34 AM, Sebastian Moeller <moeller0@gmx.de <mailto:moeller0@gmx.de>> wrote:
>> But 444.35 + 443.65 = 888, no?
> 
> My bad. I miss-read the test setup. Pre-coffee here, though, that
> caused an adrenalin spike.
> 
> Yea! per host fairness 1v12! and correct bandwidth on this cpu. :whew:
> 
>> 
>>> On Nov 27, 2017, at 18:33, Dave Taht <dave.taht@gmail.com> wrote:
>>> 
>>> georgios
>>> 
>>> the result you got was "fair", but you shoul have seen something
>>> closer to 900mbit than 400.
>>> 
>>> On Mon, Nov 27, 2017 at 8:17 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
>>>> Dear Pete,
>>>> 
>>>> I am trying to replicate the unfair behaviour you are seeing with
>>>> dual-{src,dst}host, albeit on different hardware and I am getting a fair
>>>> distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients). All
>>>> running Archlinux, latest cake and patched iproute2-4.14.1, connected with
>>>> Gbit ethernet, TSO/GSO/GRO enabled.
>>>> 
>>>> Qdisc setup:
>>>> ----------------
>>>> Router:
>>>> qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3
>>>> dual-dsthost rtt 100.0ms raw
>>>> 
>>>> Client A(kernel default):
>>>> qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantum
>>>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
>>>> 
>>>> Client B (kernel default):
>>>> qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quantum
>>>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
>>>> ----------------
>>>> 
>>>> 
>>>> Cli:
>>>> ----------------
>>>> Router:
>>>> netserver &
>>>> 
>>>> Client A:
>>>> flent tcp_1down -H router
>>>> 
>>>> Client B:
>>>> flent tcp_12down -H router
>>>> ----------------
>>>> 
>>>> 
>>>> Results:
>>>> ----------------
>>>> Router:
>>>> qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt
>>>> 100.0ms raw
>>>> Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 requeues
>>>> 0)
>>>> backlog 0b 0p requeues 0
>>>> memory used: 1224872b of 15140Kb
>>>> capacity estimate: 900Mbit
>>>>                Bulk   Best Effort      Voice
>>>> thresh     56250Kbit     900Mbit     225Mbit
>>>> target         5.0ms       5.0ms       5.0ms
>>>> interval     100.0ms     100.0ms     100.0ms
>>>> pk_delay        14us       751us         7us
>>>> av_delay         2us       642us         1us
>>>> sp_delay         1us         1us         1us
>>>> pkts          109948     4601651       14315
>>>> bytes      160183242  6964893773     1618242
>>>> way_inds           0       21009           0
>>>> way_miss         160         188           5
>>>> way_cols           0           0           0
>>>> drops              0          10           0
>>>> marks              0           0           0
>>>> ack_drop           0           0           0
>>>> sp_flows           0           1           1
>>>> bk_flows           1           0           0
>>>> un_flows           0           0           0
>>>> max_len         7570       68130        1022
>>>> 
>>>> 
>>>> Client A:
>>>>                          avg       median          # data pts
>>>> Ping (ms) ICMP :         0.11         0.08 ms              350
>>>> TCP download   :       443.65       430.38 Mbits/s         301
>>>> 
>>>> 
>>>> Client B:
>>>>                            avg       median          # data pts
>>>> Ping (ms) ICMP   :         0.09         0.06 ms              350
>>>> TCP download avg :        37.03        35.87 Mbits/s         301
>>>> TCP download sum :       444.35       430.40 Mbits/s         301
>>>> TCP download::1  :        37.00        35.87 Mbits/s         301
>>>> TCP download::10 :        37.01        35.87 Mbits/s         301
>>>> TCP download::11 :        37.02        35.87 Mbits/s         301
>>>> TCP download::12 :        37.00        35.87 Mbits/s         301
>>>> TCP download::2  :        37.03        35.87 Mbits/s         301
>>>> TCP download::3  :        36.99        35.87 Mbits/s         301
>>>> TCP download::4  :        37.03        35.87 Mbits/s         301
>>>> TCP download::5  :        37.07        35.87 Mbits/s         301
>>>> TCP download::6  :        37.00        35.87 Mbits/s         301
>>>> TCP download::7  :        37.12        35.87 Mbits/s         301
>>>> TCP download::8  :        37.05        35.87 Mbits/s         301
>>>> TCP download::9  :        37.03        35.87 Mbits/s         301
>>>> ----------------
>>>> 
>>>> Does this suggest that it is indeed a problem of an underpowered CPU in your
>>>> case?
>>>> 
>>>> George
>>>> 
>>>> 
>>>> On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist <peteheist@gmail.com> wrote:
>>>>> 
>>>>> 
>>>>> On Nov 27, 2017, at 3:48 PM, Jonathan Morton <chromatix99@gmail.com>
>>>>> wrote:
>>>>> 
>>>>> It's not at all obvious how we'd detect that.  Packets are staying in the
>>>>> queue for less time than the codel target, which is exactly what you'd get
>>>>> if you weren't saturated at all.
>>>>> 
>>>>> That makes complete sense when you put it that way. Cake has no way of
>>>>> knowing why the input rate is lower than expected, even if it’s part of the
>>>>> cause.
>>>>> 
>>>>> I don’t think flent can know this either. It can’t easily know the cause
>>>>> for its total output to be lower than expected.
>>>>> 
>>>>> All I know is, this is a common problem in deployments, particularly on
>>>>> low-end hardware like ER-Xs, that can be tricky for users to figure out.
>>>>> 
>>>>> I don’t even think monitoring CPU in general would work. The CPU could be
>>>>> high because it’s doing other calculations, but there’s still enough for
>>>>> cake at a low rate, and there’s no need to warn in that case. I’d be
>>>>> interested in any ideas on how to know this is happening in the system as a
>>>>> whole. So far, there are just various clues that one needs to piece together
>>>>> (no or few drops or marks, less total throughput that expected, high cpu
>>>>> without other external usage, etc). Then it needs to be proven with a test.
>>>>> 
>>>>> Anyway thanks, your clue was what I needed! I need to remember to review
>>>>> the qdisc stats when something unexpected happens.
>>>>> 
>>>>> _______________________________________________
>>>>> Cake mailing list
>>>>> Cake@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Cake mailing list
>>>> Cake@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/cake
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> Dave Täht
>>> CEO, TekLibre, LLC
>>> http://www.teklibre.com
>>> Tel: 1-669-226-2619
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>> 
> 
> 
> 
> -- 
> 
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com <http://www.teklibre.com/>
> Tel: 1-669-226-2619
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
> https://lists.bufferbloat.net/listinfo/cake <https://lists.bufferbloat.net/listinfo/cake>

[-- Attachment #2: Type: text/html, Size: 26081 bytes --]

Re: [Cake] cake flenter results round 1

[Pete Heist]

Saw that too, but I think his test was only for download, with one client with 1 flow and the second client with 12, so the total of the two seems correct...

> On Nov 27, 2017, at 6:33 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> georgios
> 
> the result you got was "fair", but you shoul have seen something
> closer to 900mbit than 400.
> 
> On Mon, Nov 27, 2017 at 8:17 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
>> Dear Pete,
>> 
>> I am trying to replicate the unfair behaviour you are seeing with
>> dual-{src,dst}host, albeit on different hardware and I am getting a fair
>> distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients). All
>> running Archlinux, latest cake and patched iproute2-4.14.1, connected with
>> Gbit ethernet, TSO/GSO/GRO enabled.
>> 
>> Qdisc setup:
>> ----------------
>> Router:
>> qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3
>> dual-dsthost rtt 100.0ms raw
>> 
>> Client A(kernel default):
>> qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantum
>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
>> 
>> Client B (kernel default):
>> qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quantum
>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn
>> ----------------
>> 
>> 
>> Cli:
>> ----------------
>> Router:
>> netserver &
>> 
>> Client A:
>> flent tcp_1down -H router
>> 
>> Client B:
>> flent tcp_12down -H router
>> ----------------
>> 
>> 
>> Results:
>> ----------------
>> Router:
>> qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt
>> 100.0ms raw
>> Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 requeues
>> 0)
>> backlog 0b 0p requeues 0
>> memory used: 1224872b of 15140Kb
>> capacity estimate: 900Mbit
>>                 Bulk   Best Effort      Voice
>>  thresh     56250Kbit     900Mbit     225Mbit
>>  target         5.0ms       5.0ms       5.0ms
>>  interval     100.0ms     100.0ms     100.0ms
>>  pk_delay        14us       751us         7us
>>  av_delay         2us       642us         1us
>>  sp_delay         1us         1us         1us
>>  pkts          109948     4601651       14315
>>  bytes      160183242  6964893773     1618242
>>  way_inds           0       21009           0
>>  way_miss         160         188           5
>>  way_cols           0           0           0
>>  drops              0          10           0
>>  marks              0           0           0
>>  ack_drop           0           0           0
>>  sp_flows           0           1           1
>>  bk_flows           1           0           0
>>  un_flows           0           0           0
>>  max_len         7570       68130        1022
>> 
>> 
>> Client A:
>>                           avg       median          # data pts
>> Ping (ms) ICMP :         0.11         0.08 ms              350
>> TCP download   :       443.65       430.38 Mbits/s         301
>> 
>> 
>> Client B:
>>                             avg       median          # data pts
>> Ping (ms) ICMP   :         0.09         0.06 ms              350
>> TCP download avg :        37.03        35.87 Mbits/s         301
>> TCP download sum :       444.35       430.40 Mbits/s         301
>> TCP download::1  :        37.00        35.87 Mbits/s         301
>> TCP download::10 :        37.01        35.87 Mbits/s         301
>> TCP download::11 :        37.02        35.87 Mbits/s         301
>> TCP download::12 :        37.00        35.87 Mbits/s         301
>> TCP download::2  :        37.03        35.87 Mbits/s         301
>> TCP download::3  :        36.99        35.87 Mbits/s         301
>> TCP download::4  :        37.03        35.87 Mbits/s         301
>> TCP download::5  :        37.07        35.87 Mbits/s         301
>> TCP download::6  :        37.00        35.87 Mbits/s         301
>> TCP download::7  :        37.12        35.87 Mbits/s         301
>> TCP download::8  :        37.05        35.87 Mbits/s         301
>> TCP download::9  :        37.03        35.87 Mbits/s         301
>> ----------------
>> 
>> Does this suggest that it is indeed a problem of an underpowered CPU in your
>> case?
>> 
>> George
>> 
>> 
>> On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist <peteheist@gmail.com> wrote:
>>> 
>>> 
>>> On Nov 27, 2017, at 3:48 PM, Jonathan Morton <chromatix99@gmail.com>
>>> wrote:
>>> 
>>> It's not at all obvious how we'd detect that.  Packets are staying in the
>>> queue for less time than the codel target, which is exactly what you'd get
>>> if you weren't saturated at all.
>>> 
>>> That makes complete sense when you put it that way. Cake has no way of
>>> knowing why the input rate is lower than expected, even if it’s part of the
>>> cause.
>>> 
>>> I don’t think flent can know this either. It can’t easily know the cause
>>> for its total output to be lower than expected.
>>> 
>>> All I know is, this is a common problem in deployments, particularly on
>>> low-end hardware like ER-Xs, that can be tricky for users to figure out.
>>> 
>>> I don’t even think monitoring CPU in general would work. The CPU could be
>>> high because it’s doing other calculations, but there’s still enough for
>>> cake at a low rate, and there’s no need to warn in that case. I’d be
>>> interested in any ideas on how to know this is happening in the system as a
>>> whole. So far, there are just various clues that one needs to piece together
>>> (no or few drops or marks, less total throughput that expected, high cpu
>>> without other external usage, etc). Then it needs to be proven with a test.
>>> 
>>> Anyway thanks, your clue was what I needed! I need to remember to review
>>> the qdisc stats when something unexpected happens.
>>> 
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>>> 
>> 
>> 
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
>> 
> 
> 
> 
> -- 
> 
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 1

[Dave Taht]

Pete Heist <peteheist@gmail.com> writes:

>> On Nov 27, 2017, at 12:10 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> Pete Heist <peteheist@gmail.com> writes:
>> 
>>> * I wonder if the UDP flood tests really work at 900mbit:
>> 
>> Did you set the UDP bandwidth? --test-parameter udp_bandwidth=1000M for
>> instance
>
> Aha, that’s undoubtedly the problem. Will do that next time, thanks!

If you set it to 1000M on the box driving the test, you will
saturate at the box driving the test.


>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake flenter results round 1

[Pete Heist]

> On Nov 27, 2017, at 7:13 PM, Dave Taht <dave@taht.net> wrote:
> 
> Pete Heist <peteheist@gmail.com> writes:
> 
>>> On Nov 27, 2017, at 12:10 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>> 
>>> Pete Heist <peteheist@gmail.com> writes:
>>> 
>>>> * I wonder if the UDP flood tests really work at 900mbit:
>>> 
>>> Did you set the UDP bandwidth? --test-parameter udp_bandwidth=1000M for
>>> instance
>> 
>> Aha, that’s undoubtedly the problem. Will do that next time, thanks!
> 
> If you set it to 1000M on the box driving the test, you will
> saturate at the box driving the test.

In my setup, flent and the egress queue under test are running on the same box. On the other box is netserver and another egress queue under test.

Re: [Cake] cake flenter results round 1

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1210 bytes --]


> On Nov 27, 2017, at 12:04 PM, Pete Heist <peteheist@gmail.com> wrote:
> 
> * And then above 200mbit, fq_codel performs considerably better than cake at the 32/32 flow tests. At 900mbit, UDP/ping is 1.1ms for fq_codel and 10ms for cake. TCP RTT is ~6.5ms for fq_codel and ~12ms for cake. Dave’s earlier explanation probably applies here: "Since fq_codel supports superpackets and cake peels them, we have a cpu and latency hit that originates from that. Also the code derived algorithm in cake differs quite significantly from mainline codel, and my principal gripe about it has been that it has not been extensively tested against higher delays."
> 
> http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_fq_codel_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_fq_codel_900mbit/index.html>
> http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_cakeeth_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round1/32flows_eg_cakeeth_900mbit/index.html>
I would not be surprised to find out that this result was also due to lack of CPU, since there’s a steady degradation in Cake’s performance above 200mbit. Next time I’ll try 8/8 flows in addition.

[-- Attachment #2: Type: text/html, Size: 1897 bytes --]

Re: [Cake] cake flenter results round 1

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 4589 bytes --]

I reran the test under different cake setup at the server.
----------------------

With "ethernet":
qdisc cake 800f: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt
100.0ms noatm overhead 38 via-ethernet mpu 84

Client A:
                           avg       median          # data pts
 Ping (ms) ICMP :         0.12         0.08 ms              350
 TCP download   :       439.82       417.15 Mbits/s         301

Client B:
                             avg       median          # data pts
 Ping (ms) ICMP   :         0.09         0.06 ms              350
 TCP download avg :        36.73        34.76 Mbits/s         301
 TCP download sum :       440.72       417.15 Mbits/s         301
 TCP download::1  :        36.71        34.76 Mbits/s         301
 TCP download::10 :        36.75        34.76 Mbits/s         301
 TCP download::11 :        36.70        34.76 Mbits/s         301
 TCP download::12 :        36.75        34.76 Mbits/s         301
 TCP download::2  :        36.77        34.76 Mbits/s         301
 TCP download::3  :        36.71        34.76 Mbits/s         301
 TCP download::4  :        36.75        34.76 Mbits/s         301
 TCP download::5  :        36.73        34.76 Mbits/s         301
 TCP download::6  :        36.68        34.76 Mbits/s         301
 TCP download::7  :        36.74        34.76 Mbits/s         301
 TCP download::8  :        36.74        34.76 Mbits/s         301
 TCP download::9  :        36.69        34.76 Mbits/s         301
----------------------

With "ethernet lan":
qdisc cake 8010: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt
1.0ms noatm overhead 38 via-ethernet mpu 84

Client A:
                           avg       median          # data pts
 Ping (ms) ICMP :         0.28         0.27 ms              350
 TCP download   :       333.33       311.52 Mbits/s         301

Client B:
                             avg       median          # data pts
 Ping (ms) ICMP   :         0.26         0.23 ms              350
 TCP download avg :        43.12        42.19 Mbits/s         301
 TCP download sum :       517.41       506.28 Mbits/s         301
 TCP download::1  :        43.59        42.16 Mbits/s         301
 TCP download::10 :        43.13        42.23 Mbits/s         301
 TCP download::11 :        43.12        42.22 Mbits/s         301
 TCP download::12 :        43.10        42.13 Mbits/s         301
 TCP download::2  :        43.21        42.18 Mbits/s         301
 TCP download::3  :        42.94        42.17 Mbits/s         301
 TCP download::4  :        43.04        42.12 Mbits/s         301
 TCP download::5  :        43.17        42.16 Mbits/s         301
 TCP download::6  :        43.01        42.12 Mbits/s         301
 TCP download::7  :        43.04        42.17 Mbits/s         301
 TCP download::8  :        42.96        42.17 Mbits/s         301
 TCP download::9  :        43.10        42.20 Mbits/s         301
----------------------

With "ethernet rtt 10ms":
qdisc cake 8011: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt
10.0ms noatm overhead 38 via-ethernet mpu 84

Client A:
                           avg       median          # data pts
 Ping (ms) ICMP :         0.16         0.13 ms              350
 TCP download   :       428.05       417.06 Mbits/s         301

Client B:
                             avg       median          # data pts
 Ping (ms) ICMP   :         0.14         0.10 ms              350
 TCP download avg :        35.86        34.76 Mbits/s         301
 TCP download sum :       430.30       417.14 Mbits/s         301
 TCP download::1  :        35.93        34.77 Mbits/s         301
 TCP download::10 :        35.78        34.76 Mbits/s         301
 TCP download::11 :        35.90        34.77 Mbits/s         301
 TCP download::12 :        35.88        34.76 Mbits/s         301
 TCP download::2  :        35.77        34.76 Mbits/s         300
 TCP download::3  :        35.77        34.76 Mbits/s         300
 TCP download::4  :        36.00        34.77 Mbits/s         301
 TCP download::5  :        35.83        34.76 Mbits/s         301
 TCP download::6  :        35.74        34.76 Mbits/s         301
 TCP download::7  :        35.91        34.76 Mbits/s         301
 TCP download::8  :        35.89        34.77 Mbits/s         301
 TCP download::9  :        35.90        34.76 Mbits/s         301
----------------------

Conclusions:
1) "ethernet" does not seem to make a difference.
2) "lan" deteriorates fairness, probably due to kernel timing limitations
as suggested before
3) "rtt 10ms" restores fairness

George

[-- Attachment #2: Type: text/html, Size: 6786 bytes --]

Re: [Cake] cake flenter results round 1

[Toke Høiland-Jørgensen]

> I would not be surprised to find out that this result was also due to
> lack of CPU, since there’s a steady degradation in Cake’s performance
> above 200mbit. Next time I’ll try 8/8 flows in addition.

If you add --test-parameter cpu_stats_hosts=localhost you will also get
a graph of CPU usage which while somewhat rudimentary should at least
make it possible to see from the results when you are running out of CPU
entirely :)

-Toke

Re: [Cake] cake flenter results round 1

[Dave Taht]

In the flent/misc dir are several faster polling routines written in C, also.

On Mon, Nov 27, 2017 at 12:37 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> I would not be surprised to find out that this result was also due to
>> lack of CPU, since there’s a steady degradation in Cake’s performance
>> above 200mbit. Next time I’ll try 8/8 flows in addition.
>
> If you add --test-parameter cpu_stats_hosts=localhost you will also get
> a graph of CPU usage which while somewhat rudimentary should at least
> make it possible to see from the results when you are running out of CPU
> entirely :)
>
> -Toke
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 1

[Pete Heist]

> On Nov 27, 2017, at 9:37 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> If you add --test-parameter cpu_stats_hosts=localhost you will also get
> a graph of CPU usage which while somewhat rudimentary should at least
> make it possible to see from the results when you are running out of CPU
> entirely :)

That helps, I did a quick test run and see cpu_stats_localhost::load in the summary output now, but is there a plot I can add or should it show up on another plot? I didn’t see something clear in "flent rrul_be --list-plots”.

I suppose mean load should get me an idea anyway...

Re: [Cake] cake flenter results round 1

[Toke Høiland-Jørgensen]

Pete Heist <peteheist@gmail.com> writes:

>> On Nov 27, 2017, at 9:37 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>> If you add --test-parameter cpu_stats_hosts=localhost you will also get
>> a graph of CPU usage which while somewhat rudimentary should at least
>> make it possible to see from the results when you are running out of CPU
>> entirely :)
>
> That helps, I did a quick test run and see cpu_stats_localhost::load
> in the summary output now, but is there a plot I can add or should it
> show up on another plot? I didn’t see something clear in "flent
> rrul_be --list-plots”.

Yeah, the plot doesn't show up in the list because it's added
conditionally based on the data file. You have the choice of 'cpu',
'cpu_bar' and 'cpu_box' for single test plots. These should show up in
the GUI when you load a data set that includes it.

-Toke

Re: [Cake] cake flenter results round 1

[Pete Heist]

> On Nov 27, 2017, at 10:08 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Yeah, the plot doesn't show up in the list because it's added
> conditionally based on the data file. You have the choice of 'cpu',
> 'cpu_bar' and 'cpu_box' for single test plots. These should show up in
> the GUI when you load a data set that includes it.

Got it, added the ‘cpu' plot it to all of my test types and we’ll see tomorrow, thanks!