<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Yes, especially since you’ve got higher-end hardware than I. 443.65mbit vs 444.35mbit looks pretty fair. :)</div><div class=""><br class=""></div><div class="">Thanks for reproducing it. I’m going to have to review some of my flenter tests in light of this. I’m getting a handle on the limitations of the APU2 hardware. It’s quite good especially for the price, but has limits on what it can do at Gbit rates. It can actually be useful to test what happens when the CPU is overburdened, only I need to avoid being fooled by it.</div><div class=""><br class=""></div><div class="">You could also add the ‘ethernet’ keyword, which I’m going to add for this test in my next round, although that wouldn’t have fixed what I was seeing anyway...</div><div class=""><br class=""></div><div class="">Pete</div><br class=""><div><blockquote type="cite" class=""><div class="">On Nov 27, 2017, at 5:17 PM, Georgios Amanakis <<a href="mailto:gamanakis@gmail.com" class="">gamanakis@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div class="">Dear Pete,<br class=""><br class=""></div>I am trying to replicate the unfair behaviour you are seeing with dual-{src,dst}host, albeit on different hardware and I am getting a fair distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients). All running Archlinux, latest cake and patched iproute2-4.14.1, connected with Gbit ethernet, TSO/GSO/GRO enabled.<br class=""></div><div class=""><br class=""></div>Qdisc setup:</div><div class="">----------------<br class=""></div>Router: <br class=""></div><div class="">qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt 100.0ms raw<br class=""></div><br class=""></div><div class="">Client A(kernel default):</div><div class="">qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn<br class=""></div><br class="">Client B (kernel default): <br class="">qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn<br class="">----------------<br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div>Cli:<br class=""><div class="">----------------</div><div class="">Router:</div><div class="">netserver &</div><div class=""><br class=""></div><div class="">Client A:</div><div class="">flent tcp_1down -H router</div><div class=""><br class=""></div><div class="">Client B:</div><div class=""><div class="">flent tcp_12down -H router</div></div><div class="">----------------</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Results:<br class=""></div><div class="">----------------<br class=""></div><div class="">Router:</div><div class="">qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost rtt 100.0ms raw<br class=""> Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 requeues 0)<br class=""> backlog 0b 0p requeues 0<br class=""> memory used: 1224872b of 15140Kb<br class=""> capacity estimate: 900Mbit<br class=""> Bulk Best Effort Voice<br class=""> thresh 56250Kbit 900Mbit 225Mbit<br class=""> target 5.0ms 5.0ms 5.0ms<br class=""> interval 100.0ms 100.0ms 100.0ms<br class=""> pk_delay 14us 751us 7us<br class=""> av_delay 2us 642us 1us<br class=""> sp_delay 1us 1us 1us<br class=""> pkts 109948 4601651 14315<br class=""> bytes 160183242 6964893773 1618242<br class=""> way_inds 0 21009 0<br class=""> way_miss 160 188 5<br class=""> way_cols 0 0 0<br class=""> drops 0 10 0<br class=""> marks 0 0 0<br class=""> ack_drop 0 0 0<br class=""> sp_flows 0 1 1<br class=""> bk_flows 1 0 0<br class=""> un_flows 0 0 0<br class=""> max_len 7570 68130 1022<br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""></div><div class="">Client A:</div><div class=""> avg median # data pts<br class=""> Ping (ms) ICMP : 0.11 0.08 ms 350<br class=""> TCP download : 443.65 430.38 Mbits/s 301<br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""></div><div class="">Client B:</div><div class=""> avg median # data pts<br class=""> Ping (ms) ICMP : 0.09 0.06 ms 350<br class=""> TCP download avg : 37.03 35.87 Mbits/s 301<br class=""> TCP download sum : 444.35 430.40 Mbits/s 301<br class=""> TCP download::1 : 37.00 35.87 Mbits/s 301<br class=""> TCP download::10 : 37.01 35.87 Mbits/s 301<br class=""> TCP download::11 : 37.02 35.87 Mbits/s 301<br class=""> TCP download::12 : 37.00 35.87 Mbits/s 301<br class=""> TCP download::2 : 37.03 35.87 Mbits/s 301<br class=""> TCP download::3 : 36.99 35.87 Mbits/s 301<br class=""> TCP download::4 : 37.03 35.87 Mbits/s 301<br class=""> TCP download::5 : 37.07 35.87 Mbits/s 301<br class=""> TCP download::6 : 37.00 35.87 Mbits/s 301<br class=""> TCP download::7 : 37.12 35.87 Mbits/s 301<br class=""> TCP download::8 : 37.05 35.87 Mbits/s 301<br class=""> TCP download::9 : 37.03 35.87 Mbits/s 301<br class="">----------------</div><div class=""><br class=""></div><div class="">Does this suggest that it is indeed a problem of an underpowered CPU in your case?</div><div class=""><br class=""></div><div class="">George<br class=""></div><div class=""><br class=""></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist <span dir="ltr" class=""><<a href="mailto:peteheist@gmail.com" target="_blank" class="">peteheist@gmail.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><span class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Nov 27, 2017, at 3:48 PM, Jonathan Morton <<a href="mailto:chromatix99@gmail.com" target="_blank" class="">chromatix99@gmail.com</a>> wrote:</div><div class=""><p dir="ltr" class="">It's not at all obvious how we'd detect that. Packets are staying in the queue for less time than the codel target, which is exactly what you'd get if you weren't saturated at all.</p></div></blockquote></div></span><div class="">That makes complete sense when you put it that way. Cake has no way of knowing why the input rate is lower than expected, even if it’s part of the cause.</div><div class=""><br class=""></div><div class="">I don’t think flent can know this either. It can’t easily know the cause for its total output to be lower than expected.</div><div class=""><br class=""></div><div class="">All I know is, this is a common problem in deployments, particularly on low-end hardware like ER-Xs, that can be tricky for users to figure out.</div><div class=""><br class=""></div><div class="">I don’t even think monitoring CPU in general would work. The CPU could be high because it’s doing other calculations, but there’s still enough for cake at a low rate, and there’s no need to warn in that case. I’d be interested in any ideas on how to know this is happening in the system as a whole. So far, there are just various clues that one needs to piece together (no or few drops or marks, less total throughput that expected, high cpu without other external usage, etc). Then it needs to be proven with a test.</div><div class=""><br class=""></div><div class="">Anyway thanks, your clue was what I needed! I need to remember to review the qdisc stats when something unexpected happens.</div></div><br class="">______________________________<wbr class="">_________________<br class="">
Cake mailing list<br class="">
<a href="mailto:Cake@lists.bufferbloat.net" class="">Cake@lists.bufferbloat.net</a><br class="">
<a href="https://lists.bufferbloat.net/listinfo/cake" rel="noreferrer" target="_blank" class="">https://lists.bufferbloat.net/<wbr class="">listinfo/cake</a><br class="">
<br class=""></blockquote></div><br class=""></div>
</div></blockquote></div><br class=""></body></html>