<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jan 30, 2017, at 10:44 PM, Toke Høiland-Jørgensen <<a href="mailto:toke@toke.dk" class="">toke@toke.dk</a>> wrote:</div><div class=""><div class=""><br class="">Oh my, this is quite a lot of tests. Nice :)<br class=""></div></div></blockquote><div><br class=""></div><div>It’s also a thumbs up for the ath9k driver changes that nothing went wrong during the testing. It takes about 15 hours for a full run and I probably did that 4-5 times total.</div><br class=""><blockquote type="cite" class=""><div class=""><div class="">Few general points on running tests:<br class=""><br class="">- Yeah, as you note Flent has a batch facility. Did you not use this<br class=""> simply because you couldn't find it, or was there some other reason?<br class=""> Would love some feedback on what I can do to make that more useful to<br class=""> people... While I have no doubt that your 'flenter.py' works, wrapping<br class=""> a wrapper in this sense makes me cringe a little bit ;)<br class=""></div></div></blockquote><div><br class=""></div><div>I actually didn’t notice it existed until I was about 85% done and scanning the Flent man page for some other reason. I cringed, but at that point I just stuck with what I had. I don’t know if Flent can also make some basic html report with the graphs and setup output, but that was useful to write for myself. Flent’s metadata feature sounds useful and I’ll try that.</div><br class=""><blockquote type="cite" class=""><div class=""><div class="">- I'm not sure if you're checking that applying your qdiscs actually<br class=""> works? For the WiFi interfaces with 'noqueue' you *cannot* apply a<br class=""> different qdisc (which also answers your question #2).<br class=""></div></div></blockquote><div><br class=""></div><div>Hmm. Unless I’m missing something, what I’m seeing is that I _can_ add another qdisc, only that it’s ineffective unless soft rate limiting is used. As evidence, here's my nolimit test of fq_codel:</div><div><br class=""></div><div><a href="http://www.drhleny.cz/bufferbloat/fq_codel_nolimit/index.html" class="">http://www.drhleny.cz/bufferbloat/fq_codel_nolimit/index.html</a></div><div><br class=""></div><div>Under the sections qos_om1 and qos_om2, you can see that fq_codel has been added to wlan0 from the tc output. But the latency is the same 25 ms as the default, so I’m not controlling the queue, and that was true for any qdisc without rate limiting.</div><div><br class=""></div><div>But, here's ‘40mbit full-duplex rate limiting on the AP and station with htb+fq_codel’:</div><div><br class=""></div><div><a href="http://www.drhleny.cz/bufferbloat/fq_codel_fd-wifi-both_40mbit/index.html" class="">http://www.drhleny.cz/bufferbloat/fq_codel_fd-wifi-both_40mbit/index.html</a></div><div><br class=""></div><div><div>With this, I could reduce latency to about 9ms, so it did “something”. And pfifo with 40mbit rate limiting did the “something” that it usually does, bloating things up to 200+ ms:</div></div><div><br class=""></div><div><a href="http://www.drhleny.cz/bufferbloat/pfifo_fd-wifi-both_40mbit/index.html" class="">http://www.drhleny.cz/bufferbloat/pfifo_fd-wifi-both_40mbit/index.html</a></div><div><br class=""></div><div>So I don’t see any evidence that I can’t add a qdisc to the Wi-Fi devices, only that I have to use soft limiting for it to be effective.</div><div><br class=""></div><div>Now, HTB rate limiting seems to break down on the OM2P-HS at bitrates above about 50mbit, when the actual throughput starts to fall away from the set limit, but that’s another story.</div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div class="">Question 1 (and partly #13): Yeah, the version of LEDE you're running<br class="">already has the FQ-CoDel-based queueing in the ath9k driver. The<br class="">baseline you're seeing is consistent with the results we've been getting<br class="">in testing. This is also seen by any gains you get being paired with<br class="">quite a hefty hit in throughput. So with this driver, I would say it's<br class="">not worth it. However, this is going to be different on a setup without<br class="">the WiFi queueing fixes.<br class=""></div></div></blockquote><div><br class=""></div><div>Ok, that explains a lot, thanks. I was still able to see about a 50% reduction in latency (from ~25 ms to ~12ms) with a 13% drop in throughput (from ~92 Mbps to ~80Mbps), when doing half-duplex rate limiting to 85Mbps and fq_codel’ing on the external router. See:</div><div><br class=""></div><div><a href="http://www.drhleny.cz/bufferbloat/fq_codel_hd-eth-ap_85mbit/index.html" class="">http://www.drhleny.cz/bufferbloat/fq_codel_hd-eth-ap_85mbit/index.html</a></div><div><br class=""></div><div>vs the default:</div><div><br class=""></div><div><a href="http://www.drhleny.cz/bufferbloat/default/index.html" class="">http://www.drhleny.cz/bufferbloat/default/index.html</a></div><div><br class=""></div><div>I can get down to 10ms if I give up another 5 Mbps, or lower values with more severe throughput sacrifices.</div><div><br class=""></div><div>But this is with a stable RSSI of around -50 and low noise. I understand that fq-codel’ing in the driver must be superior in its handling of rate changes, retries or other external factors, and that point-to-multipoint is a different story. But maybe some of FreeNet’s line-of-sight point-to-point links may also be stable enough such that fixed software rate limiting is usable for them, I’m not sure yet.</div><div><br class=""></div><div>It’s not critical, but why am I able to see this level of reduction when there’s already fq-codel in the driver? 25ms is very good, I only wonder where I’m getting the extra 10-15ms from, out of interest. :)</div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div class="">Question 5: For TCP you can't get packet loss from user space; you'll<br class="">need packet captures for that. So no way to get it from Flent either.<br class="">You can, however, get average throughput. Look at the box plots; if you<br class="">run multiple iterations of the same test, you can plot several data<br class="">files in a single box_combine plot, to get error bars. `flent<br class="">file.flent.gz -f summary` (which is the default if you don't specify a<br class="">plot) will get you averages per data series; or you can extract it from<br class="">the metadata.<br class=""></div></div></blockquote><br class=""></div><div>Ok, so far, I was doing `cat file.flent.gz | grep null | wc -l`, which is a very crude count of the nulls recorded, which seem to happen for the udp and icmp flows with packet loss. There are always some nulls from before the test starts and after it ends, but if the count jumps up I speculate that there’s more packet loss. It’s pretty weak but it’s a hint.</div><div><br class=""></div><div>I see the box plots now, that’s nice, thanks!</div><br class=""></body></html>