<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div><blockquote type="cite" class=""><div class="">On Feb 12, 2017, at 2:08 PM, Dave Taht <<a href="mailto:dave.taht@gmail.com" class="">dave.taht@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Disable offloads on the sky hardware and see what happens?<br class=""><br class="">ethtool -K gro off tso off gso off your_device<br class=""></div></div></blockquote><div><br class=""></div><div>I’d already had them disabled for testing in /etc/network/interfaces:</div><div><br class=""></div><div>post-up ethtool -K eth0 tso off gso off gro off sg off</div><div><br class=""></div><div>On a whim I tried _enabling_ offloads again but it happens in both cases.</div><br class=""><blockquote type="cite" class=""><div class=""><div class="">How old is the OS on that hardware - offloads have always been tricksy.<br class=""></div></div></blockquote><div><br class=""></div><div>Pretty new: Ubuntu 16.10 (GNU/Linux 4.8.0-37-generic x86_64)</div><br class=""><blockquote type="cite" class=""><div class=""><div class="">as to why you might be seeing it more with cake, with this stuff on,<br class="">you are not necessarily checking every packet for checksums, and flows<br class="">are "finer" - more mixed up packets.<br class=""><br class="">capturing these events with tcpdump at various points on the path might help.<br class=""><br class="">Still, these are the kinds of baseline deployment issues that block<br class="">progress elsewhere. The whole first stage of the rocket has to succeed<br class="">in order to test the second. Doesn't matter how good your second stage<br class="">is, if you RUD the first.<br class=""></div></div></blockquote><div><br class=""></div><div>It must be a challenge for you guys sometimes! Unless I can find an obvious solution soon it’s probably going to mean a hardware change for me. But there are only a few options I see with what’s available to me now:</div><div><br class=""></div><div>1) Using my Apple USB Ethernet adapter for testing instead of just management. Not excited about that- no BQL? USB latency? fq_codel on this adapter over Ethernet reduces Flent RRUL average latency to a pretty solid 1ms, looks sufficient? (Perhaps no coincidence that USB 2.0 start-of-frame is sent every 1 ms.)</div><div><br class=""></div><div>2) Using a 1.25 GHz Mac Mini PPC G4 I have laying around. I successfully ran fq_codel for ADSL on that box in the past, but at 5 / 0.5 Mbps. Accurate Flent results running Cake at 80 Mbps? Timer issues? Also I think no BQL support with the Sun GEM chipset: <a href="https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/sun/sungem.c" class="">https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/sun/sungem.c</a>.</div><div><br class=""></div><div>3) Using two of these for my routers instead: <a href="https://pcengines.ch/alix2d2.htm" class="">https://pcengines.ch/alix2d2.htm</a>, which I’ll want to test later anyway. They’re not new. 500 MHz AMD Geode LX800. Pre-Obama (June 2008). Not even sure yet if I’ll rate limit properly at 80-90 Mbit with these.</div><div><br class=""></div><div>Any opinion on a ‘best’ alternative among these? I’m leaning towards #1 for ease. Otherwise I’ll make my way, and may have to dig up some better hardware.</div><div><br class=""></div><div>Pete</div><div><br class=""></div></div></body></html>