Re: [Cake] Cake latency update

[Pete Heist <peteheist@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2961 bytes --]

Or, I can do: ethtool -K eth0 tx off rx off

and disable the checksums entirely. That stops the messages, but unfortunately it doesn’t appear to be the end of the throughput shifts.

But this experience has made me want to look at all of this on other hardware, so that’s next.

> On Feb 12, 2017, at 3:12 PM, Pete Heist <peteheist@gmail.com> wrote:
> 
>> On Feb 12, 2017, at 2:08 PM, Dave Taht <dave.taht@gmail.com <mailto:dave.taht@gmail.com>> wrote:
>> 
>> Disable offloads on the sky hardware and see what happens?
>> 
>> ethtool -K gro off tso off gso off your_device
> 
> I’d already had them disabled for testing in /etc/network/interfaces:
> 
> post-up ethtool -K eth0 tso off gso off gro off sg off
> 
> On a whim I tried _enabling_ offloads again but it happens in both cases.
> 
>> How old is the OS on that hardware - offloads have always been tricksy.
> 
> Pretty new: Ubuntu 16.10 (GNU/Linux 4.8.0-37-generic x86_64)
> 
>> as to why you might be seeing it more with cake, with this stuff on,
>> you are not necessarily checking every packet for checksums, and flows
>> are "finer" - more mixed up packets.
>> 
>> capturing these events with tcpdump at various points on the path might help.
>> 
>> Still, these are the kinds of baseline deployment issues that block
>> progress elsewhere. The whole first stage of the rocket has to succeed
>> in order to test the second. Doesn't matter how good your second stage
>> is, if you RUD the first.
> 
> It must be a challenge for you guys sometimes! Unless I can find an obvious solution soon it’s probably going to mean a hardware change for me. But there are only a few options I see with what’s available to me now:
> 
> 1) Using my Apple USB Ethernet adapter for testing instead of just management. Not excited about that- no BQL? USB latency? fq_codel on this adapter over Ethernet reduces Flent RRUL average latency to a pretty solid 1ms, looks sufficient? (Perhaps no coincidence that USB 2.0 start-of-frame is sent every 1 ms.)
> 
> 2) Using a 1.25 GHz Mac Mini PPC G4 I have laying around. I successfully ran fq_codel for ADSL on that box in the past, but at 5 / 0.5 Mbps. Accurate Flent results running Cake at 80 Mbps? Timer issues? Also I think no BQL support with the Sun GEM chipset: https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/sun/sungem.c <https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/sun/sungem.c>.
> 
> 3) Using two of these for my routers instead: https://pcengines.ch/alix2d2.htm <https://pcengines.ch/alix2d2.htm>, which I’ll want to test later anyway. They’re not new. 500 MHz AMD Geode LX800. Pre-Obama (June 2008). Not even sure yet if I’ll rate limit properly at 80-90 Mbit with these.
> 
> Any opinion on a ‘best’ alternative among these? I’m leaning towards #1 for ease. Otherwise I’ll make my way, and may have to dig up some better hardware.
> 
> Pete
> 


[-- Attachment #2: Type: text/html, Size: 4702 bytes --]