Yep.. downgrading systemd from 232.15 to 231.10 fixed it. There you have it. Systemd 232 not only got the ability to set offloads, but it also disables tx-tcp-segmentation by default for both physical NICs and bonds. These packages from Debian Snapshots did the trick: libsystemd0_231-10_amd64.deb libudev1_231-10_amd64.deb systemd_231-10_amd64.deb udev_231-10_amd64.deb This is ...fun.... No wonder file my server performance got more unreliable the last couple of months. On 13 February 2017 at 18:39, Hans-Kristian Bakke wrote: > After some further testing i see that this is not igb specific! Also bond > interfaces and e1000e have tso disabled like this: > > tx-tcp-segmentation: off > tx-tcp-mangleid-segmentation: off [requested on] > > bonded vlan interfaces however have tso enabled, but again vlan interfaces > seems to have everything enabled. > > As downgrading the kernel to a known good did not do the trick there must > be something else systemwide in Debian Stretch. Perhaps I should really try > to get all the dependencies together to downgrade systemd. > > Regards, > Hans-Kristian > > On 13 February 2017 at 17:54, Hans-Kristian Bakke > wrote: > >> In a previous thread on this mailing list (Excessive throttling with fq) >> it was concluded that the reason for the bad performance was that tso was >> somewhat inconcistent between the bond and the physical interfaces. I never >> really knew why, but I knew I had been experimenting with traffic shaping >> and offloads so I naturally thought that was the culprit. >> >> However, I have now rebooted some of my systems and in both of my Debian >> Stretch (testing) systems (I tried both kernel 4.8 and 4.9, one with >> fq_codel and one with sch_fq) I end up with this combination of >> segmentation offload settings: >> >> ... >> tcp-segmentation-offload: on >> tx-tcp-segmentation: off <---- ??? >> tx-tcp-ecn-segmentation: off [fixed] >> tx-tcp-mangleid-segmentation: off >> tx-tcp6-segmentation: on >> udp-fragmentation-offload: off [fixed] >> generic-segmentation-offload: on >> ... >> >> If I do the following... >> ethtool -K eth0 tx-tcp-segmentation on >> >> ...I end up with what I expect and the performance returns for sch_fq >> with pacing. >> ... >> tcp-segmentation-offload: on >> tx-tcp-segmentation: on >> tx-tcp-ecn-segmentation: off [fixed] >> tx-tcp-mangleid-segmentation: off >> tx-tcp6-segmentation: on >> udp-fragmentation-offload: off [fixed] >> generic-segmentation-offload: on >> ... >> >> I have some other Debian Jessie systems also using the igb-driver and tso >> is always enabled here by default (but not the same NIC). (Kernel 3.16) >> >> I also have a Proxmox VE system on kernel 4.4 with exactly the same quad >> NIC that i suspect do not use the kernel source igb-drivers as it has a lot >> more params that also has tso enabled correctly. >> >> # Proxmox VE system (full TSO by default) >> #ethtool -i eth1 >> driver: igb >> version: 5.3.5.3 >> firmware-version: 3.19, 0x00013cbf >> bus-info: 0000:03:00.1 >> supports-statistics: yes >> supports-test: yes >> supports-eeprom-access: yes >> supports-register-dump: yes >> supports-priv-flags: no >> >> # Debian Stretch systems (half-enabled TSO by default) >> # ethtool -i eth2 >> driver: igb >> version: 5.4.0-k >> firmware-version: 0.0.0 >> expansion-rom-version: >> bus-info: 0000:00:14.2 >> supports-statistics: yes >> supports-test: yes >> supports-eeprom-access: yes >> supports-register-dump: yes >> supports-priv-flags: no >> >> The NIC that the "working" and the "non-working" system have is HP >> NC365T, which is based on intel 82580. >> The other "non-working" Debian Stretch system is a Intel Rangeley Atom >> c2758 supermicros system with a quad intel I354 network controller embedded. >> >> I tried booting the stretch system with kernel 3.16 from Debian Jessie to >> get the same drivers like my working systems but no change. Then I noticed >> that systemd got the ability to change exactly these offloads in the last >> systemd version 232 which arrived in testing in the end of 2016 so I am >> thinking that this might have something to do with this, but I have not >> found anything (mostly because I don't even really know where to look) >> >> I also tried to not having one of the interfaces in any kind of bond or >> vlan subinterface in case those were changing some stuff but no changes in >> behaviour. >> >> So to my questions: >> - Can anyone think of a reason why tx-tcp-segmentation offload is >> disabled by default (and not tcp segmentation offload in general) >> - Do you have any tips to troubleshoot this? >> - As this default combination of out-of-the-box settings is bad for fq >> with pacing performance, and it seems to somewhat be the default now for >> two different igb-systems I thought I should give you a heads up. >> >> Regards, >> Hans-Kristian >> > >