In a previous thread on this mailing list (Excessive throttling with fq) it was concluded that the reason for the bad performance was that tso was somewhat inconcistent between the bond and the physical interfaces. I never really knew why, but I knew I had been experimenting with traffic shaping and offloads so I naturally thought that was the culprit.

However, I have now rebooted some of my systems and in both of my Debian Stretch (testing) systems (I tried both kernel 4.8 and 4.9, one with fq_codel and one with sch_fq) I end up with this combination of segmentation offload settings:

...

tcp-segmentation-offload: on

tx-tcp-segmentation: off <---- ???

tx-tcp-ecn-segmentation: off [fixed]

tx-tcp-mangleid-segmentation: off

tx-tcp6-segmentation: on

udp-fragmentation-offload: off [fixed]

generic-segmentation-offload: on

...

If I do the following...

ethtool -K eth0 tx-tcp-segmentation on

...I end up with what I expect and the performance returns for sch_fq with pacing.

...

tcp-segmentation-offload: on

tx-tcp-segmentation: on

tx-tcp-ecn-segmentation: off [fixed]

tx-tcp-mangleid-segmentation: off

tx-tcp6-segmentation: on

udp-fragmentation-offload: off [fixed]

generic-segmentation-offload: on

...

I have some other Debian Jessie systems also using the igb-driver and tso is always enabled here by default (but not the same NIC). (Kernel 3.16)

I also have a Proxmox VE system on kernel 4.4 with exactly the same quad NIC that i suspect do not use the kernel source igb-drivers as it has a lot more params that also has tso enabled correctly.

# Proxmox VE system (full TSO by default)

#ethtool -i eth1

driver: igb

version: 5.3.5.3

firmware-version: 3.19, 0x00013cbf

bus-info: 0000:03:00.1

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: no

# Debian Stretch systems (half-enabled TSO by default)

# ethtool -i eth2

driver: igb

version: 5.4.0-k

firmware-version: 0.0.0

expansion-rom-version:

bus-info: 0000:00:14.2

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: no

The NIC that the "working" and the "non-working" system have is HP NC365T, which is based on intel 82580.

The other "non-working" Debian Stretch system is a Intel Rangeley Atom c2758 supermicros system with a quad intel I354 network controller embedded.

I tried booting the stretch system with kernel 3.16 from Debian Jessie to get the same drivers like my working systems but no change. Then I noticed that systemd got the ability to change exactly these offloads in the last systemd version 232 which arrived in testing in the end of 2016 so I am thinking that this might have something to do with this, but I have not found anything (mostly because I don't even really know where to look)

I also tried to not having one of the interfaces in any kind of bond or vlan subinterface in case those were changing some stuff but no changes in behaviour.

So to my questions:

- Can anyone think of a reason why tx-tcp-segmentation offload is disabled by default (and not tcp segmentation offload in general)

- Do you have any tips to troubleshoot this?

- As this default combination of out-of-the-box settings is bad for fq with pacing performance, and it seems to somewhat be the default now for two different igb-systems I thought I should give you a heads up.

Regards,

Hans-Kristian