[Bloat] sch_fq, pacing and inconsistent TSO continued...

[Hans-Kristian Bakke]

After some further testing i see that this is not igb specific! Also bond
interfaces and e1000e have tso disabled like this:

tx-tcp-segmentation: off
tx-tcp-mangleid-segmentation: off [requested on]

bonded vlan interfaces however have tso enabled, but again vlan interfaces
seems to have everything enabled.

As downgrading the kernel to a known good did not do the trick there must
be something else systemwide in Debian Stretch. Perhaps I should really try
to get all the dependencies together to downgrade systemd.

Regards,
Hans-Kristian

On 13 February 2017 at 17:54, Hans-Kristian Bakke <hkbakke at gmail.com> wrote:

> In a previous thread on this mailing list (Excessive throttling with fq)
> it was concluded that the reason for the bad performance was that tso was
> somewhat inconcistent between the bond and the physical interfaces. I never
> really knew why, but I knew I had been experimenting with traffic shaping
> and offloads so I naturally thought that was the culprit.
>
> However, I have now rebooted some of my systems and in both of my Debian
> Stretch (testing) systems (I tried both kernel 4.8 and 4.9, one with
> fq_codel and one with sch_fq) I end up with this combination of
> segmentation offload settings:
>
> ...
> tcp-segmentation-offload: on
>         tx-tcp-segmentation: off <---- ???
>         tx-tcp-ecn-segmentation: off [fixed]
>         tx-tcp-mangleid-segmentation: off
>         tx-tcp6-segmentation: on
> udp-fragmentation-offload: off [fixed]
> generic-segmentation-offload: on
> ...
>
> If I do the following...
> ethtool -K eth0 tx-tcp-segmentation on
>
> ...I end up with what I expect and the performance returns for sch_fq with
> pacing.
> ...
> tcp-segmentation-offload: on
>         tx-tcp-segmentation: on
>         tx-tcp-ecn-segmentation: off [fixed]
>         tx-tcp-mangleid-segmentation: off
>         tx-tcp6-segmentation: on
> udp-fragmentation-offload: off [fixed]
> generic-segmentation-offload: on
> ...
>
> I have some other Debian Jessie systems also using the igb-driver and tso
> is always enabled here by default (but not the same NIC). (Kernel 3.16)
>
> I also have a Proxmox VE system on kernel 4.4 with exactly the same quad
> NIC that i suspect do not use the kernel source igb-drivers as it has a lot
> more params that also has tso enabled correctly.
>
> # Proxmox VE system (full TSO by default)
> #ethtool -i eth1
> driver: igb
> version: 5.3.5.3
> firmware-version: 3.19, 0x00013cbf
> bus-info: 0000:03:00.1
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
>
> # Debian Stretch systems (half-enabled TSO by default)
> # ethtool -i eth2
> driver: igb
> version: 5.4.0-k
> firmware-version: 0.0.0
> expansion-rom-version:
> bus-info: 0000:00:14.2
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
>
> The NIC that the "working" and the "non-working" system have is HP NC365T,
> which is based on intel 82580.
> The other "non-working" Debian Stretch system is a Intel Rangeley Atom
> c2758 supermicros system with a quad intel I354 network controller embedded.
>
> I tried booting the stretch system with kernel 3.16 from Debian Jessie to
> get the same drivers like my working systems but no change. Then I noticed
> that systemd got the ability to change exactly these offloads in the last
> systemd version 232 which arrived in testing in the end of 2016 so I am
> thinking that this might have something to do with this, but I have not
> found anything (mostly because I don't even really know where to look)
>
> I also tried to not having one of the interfaces in any kind of bond or
> vlan subinterface in case those were changing some stuff but no changes in
> behaviour.
>
> So to my questions:
> - Can anyone think of a reason why tx-tcp-segmentation offload is disabled
> by default (and not tcp segmentation offload in general)
> - Do you have any tips to troubleshoot this?
> - As this default combination of out-of-the-box settings is bad for fq
> with pacing performance, and it seems to somewhat be the default now for
> two different igb-systems I thought I should give you a heads up.
>
> Regards,
> Hans-Kristian
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20170213/9d9555c2/attachment.html>