General list for discussing Bufferbloat
 help / color / mirror / Atom feed
From: Hans-Kristian Bakke <hkbakke@gmail.com>
To: bloat <bloat@lists.bufferbloat.net>
Subject: Re: [Bloat] sch_fq, pacing and inconsistent TSO continued...
Date: Mon, 13 Feb 2017 18:39:16 +0100	[thread overview]
Message-ID: <CAD_cGvE+LYmiswo53yqDMZVKgrJrSdUz73d0_cLMZ-PMLi7=SA@mail.gmail.com> (raw)
In-Reply-To: <CAD_cGvFq5wgnRtDt3vRuECVH5gKwVM+S-qfUCFLhq7tWFe0+-A@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 4275 bytes --]

After some further testing i see that this is not igb specific! Also bond
interfaces and e1000e have tso disabled like this:

tx-tcp-segmentation: off
tx-tcp-mangleid-segmentation: off [requested on]

bonded vlan interfaces however have tso enabled, but again vlan interfaces
seems to have everything enabled.

As downgrading the kernel to a known good did not do the trick there must
be something else systemwide in Debian Stretch. Perhaps I should really try
to get all the dependencies together to downgrade systemd.

Regards,
Hans-Kristian

On 13 February 2017 at 17:54, Hans-Kristian Bakke <hkbakke@gmail.com> wrote:

> In a previous thread on this mailing list (Excessive throttling with fq)
> it was concluded that the reason for the bad performance was that tso was
> somewhat inconcistent between the bond and the physical interfaces. I never
> really knew why, but I knew I had been experimenting with traffic shaping
> and offloads so I naturally thought that was the culprit.
>
> However, I have now rebooted some of my systems and in both of my Debian
> Stretch (testing) systems (I tried both kernel 4.8 and 4.9, one with
> fq_codel and one with sch_fq) I end up with this combination of
> segmentation offload settings:
>
> ...
> tcp-segmentation-offload: on
>         tx-tcp-segmentation: off <---- ???
>         tx-tcp-ecn-segmentation: off [fixed]
>         tx-tcp-mangleid-segmentation: off
>         tx-tcp6-segmentation: on
> udp-fragmentation-offload: off [fixed]
> generic-segmentation-offload: on
> ...
>
> If I do the following...
> ethtool -K eth0 tx-tcp-segmentation on
>
> ...I end up with what I expect and the performance returns for sch_fq with
> pacing.
> ...
> tcp-segmentation-offload: on
>         tx-tcp-segmentation: on
>         tx-tcp-ecn-segmentation: off [fixed]
>         tx-tcp-mangleid-segmentation: off
>         tx-tcp6-segmentation: on
> udp-fragmentation-offload: off [fixed]
> generic-segmentation-offload: on
> ...
>
> I have some other Debian Jessie systems also using the igb-driver and tso
> is always enabled here by default (but not the same NIC). (Kernel 3.16)
>
> I also have a Proxmox VE system on kernel 4.4 with exactly the same quad
> NIC that i suspect do not use the kernel source igb-drivers as it has a lot
> more params that also has tso enabled correctly.
>
> # Proxmox VE system (full TSO by default)
> #ethtool -i eth1
> driver: igb
> version: 5.3.5.3
> firmware-version: 3.19, 0x00013cbf
> bus-info: 0000:03:00.1
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
>
> # Debian Stretch systems (half-enabled TSO by default)
> # ethtool -i eth2
> driver: igb
> version: 5.4.0-k
> firmware-version: 0.0.0
> expansion-rom-version:
> bus-info: 0000:00:14.2
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
>
> The NIC that the "working" and the "non-working" system have is HP NC365T,
> which is based on intel 82580.
> The other "non-working" Debian Stretch system is a Intel Rangeley Atom
> c2758 supermicros system with a quad intel I354 network controller embedded.
>
> I tried booting the stretch system with kernel 3.16 from Debian Jessie to
> get the same drivers like my working systems but no change. Then I noticed
> that systemd got the ability to change exactly these offloads in the last
> systemd version 232 which arrived in testing in the end of 2016 so I am
> thinking that this might have something to do with this, but I have not
> found anything (mostly because I don't even really know where to look)
>
> I also tried to not having one of the interfaces in any kind of bond or
> vlan subinterface in case those were changing some stuff but no changes in
> behaviour.
>
> So to my questions:
> - Can anyone think of a reason why tx-tcp-segmentation offload is disabled
> by default (and not tcp segmentation offload in general)
> - Do you have any tips to troubleshoot this?
> - As this default combination of out-of-the-box settings is bad for fq
> with pacing performance, and it seems to somewhat be the default now for
> two different igb-systems I thought I should give you a heads up.
>
> Regards,
> Hans-Kristian
>

[-- Attachment #2: Type: text/html, Size: 7232 bytes --]

  reply	other threads:[~2017-02-13 17:39 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-02-13 16:54 Hans-Kristian Bakke
2017-02-13 17:39 ` Hans-Kristian Bakke [this message]
2017-02-13 17:51   ` Hans-Kristian Bakke

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.bufferbloat.net/postorius/lists/bloat.lists.bufferbloat.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAD_cGvE+LYmiswo53yqDMZVKgrJrSdUz73d0_cLMZ-PMLi7=SA@mail.gmail.com' \
    --to=hkbakke@gmail.com \
    --cc=bloat@lists.bufferbloat.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox