[Bloat] Intel recommends against LRO in routing and bridging.

[Bloat] Intel recommends against LRO in routing and bridging.

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 664 bytes --]

Just came across this at the top of the README for the ixgbe driver: (
http://downloadmirror.intel.com/22919/eng/README.txt)

WARNING:  The ixgbe driver compiles by default with the LRO (Large Receive
Offload) feature enabled.  This option offers the lowest CPU utilization for
receives, but is completely incompatible with *routing/ip forwarding* and
*bridging*.  If enabling ip forwarding or bridging is a requirement, it is
necessary to disable LRO using compile time options as noted in the LRO
section later in this document.  The result of not disabling LRO when combined
with ip forwarding or bridging can be low throughput or even a kernel panic.


-Aaron

[-- Attachment #2: Type: text/html, Size: 894 bytes --]

Re: [Bloat] Intel recommends against LRO in routing and bridging.

[Dave Täht]

On 8/9/16 12:13 AM, Aaron Wood wrote:
> Just came across this at the top of the README for the ixgbe driver:
> (http://downloadmirror.intel.com/22919/eng/README.txt)
> 
> WARNING:  The ixgbe driver compiles by default with the LRO (Large Receive
> Offload) feature enabled.  This option offers the lowest CPU utilization for
> receives, but is completely incompatible with *routing/ip forwarding* and
> *bridging*.  If enabling ip forwarding or bridging is a requirement, it is
> necessary to disable LRO using compile time options as noted in the LRO
> section later in this document.  The result of not disabling LRO when combined
> with ip forwarding or bridging can be low throughput or even a kernel panic.

I am not sure how true this remains. (?) We certainly have gone through
hell with TSO and GRO, but it was my hope that most of those issues were
fixed over the past 2 years, in the drivers, and htb/cake/hfsc, etc.

I don't think that the existing pie in linux head is handling 64k
superpackets[1] well. tbf is doing more of the right things here now,,
but wasn't till about a year back. Haven't tracked htb, and certainly
hfsc has shown some problems that I don't know are fixed or not (there
was a kernel bug filed on it some time back with stephen taking the lead
on it)

Cake's "peeling" mechanism for superpackets is also overly aggressive,
but we have not put much effort into tuning it yet. We have a variety of
potential solutions for that...

BQL tends to get a very big number for its estimator when superpackets
are in use, usually 4x what happens without superpackets turned on via
ethtool - but NOT having superpackets turned on can really hurt throughput.

The code we tested a while back on the bsd implementations (which
generally lack superpacket support) had trouble hitting fifo speeds at
high rates on the hardware under test (>400mbits).

[1] For want of a word, I'll use superpackets to refer to
TSO/GSO/GRO/LSO universally.

> 
> -Aaron
> 
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>