[Cake] [PATCH net-next v3] Add Common Applications Kept Enhanced (cake) qdisc

[Toke Høiland-Jørgensen]

Eric Dumazet <eric.dumazet at gmail.com> writes:

> On 04/25/2018 09:52 AM, Jonathan Morton wrote:
>>> We can see here the high cost of forcing software GSO :/
>>>
>>> Really, this should be done only :
>>> 1) If requested by the admin ( tc .... gso ....)
>>>
>>> 2) If packet size is above a threshold.
>>>  The threshold could be set by the admin, and/or based on a fraction of the bandwidth parameter.
>>>
>>> I totally understand why you prefer to segment yourself for < 100 Mbit links.
>>>
>>> But this makes no sense on 10Gbit+
>> 
>> It is absolutely necessary, so far as I can see, to segment GSO
>> superpackets when overhead compensation is selected - as it very
>> often should be, even on pure Ethernet links. Without that, the
>> calculation of link occupancy time will be wrong. (The actual
>> transmission time of an Ethernet frame is rather more than just 14
>> bytes longer than the underlying IP packet.)
>
> Just fix the overhead compensation computation in the code.
>
> skb in a qdisc have everything you need.
>
> qdisc_pkt_len_init() has initialized qdisc_skb_cb(skb)->pkt_len with
> the exact bytes on the wire, and you have gso_segs to perform any
> adjustement you need to do.

The problem is that may not be the right values. For example, in many
CPEs there's a built-in switch that strips VLAN tags before the packet
actually hits the wire. So we do need to be able to get the actual
packet size. Is it possible to get the sizes of the individual segments
of a GSO packet? That way we could do the calculation for the whole
super-packet...

> Do not kill GSO only because you do not want to deal with it.

It's not (just) that we don't want to deal with it, it is also a very
aggressive optimisation for latency, which makes a lot of sense at
residential internet bandwidths.

>> Another reason to apply GSO segmentation is to achieve maximal
>> smoothness of flow isolation. This should still be achievable within
>> some tolerance at high link rates, but calculating this tolerance is
>> complicated by the triple-isolate algorithm.
>> 
>> If there's a way to obtain the individual packet sizes without
>> incurring the full segmentation overhead, it may be worth considering
>> (at high link rates only). I would want to leave it on by default,
>> because some of Cake's demonstrably superior latency performance
>> depends on seeing the real packets, not the aggregates, and the
>> overhead only becomes significant above 100Mbps on weak MIPS CPUs and
>> 1Gbps on vaguely modern x86 stuff.
>
> Again, these arguments are moot on 10Gbit+.
>
> Lets build the future, do not pretend GSO/TSO is not part of it.

I'm all for being future proof, but we also do want well-tested code.
Reworking this feature to function correctly with GSO is going to take
some work. Would you accept gating GSO splitting on bandwidth for now,
and adding correct overhead compensation for GSO packets later?

something like:

if (shaper_active() && (bandwidth <= 1Gbps || overhead_configured))
  split_gso()

-Toke