Re: [Codel] fq_codel_drop vs a udp flood

[moeller0 <moeller0@gmx.de>]

Hi Eric,

> On May 6, 2016, at 15:25 , Eric Dumazet <eric.dumazet@gmail.com> wrote:
> 
> On Fri, 2016-05-06 at 13:46 +0200, moeller0 wrote:
>> Hi Jesper,
>> 
>>> On May 6, 2016, at 13:33 , Jesper Dangaard Brouer
>> <brouer@redhat.com> wrote:
>>> 
>>> 
>>> On Fri, 6 May 2016 10:41:53 +0200 moeller0 <moeller0@gmx.de> wrote:
>>> 
>>>> 	Speaking out of total ignorance, I ask why not account
>>>> GRO/GSO packets by the number of their fragments against the packet
>>>> limit? Counting a 64kB packets as equivalent to a 64B packet
>> probably
>>>> is the right thing if one tries to account for the work the OS
>> needs
>>>> to perform to figure out what to do with the packet, but for
>> limiting
>>>> the memory consumption it introduces an impressive/manly level of
>>>> uncertainty (2 orders of magnitude). 
>>> 
>>> Looking at the drop code in fq_codel:
>>> 
>> https://github.com/torvalds/linux/blob/v4.6-rc6/net/sched/sch_fq_codel.c#L136
>>> 
>>> It looks like we are finding the "fat" flow to drop from based on
>>> number of bytes queued.  And AFAIK skb->len also account for the
>> total
>>> length of all GSO packets (Eric?)
>>> 
>>> Even better, we are using qdisc_pkt_len(skb), which also account for
>>> the GSO headers in qdisc_pkt_len_init(). 
>>> https://github.com/torvalds/linux/blob/v4.6-rc6/net/core/dev.c#L2993
>>> 
>>> If anything, the GSO packets get hit harder by the fq_codel_drop
>>> function, as we drop the entire GSO skb.
>> 
>> 	This sounds all very reassuring!
>> 
>>> 
>>> 
>>> Is the issue you are raising that the 1024 packet limit, would allow
>>> 1024 x 64K bytes to be queued before the drop kicks in? (Resulting
>> in
>>> using too much memory on your small device).
>> 
>> 	Yes, I guess I need to explain better. My wndr3700v7 only sports a
>> measly 64MB ram total, so in the past with the default 10240 limit I
>> could force OOM-initiated reboots. So my angle on this issue is
>> always, I want my router to survive even if the the network is
>> over-saturated (I do not expect the router to work well under those
>> circumstances, but I somehow think it should at least not reboot
>> during DOS conditions…). Cake allows to specify a hard memory limit
>> that looks a skb-truesize to allow stricter memory control for people
>> like me, making the whole GRO/GSO issue go away, at least as far as
>> OOM is concerned. And as I begin to understand once a link reaches a
>> certain bandwidth GRO/GSO will become helpful again, especially on
>> small routers as they help reduce the work the kernel needs to to per
>> byte…
> 
> Angles of attack :
> 
> 1) I will provide a per device /sys/class/net/eth0/gro_max_frags so that
> we can more easily control amount of segs per GRO packets. It makes
> sense to have GRO, but not so much allowing it to cook big packets that
> might hurt FQ.

	This sounds great, so we can teach, say sqm to set this to a reasonable value given the (shaped) bandwidth of a given interface. Would something like this also make sense/is possible on the send side for GSO/TSO?

> 
> 2) Tracking skb->truesize looks mandatory for small devices.
> I will add this to fq_codel.

	Thanks.

> 
> 3) Making sure skb->truesize is accurate is a long term effort we want
> to constantly monitor, since some drivers are doing under estimations.

	Is there an easy way to test/measure this? Say if 2) is implemented how can one figure out how much memory is actually allocated for a given queue?

Best Regards & Merci beaucoup
	Sebastian

> 
> Thanks.
> 
> 
> 
> 
>