Re: [Make-wifi-fast] [Codel] fq_codel_drop vs a udp flood

[Eric Dumazet <eric.dumazet@gmail.com>]

On Fri, 2016-05-06 at 13:46 +0200, moeller0 wrote:
> Hi Jesper,
> 
> > On May 6, 2016, at 13:33 , Jesper Dangaard Brouer
> <brouer@redhat.com> wrote:
> > 
> > 
> > On Fri, 6 May 2016 10:41:53 +0200 moeller0 <moeller0@gmx.de> wrote:
> > 
> >> 	Speaking out of total ignorance, I ask why not account
> >> GRO/GSO packets by the number of their fragments against the packet
> >> limit? Counting a 64kB packets as equivalent to a 64B packet
> probably
> >> is the right thing if one tries to account for the work the OS
> needs
> >> to perform to figure out what to do with the packet, but for
> limiting
> >> the memory consumption it introduces an impressive/manly level of
> >> uncertainty (2 orders of magnitude). 
> > 
> > Looking at the drop code in fq_codel:
> >
> https://github.com/torvalds/linux/blob/v4.6-rc6/net/sched/sch_fq_codel.c#L136
> > 
> > It looks like we are finding the "fat" flow to drop from based on
> > number of bytes queued.  And AFAIK skb->len also account for the
> total
> > length of all GSO packets (Eric?)
> > 
> > Even better, we are using qdisc_pkt_len(skb), which also account for
> > the GSO headers in qdisc_pkt_len_init(). 
> > https://github.com/torvalds/linux/blob/v4.6-rc6/net/core/dev.c#L2993
> > 
> > If anything, the GSO packets get hit harder by the fq_codel_drop
> > function, as we drop the entire GSO skb.
> 
> 	This sounds all very reassuring!
> 
> > 
> > 
> > Is the issue you are raising that the 1024 packet limit, would allow
> > 1024 x 64K bytes to be queued before the drop kicks in? (Resulting
> in
> > using too much memory on your small device).
> 
> 	Yes, I guess I need to explain better. My wndr3700v7 only sports a
> measly 64MB ram total, so in the past with the default 10240 limit I
> could force OOM-initiated reboots. So my angle on this issue is
> always, I want my router to survive even if the the network is
> over-saturated (I do not expect the router to work well under those
> circumstances, but I somehow think it should at least not reboot
> during DOS conditions…). Cake allows to specify a hard memory limit
> that looks a skb-truesize to allow stricter memory control for people
> like me, making the whole GRO/GSO issue go away, at least as far as
> OOM is concerned. And as I begin to understand once a link reaches a
> certain bandwidth GRO/GSO will become helpful again, especially on
> small routers as they help reduce the work the kernel needs to to per
> byte…

Angles of attack :

1) I will provide a per device /sys/class/net/eth0/gro_max_frags so that
we can more easily control amount of segs per GRO packets. It makes
sense to have GRO, but not so much allowing it to cook big packets that
might hurt FQ.

2) Tracking skb->truesize looks mandatory for small devices.
I will add this to fq_codel.

3) Making sure skb->truesize is accurate is a long term effort we want
to constantly monitor, since some drivers are doing under estimations.

Thanks.