From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-x229.google.com (mail-pf0-x229.google.com [IPv6:2607:f8b0:400e:c00::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 6F5C43B25E; Fri, 6 May 2016 09:25:59 -0400 (EDT) Received: by mail-pf0-x229.google.com with SMTP id y69so49595389pfb.1; Fri, 06 May 2016 06:25:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:in-reply-to:references :mime-version:content-transfer-encoding; bh=XJQR8jbx7ZdQt8uzfS7KEp0ejuVNPBtWcVkrx3CW+bM=; b=e3B382Ybsf8pOahHnq8hihuOynmFEQEbZ9CUTz3W7rjdwHjA7Yx/2lAr/sOi80PzGL LhaJTZth6AVrF+ih0ss4BrcMkt7Fb0Nkz0iGGJnc5t7TmRsDK1nJHGTtH+X4o14F18Wy rd/GFEiHYeXoV9uf5vGYl/Bm1QUxVvyjbzlwb8xn2Zb6TFgmG06qCvjni4MXtn3xDrC2 zJ9t0GNVobIczSkq32JeVTGeKZ5p/TCEhqLA3OEPVauxBXRgHERGMVmFr3s/CaGdJG/3 IGQmZotThUarNmPsx8GCqzGz23JG668hSXb+YKUUpSdExB+5vzfbshCHl71Tufgm+F5d 11Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=XJQR8jbx7ZdQt8uzfS7KEp0ejuVNPBtWcVkrx3CW+bM=; b=MUs8eBMdaqL0VfFIYYt6IDiU9t7SrTzQGV6Napb4i+TTcCvbfqDXM0c0Ys9OG69lgS /bLvuqyX2iwfjQT0HN8fMp3W5rN+mjgr4Sb3lUacVnsVDbbavreUWbS5WUB/5urZ+zAI Uj3ZabCqj5WwwFg3YMjFVhw5rO+/i/y+KAcPVD05RiruFuIfnPtZqouFqrdpI0hgB6w4 wf09s/wS328j3CTa5KUBAi8Kn0edmCCPrSX+P0mc/8qISlRbQG63E0m/cdkgPUgfggIv /JN4mrjq5NisqHPisV1VVBz6ZlhMpXO/c+As7sMAWckR9dq4/aiE0N+1h1wlQFhfQhlL /sZQ== X-Gm-Message-State: AOPr4FUR5fszsHsTOmGiKmE2PzDiFJJhoL4etFDmLBkfhzuqCFEo0yYH3/vOqlcQr2k5ug== X-Received: by 10.98.53.6 with SMTP id c6mr29129233pfa.89.1462541158402; Fri, 06 May 2016 06:25:58 -0700 (PDT) Received: from [172.26.49.69] ([172.26.49.69]) by smtp.googlemail.com with ESMTPSA id s23sm21207752pfj.86.2016.05.06.06.25.56 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 06 May 2016 06:25:57 -0700 (PDT) Message-ID: <1462541156.13075.34.camel@edumazet-glaptop3.roam.corp.google.com> From: Eric Dumazet To: moeller0 Cc: Jesper Dangaard Brouer , Dave =?ISO-8859-1?Q?T=E4ht?= , make-wifi-fast@lists.bufferbloat.net, ath10k , "codel@lists.bufferbloat.net" , Jonathan Morton , Roman Yeryomin Date: Fri, 06 May 2016 06:25:56 -0700 In-Reply-To: References: <865DA393-262D-40B6-A9D3-1B978CD5F6C6@gmail.com> <1462128385.5535.200.camel@edumazet-glaptop3.roam.corp.google.com> <1462136140.5535.219.camel@edumazet-glaptop3.roam.corp.google.com> <1462201620.5535.250.camel@edumazet-glaptop3.roam.corp.google.com> <1462205669.5535.254.camel@edumazet-glaptop3.roam.corp.google.com> <1462464776.13075.18.camel@edumazet-glaptop3.roam.corp.google.com> <1462476207.13075.20.camel@edumazet-glaptop3.roam.corp.google.com> <542135C7-D7CC-4E33-B35B-C2AD259FA5AB@gmx.de> <20160506133323.0b190f47@redhat.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.10.4-0ubuntu2 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: Re: [Make-wifi-fast] [Codel] fq_codel_drop vs a udp flood X-BeenThere: make-wifi-fast@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 May 2016 13:25:59 -0000 On Fri, 2016-05-06 at 13:46 +0200, moeller0 wrote: > Hi Jesper, > > > On May 6, 2016, at 13:33 , Jesper Dangaard Brouer > wrote: > > > > > > On Fri, 6 May 2016 10:41:53 +0200 moeller0 wrote: > > > >> Speaking out of total ignorance, I ask why not account > >> GRO/GSO packets by the number of their fragments against the packet > >> limit? Counting a 64kB packets as equivalent to a 64B packet > probably > >> is the right thing if one tries to account for the work the OS > needs > >> to perform to figure out what to do with the packet, but for > limiting > >> the memory consumption it introduces an impressive/manly level of > >> uncertainty (2 orders of magnitude). > > > > Looking at the drop code in fq_codel: > > > https://github.com/torvalds/linux/blob/v4.6-rc6/net/sched/sch_fq_codel.c#L136 > > > > It looks like we are finding the "fat" flow to drop from based on > > number of bytes queued. And AFAIK skb->len also account for the > total > > length of all GSO packets (Eric?) > > > > Even better, we are using qdisc_pkt_len(skb), which also account for > > the GSO headers in qdisc_pkt_len_init(). > > https://github.com/torvalds/linux/blob/v4.6-rc6/net/core/dev.c#L2993 > > > > If anything, the GSO packets get hit harder by the fq_codel_drop > > function, as we drop the entire GSO skb. > > This sounds all very reassuring! > > > > > > > Is the issue you are raising that the 1024 packet limit, would allow > > 1024 x 64K bytes to be queued before the drop kicks in? (Resulting > in > > using too much memory on your small device). > > Yes, I guess I need to explain better. My wndr3700v7 only sports a > measly 64MB ram total, so in the past with the default 10240 limit I > could force OOM-initiated reboots. So my angle on this issue is > always, I want my router to survive even if the the network is > over-saturated (I do not expect the router to work well under those > circumstances, but I somehow think it should at least not reboot > during DOS conditions…). Cake allows to specify a hard memory limit > that looks a skb-truesize to allow stricter memory control for people > like me, making the whole GRO/GSO issue go away, at least as far as > OOM is concerned. And as I begin to understand once a link reaches a > certain bandwidth GRO/GSO will become helpful again, especially on > small routers as they help reduce the work the kernel needs to to per > byte… Angles of attack : 1) I will provide a per device /sys/class/net/eth0/gro_max_frags so that we can more easily control amount of segs per GRO packets. It makes sense to have GRO, but not so much allowing it to cook big packets that might hurt FQ. 2) Tracking skb->truesize looks mandatory for small devices. I will add this to fq_codel. 3) Making sure skb->truesize is accurate is a long term effort we want to constantly monitor, since some drivers are doing under estimations. Thanks.