From: "Toke Høiland-Jørgensen" <toke@toke.dk>
To: Jamal Hadi Salim <jhs@mojatatu.com>, netdev@vger.kernel.org
Cc: "Jamal Hadi Salim" <jhs@mojatatu.com>,
"Jiri Pirko" <jiri@resnulli.us>,
"David S . Miller" <davem@davemloft.net>,
"Eric Dumazet" <edumazet@kernel.org>,
"Jakub Kicinski" <kuba@kernel.org>,
"Paolo Abeni" <pabeni@redhat.com>,
"Simon Horman" <horms@kernel.org>,
"Victor Nogueira" <victor@mojatatu.com>,
"Björn Töpel" <bjorn@kernel.org>,
cake@lists.bufferbloat.net, Sashiko <sashiko-bot@kernel.org>
Subject: [Cake] Re: [PATCH net-next v7 3/4] net/sched: cap the accounted backlog before it can wrap
Date: Mon, 19 Oct 2026 20:20:37 +0200 [thread overview]
Message-ID: <87qzhlwo0a.fsf@toke.dk> (raw)
In-Reply-To: <QDISC-BA27.v7.r0.20261019115505-3@mojatatu.com>
Jamal Hadi Salim <jhs@mojatatu.com> writes:
> This is a followup of commit 709f34f7c28d ("net/sched: fq: add overflow
> bounds to quantum and initial quantum") as reported by Sashiko.
The rest of this commit message is somewhat obtuse - too much run-on
information. I think it can be pretty easily fixed by a couple of
rearrangements and paragraph breaks, like this:
> The issue: fq_codel, cake, codel, pie, fq_pie, dualpi2, RED, pfifo,
> plug and (non-WRED) gred account every enqueued packet's stab-adjusted
> length into the 32-bit sch->qstats.backlog. A crafted tc size table
> inflates qdisc_pkt_len() up to QDISC_PKT_LEN_MAX (1 MiB), and a dodgy
> GSO skb can account even more through qdisc_pkt_len_segs_init(), so a
> few thousand packets wrap the counter mod 2^32.
>
> The AQM then reads a small backlog and makes the wrong drop decision,
> and the dequeue-side subtractions leave it corrupt (for non-WRED gred
> it is the reported stats and the dequeue-side subtractions that go
> wrong instead). In fq_codel a wrapped per-flow backlog can also make
> fq_codel_drop()'s scan pick an empty flow, dereferencing a NULL
> flow->head.
>
> The Fix: drop at enqueue once the stored backlog cannot represent the
> incoming packet's accounted length.
>
> Ordinary packets (qdisc_pkt_len() at most QDISC_PKT_LEN_MAX) are
> covered by a standing ceiling: drop once the backlog is within
> QDISC_MAX_BACKLOG (U32_MAX - QDISC_PKT_LEN_MAX) of the wrap point, the
> largest backlog one more 1 MiB packet cannot wrap, so there is no
> per-enqueue 64-bit add.
>
> A larger wire estimate is checked against the space that actually
> remains (len > U32_MAX - backlog): qdisc_pkt_len_segs_init() adds the
> headers replicated on every output segment, so a supported small-MSS
> BIG TCP aggregate can carry a wire estimate above 1 MiB while its raw
> length stays within GSO_MAX_SIZE, and such traffic must pass on an
> empty queue.
>
> qdisc_pkt_len() itself stays truthful for every input: it is not
> clamped and the initializer keeps succeeding, so BPF
> __sk_buff.wire_len, act_police and the HTB/TBF shapers read the real
> length, and sch_handle_ingress() -- which ignores the initializer's
> return value and reads pkt_segs for packet statistics -- keeps
> complete metadata.
>
> Cover sch_plug and non-WRED sch_gred too: plug admits on skb->len
> while qdisc_enqueue_tail() adds qdisc_pkt_len(), and gred bounds each
> virtual queue but adds every VQ into the one aggregate. pfifo is
> included because RED reads a grafted child's backlog into
> red_calc_qavg(); it can be reached standalone or as the default child
> of SFB. In RED, account the packet to the aggregate before handing it
> to the child, because a duplicating netem child re-enqueues its clone
> at the root while the stored backlog is still un-updated -- one guard
> check must not authorize two additions.
>
> The guard bounds each qdisc's own enqueue path: a child that segments a
> GSO skb (tbf, netem) can still grow a parent's aggregate through
> qdisc_tree_reduce_backlog()'s unclamped byte delta, which is left to a
> separate change.
>
> Conditions to recreate the bug:
> ip tuntap add tun0 mode tun
> ip link set tun0 txqueuelen 32 up
> ip addr add 10.99.0.1/24 dev tun0
> tc qdisc add dev tun0 root handle 1: stab overhead 2000000000 \
> fq_codel flows 1 limit 20000 ecn
> # hold the tun fd open without reading so the backlog persists, then
> # send 6000 packets: the counter wraps and the unfixed kernel reports
> # a wrapped backlog, while the fixed one drops at the ceiling.
>
> Reported-by: Sashiko (nipa) <sashiko-bot@kernel.org>
> Closes: https://lore.kernel.org/netdev/20260818101130.16203-1-jhs@mojatatu.com/
> Link: https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20260818101130.16203-1-jhs@mojatatu.com
> Link: https://lore.kernel.org/netdev/CANn89iLfMJV7ancKH1Gjzzm7ZUG-gKcrczkJjJEWN5sCuWd-ug@mail.gmail.com/
> Link: https://lore.kernel.org/netdev/CANn89i+GOFH_8g+vSV0jUj0aBqLYVWuFOrHuDggZu9kPC_m6tQ@mail.gmail.com/
> Link: https://lore.kernel.org/netdev/87a4oottcj.fsf@toke.dk/
> Link: https://lore.kernel.org/netdev/87pkxks68f.fsf@toke.dk/
> Link: https://lore.kernel.org/netdev/179231932946.3135.7497478990186385688@kernel.org/
> Reviewed-by: Victor Nogueira <victor@mojatatu.com>
> Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
With the above rewording:
Reviewed-by: Toke Høiland-Jørgensen <toke@toke.dk>
next prev parent reply other threads:[~2026-10-19 18:20 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-19 16:00 [Cake] [PATCH net-next v7 0/4] net/sched: cap the accounted backlog before it can wrap Jamal Hadi Salim
2026-10-19 16:00 ` [Cake] [PATCH net-next v7 1/4] net/sched: cake, fq_codel: return the classifier result to the caller Jamal Hadi Salim
2026-10-19 18:11 ` [Cake] " Toke Høiland-Jørgensen
2026-10-19 21:05 ` Jamal Hadi Salim
2026-10-20 8:58 ` Toke Høiland-Jørgensen
2026-10-20 10:06 ` Jamal Hadi Salim
2026-10-19 16:00 ` [Cake] [PATCH net-next v7 2/4] net/sched: cake: use a dedicated split-GSO list iterator Jamal Hadi Salim
2026-10-19 18:11 ` [Cake] " Toke Høiland-Jørgensen
2026-10-19 16:00 ` [Cake] [PATCH net-next v7 3/4] net/sched: cap the accounted backlog before it can wrap Jamal Hadi Salim
2026-10-19 18:20 ` Toke Høiland-Jørgensen [this message]
2026-10-19 21:08 ` [Cake] " Jamal Hadi Salim
2026-10-19 16:00 ` [Cake] [PATCH net-next v7 4/4] net/sched: cake: hash before segmenting and reject an overlong split Jamal Hadi Salim
2026-10-19 18:32 ` [Cake] " Toke Høiland-Jørgensen
2026-10-19 21:35 ` Jamal Hadi Salim
2026-10-20 8:47 ` Toke Høiland-Jørgensen
2026-10-20 10:08 ` Jamal Hadi Salim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://lists.bufferbloat.net/postorius/lists/cake.lists.bufferbloat.net/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87qzhlwo0a.fsf@toke.dk \
--to=toke@toke.dk \
--cc=bjorn@kernel.org \
--cc=cake@lists.bufferbloat.net \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=horms@kernel.org \
--cc=jhs@mojatatu.com \
--cc=jiri@resnulli.us \
--cc=kuba@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=sashiko-bot@kernel.org \
--cc=victor@mojatatu.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox