[Cake] [PATCH net-next] net/sched: sch_cake: skip clearing unused tins during rate adjustment

[Cake] [PATCH net-next] net/sched: sch_cake: skip clearing unused tins during rate adjustment

[Jonas Köppeler]

When cake_configure_rates() is called from the dequeue path with
rate_adjust=true, it only needs to update the rate parameters. The
loop that clears the unused tins is both unnecessary and harmful in
this path:

 - cake_clear_tin() overwrites q->cur_tin and q->cur_flow, which are
   actively used by cake_dequeue(), corrupting the dequeue state.
 - iterating over the unused tins and their internal queues to purge
   packets adds needless overhead to the hot path.

Skip the entire loop when rate_adjust is set, as neither
cake_clear_tin() nor the mtu_time update are needed when only the
rate changes.

Fixes: 15c2715a5264 ("net/sched: sch_cake: fixup cake_mq rate adjustment for diffserv config")
Signed-off-by: Jonas Köppeler <j.koeppeler@tu-berlin.de>
Tested-by: Mike Pham <mikepham4321@gmail.com>
---
 net/sched/sch_cake.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index f78f8e950776..845e1c017714 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -2609,9 +2609,11 @@ static void cake_configure_rates(struct Qdisc *sch, u64 rate, bool rate_adjust)
 		break;
 	}
 
-	for (c = qd->tin_cnt; c < CAKE_MAX_TINS; c++) {
-		cake_clear_tin(sch, c);
-		qd->tins[c].cparams.mtu_time = qd->tins[ft].cparams.mtu_time;
+	if (!rate_adjust) {
+		for (c = qd->tin_cnt; c < CAKE_MAX_TINS; c++) {
+			cake_clear_tin(sch, c);
+			qd->tins[c].cparams.mtu_time = qd->tins[ft].cparams.mtu_time;
+		}
 	}
 
 	qd->rate_ns   = qd->tins[ft].tin_rate_ns;

---
base-commit: f6f3b36c15ed44de1fbb44e645e4fae8c4a4453e
change-id: 20260716-sch_cake-skip-clearing-tins-856812586cde

Best regards,
--  
Jonas Köppeler <j.koeppeler@tu-berlin.de>

[Cake] Re: [PATCH net-next] net/sched: sch_cake: skip clearing unused tins during rate adjustment

[Toke Høiland-Jørgensen]

Jonas Köppeler <j.koeppeler@tu-berlin.de> writes:

> When cake_configure_rates() is called from the dequeue path with
> rate_adjust=true, it only needs to update the rate parameters. The
> loop that clears the unused tins is both unnecessary and harmful in
> this path:
>
>  - cake_clear_tin() overwrites q->cur_tin and q->cur_flow, which are
>    actively used by cake_dequeue(), corrupting the dequeue state.
>  - iterating over the unused tins and their internal queues to purge
>    packets adds needless overhead to the hot path.
>
> Skip the entire loop when rate_adjust is set, as neither
> cake_clear_tin() nor the mtu_time update are needed when only the
> rate changes.
>
> Fixes: 15c2715a5264 ("net/sched: sch_cake: fixup cake_mq rate adjustment for diffserv config")
> Signed-off-by: Jonas Köppeler <j.koeppeler@tu-berlin.de>
> Tested-by: Mike Pham <mikepham4321@gmail.com>

Do you have any performance numbers to show the impact of this?

-Toke

[Cake] Re: [PATCH net-next] net/sched: sch_cake: skip clearing unused tins during rate adjustment

[Jonas Köppeler]

On 7/17/26 10:31, Toke Høiland-Jørgensen wrote:
> Jonas Köppeler <j.koeppeler@tu-berlin.de> writes:
> 
>> When cake_configure_rates() is called from the dequeue path with
>> rate_adjust=true, it only needs to update the rate parameters. The
>> loop that clears the unused tins is both unnecessary and harmful in
>> this path:
>>
>>   - cake_clear_tin() overwrites q->cur_tin and q->cur_flow, which are
>>     actively used by cake_dequeue(), corrupting the dequeue state.
>>   - iterating over the unused tins and their internal queues to purge
>>     packets adds needless overhead to the hot path.
>>
>> Skip the entire loop when rate_adjust is set, as neither
>> cake_clear_tin() nor the mtu_time update are needed when only the
>> rate changes.
>>
>> Fixes: 15c2715a5264 ("net/sched: sch_cake: fixup cake_mq rate adjustment for diffserv config")
>> Signed-off-by: Jonas Köppeler <j.koeppeler@tu-berlin.de>
>> Tested-by: Mike Pham <mikepham4321@gmail.com>
> 
> Do you have any performance numbers to show the impact of this?
Yes, the table below shows results from a test setup using vng with
2 network namespaces, with cake/cake_mq attached in one of them:

     ns1 -> cake/cake_mq -> ns2

- veth devices are configured with 8 rx/tx queues.
- cake/cake_mq is configured with a 2 Gbit rate limit.
- Running flent's rrul and tcp_nup tests with 32 TCP upstreams:

legend: qdisc mq = cake_mq; mode be = besteffort, ds3 = diffserv3
         test nup = tcp_nup; base/load = idle/loaded RTT (ms); tput = Mbit/s

+---------------------+-------+------+------+-------+-------+---------+
| kernel              | qdisc | mode | test |  base |  load |    tput |
+---------------------+-------+------+------+-------+-------+---------+
| net-next            | cake  | be   | rrul | 0.075 |  4.76 | 1473.69 |
| net-next            | cake  | be   | nup  | 0.078 |  6.23 | 1550.79 |
| net-next            | cake  | ds3  | rrul | 0.063 |  5.81 | 1526.75 |
| net-next            | cake  | ds3  | nup  | 0.046 |  6.09 | 1761.45 |
+---------------------+-------+------+------+-------+-------+---------+
| net-next            | mq    | be   | rrul | 0.810 | 11.78 | 1469.67 |
| net-next            | mq    | be   | nup  | 0.637 | 85.71 | 1243.15 |
| net-next            | mq    | ds3  | rrul | 0.397 | 15.28 | 1770.06 |
| net-next            | mq    | ds3  | nup  | 0.351 | 15.98 | 1799.39 |
+---------------------+-------+------+------+-------+-------+---------+
| this patch          | mq    | be   | rrul | 0.092 |  0.56 | 1873.40 |
| this patch          | mq    | be   | nup  | 0.109 |  1.82 | 1869.12 |
| this patch          | mq    | ds3  | rrul | 0.097 |  0.98 | 1866.10 |
| this patch          | mq    | ds3  | nup  | 0.101 |  0.51 | 1861.79 |
+---------------------+-------+------+------+-------+-------+---------+
| before 15c2715a5264 | mq    | be   | rrul | 0.073 |  0.30 | 1895.45 |
| before 15c2715a5264 | mq    | be   | nup  | 0.076 |  0.49 | 1905.57 |
| before 15c2715a5264 | mq    | ds3  | rrul | 0.069 |  0.31 | 1896.59 |
| before 15c2715a5264 | mq    | ds3  | nup  | 0.058 |  0.86 | 1884.01 |
+---------------------+-------+------+------+-------+-------+---------+

Not only is p99 latency drastically reduced -- nearly matching
pre-15c2715a5264 results -- but on current upstream cake_mq,
throughput also increases as a cake mode uses more tins. This points
directly to cake_clear_tin() during reconfig as the cause, since it
clears (max_tins - cur_tins) tins each time. So the fewer tins the
current mode uses, the more get cleared on every reconfig.

Mike ran also some test on OpenWrt, on an IPQ8074A with 4 rx/tx
queues, and saw similar trends. cake_mq is configured with a 2.2 Gbit
rate limit.

Unfortunately, we only have data for 128 TCP upstreams on net-next,
and 64 TCP upstreams for 'this patch'.

+---------------------+-------+------+------+---------+----------+
| kernel              | qdisc | mode | test |    load |     tput |
+---------------------+-------+------+------+---------+----------+
| net-next            | mq    | be   | nup  |  468.50 |    50.90 |
| net-next            | mq    | ds3  | nup  |  355.22 |    98.21 |
| net-next            | mq    | ds4  | nup  |  268.28 |   255.84 |
| net-next            | mq    | ds8  | nup  |    7.48 |  2023.66 |
+---------------------+-------+------+------+---------+----------+
| this patch          | mq    | be   | nup  |    4.24 |   944.35 |
| this patch          | mq    | ds3  | nup  |    4.27 |   937.75 |
| this patch          | mq    | ds4  | nup  |    4.24 |   936.97 |
| this patch          | mq    | ds8  | nup  |    4.32 |   927.89 |
+---------------------+-------+------+------+---------+----------+

This again shows the same trend: throughput increases and latency
drops as cake_mq is configured with more tins. We're still looking
into why net-next+ds8 reaches close to 2 Gbit/s, while this patch
tops out around 928 Mbit/s.

That said, this patch doesn't solve every issue yet, but it does
remove the regression introduced by commit 15c2715a5264
("net/sched: sch_cake: fixup cake_mq rate adjustment for diffserv
config").

We're continuing to look into further improvements. Let us know if
you'd like to see additional tests :)

- Jonas

> 
> -Toke

[Cake] Re: [PATCH net-next] net/sched: sch_cake: skip clearing unused tins during rate adjustment

[Toke Høiland-Jørgensen]

On 18 July 2026 17.06.28 CEST, "Jonas Köppeler" <j.koeppeler@tu-berlin.de> wrote:
>On 7/17/26 10:31, Toke Høiland-Jørgensen wrote:
>> Jonas Köppeler <j.koeppeler@tu-berlin.de> writes:
>> 
>>> When cake_configure_rates() is called from the dequeue path with
>>> rate_adjust=true, it only needs to update the rate parameters. The
>>> loop that clears the unused tins is both unnecessary and harmful in
>>> this path:
>>> 
>>>   - cake_clear_tin() overwrites q->cur_tin and q->cur_flow, which are
>>>     actively used by cake_dequeue(), corrupting the dequeue state.
>>>   - iterating over the unused tins and their internal queues to purge
>>>     packets adds needless overhead to the hot path.
>>> 
>>> Skip the entire loop when rate_adjust is set, as neither
>>> cake_clear_tin() nor the mtu_time update are needed when only the
>>> rate changes.
>>> 
>>> Fixes: 15c2715a5264 ("net/sched: sch_cake: fixup cake_mq rate adjustment for diffserv config")
>>> Signed-off-by: Jonas Köppeler <j.koeppeler@tu-berlin.de>
>>> Tested-by: Mike Pham <mikepham4321@gmail.com>
>> 
>> Do you have any performance numbers to show the impact of this?
>Yes, the table below shows results from a test setup using vng with
>2 network namespaces, with cake/cake_mq attached in one of them:
>
>    ns1 -> cake/cake_mq -> ns2
>
>- veth devices are configured with 8 rx/tx queues.
>- cake/cake_mq is configured with a 2 Gbit rate limit.
>- Running flent's rrul and tcp_nup tests with 32 TCP upstreams:
>
>legend: qdisc mq = cake_mq; mode be = besteffort, ds3 = diffserv3
>        test nup = tcp_nup; base/load = idle/loaded RTT (ms); tput = Mbit/s
>
>+---------------------+-------+------+------+-------+-------+---------+
>| kernel              | qdisc | mode | test |  base |  load |    tput |
>+---------------------+-------+------+------+-------+-------+---------+
>| net-next            | cake  | be   | rrul | 0.075 |  4.76 | 1473.69 |
>| net-next            | cake  | be   | nup  | 0.078 |  6.23 | 1550.79 |
>| net-next            | cake  | ds3  | rrul | 0.063 |  5.81 | 1526.75 |
>| net-next            | cake  | ds3  | nup  | 0.046 |  6.09 | 1761.45 |
>+---------------------+-------+------+------+-------+-------+---------+
>| net-next            | mq    | be   | rrul | 0.810 | 11.78 | 1469.67 |
>| net-next            | mq    | be   | nup  | 0.637 | 85.71 | 1243.15 |
>| net-next            | mq    | ds3  | rrul | 0.397 | 15.28 | 1770.06 |
>| net-next            | mq    | ds3  | nup  | 0.351 | 15.98 | 1799.39 |
>+---------------------+-------+------+------+-------+-------+---------+
>| this patch          | mq    | be   | rrul | 0.092 |  0.56 | 1873.40 |
>| this patch          | mq    | be   | nup  | 0.109 |  1.82 | 1869.12 |
>| this patch          | mq    | ds3  | rrul | 0.097 |  0.98 | 1866.10 |
>| this patch          | mq    | ds3  | nup  | 0.101 |  0.51 | 1861.79 |
>+---------------------+-------+------+------+-------+-------+---------+
>| before 15c2715a5264 | mq    | be   | rrul | 0.073 |  0.30 | 1895.45 |
>| before 15c2715a5264 | mq    | be   | nup  | 0.076 |  0.49 | 1905.57 |
>| before 15c2715a5264 | mq    | ds3  | rrul | 0.069 |  0.31 | 1896.59 |
>| before 15c2715a5264 | mq    | ds3  | nup  | 0.058 |  0.86 | 1884.01 |
>+---------------------+-------+------+------+-------+-------+---------+
>
>Not only is p99 latency drastically reduced -- nearly matching
>pre-15c2715a5264 results -- but on current upstream cake_mq,
>throughput also increases as a cake mode uses more tins. This points
>directly to cake_clear_tin() during reconfig as the cause, since it
>clears (max_tins - cur_tins) tins each time. So the fewer tins the
>current mode uses, the more get cleared on every reconfig.
>
>Mike ran also some test on OpenWrt, on an IPQ8074A with 4 rx/tx
>queues, and saw similar trends. cake_mq is configured with a 2.2 Gbit
>rate limit.
>
>Unfortunately, we only have data for 128 TCP upstreams on net-next,
>and 64 TCP upstreams for 'this patch'.
>
>+---------------------+-------+------+------+---------+----------+
>| kernel              | qdisc | mode | test |    load |     tput |
>+---------------------+-------+------+------+---------+----------+
>| net-next            | mq    | be   | nup  |  468.50 |    50.90 |
>| net-next            | mq    | ds3  | nup  |  355.22 |    98.21 |
>| net-next            | mq    | ds4  | nup  |  268.28 |   255.84 |
>| net-next            | mq    | ds8  | nup  |    7.48 |  2023.66 |
>+---------------------+-------+------+------+---------+----------+
>| this patch          | mq    | be   | nup  |    4.24 |   944.35 |
>| this patch          | mq    | ds3  | nup  |    4.27 |   937.75 |
>| this patch          | mq    | ds4  | nup  |    4.24 |   936.97 |
>| this patch          | mq    | ds8  | nup  |    4.32 |   927.89 |
>+---------------------+-------+------+------+---------+----------+
>
>This again shows the same trend: throughput increases and latency
>drops as cake_mq is configured with more tins. We're still looking
>into why net-next+ds8 reaches close to 2 Gbit/s, while this patch
>tops out around 928 Mbit/s.
>
>That said, this patch doesn't solve every issue yet, but it does
>remove the regression introduced by commit 15c2715a5264
>("net/sched: sch_cake: fixup cake_mq rate adjustment for diffserv
>config").
>
>We're continuing to look into further improvements. Let us know if
>you'd like to see additional tests :)

Cool! Could you please respin the patch with this data in the commit message?

Doesn't have to be all of it, but some indication of the benefit would be good to have on hand for future reference :)

-Toke