From frantisek.borsik at gmail.com  Mon Dec  2 06:38:57 2024
From: frantisek.borsik at gmail.com (Frantisek Borsik)
Date: Mon, 2 Dec 2024 12:38:57 +0100
Subject: [Cake] Registration for Understanding Latency 3.0 is open!
In-Reply-To: <CAJUtOOhwZ71B55YbeKTDqfCZG5yy2oMu5Zfyhev-UwXbjTGhZg@mail.gmail.com>
References: <42f9fdfad6a7bbc3af0a33f41.7e1d8e665c.20241111144952.146b035395.78514b4b@mail243.atl61.mcsv.net>
 <CAJUtOOhwZ71B55YbeKTDqfCZG5yy2oMu5Zfyhev-UwXbjTGhZg@mail.gmail.com>
Message-ID: <CAJUtOOjwGP5KdF0PtLOjSghrX0A5UY4w15r=Mvv3=dKGazgo_A@mail.gmail.com>

See you all next week, starting on Monday, December 9!
https://understandinglatency.com

[image: GcbTc-MW8AAXD0b.jpeg]

All the best,

Frank

Frantisek (Frank) Borsik



https://www.linkedin.com/in/frantisekborsik

Signal, Telegram, WhatsApp: +421919416714

iMessage, mobile: +420775230885

Skype: casioa5302ca

frantisek.borsik at gmail.com


On Mon, Nov 11, 2024 at 6:13 PM Frantisek Borsik <frantisek.borsik at gmail.com>
wrote:

> Hello to all,
>
> Register for the 3rd run of the FREE webinar series, Understanding Latency
> https://understandinglatency.com, organized by Domos.ai - if you are
> interested in the 1.0 and 2.0 runs, here you are:
> https://www.youtube.com/@domoslabs/videos
>
> PS: LibreQoS/Dave Taht will be there!
>
> All the best,
>
> Frank
>
> Frantisek (Frank) Borsik
>
>
>
> https://www.linkedin.com/in/frantisekborsik
>
> Signal, Telegram, WhatsApp: +421919416714
>
> iMessage, mobile: +420775230885
>
> Skype: casioa5302ca
>
> frantisek.borsik at gmail.com
>
>
> ---------- Forwarded message ---------
> From: Domos <info at domos.no>
> Date: Mon, Nov 11, 2024 at 2:50 PM
> Subject: Registration for Understanding Latency 3.0 is open!
> To: <frantisek.borsik at gmail.com>
>
>
> View this email in your browser
> <https://mailchi.mp/domos/your-links-to-the-understanding-latency-webinar-series-6417399?e=7e1d8e665c>
>
> <https://domos.us11.list-manage.com/track/click?u=42f9fdfad6a7bbc3af0a33f41&id=74aa87c21f&e=7e1d8e665c>
>
> FINALLY: Understanding Latency
> <https://domos.us11.list-manage.com/track/click?u=42f9fdfad6a7bbc3af0a33f41&id=8aaef5c0fe&e=7e1d8e665c>
> is back!
>
>    - 9-11 December 2024
>    - Online
>    - The Biggest Expert-Led Knowledge-Sharing Event on Network Latency
>
> Our biggest event of the year is back, with top industry speakers already
> lining up. Expect expert insights, practical advice, and a whole lot of
> knowledge-sharing.
>
> Reserve your spot now! *Register here: understandinglatency.com/register
> <https://domos.us11.list-manage.com/track/click?u=42f9fdfad6a7bbc3af0a33f41&id=147ea0f8e2&e=7e1d8e665c>*
>
> Best,
> The Understanding Latency team
> *You are receiving this email because you previously signed up to stay
> informed about our latest updates and events.*
>
> Want to change how you receive these emails?
> You can update your preferences
> <https://domos.us11.list-manage.com/profile?u=42f9fdfad6a7bbc3af0a33f41&id=01cb58b2d8&e=7e1d8e665c&c=146b035395>
> or unsubscribe from this list
> <https://domos.us11.list-manage.com/unsubscribe?u=42f9fdfad6a7bbc3af0a33f41&id=01cb58b2d8&t=b&e=7e1d8e665c&c=146b035395>.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20241202/1ba10e00/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: GcbTc-MW8AAXD0b.jpeg
Type: image/jpeg
Size: 130224 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20241202/1ba10e00/attachment-0001.jpeg>

From dave.seddon.ca at gmail.com  Fri Dec  6 18:43:17 2024
From: dave.seddon.ca at gmail.com (dave seddon)
Date: Fri, 6 Dec 2024 15:43:17 -0800
Subject: [Cake] cake at 8gbit
In-Reply-To: <CAA93jw6MdQkJXU=4E4QT6t6chDO__sv4dEPY5_A_FUW1xw1aHw@mail.gmail.com>
References: <CAA93jw6MdQkJXU=4E4QT6t6chDO__sv4dEPY5_A_FUW1xw1aHw@mail.gmail.com>
Message-ID: <CANypexQwc7OVZQWJU+SPdPg9ZfX6ucML+B4swhY_DDeMoC-BYg@mail.gmail.com>

Thanks again for this link.  Super interesting.

Regarding monitoring socket performance with Kubernetes clusters, I've
actually been working on deploying the xTCP socket monitoring tool into
kubernetes clusters.  The intention would be to stream the tcp_diag data
out of all the PoD on a regular basis.

The challenge is that ideally from a single process, you could open a
netlink socket into every PoD on the kubernetes node.  This is not
possible, as far as I understand, because a process can only live in a
single name space at any given time.

e.g. You can't do this:

[image: image.png]

The simple solution would be too run many versions of xTCP as a "sidecar"
in each PoD, like this:

[image: image.png]

This isn't great because then xTCP would be duplicated many times, so it
would waste RAMs, and you would have lots of Kafka sockets streaming the
socket data out out each PoD.

An alternative I was thinking would be to possibly have a small unix domain
socket (UDS) to netlink proxy in each PoD.  Over the UDS, the xTCP
daemonset ( single instance per node) could read and write the tcp_diag
data.

(e.g. I was thinking of a little rust binary that would essentially open a
UDS socket and a netlink socket, and then essentially copy from one to the
other. )


[image: image.png]


I don't really know if this is a good idea, or if I'm missing some other
way to extract the socket data from many PoDs.  Happy to hear ideas please!
:)

Full xtCP slides, including these "xTCP for kubernetes" here:
https://docs.google.com/presentation/d/11rixKNfIBCdofUpPL2wOuiWJXa40x4I0A1wuTMMv4Uo/edit#slide=id.g31cb19b2f14_0_87

Thanks,
Dave Seddon

On Tue, Nov 19, 2024 at 8:07 AM Dave Taht via Cake <
cake at lists.bufferbloat.net> wrote:

> https://github.com/cilium/cilium/issues/29083#issuecomment-2485142294
>
> --
> Dave Täht CSO, LibreQos
> _______________________________________________
> Cake mailing list
> Cake at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>


-- 
Regards,
Dave Seddon
+1 415 857 5102
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20241206/a5690e9d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 129740 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20241206/a5690e9d/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 183206 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20241206/a5690e9d/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 164177 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20241206/a5690e9d/attachment-0005.png>

From toke at redhat.com  Mon Dec  9 07:02:18 2024
From: toke at redhat.com (=?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?=)
Date: Mon, 09 Dec 2024 13:02:18 +0100
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
Message-ID: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>

Add three qdisc-specific drop reasons for sch_cake:

 1) SKB_DROP_REASON_CAKE_CONGESTED
    Whenever a packet is dropped by the CAKE AQM algorithm because
    congestion is detected.

 2) SKB_DROP_REASON_CAKE_FLOOD
    Whenever a packet is dropped by the flood protection part of the
    CAKE AQM algorithm (BLUE).

 3) SKB_DROP_REASON_CAKE_OVERLIMIT
    Whenever the total queue limit for a CAKE instance is exceeded and a
    packet is dropped to make room.

Also use the existing SKB_DROP_REASON_QUEUE_PURGE in cake_clear_tin().

Reasons show up as:

perf record -a -e skb:kfree_skb sleep 1; perf script

          iperf3     665 [005]   848.656964: skb:kfree_skb: skbaddr=0xffff98168a333500 rx_sk=(nil) protocol=34525 location=__dev_queue_xmit+0x10f0 reason: CAKE_OVERLIMIT
         swapper       0 [001]   909.166055: skb:kfree_skb: skbaddr=0xffff98168280cee0 rx_sk=(nil) protocol=34525 location=cake_dequeue+0x5ef reason: CAKE_CONGESTED

Signed-off-by: Toke Høiland-Jørgensen <toke at redhat.com>
---
 include/net/dropreason-core.h | 18 ++++++++++++++++++
 net/sched/sch_cake.c          | 43 +++++++++++++++++++++++--------------------
 2 files changed, 41 insertions(+), 20 deletions(-)

diff --git a/include/net/dropreason-core.h b/include/net/dropreason-core.h
index c29282fabae6cdf9dd79f698b92b4b8f57156b1e..a9be76be11ad67d6cc7175f1a643314a7dcdf0b8 100644
--- a/include/net/dropreason-core.h
+++ b/include/net/dropreason-core.h
@@ -61,6 +61,9 @@
 	FN(FQ_BAND_LIMIT)		\
 	FN(FQ_HORIZON_LIMIT)		\
 	FN(FQ_FLOW_LIMIT)		\
+	FN(CAKE_CONGESTED)		\
+	FN(CAKE_FLOOD)			\
+	FN(CAKE_OVERLIMIT)		\
 	FN(CPU_BACKLOG)			\
 	FN(XDP)				\
 	FN(TC_INGRESS)			\
@@ -329,6 +332,21 @@ enum skb_drop_reason {
 	 * exceeds its limits.
 	 */
 	SKB_DROP_REASON_FQ_FLOW_LIMIT,
+	/**
+	 * @SKB_DROP_REASON_CAKE_CONGESTED: dropped by the CAKE qdisc AQM
+	 * algorithm due to congestion.
+	 */
+	SKB_DROP_REASON_CAKE_CONGESTED,
+	/**
+	 * @SKB_DROP_REASON_CAKE_FLOOD: dropped by the flood protection part of
+	 * CAKE qdisc AQM algorithm (BLUE).
+	 */
+	SKB_DROP_REASON_CAKE_FLOOD,
+	/**
+	 * @SKB_DROP_REASON_CAKE_OVERLIMIT: dropped by CAKE qdisc when a qdisc
+	 * instance exceeds its total buffer size limit.
+	 */
+	SKB_DROP_REASON_CAKE_OVERLIMIT,
 	/**
 	 * @SKB_DROP_REASON_CPU_BACKLOG: failed to enqueue the skb to the per CPU
 	 * backlog queue. This can be caused by backlog queue full (see
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index 8d8b2db4653c0c9f271f9c1953e8c61175d8f76b..a57bdc771dd14e81fb0cdf54ca7890ac96f1e311 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -484,13 +484,14 @@ static bool cobalt_queue_empty(struct cobalt_vars *vars,
 /* Call this with a freshly dequeued packet for possible congestion marking.
  * Returns true as an instruction to drop the packet, false for delivery.
  */
-static bool cobalt_should_drop(struct cobalt_vars *vars,
-			       struct cobalt_params *p,
-			       ktime_t now,
-			       struct sk_buff *skb,
-			       u32 bulk_flows)
+static enum skb_drop_reason cobalt_should_drop(struct cobalt_vars *vars,
+					       struct cobalt_params *p,
+					       ktime_t now,
+					       struct sk_buff *skb,
+					       u32 bulk_flows)
 {
-	bool next_due, over_target, drop = false;
+	enum skb_drop_reason reason = SKB_NOT_DROPPED_YET;
+	bool next_due, over_target;
 	ktime_t schedule;
 	u64 sojourn;
 
@@ -533,7 +534,8 @@ static bool cobalt_should_drop(struct cobalt_vars *vars,
 
 	if (next_due && vars->dropping) {
 		/* Use ECN mark if possible, otherwise drop */
-		drop = !(vars->ecn_marked = INET_ECN_set_ce(skb));
+		if (!(vars->ecn_marked = INET_ECN_set_ce(skb)))
+			reason = SKB_DROP_REASON_CAKE_CONGESTED;
 
 		vars->count++;
 		if (!vars->count)
@@ -556,16 +558,17 @@ static bool cobalt_should_drop(struct cobalt_vars *vars,
 	}
 
 	/* Simple BLUE implementation.  Lack of ECN is deliberate. */
-	if (vars->p_drop)
-		drop |= (get_random_u32() < vars->p_drop);
+	if (vars->p_drop && reason == SKB_NOT_DROPPED_YET &&
+	    get_random_u32() < vars->p_drop)
+		reason = SKB_DROP_REASON_CAKE_FLOOD;
 
 	/* Overload the drop_next field as an activity timeout */
 	if (!vars->count)
 		vars->drop_next = ktime_add_ns(now, p->interval);
-	else if (ktime_to_ns(schedule) > 0 && !drop)
+	else if (ktime_to_ns(schedule) > 0 && reason == SKB_NOT_DROPPED_YET)
 		vars->drop_next = now;
 
-	return drop;
+	return reason;
 }
 
 static bool cake_update_flowkeys(struct flow_keys *keys,
@@ -1528,12 +1531,11 @@ static unsigned int cake_drop(struct Qdisc *sch, struct sk_buff **to_free)
 
 	flow->dropped++;
 	b->tin_dropped++;
-	sch->qstats.drops++;
 
 	if (q->rate_flags & CAKE_FLAG_INGRESS)
 		cake_advance_shaper(q, b, skb, now, true);
 
-	__qdisc_drop(skb, to_free);
+	qdisc_drop_reason(skb, sch, to_free, SKB_DROP_REASON_CAKE_OVERLIMIT);
 	sch->q.qlen--;
 	qdisc_tree_reduce_backlog(sch, 1, len);
 
@@ -1926,7 +1928,7 @@ static void cake_clear_tin(struct Qdisc *sch, u16 tin)
 	q->cur_tin = tin;
 	for (q->cur_flow = 0; q->cur_flow < CAKE_QUEUES; q->cur_flow++)
 		while (!!(skb = cake_dequeue_one(sch)))
-			kfree_skb(skb);
+			kfree_skb_reason(skb, SKB_DROP_REASON_QUEUE_PURGE);
 }
 
 static struct sk_buff *cake_dequeue(struct Qdisc *sch)
@@ -1934,6 +1936,7 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
 	struct cake_sched_data *q = qdisc_priv(sch);
 	struct cake_tin_data *b = &q->tins[q->cur_tin];
 	struct cake_host *srchost, *dsthost;
+	enum skb_drop_reason reason;
 	ktime_t now = ktime_get();
 	struct cake_flow *flow;
 	struct list_head *head;
@@ -2143,12 +2146,12 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
 			goto begin;
 		}
 
+		reason = cobalt_should_drop(&flow->cvars, &b->cparams, now, skb,
+					    (b->bulk_flow_count *
+					     !!(q->rate_flags &
+						CAKE_FLAG_INGRESS)));
 		/* Last packet in queue may be marked, shouldn't be dropped */
-		if (!cobalt_should_drop(&flow->cvars, &b->cparams, now, skb,
-					(b->bulk_flow_count *
-					 !!(q->rate_flags &
-					    CAKE_FLAG_INGRESS))) ||
-		    !flow->head)
+		if (reason == SKB_NOT_DROPPED_YET || !flow->head)
 			break;
 
 		/* drop this packet, get another one */
@@ -2162,7 +2165,7 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
 		b->tin_dropped++;
 		qdisc_tree_reduce_backlog(sch, 1, qdisc_pkt_len(skb));
 		qdisc_qstats_drop(sch);
-		kfree_skb(skb);
+		kfree_skb_reason(skb, reason);
 		if (q->rate_flags & CAKE_FLAG_INGRESS)
 			goto retry;
 	}

---
base-commit: 7ea2745766d776866cfbc981b21ed3cfdf50124e
change-id: 20241205-cake-drop-reason-b1661e1e7f0a


From edumazet at google.com  Mon Dec  9 08:26:10 2024
From: edumazet at google.com (Eric Dumazet)
Date: Mon, 9 Dec 2024 14:26:10 +0100
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
In-Reply-To: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
References: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
Message-ID: <CANn89iJg=NBG4ubKA77iRZ0zxLk2isMRsC3tzJew4rBW3OO0kQ@mail.gmail.com>

On Mon, Dec 9, 2024 at 1:02 PM Toke Høiland-Jørgensen <toke at redhat.com> wrote:
>
> Add three qdisc-specific drop reasons for sch_cake:
>
>  1) SKB_DROP_REASON_CAKE_CONGESTED
>     Whenever a packet is dropped by the CAKE AQM algorithm because
>     congestion is detected.
>
>  2) SKB_DROP_REASON_CAKE_FLOOD
>     Whenever a packet is dropped by the flood protection part of the
>     CAKE AQM algorithm (BLUE).
>
>  3) SKB_DROP_REASON_CAKE_OVERLIMIT
>     Whenever the total queue limit for a CAKE instance is exceeded and a
>     packet is dropped to make room.
>
> Also use the existing SKB_DROP_REASON_QUEUE_PURGE in cake_clear_tin().
>
> Reasons show up as:
>
> perf record -a -e skb:kfree_skb sleep 1; perf script
>
>           iperf3     665 [005]   848.656964: skb:kfree_skb: skbaddr=0xffff98168a333500 rx_sk=(nil) protocol=34525 location=__dev_queue_xmit+0x10f0 reason: CAKE_OVERLIMIT
>          swapper       0 [001]   909.166055: skb:kfree_skb: skbaddr=0xffff98168280cee0 rx_sk=(nil) protocol=34525 location=cake_dequeue+0x5ef reason: CAKE_CONGESTED
>
> Signed-off-by: Toke Høiland-Jørgensen <toke at redhat.com>

Reviewed-by: Eric Dumazet <edumazet at google.com>

From jhs at mojatatu.com  Mon Dec  9 18:00:44 2024
From: jhs at mojatatu.com (Jamal Hadi Salim)
Date: Mon, 9 Dec 2024 18:00:44 -0500
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
In-Reply-To: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
References: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
Message-ID: <CAM0EoM=KWpQMcFrHiDPH=FM1raL47McVkCYBMGHBcScXGY_z3Q@mail.gmail.com>

On Mon, Dec 9, 2024 at 7:02 AM Toke Høiland-Jørgensen <toke at redhat.com> wrote:
>
> Add three qdisc-specific drop reasons for sch_cake:
>
>  1) SKB_DROP_REASON_CAKE_CONGESTED
>     Whenever a packet is dropped by the CAKE AQM algorithm because
>     congestion is detected.
>
>  2) SKB_DROP_REASON_CAKE_FLOOD
>     Whenever a packet is dropped by the flood protection part of the
>     CAKE AQM algorithm (BLUE).
>
>  3) SKB_DROP_REASON_CAKE_OVERLIMIT
>     Whenever the total queue limit for a CAKE instance is exceeded and a
>     packet is dropped to make room.
>
> Also use the existing SKB_DROP_REASON_QUEUE_PURGE in cake_clear_tin().
>
> Reasons show up as:
>
> perf record -a -e skb:kfree_skb sleep 1; perf script
>
>           iperf3     665 [005]   848.656964: skb:kfree_skb: skbaddr=0xffff98168a333500 rx_sk=(nil) protocol=34525 location=__dev_queue_xmit+0x10f0 reason: CAKE_OVERLIMIT
>          swapper       0 [001]   909.166055: skb:kfree_skb: skbaddr=0xffff98168280cee0 rx_sk=(nil) protocol=34525 location=cake_dequeue+0x5ef reason: CAKE_CONGESTED
>
> Signed-off-by: Toke Høiland-Jørgensen <toke at redhat.com>

Reviewed-by: Jamal Hadi Salim <jhs at mojatatu.com>

cheers,
jamal

From dave.taht at gmail.com  Mon Dec  9 18:14:58 2024
From: dave.taht at gmail.com (Dave Taht)
Date: Mon, 9 Dec 2024 15:14:58 -0800
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
In-Reply-To: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
References: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
Message-ID: <CAA93jw4S_fLpZDvk04QxeAaBaH5m6d8px1jE-B4KKqR-C88Khw@mail.gmail.com>

On Mon, Dec 9, 2024 at 4:02 AM Toke Høiland-Jørgensen via Cake
<cake at lists.bufferbloat.net> wrote:
>
> Add three qdisc-specific drop reasons for sch_cake:
>
>  1) SKB_DROP_REASON_CAKE_CONGESTED
>     Whenever a packet is dropped by the CAKE AQM algorithm because
>     congestion is detected.
>
>  2) SKB_DROP_REASON_CAKE_FLOOD
>     Whenever a packet is dropped by the flood protection part of the
>     CAKE AQM algorithm (BLUE).
>
>  3) SKB_DROP_REASON_CAKE_OVERLIMIT
>     Whenever the total queue limit for a CAKE instance is exceeded and a
>     packet is dropped to make room.
>
> Also use the existing SKB_DROP_REASON_QUEUE_PURGE in cake_clear_tin().
>
> Reasons show up as:
>
> perf record -a -e skb:kfree_skb sleep 1; perf script
>
>           iperf3     665 [005]   848.656964: skb:kfree_skb: skbaddr=0xffff98168a333500 rx_sk=(nil) protocol=34525 location=__dev_queue_xmit+0x10f0 reason: CAKE_OVERLIMIT
>          swapper       0 [001]   909.166055: skb:kfree_skb: skbaddr=0xffff98168280cee0 rx_sk=(nil) protocol=34525 location=cake_dequeue+0x5ef reason: CAKE_CONGESTED
>
> Signed-off-by: Toke Høiland-Jørgensen <toke at redhat.com>

Acked-by: Dave Taht <dave.taht at gmail.com>

> ---
>  include/net/dropreason-core.h | 18 ++++++++++++++++++
>  net/sched/sch_cake.c          | 43 +++++++++++++++++++++++--------------------
>  2 files changed, 41 insertions(+), 20 deletions(-)
>
> diff --git a/include/net/dropreason-core.h b/include/net/dropreason-core.h
> index c29282fabae6cdf9dd79f698b92b4b8f57156b1e..a9be76be11ad67d6cc7175f1a643314a7dcdf0b8 100644
> --- a/include/net/dropreason-core.h
> +++ b/include/net/dropreason-core.h
> @@ -61,6 +61,9 @@
>         FN(FQ_BAND_LIMIT)               \
>         FN(FQ_HORIZON_LIMIT)            \
>         FN(FQ_FLOW_LIMIT)               \
> +       FN(CAKE_CONGESTED)              \
> +       FN(CAKE_FLOOD)                  \
> +       FN(CAKE_OVERLIMIT)              \
>         FN(CPU_BACKLOG)                 \
>         FN(XDP)                         \
>         FN(TC_INGRESS)                  \
> @@ -329,6 +332,21 @@ enum skb_drop_reason {
>          * exceeds its limits.
>          */
>         SKB_DROP_REASON_FQ_FLOW_LIMIT,
> +       /**
> +        * @SKB_DROP_REASON_CAKE_CONGESTED: dropped by the CAKE qdisc AQM
> +        * algorithm due to congestion.
> +        */
> +       SKB_DROP_REASON_CAKE_CONGESTED,
> +       /**
> +        * @SKB_DROP_REASON_CAKE_FLOOD: dropped by the flood protection part of
> +        * CAKE qdisc AQM algorithm (BLUE).
> +        */
> +       SKB_DROP_REASON_CAKE_FLOOD,
> +       /**
> +        * @SKB_DROP_REASON_CAKE_OVERLIMIT: dropped by CAKE qdisc when a qdisc
> +        * instance exceeds its total buffer size limit.
> +        */
> +       SKB_DROP_REASON_CAKE_OVERLIMIT,
>         /**
>          * @SKB_DROP_REASON_CPU_BACKLOG: failed to enqueue the skb to the per CPU
>          * backlog queue. This can be caused by backlog queue full (see
> diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
> index 8d8b2db4653c0c9f271f9c1953e8c61175d8f76b..a57bdc771dd14e81fb0cdf54ca7890ac96f1e311 100644
> --- a/net/sched/sch_cake.c
> +++ b/net/sched/sch_cake.c
> @@ -484,13 +484,14 @@ static bool cobalt_queue_empty(struct cobalt_vars *vars,
>  /* Call this with a freshly dequeued packet for possible congestion marking.
>   * Returns true as an instruction to drop the packet, false for delivery.
>   */
> -static bool cobalt_should_drop(struct cobalt_vars *vars,
> -                              struct cobalt_params *p,
> -                              ktime_t now,
> -                              struct sk_buff *skb,
> -                              u32 bulk_flows)
> +static enum skb_drop_reason cobalt_should_drop(struct cobalt_vars *vars,
> +                                              struct cobalt_params *p,
> +                                              ktime_t now,
> +                                              struct sk_buff *skb,
> +                                              u32 bulk_flows)
>  {
> -       bool next_due, over_target, drop = false;
> +       enum skb_drop_reason reason = SKB_NOT_DROPPED_YET;
> +       bool next_due, over_target;
>         ktime_t schedule;
>         u64 sojourn;
>
> @@ -533,7 +534,8 @@ static bool cobalt_should_drop(struct cobalt_vars *vars,
>
>         if (next_due && vars->dropping) {
>                 /* Use ECN mark if possible, otherwise drop */
> -               drop = !(vars->ecn_marked = INET_ECN_set_ce(skb));
> +               if (!(vars->ecn_marked = INET_ECN_set_ce(skb)))
> +                       reason = SKB_DROP_REASON_CAKE_CONGESTED;
>
>                 vars->count++;
>                 if (!vars->count)
> @@ -556,16 +558,17 @@ static bool cobalt_should_drop(struct cobalt_vars *vars,
>         }
>
>         /* Simple BLUE implementation.  Lack of ECN is deliberate. */
> -       if (vars->p_drop)
> -               drop |= (get_random_u32() < vars->p_drop);
> +       if (vars->p_drop && reason == SKB_NOT_DROPPED_YET &&
> +           get_random_u32() < vars->p_drop)
> +               reason = SKB_DROP_REASON_CAKE_FLOOD;
>
>         /* Overload the drop_next field as an activity timeout */
>         if (!vars->count)
>                 vars->drop_next = ktime_add_ns(now, p->interval);
> -       else if (ktime_to_ns(schedule) > 0 && !drop)
> +       else if (ktime_to_ns(schedule) > 0 && reason == SKB_NOT_DROPPED_YET)
>                 vars->drop_next = now;
>
> -       return drop;
> +       return reason;
>  }
>
>  static bool cake_update_flowkeys(struct flow_keys *keys,
> @@ -1528,12 +1531,11 @@ static unsigned int cake_drop(struct Qdisc *sch, struct sk_buff **to_free)
>
>         flow->dropped++;
>         b->tin_dropped++;
> -       sch->qstats.drops++;
>
>         if (q->rate_flags & CAKE_FLAG_INGRESS)
>                 cake_advance_shaper(q, b, skb, now, true);
>
> -       __qdisc_drop(skb, to_free);
> +       qdisc_drop_reason(skb, sch, to_free, SKB_DROP_REASON_CAKE_OVERLIMIT);
>         sch->q.qlen--;
>         qdisc_tree_reduce_backlog(sch, 1, len);
>
> @@ -1926,7 +1928,7 @@ static void cake_clear_tin(struct Qdisc *sch, u16 tin)
>         q->cur_tin = tin;
>         for (q->cur_flow = 0; q->cur_flow < CAKE_QUEUES; q->cur_flow++)
>                 while (!!(skb = cake_dequeue_one(sch)))
> -                       kfree_skb(skb);
> +                       kfree_skb_reason(skb, SKB_DROP_REASON_QUEUE_PURGE);
>  }
>
>  static struct sk_buff *cake_dequeue(struct Qdisc *sch)
> @@ -1934,6 +1936,7 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
>         struct cake_sched_data *q = qdisc_priv(sch);
>         struct cake_tin_data *b = &q->tins[q->cur_tin];
>         struct cake_host *srchost, *dsthost;
> +       enum skb_drop_reason reason;
>         ktime_t now = ktime_get();
>         struct cake_flow *flow;
>         struct list_head *head;
> @@ -2143,12 +2146,12 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
>                         goto begin;
>                 }
>
> +               reason = cobalt_should_drop(&flow->cvars, &b->cparams, now, skb,
> +                                           (b->bulk_flow_count *
> +                                            !!(q->rate_flags &
> +                                               CAKE_FLAG_INGRESS)));
>                 /* Last packet in queue may be marked, shouldn't be dropped */
> -               if (!cobalt_should_drop(&flow->cvars, &b->cparams, now, skb,
> -                                       (b->bulk_flow_count *
> -                                        !!(q->rate_flags &
> -                                           CAKE_FLAG_INGRESS))) ||
> -                   !flow->head)
> +               if (reason == SKB_NOT_DROPPED_YET || !flow->head)
>                         break;
>
>                 /* drop this packet, get another one */
> @@ -2162,7 +2165,7 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
>                 b->tin_dropped++;
>                 qdisc_tree_reduce_backlog(sch, 1, qdisc_pkt_len(skb));
>                 qdisc_qstats_drop(sch);
> -               kfree_skb(skb);
> +               kfree_skb_reason(skb, reason);
>                 if (q->rate_flags & CAKE_FLAG_INGRESS)
>                         goto retry;
>         }
>
> ---
> base-commit: 7ea2745766d776866cfbc981b21ed3cfdf50124e
> change-id: 20241205-cake-drop-reason-b1661e1e7f0a
>
> _______________________________________________
> Cake mailing list
> Cake at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



-- 
Dave Täht CSO, LibreQos

From kuba at kernel.org  Mon Dec  9 18:51:57 2024
From: kuba at kernel.org (Jakub Kicinski)
Date: Mon, 9 Dec 2024 15:51:57 -0800
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
In-Reply-To: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
References: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
Message-ID: <20241209155157.6a817bc5@kernel.org>

On Mon, 09 Dec 2024 13:02:18 +0100 Toke Høiland-Jørgensen wrote:
> Add three qdisc-specific drop reasons for sch_cake:
> 
>  1) SKB_DROP_REASON_CAKE_CONGESTED
>     Whenever a packet is dropped by the CAKE AQM algorithm because
>     congestion is detected.
> 
>  2) SKB_DROP_REASON_CAKE_FLOOD
>     Whenever a packet is dropped by the flood protection part of the
>     CAKE AQM algorithm (BLUE).
> 
>  3) SKB_DROP_REASON_CAKE_OVERLIMIT
>     Whenever the total queue limit for a CAKE instance is exceeded and a
>     packet is dropped to make room.

Eric's patch was adding fairly FQ-specific reasons, other than flood
this seems like generic AQM stuff, no? From a very quick look the
congestion looks like fairly standard AQM, overlimit is also typical
for qdics?

From dave.taht at gmail.com  Mon Dec  9 19:25:01 2024
From: dave.taht at gmail.com (Dave Taht)
Date: Mon, 9 Dec 2024 16:25:01 -0800
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
In-Reply-To: <20241209155157.6a817bc5@kernel.org>
References: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
 <20241209155157.6a817bc5@kernel.org>
Message-ID: <CAA93jw4chUsQ40LQStvJBeOEENydbv58gOWz8y7fFPJkATa9tA@mail.gmail.com>

On Mon, Dec 9, 2024 at 3:52 PM Jakub Kicinski via Cake
<cake at lists.bufferbloat.net> wrote:
>
> On Mon, 09 Dec 2024 13:02:18 +0100 Toke Høiland-Jørgensen wrote:
> > Add three qdisc-specific drop reasons for sch_cake:
> >
> >  1) SKB_DROP_REASON_CAKE_CONGESTED
> >     Whenever a packet is dropped by the CAKE AQM algorithm because
> >     congestion is detected.
> >
> >  2) SKB_DROP_REASON_CAKE_FLOOD
> >     Whenever a packet is dropped by the flood protection part of the
> >     CAKE AQM algorithm (BLUE).
> >
> >  3) SKB_DROP_REASON_CAKE_OVERLIMIT
> >     Whenever the total queue limit for a CAKE instance is exceeded and a
> >     packet is dropped to make room.
>
> Eric's patch was adding fairly FQ-specific reasons, other than flood
> this seems like generic AQM stuff, no? From a very quick look the
> congestion looks like fairly standard AQM, overlimit is also typical
> for qdics?

While I initially agreed with making this generic, preserving the qdisc from
where the drop came lets you safely inspect the cb block (timestamp, etc),
format of which varies by qdisc. You also get insight as to which
qdisc was dropping.

Downside is we'll end up with SKB_DROP_REASON_XXX_OVERLIMIT for
each of the qdiscs. Etc.

> _______________________________________________
> Cake mailing list
> Cake at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



-- 
Dave Täht CSO, LibreQos

From toke at redhat.com  Tue Dec 10 03:42:55 2024
From: toke at redhat.com (Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?=)
Date: Tue, 10 Dec 2024 09:42:55 +0100
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
In-Reply-To: <CAA93jw4chUsQ40LQStvJBeOEENydbv58gOWz8y7fFPJkATa9tA@mail.gmail.com>
References: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
 <20241209155157.6a817bc5@kernel.org>
 <CAA93jw4chUsQ40LQStvJBeOEENydbv58gOWz8y7fFPJkATa9tA@mail.gmail.com>
Message-ID: <87a5d46i9c.fsf@toke.dk>

Dave Taht <dave.taht at gmail.com> writes:

> On Mon, Dec 9, 2024 at 3:52 PM Jakub Kicinski via Cake
> <cake at lists.bufferbloat.net> wrote:
>>
>> On Mon, 09 Dec 2024 13:02:18 +0100 Toke Høiland-Jørgensen wrote:
>> > Add three qdisc-specific drop reasons for sch_cake:
>> >
>> >  1) SKB_DROP_REASON_CAKE_CONGESTED
>> >     Whenever a packet is dropped by the CAKE AQM algorithm because
>> >     congestion is detected.
>> >
>> >  2) SKB_DROP_REASON_CAKE_FLOOD
>> >     Whenever a packet is dropped by the flood protection part of the
>> >     CAKE AQM algorithm (BLUE).
>> >
>> >  3) SKB_DROP_REASON_CAKE_OVERLIMIT
>> >     Whenever the total queue limit for a CAKE instance is exceeded and a
>> >     packet is dropped to make room.
>>
>> Eric's patch was adding fairly FQ-specific reasons, other than flood
>> this seems like generic AQM stuff, no? From a very quick look the
>> congestion looks like fairly standard AQM, overlimit is also typical
>> for qdics?
>
> While I initially agreed with making this generic, preserving the qdisc from
> where the drop came lets you safely inspect the cb block (timestamp, etc),
> format of which varies by qdisc. You also get insight as to which
> qdisc was dropping.
>
> Downside is we'll end up with SKB_DROP_REASON_XXX_OVERLIMIT for
> each of the qdiscs. Etc.

Yeah, I agree that a generic "dropped by AQM" reason will be too generic
without knowing which qdisc dropped it. I guess any calls directly to
kfree_skb_reason() from the qdisc will provide the calling function, but
for qdisc_drop_reason() the drop will be deferred to __dev_queue_xmit(),
so no way of knowing where the drop came from, AFAICT?

-Toke


From chromatix99 at gmail.com  Tue Dec 10 04:10:57 2024
From: chromatix99 at gmail.com (Jonathan Morton)
Date: Tue, 10 Dec 2024 11:10:57 +0200
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
In-Reply-To: <87a5d46i9c.fsf@toke.dk>
References: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
 <20241209155157.6a817bc5@kernel.org>
 <CAA93jw4chUsQ40LQStvJBeOEENydbv58gOWz8y7fFPJkATa9tA@mail.gmail.com>
 <87a5d46i9c.fsf@toke.dk>
Message-ID: <C2055FB9-8EA4-486D-9654-84ED422A4A0C@gmail.com>

> On 10 Dec, 2024, at 10:42 am, Toke Høiland-Jørgensen via Cake <cake at lists.bufferbloat.net> wrote:
> 
>>> On Mon, 09 Dec 2024 13:02:18 +0100 Toke Høiland-Jørgensen wrote:
>>>> Add three qdisc-specific drop reasons for sch_cake:
>>>> 
>>>> 1) SKB_DROP_REASON_CAKE_CONGESTED
>>>>    Whenever a packet is dropped by the CAKE AQM algorithm because
>>>>    congestion is detected.
>>>> 
>>>> 2) SKB_DROP_REASON_CAKE_FLOOD
>>>>    Whenever a packet is dropped by the flood protection part of the
>>>>    CAKE AQM algorithm (BLUE).
>>>> 
>>>> 3) SKB_DROP_REASON_CAKE_OVERLIMIT
>>>>    Whenever the total queue limit for a CAKE instance is exceeded and a
>>>>    packet is dropped to make room.
>>> 
>>> Eric's patch was adding fairly FQ-specific reasons, other than flood
>>> this seems like generic AQM stuff, no? From a very quick look the
>>> congestion looks like fairly standard AQM, overlimit is also typical
>>> for qdics?
>> 
>> While I initially agreed with making this generic, preserving the qdisc from
>> where the drop came lets you safely inspect the cb block (timestamp, etc),
>> format of which varies by qdisc. You also get insight as to which
>> qdisc was dropping.
>> 
>> Downside is we'll end up with SKB_DROP_REASON_XXX_OVERLIMIT for
>> each of the qdiscs. Etc.
> 
> Yeah, I agree that a generic "dropped by AQM" reason will be too generic
> without knowing which qdisc dropped it. I guess any calls directly to
> kfree_skb_reason() from the qdisc will provide the calling function, but
> for qdisc_drop_reason() the drop will be deferred to __dev_queue_xmit(),
> so no way of knowing where the drop came from, AFAICT?

Would it make sense to be able to extract a "generic" code by applying a bitmask?  Leave code space for "qdisc specific" reasons within that mask.  Then people who don't care about qdisc internals can still reliably interpret the codes, even for future qdiscs.

 - Jonathan Morton

From kuba at kernel.org  Tue Dec 10 20:28:02 2024
From: kuba at kernel.org (Jakub Kicinski)
Date: Tue, 10 Dec 2024 17:28:02 -0800
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
In-Reply-To: <87a5d46i9c.fsf@toke.dk>
References: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
 <20241209155157.6a817bc5@kernel.org>
 <CAA93jw4chUsQ40LQStvJBeOEENydbv58gOWz8y7fFPJkATa9tA@mail.gmail.com>
 <87a5d46i9c.fsf@toke.dk>
Message-ID: <20241210172802.410c76a6@kernel.org>

On Tue, 10 Dec 2024 09:42:55 +0100 Toke Høiland-Jørgensen wrote:
> > While I initially agreed with making this generic, preserving the qdisc from
> > where the drop came lets you safely inspect the cb block (timestamp, etc),
> > format of which varies by qdisc. You also get insight as to which
> > qdisc was dropping.
> >
> > Downside is we'll end up with SKB_DROP_REASON_XXX_OVERLIMIT for
> > each of the qdiscs. Etc.  
> 
> Yeah, I agree that a generic "dropped by AQM" reason will be too generic
> without knowing which qdisc dropped it.

Why does type of the qdisc matter if the qdisc was overlimit?

> I guess any calls directly to kfree_skb_reason() from the qdisc will
> provide the calling function, but for qdisc_drop_reason() the drop
> will be deferred to __dev_queue_xmit(), so no way of knowing where
> the drop came from, AFAICT?

Can you tell me why I'd need to inspect the skb->cb[] in cake if packet
is overlimit? Actually, none of the fields of the cb are initialized
when the packet is dropped for overlimit, AFAIU.

If someone is doing serious / advanced debug they mostly care about
access to the qdisc and can trivially check if its ops match the
expected symbol. (Speaking from experience, I've been debugging FQ
packet loss on Nov 23rd.)

If someone is just doing high level drop attribution having to list all
possible qdiscs under "qdisc discard" is purely pain.

Can we start with OVERLIMIT and CONGESTION as generic values and we can
specialize if anyone has a clear need?

From toke at redhat.com  Wed Dec 11 04:55:03 2024
From: toke at redhat.com (Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?=)
Date: Wed, 11 Dec 2024 10:55:03 +0100
Subject: [Cake] [PATCH net-next] net_sched: sch_cake: Add drop reasons
In-Reply-To: <20241210172802.410c76a6@kernel.org>
References: <20241209-cake-drop-reason-v1-1-19205f6d1f19@redhat.com>
 <20241209155157.6a817bc5@kernel.org>
 <CAA93jw4chUsQ40LQStvJBeOEENydbv58gOWz8y7fFPJkATa9tA@mail.gmail.com>
 <87a5d46i9c.fsf@toke.dk> <20241210172802.410c76a6@kernel.org>
Message-ID: <87sequ5ytk.fsf@toke.dk>

Jakub Kicinski <kuba at kernel.org> writes:

> On Tue, 10 Dec 2024 09:42:55 +0100 Toke Høiland-Jørgensen wrote:
>> > While I initially agreed with making this generic, preserving the qdisc from
>> > where the drop came lets you safely inspect the cb block (timestamp, etc),
>> > format of which varies by qdisc. You also get insight as to which
>> > qdisc was dropping.
>> >
>> > Downside is we'll end up with SKB_DROP_REASON_XXX_OVERLIMIT for
>> > each of the qdiscs. Etc.  
>> 
>> Yeah, I agree that a generic "dropped by AQM" reason will be too generic
>> without knowing which qdisc dropped it.
>
> Why does type of the qdisc matter if the qdisc was overlimit?

Well, I was thinking you'd want to figure out which device it was
dropped from, but I guess you have skb->dev for that (and counters).

>> I guess any calls directly to kfree_skb_reason() from the qdisc will
>> provide the calling function, but for qdisc_drop_reason() the drop
>> will be deferred to __dev_queue_xmit(), so no way of knowing where
>> the drop came from, AFAICT?
>
> Can you tell me why I'd need to inspect the skb->cb[] in cake if packet
> is overlimit? Actually, none of the fields of the cb are initialized
> when the packet is dropped for overlimit, AFAIU.
>
> If someone is doing serious / advanced debug they mostly care about
> access to the qdisc and can trivially check if its ops match the
> expected symbol. (Speaking from experience, I've been debugging FQ
> packet loss on Nov 23rd.)
>
> If someone is just doing high level drop attribution having to list all
> possible qdiscs under "qdisc discard" is purely pain.
>
> Can we start with OVERLIMIT and CONGESTION as generic values and we can
> specialize if anyone has a clear need?

OK, I'll respin and drop CAKE from the names of those two...

-Toke


From toke at redhat.com  Wed Dec 11 05:17:09 2024
From: toke at redhat.com (=?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?=)
Date: Wed, 11 Dec 2024 11:17:09 +0100
Subject: [Cake] [PATCH net-next v2] net_sched: sch_cake: Add drop reasons
Message-ID: <20241211-cake-drop-reason-v2-1-920afadf4d1b@redhat.com>

Add three qdisc-specific drop reasons and use them in sch_cake:

 1) SKB_DROP_REASON_QDISC_OVERLIMIT
    Whenever the total queue limit for a qdisc instance is exceeded
    and a packet is dropped to make room.

 2) SKB_DROP_REASON_QDISC_CONGESTED
    Whenever a packet is dropped by the qdisc AQM algorithm because
    congestion is detected.

 3) SKB_DROP_REASON_CAKE_FLOOD
    Whenever a packet is dropped by the flood protection part of the
    CAKE AQM algorithm (BLUE).

Also use the existing SKB_DROP_REASON_QUEUE_PURGE in cake_clear_tin().

Reasons show up as:

perf record -a -e skb:kfree_skb sleep 1; perf script

          iperf3     665 [005]   848.656964: skb:kfree_skb: skbaddr=0xffff98168a333500 rx_sk=(nil) protocol=34525 location=__dev_queue_xmit+0x10f0 reason: QDISC_OVERLIMIT
         swapper       0 [001]   909.166055: skb:kfree_skb: skbaddr=0xffff98168280cee0 rx_sk=(nil) protocol=34525 location=cake_dequeue+0x5ef reason: QDISC_CONGESTED

Reviewed-by: Eric Dumazet <edumazet at google.com>
Reviewed-by: Jamal Hadi Salim <jhs at mojatatu.com>
Acked-by: Dave Taht <dave.taht at gmail.com>
Signed-off-by: Toke Høiland-Jørgensen <toke at redhat.com>
---
Changes in v2:
- Make CONGESTED and OVERLIMIT qdisc-generic instead of specific to CAKE (Jakub)
- Link to v1: https://lore.kernel.org/r/20241209-cake-drop-reason-v1-1-19205f6d1f19 at redhat.com
---
 include/net/dropreason-core.h | 18 ++++++++++++++++++
 net/sched/sch_cake.c          | 43 +++++++++++++++++++++++--------------------
 2 files changed, 41 insertions(+), 20 deletions(-)

diff --git a/include/net/dropreason-core.h b/include/net/dropreason-core.h
index c29282fabae6cdf9dd79f698b92b4b8f57156b1e..ead4170a1d0a9d3198876fabf453130bf11953be 100644
--- a/include/net/dropreason-core.h
+++ b/include/net/dropreason-core.h
@@ -58,6 +58,9 @@
 	FN(TC_EGRESS)			\
 	FN(SECURITY_HOOK)		\
 	FN(QDISC_DROP)			\
+	FN(QDISC_OVERLIMIT)		\
+	FN(QDISC_CONGESTED)		\
+	FN(CAKE_FLOOD)			\
 	FN(FQ_BAND_LIMIT)		\
 	FN(FQ_HORIZON_LIMIT)		\
 	FN(FQ_FLOW_LIMIT)		\
@@ -314,6 +317,21 @@ enum skb_drop_reason {
 	 * failed to enqueue to current qdisc)
 	 */
 	SKB_DROP_REASON_QDISC_DROP,
+	/**
+	 * @SKB_DROP_REASON_QDISC_OVERLIMIT: dropped by qdisc when a qdisc
+	 * instance exceeds its total buffer size limit.
+	 */
+	SKB_DROP_REASON_QDISC_OVERLIMIT,
+	/**
+	 * @SKB_DROP_REASON_QDISC_CONGESTED: dropped by a qdisc AQM algorithm
+	 * due to congestion.
+	 */
+	SKB_DROP_REASON_QDISC_CONGESTED,
+	/**
+	 * @SKB_DROP_REASON_CAKE_FLOOD: dropped by the flood protection part of
+	 * CAKE qdisc AQM algorithm (BLUE).
+	 */
+	SKB_DROP_REASON_CAKE_FLOOD,
 	/**
 	 * @SKB_DROP_REASON_FQ_BAND_LIMIT: dropped by fq qdisc when per band
 	 * limit is reached.
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index 8d8b2db4653c0c9f271f9c1953e8c61175d8f76b..deb0925f536dda69469738e45e1dbf8ed59a6820 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -484,13 +484,14 @@ static bool cobalt_queue_empty(struct cobalt_vars *vars,
 /* Call this with a freshly dequeued packet for possible congestion marking.
  * Returns true as an instruction to drop the packet, false for delivery.
  */
-static bool cobalt_should_drop(struct cobalt_vars *vars,
-			       struct cobalt_params *p,
-			       ktime_t now,
-			       struct sk_buff *skb,
-			       u32 bulk_flows)
+static enum skb_drop_reason cobalt_should_drop(struct cobalt_vars *vars,
+					       struct cobalt_params *p,
+					       ktime_t now,
+					       struct sk_buff *skb,
+					       u32 bulk_flows)
 {
-	bool next_due, over_target, drop = false;
+	enum skb_drop_reason reason = SKB_NOT_DROPPED_YET;
+	bool next_due, over_target;
 	ktime_t schedule;
 	u64 sojourn;
 
@@ -533,7 +534,8 @@ static bool cobalt_should_drop(struct cobalt_vars *vars,
 
 	if (next_due && vars->dropping) {
 		/* Use ECN mark if possible, otherwise drop */
-		drop = !(vars->ecn_marked = INET_ECN_set_ce(skb));
+		if (!(vars->ecn_marked = INET_ECN_set_ce(skb)))
+			reason = SKB_DROP_REASON_QDISC_CONGESTED;
 
 		vars->count++;
 		if (!vars->count)
@@ -556,16 +558,17 @@ static bool cobalt_should_drop(struct cobalt_vars *vars,
 	}
 
 	/* Simple BLUE implementation.  Lack of ECN is deliberate. */
-	if (vars->p_drop)
-		drop |= (get_random_u32() < vars->p_drop);
+	if (vars->p_drop && reason == SKB_NOT_DROPPED_YET &&
+	    get_random_u32() < vars->p_drop)
+		reason = SKB_DROP_REASON_CAKE_FLOOD;
 
 	/* Overload the drop_next field as an activity timeout */
 	if (!vars->count)
 		vars->drop_next = ktime_add_ns(now, p->interval);
-	else if (ktime_to_ns(schedule) > 0 && !drop)
+	else if (ktime_to_ns(schedule) > 0 && reason == SKB_NOT_DROPPED_YET)
 		vars->drop_next = now;
 
-	return drop;
+	return reason;
 }
 
 static bool cake_update_flowkeys(struct flow_keys *keys,
@@ -1528,12 +1531,11 @@ static unsigned int cake_drop(struct Qdisc *sch, struct sk_buff **to_free)
 
 	flow->dropped++;
 	b->tin_dropped++;
-	sch->qstats.drops++;
 
 	if (q->rate_flags & CAKE_FLAG_INGRESS)
 		cake_advance_shaper(q, b, skb, now, true);
 
-	__qdisc_drop(skb, to_free);
+	qdisc_drop_reason(skb, sch, to_free, SKB_DROP_REASON_QDISC_OVERLIMIT);
 	sch->q.qlen--;
 	qdisc_tree_reduce_backlog(sch, 1, len);
 
@@ -1926,7 +1928,7 @@ static void cake_clear_tin(struct Qdisc *sch, u16 tin)
 	q->cur_tin = tin;
 	for (q->cur_flow = 0; q->cur_flow < CAKE_QUEUES; q->cur_flow++)
 		while (!!(skb = cake_dequeue_one(sch)))
-			kfree_skb(skb);
+			kfree_skb_reason(skb, SKB_DROP_REASON_QUEUE_PURGE);
 }
 
 static struct sk_buff *cake_dequeue(struct Qdisc *sch)
@@ -1934,6 +1936,7 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
 	struct cake_sched_data *q = qdisc_priv(sch);
 	struct cake_tin_data *b = &q->tins[q->cur_tin];
 	struct cake_host *srchost, *dsthost;
+	enum skb_drop_reason reason;
 	ktime_t now = ktime_get();
 	struct cake_flow *flow;
 	struct list_head *head;
@@ -2143,12 +2146,12 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
 			goto begin;
 		}
 
+		reason = cobalt_should_drop(&flow->cvars, &b->cparams, now, skb,
+					    (b->bulk_flow_count *
+					     !!(q->rate_flags &
+						CAKE_FLAG_INGRESS)));
 		/* Last packet in queue may be marked, shouldn't be dropped */
-		if (!cobalt_should_drop(&flow->cvars, &b->cparams, now, skb,
-					(b->bulk_flow_count *
-					 !!(q->rate_flags &
-					    CAKE_FLAG_INGRESS))) ||
-		    !flow->head)
+		if (reason == SKB_NOT_DROPPED_YET || !flow->head)
 			break;
 
 		/* drop this packet, get another one */
@@ -2162,7 +2165,7 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
 		b->tin_dropped++;
 		qdisc_tree_reduce_backlog(sch, 1, qdisc_pkt_len(skb));
 		qdisc_qstats_drop(sch);
-		kfree_skb(skb);
+		kfree_skb_reason(skb, reason);
 		if (q->rate_flags & CAKE_FLAG_INGRESS)
 			goto retry;
 	}

---
base-commit: 7ea2745766d776866cfbc981b21ed3cfdf50124e
change-id: 20241205-cake-drop-reason-b1661e1e7f0a


From dave.taht at gmail.com  Fri Dec 20 18:15:59 2024
From: dave.taht at gmail.com (Dave Taht)
Date: Fri, 20 Dec 2024 15:15:59 -0800
Subject: [Cake] rl configuration for sim
Message-ID: <CAA93jw7M4TzPDOd1vfnUKTpxCXb4CE5_HO_gDdL=NcsGABxZYQ@mail.gmail.com>

https://ieeexplore.ieee.org/abstract/document/10754744

-- 
Dave Täht CSO, LibreQos