From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AF6313B29D for ; Sat, 5 Nov 2022 12:29:18 -0400 (EDT) Received: by mail-ej1-x62c.google.com with SMTP id bj12so20324906ejb.13 for ; Sat, 05 Nov 2022 09:29:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jackrabbitwireless.com; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=QKv2EAGn1IwT0C2pRS1QHj1s+ZWH1NHoY1XERwa8XTQ=; b=Rx43dGe+FeE456jUflFi2Xi+gVTxUzeM6X5OvFuvL//bWZiMjqAUt8VPPjV7yhQYQC YLaa0xPEKqVemMq7vxkQOZ8b9O+iz5tYcTIvJY8331mPS25OrPNEVgoiOSHnNOo2S+7Z 0qP0POGVNgwM5E+3753IM6cEcemzERYWacnjY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QKv2EAGn1IwT0C2pRS1QHj1s+ZWH1NHoY1XERwa8XTQ=; b=krS8iid5HlhqlvmUZ5g62/2Jrwl/EkZFRYaWaUHgD8qAswVP0lDnlD8jMrL8imu2xK DF612qjpYv05fyO2c20J4452ixfCF7nnZ1KJC/fhrNtIobigCgVCGiamk1tc1mTF+yCZ sMVNty9AXEC8MxFbZO291B8JblXD579t3PZStErDsljS/QXgpb8KD+WI1rV1Mw2whQok DIpbbYkfkVinr5Di6zdSS/psDdAvUqh02fn22b250nZ2S5A6NORBrjrVXLkIA1l+YCiR 5DzbeMeZPPpfHOMuDiteneRY4hi6ocrV+l30ayl2gVoPdxyDWXfZDm1YVHdrsmTNWhZE c8EA== X-Gm-Message-State: ANoB5pnawRNe2lVa9olQYcF465FZ1lH4laufknH7BGNx1yIvp9pnY16z KXtbwf3aM11lD4qMX/gqlMbiKMeu6/fkYykAiVIm0C9fwhn2UA== X-Google-Smtp-Source: AA0mqf7Bj7CTXSodZoaBDAnchZ7c6iRJ/W+9/3mGAu+z7IFZZzx6N4un3ZTUeq+OSVV268mtaX2TliKE3Ldze7hX9Ow= X-Received: by 2002:a17:907:c086:b0:7ae:566e:3eba with SMTP id st6-20020a170907c08600b007ae566e3ebamr1850079ejc.470.1667665757539; Sat, 05 Nov 2022 09:29:17 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?Q?Robert_Chac=C3=B3n?= Date: Sat, 5 Nov 2022 10:29:06 -0600 Message-ID: To: Herbert Wolverson Cc: libreqos Content-Type: multipart/alternative; boundary="00000000000015259305ecbbb167" Subject: Re: [LibreQoS] Tracking unknown IPs (maybe for 1.4?) X-BeenThere: libreqos@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Many ISPs need the kinds of quality shaping cake can do List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Nov 2022 16:29:18 -0000 --00000000000015259305ecbbb167 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable In our particular case we use RADIUS tied to UISP so we don't have the immediate need, but I think it's an important feature to add. Perhaps cpumap-pping can have a feature to define "shaped subnets" during the filter setup, and then we could query cpumap-pping for a JSON output of IPs detected in traffic that are in the "shaped subnets" groups, but not defined in the hash map. Curious to hear what others think here. Would others need this in order to adopt LibreQoS? On Sat, Nov 5, 2022 at 7:33 AM Herbert Wolverson via LibreQoS < libreqos@lists.bufferbloat.net> wrote: > As we approach the v1.3 pre-release feature freeze, I've been thinking a > little bit about nice things to have. One thing I found useful in both > BracketQoS and Preseem was the ability to grab a list of IP addresses tha= t > had been through the shaper, but weren't mapped to a queue (obviously, on= ly > from within the "allowed IP" range - we're not trying to map the Internet= !). > > In Preseem, there's a link to download a CSV file containing all the > unmapped IP addresses and how much traffic they have consumed. BracketQoS > (pre cpumap-pping) has a report showing the IPs (no traffic). > > *Why is this useful?* > > Knowing which local IP addresses were processed but not mapped lets you > find: > > * the times that a device was installed, but the on-boarding process > wasn't completed. Yes, that shouldn't happen. And - unfortunately - it > occasionally does. If you're using RADIUS-based authentication, it's real= ly > difficult for this to happen - but not everyone is. > * If there's a bug in your shaper integration, it's helpful to see "oops, > I put X on the default" > * Just occasionally, you get a customer who needs a special setup; it's > helpful to see that it worked. > > *Current Status* > > Before cpumap-pping, Bracket was grabbing them by reading the pping outpu= t > and listing addresses that didn't match a shaping rule. That doesn't work > now: > > * xdp_pping is spitting out TC handles, rather than IP addresses. > * With a default rule in place, and handling for IPv6 and IPv4 subnets, a= n > IP address might not exactly match an entry (requires an LPM trie lookup)= - > and IPs matching a default rule (::/0 or 0.0.0.0/0) will always come back > with the "default" handle. > > It's currently pretty tricky to do. > > So I'm curious; would others like to see this? I have a few ideas for how > to make it work, but don't want to start serious planning/design if I'm t= he > only one who wants the feature. > _______________________________________________ > LibreQoS mailing list > LibreQoS@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/libreqos > --=20 Robert Chac=C3=B3n CEO | JackRabbit Wireless LLC Dev | LibreQoS.io --00000000000015259305ecbbb167 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
In our particular case we use RADIUS tied = to UISP so we don't have the immediate need, but I think it's an im= portant feature to add.

Perhaps cpumap-pping can h= ave a feature to define "shaped subnets" during the filter setup,= and then we could query cpumap-pping for a JSON output of IPs detected in = traffic that are in the "shaped subnets" groups, but not defined = in the hash map.

Curious to hear what others think= here. Would others need this in order to adopt LibreQoS?

On Sat, Nov 5, 2022 at 7:33 AM Herbert Wolverson via LibreQoS <<= a href=3D"mailto:libreqos@lists.bufferbloat.net" target=3D"_blank">libreqos= @lists.bufferbloat.net> wrote:
As we approach the v1.3 pre-rel= ease feature freeze, I've been thinking a little bit about nice things = to have. One thing I found useful in both BracketQoS and Preseem was the ab= ility to grab a list of IP addresses that had been through the shaper, but = weren't mapped to a queue (obviously, only from within the "allowe= d IP" range - we're not trying to map the Internet!).
In Preseem, there's a link to download a CSV file containi= ng all the unmapped IP addresses and how much traffic they have consumed. B= racketQoS (pre cpumap-pping) has a report showing the IPs (no traffic).
=

*Why is this useful?*

Kn= owing which local IP addresses were processed but not mapped lets you find:=

* the times that a device was installed, but the = on-boarding process wasn't completed. Yes, that shouldn't happen. A= nd - unfortunately - it occasionally does. If you're using RADIUS-based= authentication, it's really difficult for this to happen - but not eve= ryone is.
* If there's a bug in your shaper integration, it&#= 39;s helpful to see "oops, I put X on the default"
* Ju= st occasionally, you get a customer who needs a special setup; it's hel= pful to see that it worked.

*Current Status*

Before cpumap-pping, Bracket was grabbing them = by reading the pping output and listing addresses that didn't match a s= haping rule. That doesn't work now:

* xdp_ppin= g is spitting out TC handles, rather than IP addresses.
* With a = default rule in place, and handling for IPv6 and IPv4 subnets, an IP addres= s might not exactly match an entry (requires an LPM trie lookup) - and IPs = matching a default rule (::/0 or 0.0.0.0/0) will always come back with the "default" hand= le.

It's currently pretty tricky to do.
<= div>
So I'm curious; would others like to see this? I hav= e a few ideas for how to make it work, but don't want to start serious = planning/design if I'm the only one who wants the feature.
_______________________________________________
LibreQoS mailing list
LibreQo= S@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/libreqos


--
Robert Chac=C3=B3n

<= /div>
--00000000000015259305ecbbb167--