From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D81E13B2A4 for ; Sat, 8 Jun 2024 22:53:52 -0400 (EDT) Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-42172ab4b60so12903505e9.0 for ; Sat, 08 Jun 2024 19:53:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717901631; x=1718506431; darn=lists.bufferbloat.net; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=XxbaLij5X2S8/3GmAIT8UFSmQTgLlNMS0lhDpaKs29Q=; b=nSbAVlEZS6CnCSVb2xRvKFNuIUZ8jEckdIwsg4YBUUbu8ejYiK9k3bWhBXjUrdMDfU QY0APPk3ujgvBH8FlrHjCtRp3lUBcQgWG8Zz9t6evLORudW+tw7jRm9vLSImE4FCeE++ 4SHZIX12bJuXXj2wxNYtxaovtzn1ecMDyujYSAhGHo3ZuLAZGB6H1nLhw54Dn2hjFquP jo8q6n5w0Jj+PITPP6XEEZiGEKvwDf+UAom315rVZ/t80kDzSuTIr/tkapST17hbD9FQ i8K0IJO65lnw9+Owe1d2RYLEZhUdZnrSydWFTwLQzBoBUBvQm7lH4o3pqNYvNC8HhlNg 4sZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717901631; x=1718506431; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XxbaLij5X2S8/3GmAIT8UFSmQTgLlNMS0lhDpaKs29Q=; b=GWq3c9IezuhJZGlExhBlLQfDIB7HlQbhtqegoFbsw/NF0raHhaufIKQF4oi0cY25zC b4Hy7Ed86E4BNktu6c0XjaJgr5CGbrATTtgoTlsjb5VLZStc6ROCzbEha37DSUjgqWqj e6bfBHEOg38RtGMMMyHWBNPnSPZ7CLLkrCrfxNEJhENUluOKehhQbnSefBdcsSSAB+nf yHe4RUZQrUY54vFvR5kVNQax6mMOqH3hCY01mXKGn9/rI1KQL82iNULPt8t7fC68OJfi a107p81nNQD9k7LvrjUqsV/UrGiretATIA4KTFcZL44awMnvKrIYPA4qo0mq8TDIABZN tv1w== X-Gm-Message-State: AOJu0YxNlKskRxgGnups4BvNm0Xlg5q7oZHnNceoxS2uiMAENaCSUREq uQjIArDzssrWKoFDq4X5h64A6HKyRUdH8YqcYw9KJdrB3UavrV7fE6mShDvH6nF+YJr+lr9hj/4 rZXthtFquWbT0q5rAKkV3C5tQAdWfag== X-Google-Smtp-Source: AGHT+IGGU3pwPEnpO5yQ37xAoj9biSHypDNChtm1zi4IB0zXrMyKrvUZiAMUI2HxNz8F/aTboBUsmODG/ErYjNSNnDw= X-Received: by 2002:a05:600c:548b:b0:421:3674:7933 with SMTP id 5b1f17b1804b1-42164a20d7emr39967285e9.27.1717901630558; Sat, 08 Jun 2024 19:53:50 -0700 (PDT) MIME-Version: 1.0 References: <081FB299-A2AF-44F5-B534-1ACEDBC17040@gmail.com> <3939566c-c4e2-4e94-969d-cd41dd8c89ce@gmail.com> In-Reply-To: <3939566c-c4e2-4e94-969d-cd41dd8c89ce@gmail.com> From: Dave Taht Date: Sat, 8 Jun 2024 19:53:39 -0700 Message-ID: To: libreqos Content-Type: multipart/alternative; boundary="00000000000072d7cd061a6c2403" Subject: [LibreQoS] Fwd: Open source Netflow analysis for monitoring AS-to-AS traffic X-BeenThere: libreqos@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Many ISPs need the kinds of quality shaping cake can do List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Jun 2024 02:53:53 -0000 --00000000000072d7cd061a6c2403 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Yes, according to nanog this is popular. ---------- Forwarded message --------- From: Marinos Dimolianis Date: Wed, Mar 27, 2024, 4:11=E2=80=AFPM Subject: Re: Open source Netflow analysis for monitoring AS-to-AS traffic To: Andrew Hoyos , Brian Knight Cc: North American Operators' Group Brian, I have used Akvorado in an environment with ~80G of traffic and I was super happy. It can be easily set via a docker-compose file and amongst its key benefits is the user-friendly UI that allows you to gain insight into your network traffic. There is also a demo instance available to find out what to expect: https://demo.akvorado.net/ My only "concern" was that it did not provide an API for consuming data externally. - Marinos On 3/27/2024 2:55 AM, Andrew Hoyos wrote: Brian, Take a peek at Akvorado - https://github.com/akvorado/akvorado We recently set up a lab instance, and seems to check the boxes below. On Mar 26, 2024, at 19:04, Brian Knight via NANOG wrote: What's presently the most commonly used open source toolset for monitoring AS-to-AS traffic? I want to see with which ASes I am exchanging the most traffic across my transits and IX links. I want to look for opportunities to peer so I can better sell expansion of peering to upper management. Our routers are mostly $VENDOR_C_XR so Netflow support is key. In the past, I've used AS-Stats for this purpose. However, it is particularly CPU and disk IO intensive. Also, it has not been actively maintained since 2017. InfluxDB wants to sell me on Telegraf + InfluxDB + Chronograf + Kapacitor, but I can't find any clear guide on what hardware I would need for that, never mind how to set up the software. It does appear to have an open source option, however. pmacct seems to be good at gathering Netflow, but doesn't seem to analyze data. I don't see any concise howto guides for setting this up for my purpose, however. I'm aware Kentik does this very well, but I have no budget at the moment, my testing window is longer than the 30 day trial, and we are not prepared to share our Netflow data with a third party. Elastiflow appears to have been open source at one time = in the past, but no longer. Since it too appears to be hosted, I have the same objections as I do with Kentik above. On-list and off-list replies are welcome. Thanks, -Brian --00000000000072d7cd061a6c2403 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Yes, according to nanog this is popular.

---------- Forwar= ded message ---------
From: Marinos Dimolianis <dimolianis.marinos@gmail.com>
= Date: Wed, Mar 27, 2024, 4:11=E2=80=AFPM
Subject: Re: Open source Netflo= w analysis for monitoring AS-to-AS traffic
To: Andrew Hoyos <hoyosa@gmail.com>, Brian Knight <ml@knight-networks.com>
Cc= : North American Operators' Group <nanog@nanog.org>


=20 =20 =20

Brian,

I have used Akvorado in an environment with ~80G of traffic and I was super happy.

It can be easily set via a docker-compose file and amongst its key benefits is the user-friendly UI that allows you to gain insight into your network traffic.

There is also a demo instance available to find out what to expect: https://demo.akvorado.net/

My only "concern" was that it did not provide an API for consuming data externally.

- Marinos

On 3/27/2024 2:55 AM, Andrew Hoyos wrote:
=20 Brian,

Take a peek at Akvorado -=C2=A0https://github.com/akv= orado/akvorado
We recently set up a lab instance, and seems to check the boxes below.

On Mar 26, 2024, at 19:04, Brian Knight via NANOG <nanog@nanog.org> wrote:

=20
Wha= t's presently the most commonly used open source toolset for monitoring AS-to-AS traffic?

I want to see with which ASes I am exchanging the most traffic across my transits and IX links. I want to look for opportunities to peer so I can better sell expansion of peering to upper management.
=C2= =A0
Our routers are mostly $VENDOR_C_XR so Netflow support is key.

In the past, I've used AS-Stats fo= r this purpose. However, it is particularly CPU and disk IO intensive. Also, it has not been actively maintained since 2017.

InfluxDB wants to sell me on Telegraf + InfluxDB + Chronograf + Kapacitor, but I can't find any clear guide on what hardware I would need for that, never mind how to set up the software. It does appear to have an open source option, however.
=C2= =A0
pma= cct seems to be good at gathering Netflow, but doesn't seem to analyze data. I don't see any concise howto guides for setting this up for my purpose, however.
=C2= =A0
I&#= 39;m aware Kentik does this very well, but I have no budget at the moment, my testing window is longer than the 30 day trial, and we are not prepared to share our Netflow data with a third party.
=C2= =A0
=C2= =A0
On-= list and off-list replies are welcome.
=C2= =A0
Tha= nks,
=C2= =A0
-Br= ian
=C2= =A0

--00000000000072d7cd061a6c2403--