From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vs1-xe2a.google.com (mail-vs1-xe2a.google.com [IPv6:2607:f8b0:4864:20::e2a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 1D2DD3B2A4 for ; Wed, 26 Oct 2022 15:33:01 -0400 (EDT) Received: by mail-vs1-xe2a.google.com with SMTP id h3so15455371vsa.4 for ; Wed, 26 Oct 2022 12:33:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=zKoICLp3WnuT1FPJwaq7pCQPmqnUcqy/278VxQfqN8E=; b=VNv2L4afrc8RZvF8mhWbJAI6KfOK+7NDtIupHBhYDanEXhWg0E3bZI8EXhh99kv+eV pjmzw+ht4//WCzPLxU9Tc2tO3F4gMEfrjvcdKkg5S155adSnzO6FKHuyx0urRp+BBCt8 C0eheHd5PFzHWuWIJC5Ugqtw3YkdFeQComTq92eNMj4NJhL7FNLd4szUxlOxBC5SXSVq NeoyiQuuLsn4PbfsmmPwGEVN/M7Tm0ALkCa74SqY8xC0DKCtpXluI/vfPOfLyavsMVGY /yUf3CQvQivkooArLcD/+ibNqN6DewUgUQhd6pS0yw7xCvfU0lQUkFMSZudPTEAqwyVS zycA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zKoICLp3WnuT1FPJwaq7pCQPmqnUcqy/278VxQfqN8E=; b=xB/3EEx0+qLGktAQdP0TizkYYX/vnVHS8BTNzTj+rWqegOGeyuMwnlNuoZZRu0vnv3 GSMNsihXsBwM60RCKY5p6pOKsaL8A1TxDr/xaSvqGvsvuHOBF+gfGakZcWph38+FQcho swDTnJL2Oo6WfkDwYfBUnyRfI+aqWd4xA4cgz0vbmmLjU2sjfdfQHA75MRGdncO4b3zH VIYP8aN/6TBCJ7EEFFNSnCRlNK/8QSL0nXqothQyosfxyj8YmxnPggyfsgMCEbeG3sOT NFQX3IPpiGCeoIm/KPlBryIGp4hEmGDa9QHt84Gq7cnL/1l2c3UyRUDw7GFLRBg0OwxX ZNvA== X-Gm-Message-State: ACrzQf0XUtAs1tdSqyCbvN+2vCs7pgM4Z5txfUCpyDt7S+7gHKj9Dj4Z o82JOVGFaN4a0PiDkbzZdiPxpEvV4xDVrXAhBUGZyHjl X-Google-Smtp-Source: AMsMyM40glANrSTkM5CAij4E0pqgj6+XH3ZXCwwYyoXv5VaPtTr4fqUrBdTB37ZiyQqluB4nywcgHbKD004T8gaPr5A= X-Received: by 2002:a05:6a00:88f:b0:558:8186:3ec3 with SMTP id q15-20020a056a00088f00b0055881863ec3mr44145836pfj.83.1666812769403; Wed, 26 Oct 2022 12:32:49 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Herbert Wolverson Date: Wed, 26 Oct 2022 14:32:39 -0500 Message-ID: Cc: libreqos@lists.bufferbloat.net Content-Type: multipart/alternative; boundary="0000000000000708cc05ebf5172a" Subject: Re: [LibreQoS] how are you doing on ipv4 address supply? X-BeenThere: libreqos@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Many ISPs need the kinds of quality shaping cake can do List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Oct 2022 19:33:01 -0000 --0000000000000708cc05ebf5172a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The main reason I was asking about MPLS is that it looks pretty easy to extract the data, and I've seen a bunch of people on the WISP Talk FB group complaining about having to de-encapsulate before they pass things through Preseem. Where I could see that being most useful is if you have a shaper at a tower (rather than just the one), and want to shape your backhaul directly. I'll keep it on the "things to play with" list. :-) We ran MPLS/VPLS for a while; we were pretty surprised when our network sped up a lot when it was removed! (Routers got really good at routing, and it's often faster than parsing a label these days). It also made it easier for the other techs to diagnose faults, MPLS never seemed to "click" for them. On Wed, Oct 26, 2022 at 2:15 PM Robert Chac=C3=B3n < robert.chacon@jackrabbitwireless.com> wrote: > We use MPLS (VPLS tunnels back to core) with L2 isolation and DHCP > snooping everywhere. > It decapsulates at our core so we don't need MPLS parsing for LibreQoS in > our case. > It works well and minimizes the number of RFC6598 CG-NAT clients behind > public addresses. > I'd prefer simple routing, but with IPv4 exhaustion - that would lead to = a > lot of unused RFC6598 addresses (and hence high CG-NAT ratios). > Many clients on one CG-NAT address leads to a lot of problems with > services like Amazon Prime. > I'm hopeful we can switch to VXLAN as Dan suggested. It could be a great > alternative to VPLS/MPLS. > > On Wed, Oct 26, 2022 at 10:49 AM dan via LibreQoS < > libreqos@lists.bufferbloat.net> wrote: > >> MPLS is great for those that have dedicated engineering staff or hire a >> consultant to help out. It's a real PITA when the business is primarily >> techs and they have no idea how to solve an issue. I've been pushing f= or >> vxlan/evpn or srv6 for a while as a way to simplify the model. 'wisp' >> focused gear just isn't there yet though. high hopes for mikrotik's >> support for vxlan to get hardware acceleration. >> >> On Wed, Oct 26, 2022 at 10:05 AM Herbert Wolverson via LibreQoS < >> libreqos@lists.bufferbloat.net> wrote: >> >>> Actually had a conversation with our primary upstream provider yesterda= y >>> about IPv6 (and why >>> they won't provide it to us). One of their lead engineers, once plied >>> with beer, said that every >>> time they've deployed it they get 99% of it working well and 1% of >>> things mysteriously >>> stop working, or go wonky. That was similar to our experience a few >>> years ago. It mostly >>> helped a lot, but chasing down the "hey, this advertises a v6 address >>> and doesn't actually >>> support it" issues drove us crazy. >>> >>> Right now, we don't have enough IPv4 addresses, but that's being >>> rectified. We mostly >>> do CGNAT and 10.64 addresses in the meantime, with public IPs assigned >>> where they >>> are needed (mostly through a tunnel setup to avoid subnetting waste). >>> Tunnels are a pain, >>> but they work (once you chase down all of the MTU issues). >>> >>> Which reminds me, I have "can we support MPLS?" on my crazy notes list. >>> I know that >>> Preseem and similar don't try, but we're already reading deeply enough >>> into the ethernet >>> header that saying "this is an MPLS label, advance 4 bytes", "this is a >>> VPLS label, advance >>> X (I forget) bytes" looks do-able. [Note, I personally don't enjoy MPLS= . >>> It's handy when >>> you want to pretend to have a flat network on top of a large routed >>> network - and some >>> WISP consultants absolutely swear by it - but my experience is that you >>> are adding >>> complexity for the sake of it. Routing works remarkably well.] >>> >>> On Tue, Oct 25, 2022 at 10:30 PM Dave Taht via LibreQoS < >>> libreqos@lists.bufferbloat.net> wrote: >>> >>>> in my continued rip-van-winkle, living in the third world (california) >>>> way, I am curious as to how y'all are managing your >>>> ipv4 address supply and if you are deploying ipv6 to any extent? >>>> >>>> In all this discussion of multi-gbit fiber, my own direct experience >>>> is that AT&T's fiber rollout had very flaky ipv6, and more and more >>>> services (like starlink) are appearing behind cgnats, which have their >>>> own capex and opex costs. >>>> >>>> I see a lot of rfc1918 being used as the operational overlay >>>> elsewhere, tons of tunnels, also. >>>> >>>> -- >>>> This song goes out to all the folk that thought Stadia would work: >>>> >>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-698136= 6665607352320-FXtz >>>> Dave T=C3=A4ht CEO, TekLibre, LLC >>>> _______________________________________________ >>>> LibreQoS mailing list >>>> LibreQoS@lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/libreqos >>>> >>> _______________________________________________ >>> LibreQoS mailing list >>> LibreQoS@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/libreqos >>> >> _______________________________________________ >> LibreQoS mailing list >> LibreQoS@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/libreqos >> > > > -- > Robert Chac=C3=B3n > CEO | JackRabbit Wireless LLC > --0000000000000708cc05ebf5172a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The main reason I was asking about MPLS is that it lo= oks pretty easy to extract the data, and I've seen a bunch of people on= the WISP Talk FB group complaining about having to de-encapsulate before t= hey pass things through Preseem. Where I could see that being most useful i= s if you have a shaper at a tower (rather than just the one), and want to s= hape your backhaul directly. I'll keep it on the "things to play w= ith" list. :-)

We ran MPLS/VPLS for a while; = we were pretty surprised when our network sped up a lot when it was removed= ! (Routers got really good at routing, and it's often faster than parsi= ng a label these days). It also made it easier for the other techs to diagn= ose faults, MPLS never seemed to "click" for them.
On Wed, O= ct 26, 2022 at 2:15 PM Robert Chac=C3=B3n <robert.chacon@jackrabbitwireless.com> wro= te:
We use MPLS (VPLS tunnels back to core) with L2 isolation and DHC= P snooping everywhere.
It decapsulates at our core so we don'= t need MPLS parsing for LibreQoS in our case.
It works well a= nd minimizes the number of RFC6598 CG-NAT clients behind public addresses.<= /div>
I'd prefer simple routing, but with IPv4 exhaustion - that wo= uld lead to a lot of unused RFC6598 addresses (and hence high CG-NAT ratios= ).
Many clients on one CG-NAT address leads to a lot of problems = with services like Amazon Prime.
I'm hopeful we can switch to= =C2=A0VXLAN as Dan suggested. It could be a great alternative to VPLS/MPLS.=

On Wed, Oct 26, 2022 at 10:49 AM dan via LibreQoS <libreqos@lists.buf= ferbloat.net> wrote:
MPLS is great for those that have dedicated en= gineering staff or hire a consultant to help out.=C2=A0 It's a real PIT= A when the business is primarily techs and they have no idea how to solve a= n issue.=C2=A0 =C2=A0I've been pushing for vxlan/evpn or srv6 for a whi= le as a way to simplify the model.=C2=A0 'wisp' focused gear just i= sn't there yet though.=C2=A0 high hopes for mikrotik's support for = vxlan to get hardware acceleration.=C2=A0 =C2=A0

On Wed, Oct 26, 2022 at 10:= 05 AM Herbert Wolverson via LibreQoS <libreqos@lists.bufferbloat.net> wr= ote:
Actually had a conversation with our primary upstream provider y= esterday about IPv6 (and why
they won't provide it to us). On= e of their lead engineers, once plied with beer, said that every
= time they've deployed it they get 99% of it working well and 1% of thin= gs mysteriously
stop working, or go wonky. That was similar to ou= r experience a few years ago. It mostly
helped a lot, but chasing= down the "hey, this advertises a v6 address and doesn't actually<= /div>
support it" issues drove us crazy.

= Right now, we don't have enough IPv4 addresses, but that's being re= ctified. We mostly
do CGNAT and 10.64 addresses in the meantime, = with public IPs assigned where they
are needed (mostly through a = tunnel setup to avoid subnetting waste). Tunnels are a pain,
but = they work (once you chase down all of the MTU issues).

=
Which reminds me, I have "can we support MPLS?" on my crazy = notes list. I know that
Preseem and similar don't try, but we= 're already reading deeply enough into the ethernet
header th= at saying "this is an MPLS label, advance 4 bytes", "this is= a VPLS label, advance
X (I forget) bytes" looks do-able. [N= ote, I personally don't enjoy MPLS. It's handy when
you w= ant to pretend to have a flat network on top of a large routed network - an= d some
WISP consultants absolutely swear by it - but my experienc= e is that you are adding
complexity for the sake of it. Routing w= orks remarkably well.]

On Tue, Oct 25, 2022 at 10:30 PM Dave Taht = via LibreQoS <libreqos@lists.bufferbloat.net> wrote:
in my continued rip-van-winkle, l= iving in the third world (california)
way, I am curious as to how y'all are managing your
ipv4 address supply and if you are deploying ipv6 to any extent?

In all this discussion of multi-gbit fiber, my own direct experience
is that AT&T's fiber rollout had very flaky ipv6, and more and more=
services (like starlink) are appearing behind cgnats, which have their
own capex and opex costs.

I see a lot of rfc1918 being used as the operational overlay
elsewhere, tons of tunnels, also.

--
This song goes out to all the folk that thought Stadia would work:
https://www.= linkedin.com/posts/dtaht_the-mushroom-song-activity-6981366665607352320-FXt= z
Dave T=C3=A4ht CEO, TekLibre, LLC
_______________________________________________
LibreQoS mailing list
LibreQo= S@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/libreqos
_______________________________________________
LibreQoS mailing list
LibreQo= S@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/libreqos
_______________________________________________
LibreQoS mailing list
LibreQo= S@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/libreqos


--
Robert Chac=C3=B3n
CEO | JackRabbit Wireless LLC
--0000000000000708cc05ebf5172a--