From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yb1-xb2c.google.com (mail-yb1-xb2c.google.com [IPv6:2607:f8b0:4864:20::b2c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B8FE03B2A4 for ; Sat, 29 Oct 2022 20:09:26 -0400 (EDT) Received: by mail-yb1-xb2c.google.com with SMTP id z192so10098057yba.0 for ; Sat, 29 Oct 2022 17:09:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=YcE2dL1C+rY6uyuVMJp8BMH0xehk9poHJf3lgwQh9Gs=; b=ovr2rYawDIbO5+fTyh5PQ4G4oaf5T1XnemTmqcjQ/nP/jwNSPOr4EfhG05IdMV/7yx PxercG+LPtbT9bckekESb5iTnvgDP1ZMpZRKSfKDwtJ/y6Lv+3yD5UShjARZ9R83XVwr k43rg0D4ZU9dE6D8Q5LN4RTt3QgQeIXf5ogken5wLgeKol/VosHkgs/h5CSAe8PkJZGZ ZyAJZDKsq6eK6utz+00KV179acoakMBXDMazyP/9mPYVhdde9x/HuBRwIcpidVRiXURo TJEpwT9nxJ3kEIJsx8q6soFEez5awsH8NA5TgHnIdi+Aaqfqxo08xR44AZu1WWiMPs7Y LP6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YcE2dL1C+rY6uyuVMJp8BMH0xehk9poHJf3lgwQh9Gs=; b=qxd38sNbdyLVQ/8aLNgM0/3/gW+RU4pzuGzroiQ3eMu2Ug9+9QE8d2h0TtB+ZqMi/l sdMVYKUC3pf5WDuummi/J/VG6ieHJ8HyWs0Tkr1VTJwK206PaQ4+uHxqJyj8pHI4cS4A UDjf84K1GE0InPRJzPJTXXJ5tvF8L8sCPnquFEl8/hg7mePz5BCO4oFUtWKUo024gJ3V ZuiYpw/+tshBbmZM7RPKvluweU+iOrA/Xzg4rsTR605monymQYgM15y35Fmyl8JBjAmT d2fEDTba3n9JhxfcNogx+pY5N9UfWf/3Ta1G1tkjAx9AC5JLG9p3qvNwUfpik+6srP2T E3hQ== X-Gm-Message-State: ACrzQf34nJUbx+K4rfqsjmDeSZlXw0pNUAGKLcG25gc41wUkqfheVtqO 0OSa58FHzpVyNEwZ9c8LcLTHpsC0XwvPwLD5zMA= X-Google-Smtp-Source: AMsMyM4rDRsgEyz9DL+hdX0pJoD1Uc5aW2jVb57y4v6YEf4Fov3UghR6dNgb5Qy2l+n+vwI427Ej4g0E5oDT7VnHEww= X-Received: by 2002:a25:9d83:0:b0:68f:a551:ec71 with SMTP id v3-20020a259d83000000b0068fa551ec71mr5922826ybp.212.1667088566012; Sat, 29 Oct 2022 17:09:26 -0700 (PDT) MIME-Version: 1.0 References: <87sfj7vczj.wl-jch@irif.fr> <87leozuh1s.wl-jch@irif.fr> <87tu3mtgst.wl-jch@irif.fr> In-Reply-To: <87tu3mtgst.wl-jch@irif.fr> From: dan Date: Sat, 29 Oct 2022 18:09:14 -0600 Message-ID: To: Juliusz Chroboczek Cc: Herbert Wolverson , libreqos@lists.bufferbloat.net Content-Type: multipart/alternative; boundary="000000000000c950e205ec354df9" Subject: Re: [LibreQoS] routing protocols and daemons X-BeenThere: libreqos@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Many ISPs need the kinds of quality shaping cake can do List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Oct 2022 00:09:26 -0000 --000000000000c950e205ec354df9 Content-Type: text/plain; charset="UTF-8" I wish I had a fancy pony with: Routing based on diffserv. 'Hitless' adjustments to path diffserv cost. ie, if a link has a cost change, propagate that out without dumping routes. link cost with metrics matching diffserv model. ie, if diffserv 4 then a link might have: a cost for the hop a cost of it's expected latency + it's measured latency a cost for it's fixed/set throughput with some method to modify this programmatically (say radio modulation changes, admins can run scripts to adjust this) Above link cost, have a router cost. So if a router is considered to have a throughput limit of 1.5Gbps aggregate, then it might have a cost that varies based on throughput or cpu use or something like that. So we can route around saturated sites. This might also make the network be more ad-hoc, lots of sites might try to jam data through a central hub site but when it get's saturated, other paths might start looking good. type of service cost. This might be an IP map or something, but for operators to classify business vs residential vs enterprise services. and a reservation cost which may be a bit hard to describe, but something along the lines of 'if a service is being used up by a higher class of service (enterprise...) then other routers should know to try to route around that side. obviously, some method to slow adjustments and ramp up/down on various factors. Maybe some method to do some reservations with rolling updates to keep things from oscillating or routes from flapping. If a path gets hammered so latency spikes, back it off and then creep up slowly. Probably configurable on each link. An AF60LR for example has some more or less fixed modulation levels so when it starts bucking, it's going to modulate down super fast. Maybe the cost for throughput has some steps built in so it falls back to pre-defined levels quickly. Whilte AF5XHD is going to be more a noise issue, new noise might reduce modulations but a bit so the fixed numbers put in might need backed down. Then on the client side or the client's PoP it should discover it's paths for each diffserv to gateways. This could be pre-mapping 1st, 2nd, 3rd next-hops for each diffserv to each gateway for example, so failover for link downs is pretty quick. BFD capabilities, preferably built in vs separate BFD process. 'Route Oracles'. Routers that advertise that they'll store the router's cost tables for all other routers to subscribe to. Would allow routers to evaluate which other routers are even remotely in their path and then watch for cost changes. Also give operators access to routing maps to use for crazy shit like LibreQoS tree building. or show links that are underperforming vs stated fixed values etc. Finally, before I make this wish list too long, REDUNDANT packets. For instance, maybe we want to tag, for a specific customer or customer class, that anything that looks realtime should have a FEC type packet sent and the most different but usable path possible to be re-assembled on the other side. This might even be a link that is described as a 'dont use this for anything but FEC except last resort route, and still only use it for real time packets' so you could strap in an LTE or Starlink type link. The FEC packets are smaller, say 5-10% adjustable, so you don't need to entirely match the capacity, just to catch packets that get lost in a re-route due to link packet loss. ok, that's enough for now. You said I could have a pony, I asked for a clydesdale. On Sat, Oct 29, 2022 at 4:18 PM Juliusz Chroboczek wrote: > > Ultimately though, these are just shortest hop path builders and need > some > > other kit on top of them to do any sort of traffic engineering or load > > balancing. ECMP doesn't work for me, and doesn't work for a lot of wisps > > Perhaps one of you could explain what kind of traffic engineering and load > balancing you'd like to have? Please don't try to be realistic, try to > describe the pony you wish you had. > > -- Juliusz > --000000000000c950e205ec354df9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I wish I had a fancy pony with:

Routing based on di= ffserv.

'Hitless' adjustments to path diffserv cost.=C2=A0 i= e, if a link has a cost change, propagate=C2=A0that out without dumping rou= tes.

link cost with metrics matching diffserv model.=C2=A0
ie, i= f diffserv 4 then a link might have:
a cost for the hop
a cost of it&= #39;s expected latency=C2=A0+ it's measured latency
a cost for it= 9;s fixed/set throughput with some method to modify this programmatically= =C2=A0(say radio modulation changes, admins can run scripts to adjust this)=
Above link cost, have a router cost.=C2=A0 So if a router is considered= to have a throughput limit of 1.5Gbps aggregate, then it might have a cost= that varies based on throughput or cpu use or something like that.=C2=A0 S= o we can route around saturated sites.=C2=A0 This might also make the netwo= rk be more ad-hoc, lots of sites might try to jam data through a central hu= b site but when it get's=C2=A0saturated, other paths might start lookin= g good.
type of service cost.=C2=A0 This might be an IP map or something= , but for operators to classify business vs residential vs enterprise servi= ces.=C2=A0=C2=A0
and a reservation cost which may be a bit hard to descr= ibe, but something along the lines of 'if a service is being used up by= a higher class of service (enterprise...) then other routers should know t= o try to route around that side.

obviously, some method to slow adju= stments and ramp up/down on various factors.=C2=A0 Maybe some method to do = some reservations with rolling updates to keep things from oscillating or r= outes from flapping.=C2=A0 If a path gets hammered so latency spikes, back = it off and then creep up slowly.=C2=A0 =C2=A0Probably configurable on each = link.=C2=A0 An AF60LR for example has some more or less fixed modulation le= vels so when it starts bucking, it's going to modulate down super fast.= =C2=A0 Maybe the cost for throughput has some steps built in so it falls ba= ck to pre-defined levels quickly.=C2=A0 Whilte AF5XHD is going to be more a= noise issue, new noise might reduce modulations but a bit so the fixed num= bers put in might need backed down.

Then on the client side or the c= lient's PoP it should discover it's paths for each diffserv to gate= ways.=C2=A0 This could be pre-mapping 1st, 2nd, 3rd next-hops for each diff= serv to each gateway for example, so failover for link downs is pretty quic= k.

BFD capabilities, preferably built in vs separate BFD process.
'Route Oracles'.=C2=A0 Routers that advertise that they'll= store the router's cost tables for all other routers to subscribe to.= =C2=A0 Would allow routers to evaluate which other routers are even remotel= y in their path and then watch for cost changes.=C2=A0 Also give operators = access to routing maps to use for crazy shit like LibreQoS=C2=A0tree buildi= ng.=C2=A0 =C2=A0or show links that are underperforming vs stated fixed valu= es etc.

Finally, before I make this wish list too long, REDUNDANT pa= ckets.=C2=A0 For instance, maybe we want to tag, for a specific customer or= customer class, that anything that looks realtime should have a FEC type p= acket sent and the most different but usable path possible to be re-assembl= ed on the other side.=C2=A0 This might even be a link that is described as = a 'dont use this for anything but FEC except last resort route, and sti= ll only use it for real time packets' so you could strap in an LTE or S= tarlink type link.=C2=A0 The FEC packets are smaller, say 5-10% adjustable,= so you don't need to entirely match the capacity, just to catch packet= s that get lost in a re-route=C2=A0due to link packet loss.

ok, that= 's enough for now.=C2=A0 You said I could have a pony, I asked for a cl= ydesdale.=C2=A0=C2=A0

On Sat, Oct 29, 2022 at 4:18 PM Juliusz Chroboczek <= ;jch@irif.fr> wrote:
> Ultimately though, these ar= e just shortest hop path builders and need some
> other kit on top of them to do any sort of traffic engineering or load=
> balancing.=C2=A0 ECMP doesn't work for me, and doesn't work fo= r a lot of wisps

Perhaps one of you could explain what kind of traffic engineering and load<= br> balancing you'd like to have?=C2=A0 Please don't try to be realisti= c, try to
describe the pony you wish you had.

-- Juliusz
--000000000000c950e205ec354df9--