From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 921DC3B29E for ; Mon, 24 Oct 2022 19:25:42 -0400 (EDT) Received: by mail-pl1-x630.google.com with SMTP id g24so4740196plq.3 for ; Mon, 24 Oct 2022 16:25:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=LW5aA8ine50RGfeK9JmuLgP8x+4Q1YNio3Pp9x24k1Y=; b=n5sdPMI6oRWpJQHWoeTzV9x3k43nIxlEERaTmk+QwlOMbLFeRp8Jna2Jxr8OuYsBZU WrJvbAtd1DZVkt6oee/K/FVuWaSO3I3m1Zv9WpTH+OSWfiKUSyrzvBS4PHGwvMBrc9/j 6KKLcz66JbjpV0ZGRsziUL55lyQTfg9vlPN3/LD08u5k40mq9MNWEIcOux29HDC8T2oG i4VqyshMiaEFKm4V2Th7NgayBXN1ONJ6/kuNe8WMbtsSRT2rO/fw+FZFirCJJMQX6cIV MP/bysVAH9qkwfn3oO0H0iv1DF8NgnN4uwDtLQi1Z1l/llJR2YZ/ebbm9osGKojL/dZ/ DjPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=LW5aA8ine50RGfeK9JmuLgP8x+4Q1YNio3Pp9x24k1Y=; b=pNoXUORt5sSgDEE0VJL6qT0TJXn++2eQd4HwLgRS+Vfprb+MDJYJqPdjxBVDr56hqn MjKxBkG099mcw6F7glm2dn/E7okY3colQWoPmqCBDLAfiIQSHORGQTrE6ON5gfmdNAfu IHaRbP/DqnfiAsDsngpfdtV3p/XhpTpowHJ/4irsxnqLt42Eney+R7xnzOC3H27Lvnih F/zDf2m1fBT3XXird45J/yR/lzOaacZ8xoQKt5Q0yV896VzQJ3MaGMXavB9wSn0KWX8s zF9SE/ncrQST9Cm7TJv0SZn4WQtOcyf4na+ddOaYHpy1A5gfgXRYaN2nSp5XK48FgTEZ moOA== X-Gm-Message-State: ACrzQf29tlj4KoHoq9e2TS7bSrBAebJ6+W1LKp177VqsVEkNv9LfIwd4 zkP+ld4QQnjvuwJaSeBMjPzwCw+82dmLO4a4lns= X-Google-Smtp-Source: AMsMyM7Vonq1+2Upib7vZQvXIJW4TyXgDj84oLkJcGx3N6RD5P26O9fCYXaFIgxW8R3EJ84eztoNXgzupV2mMZVFyMk= X-Received: by 2002:a17:902:7792:b0:182:9404:f226 with SMTP id o18-20020a170902779200b001829404f226mr36347667pll.76.1666653941612; Mon, 24 Oct 2022 16:25:41 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Herbert Wolverson Date: Mon, 24 Oct 2022 18:25:18 -0500 Message-ID: To: Dave Taht Cc: libreqos@lists.bufferbloat.net Content-Type: multipart/alternative; boundary="00000000000027473b05ebd01c53" Subject: Re: [LibreQoS] Rain Fade (was Ack-filtering) X-BeenThere: libreqos@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Many ISPs need the kinds of quality shaping cake can do List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Oct 2022 23:25:42 -0000 --00000000000027473b05ebd01c53 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Rain fade (short version, at home now). 5.x and 2.4 barely change. 3.6, we see a tiny fade. If there is any foliage in the way, it gets a lot worse. We gave up on 24ghz, it faded if I looked too hard at it. Ubiquiti's AF60-LR seems to do better than it should, for a 60ghz - partly by running up in the 70ghz channel. We have a 2 mile link that kept carrying a gig while the flooding started. (I kinda suspect it's playing fast and loose with the rules, honestly). Other 60ghz units fade horribly. On Mon, Oct 24, 2022, 6:05 PM Dave Taht wrote: > On Mon, Oct 24, 2022 at 3:58 PM Herbert Wolverson > wrote: > > > > Hit the wrong reply button, that happens a lot when I try to type on my > phone. > > > > I'm going to test tracking many more flows early in the morning. It > should slightly increase ram usage, and have no ill effects. "Should" > doesn't always work out, hence testing! > > > > I meant "flooding" in the "oh crap, site underwater" sense. We'd need a > boat to get to one tower right now! It doesn't currently have power, whic= h > I suspect is related. > > re: flooding, yea, forgot about that kind. :) I hate to harp on it > more than once a week, but we designed fq_codel first and foremost to > cope with rain fade on the backhaul. Now that people just use it to > enforce plans :( gathering statistics as to radio behaviors when it > rains, and/or clearly identifying weather, when looking at historical > data seems apt. > > How bad are y'all's gear doing with rain fade on various techs and > bands? in 08, in nica, I'd go from a working 70 db 10 mile shot to > nothin at 5ghz when it rained, and I just laughed at the people trying > to deploy 60ghz - but times change. I see a vendor trying to ship 60 > with *really good antennas* into the office market... > > big question to ask when so busy, please ignore me. > > > > > Turned ack filter off, after some customers reported issues (others wer= e > really happy). The unhappy campers were all on Cambium devices, with good > signal and modulations. Maybe Cambium is doing some "magic"? We'll try a > unidirectional test tomorrow. > > Maybe an exceptions db for stuff that is too smart or weird. TCP > "accellerator"s are everywhere. > > > > > On Mon, Oct 24, 2022, 4:57 PM Dave Taht wrote: > >> > >> On Mon, Oct 24, 2022 at 2:19 PM Herbert Wolverson via LibreQoS > >> wrote: > >> > > >> > Highly un-scientific (we need to let it run for a bit and do a prope= r > before-after comparison that includes a decent timeframe), but I like the > quick'n'dirty results of testing "ack-filter": > >> > > >> > We've been having a Bad Network Day (TM), with sudden flooding makin= g > us use some pretty constrained > >> > >> I've been looking at various ddos mitigation schemes of late. Are you > using any? > >> > >> >- so our latencies were really suffering in one region. That region > just happens to be the worst part of our network (we haven't finished > digesting an acquisition; there's even Bullet M2 omnis up there!). Lots o= f > relatively low-speed plans, all with big variance (10/3, 25/5, I found a > 5/1 that someone forgot to upgrade!). They seem to have benefitted greatl= y. > The parts of the network that were doing great - are still doing great, > with very little change. > >> > >> I made my previous comments in looking at the swing downwards being so > >> large, possibly not being a positive direction (my ever suspicious gut > >> was reacting, but I wasn't qualifying the numbers - been a long day > >> here too) > >> > >> I also forgot to mention that ack-filtering uses up less txops on > >> older versions of wifi. Very useful. I'd meant > >> to put it into my mt76 stuff ages ago but got overwhelmed by bugs. > >> > >> > Just a quick'n'dirty test. I'll try and put something more useful > together tomorrow, when it's had a chance to see how peak time hits it. > >> > >> :crossed fingers: > >> > >> > > >> > (Also, this digging revealed an issue with pping-cpumap in > production. It wasn't tracking enough flows, so the reporting is heavily > biased towards the top-consumers - who are likely to be monitored before > the buffer fills up and it stops counting until stats are read. So I adde= d > a "maximums.h" file to make it easy to set user limits, and made flow-cou= nt > derive from that.) > >> > >> I think polling it more frequently would be closer to the typical > >> durations of flows. Most flows last for under 3 seconds. > >> > >> What would be the harm in vastly expanding the number of flows it > tracks? > >> > >> /me hides > >> > >> > _______________________________________________ > >> > LibreQoS mailing list > >> > LibreQoS@lists.bufferbloat.net > >> > https://lists.bufferbloat.net/listinfo/libreqos > >> > >> > >> > >> -- > >> This song goes out to all the folk that thought Stadia would work: > >> > https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-698136666= 5607352320-FXtz > >> Dave T=C3=A4ht CEO, TekLibre, LLC > > > > -- > This song goes out to all the folk that thought Stadia would work: > > https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-698136666= 5607352320-FXtz > Dave T=C3=A4ht CEO, TekLibre, LLC > --00000000000027473b05ebd01c53 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Rain fade (short version, at home now). 5.x and 2.4 barel= y change. 3.6, we see a tiny fade. If there is any foliage in the way, it g= ets a lot worse.

We gave up on= 24ghz, it faded if I looked too hard at it.=C2=A0
<= br>
Ubiquiti's AF60-LR seems to do better than i= t should, for a 60ghz - partly by running up in the 70ghz channel. We have = a 2 mile link that kept carrying a gig while the flooding started. (I kinda= suspect it's playing fast and loose with the rules, honestly). Other 6= 0ghz units fade horribly.

On Mon, Oct 24, 2022, 6:05 PM Dave Taht <= dave.taht@gmail.com> wrote:
On Mon, Oct 24, 2022 at 3:58 PM Herb= ert Wolverson <herberticus@gmail.com> wrote:
>
> Hit the wrong reply button, that happens a lot when I try to type on m= y phone.
>
> I'm going to test tracking many more flows early in the morning. I= t should slightly increase ram usage, and have no ill effects. "Should= " doesn't always work out, hence testing!
>
> I meant "flooding" in the "oh crap, site underwater&quo= t; sense. We'd need a boat to get to one tower right now! It doesn'= t currently have power, which I suspect is related.

re: flooding, yea, forgot about that kind. :) I hate to harp on it
more than once a week, but we designed fq_codel first and foremost to
cope with rain fade on the backhaul. Now that people just use it to
enforce plans :( gathering statistics as to radio behaviors when it
rains, and/or clearly identifying weather, when looking at historical
data seems apt.

How bad are y'all's gear doing with rain fade on various techs and<= br> bands? in 08, in nica, I'd go from a working 70 db 10 mile shot to
nothin at 5ghz when it rained, and I just laughed at the people trying
to deploy 60ghz - but times change. I see a vendor trying to ship 60
with *really good antennas* into the office market...

big question to ask when so busy, please ignore me.

>
> Turned ack filter off, after some customers reported issues (others we= re really happy). The unhappy campers were all on Cambium devices, with goo= d signal and modulations. Maybe Cambium is doing some "magic"? We= 'll try a unidirectional test tomorrow.

Maybe an exceptions db for stuff that is too smart or weird. TCP
"accellerator"s are everywhere.

>
> On Mon, Oct 24, 2022, 4:57 PM Dave Taht <dave.taht@gmail.com&g= t; wrote:
>>
>> On Mon, Oct 24, 2022 at 2:19 PM Herbert Wolverson via LibreQoS
>> <libreqos@lists.bufferbloat.net> wrote:
>> >
>> > Highly un-scientific (we need to let it run for a bit and do = a proper before-after comparison that includes a decent timeframe), but I l= ike the quick'n'dirty results of testing "ack-filter": >> >
>> > We've been having a Bad Network Day (TM), with sudden flo= oding making us use some pretty constrained
>>
>> I've been looking at various ddos mitigation schemes of late. = Are you using any?
>>
>> >- so our latencies were really suffering in one region. That r= egion just happens to be the worst part of our network (we haven't fini= shed digesting an acquisition; there's even Bullet M2 omnis up there!).= Lots of relatively low-speed plans, all with big variance (10/3, 25/5, I f= ound a 5/1 that someone forgot to upgrade!). They seem to have benefitted g= reatly. The parts of the network that were doing great - are still doing gr= eat, with very little change.
>>
>> I made my previous comments in looking at the swing downwards bein= g so
>> large, possibly not being a positive direction (my ever suspicious= gut
>> was reacting, but I wasn't qualifying the numbers - been a lon= g day
>> here too)
>>
>> I also forgot to mention that ack-filtering uses up less txops on<= br> >> older versions of wifi. Very useful. I'd meant
>> to put it into my mt76 stuff ages ago but got overwhelmed by bugs.=
>>
>> > Just a quick'n'dirty test. I'll try and put somet= hing more useful together tomorrow, when it's had a chance to see how p= eak time hits it.
>>
>> :crossed fingers:
>>
>> >
>> > (Also, this digging revealed an issue with pping-cpumap in pr= oduction. It wasn't tracking enough flows, so the reporting is heavily = biased towards the top-consumers - who are likely to be monitored before th= e buffer fills up and it stops counting until stats are read. So I added a = "maximums.h" file to make it easy to set user limits, and made fl= ow-count derive from that.)
>>
>> I think polling it more frequently would be closer to the typical<= br> >> durations of flows. Most flows last for under 3 seconds.
>>
>> What would be the harm in vastly expanding the number of flows it = tracks?
>>
>> /me hides
>>
>> > _______________________________________________
>> > LibreQoS mailing list
>> > LibreQoS@lists.bufferbloat.net
>> > https://lists.bufferbloat.ne= t/listinfo/libreqos
>>
>>
>>
>> --
>> This song goes out to all the folk that thought Stadia would work:=
>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-698= 1366665607352320-FXtz
>> Dave T=C3=A4ht CEO, TekLibre, LLC



--
This song goes out to all the folk that thought Stadia would work:
h= ttps://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-698136666560= 7352320-FXtz
Dave T=C3=A4ht CEO, TekLibre, LLC
--00000000000027473b05ebd01c53--