[Cerowrt-devel] perverse powersave bug with sta/ap mode

[Cerowrt-devel] perverse powersave bug with sta/ap mode

[Dave Taht]

Pain shared, reduced, joy shared increased...

for weeks now I've been puzzling over why a variety of links flapped
the way they did,
routes coming and going, failing over to weird paths, and I think I
have finally isolated one
part of the problem.

In an age where adhoc does not work particularly well, and AP/sta mode
does (as in 6mbit vs 500 in one case), I've had a tendency to nail up
links in ap/sta mode.

Well, at least one ( probably several) of the devices I have in the
new lab has *very aggressive* power save, to where babel ipv6
multicast traffic either doesn't sync up to the AP's request for
multicast (or the sta's), or it is merely completely suppressed by the
stack. (or lost due to a bug!)...

Anyway...

So long as there is unicast traffic on the local part of the link, you
don't see a problem. And there's almost always a bit of traffic on the
link. So, perversely... like when I'm looking at it... ssh'd through
to the other side, running something like "watch tc", or like, pinging
from one side of the link to the other... it works. When I go away for
a bit... it fails. Eventually.

If I run a test, after getting everything all setup and verified the
network looks correct... it works.

If I walk away and run a test that has a few minutes :grump: between
runs to let things "settle down", things actually deteriorate.

Babel misses multicast traffic and gradually increases the metric on
that interface due to the loss - causing a given route, in my case, to
eventually fall over to an adhoc wifi radio elsewhere on the network,
which reduces the probability of unicast traffic through that link
still more, until ultimately the local link, otherwise nailed up,
drops off the network completely.

to "fix" this:

iw dev wlp4s0 set power_save off

worked beautifully on the ath10k driver I'm using. The babel metric
stayed stable, the route stayed stable, life was good, throughput
increased, latency dropped...

That said, I know how hard wifi device driver writers are hammering at
trying to reduce multicast effects, and save power... and I haven't
exactly found the root cause of this problem, in this driver... (or in
the ath9k on the AP side) but I think I've seen it elsewhere also,
while chasing this -l failover issue.

multicast beacons are supposed to say "hey, chips, wake up, you need
to hear this".

-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Cerowrt-devel] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

Has anyone modeled what the multicast to multiple-unicast efficiency
threshold is?  The point where you go from it being more efficient to send
multicast traffic to individual STAs instead of sending a monstrous (in
time) multicast-rate packet?

2, 5, 10 STAs?

The per-STA-queue work should make that relatively easy, by allowing the
packet to be dumped into each STA's queue...

-Aaron

[-- Attachment #2: Type: text/html, Size: 487 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[David Lang]

[-- Attachment #1: Type: TEXT/Plain, Size: 661 bytes --]

On Tue, 26 Apr 2016, Aaron Wood wrote:

> Has anyone modeled what the multicast to multiple-unicast efficiency
> threshold is?  The point where you go from it being more efficient to send
> multicast traffic to individual STAs instead of sending a monstrous (in
> time) multicast-rate packet?

is the multicast packet actually multicast over the air? or is it a lot of 
unicast packets?

When the network is encrypted, how can they encrypt the multicast packet so that 
all nodes can hear it?

David Lang

>
> 2, 5, 10 STAs?
>
> The per-STA-queue work should make that relatively easy, by allowing the
> packet to be dumped into each STA's queue...
>
> -Aaron
>

[-- Attachment #2: Type: TEXT/PLAIN, Size: 167 bytes --]

_______________________________________________
Make-wifi-fast mailing list
Make-wifi-fast@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Cerowrt-devel] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[dpreed]

Interesting stuff.  A deeper problem with WiFi-type protocols is that the very idea of "multicast" on the PHY level (air interface) is flawed, based on a model of propagation that assumes that every station can be easily addressed simultaneously, at the same bitrate, etc. Multicast is seductive to designers who ignore the realities of propagation and channel coding issues, because they think it works one way, but the reality is quite different.

So just as years were wasted in the RTP and media streaming world on router/switch layer multicast (thought to be easy and more efficient), my personal opinion is that any wireless protocol that tries to solve problems with multicast at the PHY layer is a fragile, brittle design that will waste years of effort trying to make the horse dance on its forelegs.

THe list of issues is enormous, but the most obvious ones are a) equalization, b) inability to use MIMO, and c) PHY layer acknowledgment complexity.

The usual argument is that in some special case circumstance, using multicast is "optimal".  But how much better is that "optimal" than the non-multicast general solution, and how does that "optimization" make the normal operation worse, in common conditions?

Whenever someone says that a "cross layer optimization" or a complicated special case added into a robust design is "optimal", I check that my wallet is still in my pocket.  Because "optimal" is a magic word often used to distract one's attention from what really matters.

So "multicast" considered harmful is my view.


On Tuesday, April 26, 2016 7:27pm, "Aaron Wood" <woody77@gmail.com> said:

> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast
> Has anyone modeled what the multicast to multiple-unicast efficiency
> threshold is?  The point where you go from it being more efficient to send
> multicast traffic to individual STAs instead of sending a monstrous (in
> time) multicast-rate packet?
> 
> 2, 5, 10 STAs?
> 
> The per-STA-queue work should make that relatively easy, by allowing the
> packet to be dumped into each STA's queue...
> 
> -Aaron
> 

Re: [Cerowrt-devel] [Babel-users] perverse powersave bug with sta/ap mode

[Juliusz Chroboczek]

> Well, at least one ( probably several) of the devices I have in the
> new lab has *very aggressive* power save, to where babel ipv6
> multicast traffic either doesn't sync up to the AP's request for
> multicast (or the sta's), or it is merely completely suppressed by the
> stack. (or lost due to a bug!)...

Ok, that would explain why babeld is not falling over correctly -- since
Hellos are being lost by the power management algorithm, the link quality
of the redundant link is very low, so we only fall over when the currently
used link has timed out rather than when its link quality has fallen below
a given level.

I'm still thinking it over, but right now I'm thinking that I'm not going
to work around this kind of damange at the babeld level.

-- Juliusz

Re: [Cerowrt-devel] [Babel-users] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[Juliusz Chroboczek]

> Has anyone modeled what the multicast to multiple-unicast efficiency
> threshold is?

An interesting experiment to perform, without doubt.  (Experiment would be
more interesting than modelling.)

-- Juliusz

Re: [Cerowrt-devel] [Babel-users] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[Juliusz Chroboczek]

> Multicast is seductive to designers who ignore the realities of
> propagation and channel coding issues, because they think it works one
> way, but the reality is quite different.

Hold on.

Mulsticast is used for two distinct purposes: for broadcast-style
applications (streaming), and for discovery (ARP, ND, mDNS, and of course
the Hello beaconing of routing protocols).

For broadcast-style applications, multicast can be replaced with multiple
unicast streams, as Netflix shows, although this is best done at the
routers rather than at the senders, since this allows building a single
distribution tree without the need for application-layer proxies ("CDNs").

For discovery, multicast is unavoidable -- there's simply no way you're
going to send a unicast to a node that you haven't discovered yet.

-- Juliusz

Re: [Cerowrt-devel] [Make-wifi-fast] [Babel-users] perverse powersave bug with sta/ap mode

[Toke Høiland-Jørgensen]

Juliusz Chroboczek <jch@pps.univ-paris-diderot.fr> writes:

> For discovery, multicast is unavoidable -- there's simply no way you're
> going to send a unicast to a node that you haven't discovered yet.

Presumably the access point could transparently turn IP-level multicast
into a unicast frame to each associated station? Not sure how that would
work in an IBSS network, though... Does the driver (or mac80211 stack)
maintain a list of neighbours at the mac/phy level?

-Toke

Re: [Cerowrt-devel] [Make-wifi-fast] [Babel-users] perverse powersave bug with sta/ap mode

[dpreed]

Discovery is a special case, that is not quite multicast. Discovery is "noticing".  A node wishing to be discovered must be noticed by one (or maybe more) already existent stations in a group (groups are noticed by any member being noticed by a member of another group).

So you don't need any facility to "reach all" in one message.  It's sufficient to "reach any". From that point on, it's a higher level problem of "association management" (tracking members and their reachability).  I use these general terms in quotes to step outside the frame limited to 802.11 and its rigid culture.

So the key to discovery is *anycast* not multicast.

So for example, a station that is not yet associated could follow some predictable sequence of transmissions, using a variety of MI transmissions (multiple input, i.e. multiple antennas transmitting simultaneously) with a variety of waveforms, where that sequence was determined to have a high probability of being noticed by at least one member of the group to be joined. A station noticing such a signal could then use the signal's form itself to respond and begin to bring that station into the group of stations that can hear each other, discovering further information (like mutual propagation characteristics (multipath/MIMO coefficients, attenuation (for equalization), noise)).

By conflating discovery with multicast, one loses design options for discovery and cooperative transmission. So yes, the "normative" centralized access point discovery now practiced in 802.11 nets assumes a sort of "multicast", but that is because we have "centralized" architectures, not mesh at the phy level.

On Thursday, April 28, 2016 9:43am, "Toke Høiland-Jørgensen" <toke@toke.dk> said:

> Juliusz Chroboczek <jch@pps.univ-paris-diderot.fr> writes:
> 
>> For discovery, multicast is unavoidable -- there's simply no way you're
>> going to send a unicast to a node that you haven't discovered yet.
> 
> Presumably the access point could transparently turn IP-level multicast
> into a unicast frame to each associated station? Not sure how that would
> work in an IBSS network, though... Does the driver (or mac80211 stack)
> maintain a list of neighbours at the mac/phy level?
> 
> -Toke
> 

Re: [Cerowrt-devel] [Make-wifi-fast] [Babel-users] perverse powersave bug with sta/ap mode

[Juliusz Chroboczek]

> Discovery is a special case, that is not quite multicast. [...]  So you
> don't need any facility to "reach all" in one message.

Are we speaking of the IP Internet, or of some other network?

A number of fundamental Internet protocols, such as ARP and ND, use
multicast for discovery (I see broadcast as a special case of multicast).
So if you want to implement the TCP/IP suite, your link layer needs to
support multicast.  Some people have tried to work around that (see
RFC 2022, for example), with IMHO little success.

What you seem to be arguing is that it would be possible to design
a protocol suite that uses anycast for discovery.  While an interesting
research project, your suite would no longer be TCP/IP, good luck getting
it deployed.

(So what's the solution?  As Toke suggested, push the multicast
implementation to the link layer -- have the link layer convert multicast
to multiple unicasts in a way that's invisible to the network layer.
After all, that's what the link layer is for -- hiding the idiosyncrasies
of a given physical layer from the network layer.)

-- Juliusz

Re: [Cerowrt-devel] [Make-wifi-fast] [Babel-users] perverse powersave bug with sta/ap mode

[moeller0]

> On Apr 28, 2016, at 15:43 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Juliusz Chroboczek <jch@pps.univ-paris-diderot.fr> writes:
> 
>> For discovery, multicast is unavoidable -- there's simply no way you're
>> going to send a unicast to a node that you haven't discovered yet.
> 
> Presumably the access point could transparently turn IP-level multicast
> into a unicast frame to each associated station? Not sure how that would
> work in an IBSS network, though... Does the driver (or mac80211 stack)
> maintain a list of neighbours at the mac/phy level?

I believe the openwrt developers are thinking a long similar lines, see e.g. https://lists.openwrt.org/pipermail/openwrt-devel/2015-June/033398.html

Best Regards
	Sebastian

> 
> -Toke
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] [Make-wifi-fast] [Babel-users] perverse powersave bug with sta/ap mode

[Dave Taht]

On Thu, Apr 28, 2016 at 7:59 AM, Juliusz Chroboczek
<jch@pps.univ-paris-diderot.fr> wrote:
>> Discovery is a special case, that is not quite multicast. [...]  So you
>> don't need any facility to "reach all" in one message.
>
> Are we speaking of the IP Internet, or of some other network?

Heh.  Wifi is "some other network", at this point. Perhaps it always
was. It was originally targetted at IPX/SPX.

As wifi has evolved all sorts of packets below the conventional link
layer that are invisible to IP (management frames in general), perhaps
finding saner ways of exposing these packet types and their properties
to the conventional IP stack - and the IP stack to the properties of
the wifi frames - would be of help.

For example, I just "ate" the entire 802.11-2012 and 802.11ac
specifications, notably section 9 (Aggregation stuff mostly) and annex
G - for those of you into a digestif, they are publicly available via:
http://standards.ieee.org/about/get/802/802.11.html

In my case I was mostly looking for properties of ampdus that could be
better leveraged for congestion control. It turns out that you *can*
mark certain MPDUs as QosNOACK, which means that they will not be
block acknowledged. Nobody does this... and, while you could form some
IP packets with this property there's no way to do it on the ath10k
(except in raw mode), and no hook for it on the ath9k, and no way of
the IP layer saying to the 802.11 layer, "it's totally ok you can lose
this".

Elsewhere in the stack I am seeing retries of 10 (ath9k) and 15
(Ath10k), and in the initial fq_codel implementation on ath10k, *ZERO*
loss coming from the wifi layer on a string of traces. (I was
leveraging codel's ecn marking abilities to "see" this ) The mac802.11
portion also has sw retries as a global config, not as something that
is per-station.

I am certain, out there, there is some wifi EE dancing at how perfect
they've made the wireless layer appear and "transparent" to IP, but I
look at the aircap packet traces I just got with something akin to
horror, 10s of ms of retries on stuff, eating other people's air, that
I'd just as soon throw away, which also shows up on the xplot.org
tcptraces on a wire downstream as spikes in rtt.

(there is also the needed cts random backoff in there, also, which
makes it hard to distinguish between retries at various rates and
needed backoff. I am sick of manually tearing apart aircaps....)

Now, dpreed's position on how we do wireless wrong is a great starting
point... I wish hd'd publish his 11 layer stack document somewhere...

> A number of fundamental Internet protocols, such as ARP and ND, use
> multicast for discovery (I see broadcast as a special case of multicast).
> So if you want to implement the TCP/IP suite, your link layer needs to
> support multicast.  Some people have tried to work around that (see
> RFC 2022, for example), with IMHO little success.

Sure wish more wifi folk drank with more ietf folk, more often.
Starting 2 decades back.

>
> What you seem to be arguing is that it would be possible to design
> a protocol suite that uses anycast for discovery.  While an interesting
> research project, your suite would no longer be TCP/IP, good luck getting
> it deployed.
>
> (So what's the solution?  As Toke suggested, push the multicast
> implementation to the link layer -- have the link layer convert multicast
> to multiple unicasts in a way that's invisible to the network layer.
> After all, that's what the link layer is for -- hiding the idiosyncrasies
> of a given physical layer from the network layer.)

1) Well, I have suggested that IHU messages actually be unicast rather
than bundled with the hello. That would help somewhat in this case.
(and also fix cases where multicast works and unicast doesn't).
multicast hello would become more of a discovery protocol and you
could actually signal you can "take" a unicast hello (via a new tlv)
and establish an ongoing multicast-free association that way.

Given the currently "perfect" characteristics of the underlying
unicast wireless link layer that would tend to eliminate packet loss
as a viable metric of quality. :(

2) A protocol that needs "always listening" capability could signal
the underlying stack to "make sure" these packets hit the air, and one
that also wants "please be lossy" capabl

I leave the actual implementation of that request to the fantasies of
the authors - a new dscp codepoint or three?
/me ducks

3) There are other stats from minstrel, station association, crypto
state, etc, that could be leveraged.

4) And ya know - it might merely be a (sadly common) bug. Everybody's
supposed to wake up for the multicast beacons and get a notification
there's more data to come.


> -- Juliusz
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Cerowrt-devel] [Babel-users] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[Henning Rogge]

On Thu, Apr 28, 2016 at 5:04 PM, moeller0 <moeller0@gmx.de> wrote:
>
>> On Apr 28, 2016, at 15:43 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> Presumably the access point could transparently turn IP-level multicast
>> into a unicast frame to each associated station? Not sure how that would
>> work in an IBSS network, though... Does the driver (or mac80211 stack)
>> maintain a list of neighbours at the mac/phy level?
>
> I believe the openwrt developers are thinking a long similar lines, see e.g. https://lists.openwrt.org/pipermail/openwrt-devel/2015-June/033398.html

Why not just sending IP multicast (not 802.11 management frames) with
a higher rate (lowest best linkspeed to all known neighbors)?

Henning Rogge

Re: [Cerowrt-devel] [Make-wifi-fast] [Babel-users] perverse powersave bug with sta/ap mode

[Dave Taht]

On Thu, Apr 28, 2016 at 8:44 AM, Dave Taht <dave.taht@gmail.com> wrote:
> On Thu, Apr 28, 2016 at 7:59 AM, Juliusz Chroboczek
> <jch@pps.univ-paris-diderot.fr> wrote:
>>> Discovery is a special case, that is not quite multicast. [...]  So you
>>> don't need any facility to "reach all" in one message.
>>
>> Are we speaking of the IP Internet, or of some other network?
>
> Heh.  Wifi is "some other network", at this point. Perhaps it always
> was. It was originally targetted at IPX/SPX.
>
> As wifi has evolved all sorts of packets below the conventional link
> layer that are invisible to IP (management frames in general), perhaps
> finding saner ways of exposing these packet types and their properties
> to the conventional IP stack - and the IP stack to the properties of
> the wifi frames - would be of help.
>
> For example, I just "ate" the entire 802.11-2012 and 802.11ac
> specifications, notably section 9 (Aggregation stuff mostly) and annex
> G - for those of you into a digestif, they are publicly available via:
> http://standards.ieee.org/about/get/802/802.11.html
>
> In my case I was mostly looking for properties of ampdus that could be
> better leveraged for congestion control. It turns out that you *can*
> mark certain MPDUs as QosNOACK, which means that they will not be
> block acknowledged. Nobody does this... and, while you could form some
> IP packets with this property there's no way to do it on the ath10k
> (except in raw mode), and no hook for it on the ath9k, and no way of
> the IP layer saying to the 802.11 layer, "it's totally ok you can lose
> this".
>
> Elsewhere in the stack I am seeing retries of 10 (ath9k) and 15
> (Ath10k), and in the initial fq_codel implementation on ath10k, *ZERO*
> loss coming from the wifi layer on a string of traces. (I was
> leveraging codel's ecn marking abilities to "see" this ) The mac802.11
> portion also has sw retries as a global config, not as something that
> is per-station.
>
> I am certain, out there, there is some wifi EE dancing at how perfect
> they've made the wireless layer appear and "transparent" to IP, but I
> look at the aircap packet traces I just got with something akin to
> horror, 10s of ms of retries on stuff, eating other people's air, that
> I'd just as soon throw away, which also shows up on the xplot.org
> tcptraces on a wire downstream as spikes in rtt.
>
> (there is also the needed cts random backoff in there, also, which
> makes it hard to distinguish between retries at various rates and
> needed backoff. I am sick of manually tearing apart aircaps....)
>
> Now, dpreed's position on how we do wireless wrong is a great starting
> point... I wish hd'd publish his 11 layer stack document somewhere...
>
>> A number of fundamental Internet protocols, such as ARP and ND, use
>> multicast for discovery (I see broadcast as a special case of multicast).
>> So if you want to implement the TCP/IP suite, your link layer needs to
>> support multicast.  Some people have tried to work around that (see
>> RFC 2022, for example), with IMHO little success.
>
> Sure wish more wifi folk drank with more ietf folk, more often.
> Starting 2 decades back.
>
>>
>> What you seem to be arguing is that it would be possible to design
>> a protocol suite that uses anycast for discovery.  While an interesting
>> research project, your suite would no longer be TCP/IP, good luck getting
>> it deployed.
>>
>> (So what's the solution?  As Toke suggested, push the multicast
>> implementation to the link layer -- have the link layer convert multicast
>> to multiple unicasts in a way that's invisible to the network layer.
>> After all, that's what the link layer is for -- hiding the idiosyncrasies
>> of a given physical layer from the network layer.)
>
> 1) Well, I have suggested that IHU messages actually be unicast rather
> than bundled with the hello. That would help somewhat in this case.
> (and also fix cases where multicast works and unicast doesn't).
> multicast hello would become more of a discovery protocol and you
> could actually signal you can "take" a unicast hello (via a new tlv)
> and establish an ongoing multicast-free association that way.
>
> Given the currently "perfect" characteristics of the underlying
> unicast wireless link layer that would tend to eliminate packet loss
> as a viable metric of quality. :(
>
> 2) A protocol that needs "always listening" capability could signal
> the underlying stack to "make sure" these packets hit the air, and one
> that also wants "please be lossy" capabl
>
> I leave the actual implementation of that request to the fantasies of
> the authors - a new dscp codepoint or three?
> /me ducks
>
> 3) There are other stats from minstrel, station association, crypto
> state, etc, that could be leveraged.
>
> 4) And ya know - it might merely be a (sadly common) bug. Everybody's
> supposed to wake up for the multicast beacons and get a notification
> there's more data to come.

5) be powersave aware.

turn powersave off on aps and stations with stable power sources.

Mobile units on battery then would generally not be considered as
viable routing candidates unless they were very active, and doing
unicast to them to verify they were still alive would be a "keepalive"
meshanism...



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Cerowrt-devel] [Make-wifi-fast] [Babel-users] perverse powersave bug with sta/ap mode

[Dave Taht]

On Thu, Apr 28, 2016 at 9:05 AM, Henning Rogge <hrogge@gmail.com> wrote:
> On Thu, Apr 28, 2016 at 5:04 PM, moeller0 <moeller0@gmx.de> wrote:
>>
>>> On Apr 28, 2016, at 15:43 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>> Presumably the access point could transparently turn IP-level multicast
>>> into a unicast frame to each associated station? Not sure how that would
>>> work in an IBSS network, though... Does the driver (or mac80211 stack)
>>> maintain a list of neighbours at the mac/phy level?
>>
>> I believe the openwrt developers are thinking a long similar lines, see e.g. https://lists.openwrt.org/pipermail/openwrt-devel/2015-June/033398.html
>
> Why not just sending IP multicast (not 802.11 management frames) with
> a higher rate (lowest best linkspeed to all known neighbors)?

)I've always liked this idea as an enhancement to the existing 802.11
spec. It is flawed in that the "lowest best linkspeed" in minstrel
decays to the lowest configured link rate for stations that have not
been sampled recently. (Another thing I like about unicast IHU is that
it would keep minstrel's statistics more current). I don't deeply how
other rate controllers besides the old "samplerate" work - and only
just last week finally put the minstrel paper up to review -
(http://blog.cerowrt.org/post/minstrel/). If there are other documents
on wifi rate controllers out there, like those in certain wifi chips,
I'd love to read them....

And: I have found several devices in the field that cannot take
anything but the base multicast rate, my old nexus 7 was like that.

I note I'm not seeking a solution for ND/RA/ARP at the moment - in
fact I'm trying to work on something other than routing protocols
entirely ! - I would like it if people would stop trying to treat wifi
as an ethernet equivalent and treat it as the now dominant paradigm it
is, where ethernet is the exception rather than the rule, ethernet
fallback as a "nice to have" rather than a necessity. For years I got
along just fine on wifi + ethernet using the then common ahcp/babeld
single ip methods ( http://blog.cerowrt.org/post/failing_over_faster/
)

I am unfond of turning formerly all multicast protocols into unicast,
on wifi,  as proposed for openwrt and for that matter, in the ietf.

This might make for some background reading:
https://tools.ietf.org/html/draft-yourtchenko-colitti-nd-reduce-multicast-00

Anyway, I put a couple pictures up at
http://blog.cerowrt.org/post/poking_at_powersave/ - I have some data
showing the ap/sta metric going to hell over a few minutes not in that
post yet and I still ended up with some difficulties (I have not
turned powersave off everywhere yet, repeatably, as I need to find the
right hooks to tell Ap/sta mode in
networkmanager/systemd/debian/openwrt to turn it off. (?)) - did I say
I wasn't working on meshy protocols already? :grump:

and I have a conference today and more new gear arriving to play with.

>
> Henning Rogge
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Cerowrt-devel] [Make-wifi-fast] [Babel-users] perverse powersave bug with sta/ap mode

[Henning Rogge]

On Thu, Apr 28, 2016 at 6:52 PM, Dave Taht <dave.taht@gmail.com> wrote:
>> Why not just sending IP multicast (not 802.11 management frames) with
>> a higher rate (lowest best linkspeed to all known neighbors)?
>
> )I've always liked this idea as an enhancement to the existing 802.11
> spec. It is flawed in that the "lowest best linkspeed" in minstrel
> decays to the lowest configured link rate for stations that have not
> been sampled recently.

Which might be necessary anyways because some of the neighbors might be slow.

At least it sounds like a "non-invasive" change that would not hurt.

Henning

[Cerowrt-devel] Multicast IHUs [was: perverse powersave bug with sta/ap mode]

[Juliusz Chroboczek]

> 1) Well, I have suggested that IHU messages actually be unicast rather
> than bundled with the hello.

Yes, you have suggested that before.  I answered I would implement that if
somebody volunteered to do an experimental evaluation.  Nobody volunteered.

> That would help somewhat in this case.

That's my intuition too, but I've learned to be wary of my intuitions.
Doing wireless stuff without careful evaluation is not something I'll do
again.

> 2) A protocol that needs "always listening" capability could signal
> the underlying stack to "make sure" these packets hit the air, and one
> that also wants "please be lossy" capabl
> I leave the actual implementation of that request to the fantasies of
> the authors - a new dscp codepoint or three?
> /me ducks

No need to duck, Dave, it's very similar to what was done with UDP-Lite,
where the use of a specific value in the protocol field signals the link
layer not to discard corrupted frames.  I've never seen it in the wild,
I wonder why.

> 4) And ya know - it might merely be a (sadly common) bug. Everybody's
> supposed to wake up for the multicast beacons and get a notification
> there's more data to come.

Yes, it's obviously a bug.  Just like you, I'm not suprised -- ad-hoc mode
and power save is the kind of thing that's never tested.  I suggest you
disable power saving on all your nodes and be done with it.

-- Juliusz

[Cerowrt-devel] Layering [was: perverse powersave bug with sta/ap mode]

[Juliusz Chroboczek]

> As wifi has evolved all sorts of packets below the conventional link
> layer that are invisible to IP (management frames in general), perhaps
> finding saner ways of exposing these packet types and their properties
> to the conventional IP stack - and the IP stack to the properties of
> the wifi frames - would be of help.

I think it might be useful to think why TCP/IP has eaten all the other
protocol suites for lunch.

TCP/IP is a horribly inefficient protocol suite.  Any on of us could
design something simpler, more elegant, and more efficient.  TCP/IP
wastes precious bits in multiple headers, and wastes opportunities for
optimisation by avoiding to a great extent cross-layer optimisations.

So why did TCP/IP succeed?  Because it is layered.  The price you pay for
layering is the inefficiency, but it is well worth it -- because it is
lower-layer agnostic, TCP/IP was able to adapt to new physical layers
faster than all the other protocols.  If you're not convinced, please try
running DECNET natively over ATM (encapsulating Ethernet frames within
AAL3/4 PVCs doesn't count).

I don't think it's productive to get the network layer know to much about
the details of the physical layer -- all your work will need to be redone
in five years, when the next iteration of .11 breaks your assumptions.
The link layer is where the phy-related smarts belong.

(Now babeld has some knowledge of lower-layer characteristics, and this
causes no end of trouble.  But it's not too bad, since it's just the
implementation -- no lower-layer assumptions reside in the protocol beyond
those made by IP.)

-- Juliusz

Re: [Cerowrt-devel] Multicast IHUs [was: perverse powersave bug with sta/ap mode]

[Dave Taht]

On Thu, Apr 28, 2016 at 10:10 AM, Juliusz Chroboczek
<jch@pps.univ-paris-diderot.fr> wrote:
>> 1) Well, I have suggested that IHU messages actually be unicast rather
>> than bundled with the hello.
>
> Yes, you have suggested that before.  I answered I would implement that if
> somebody volunteered to do an experimental evaluation.  Nobody volunteered.

We do need to build the size of the community.

I am perpetually giving talks about wifi in front of various
audiences. Making the comment:

"how many of you use wifi, hands up"

Always gets a laugh (from those paying attention)

(sometimes I'm sufficiently annoyed at those not paying attention to
ask "those of you that are paying attention, hands up")

"How many of you understand how it works?"

and nearly all the hands go down.

"Why is this not a problem?", and then I launch into the talk....

I guess it would be better to collect my(our) rants, problems, and
arguments, tone them down, and get something about - "wifi, the
dominant paradigm" into more widely read publications than these
mailing lists. The recent conference on wifi in DC had some data like
"3 billion wifi devices shipped last year".


>> That would help somewhat in this case.
>
> That's my intuition too, but I've learned to be wary of my intuitions.
> Doing wireless stuff without careful evaluation is not something I'll do
> again.

Yep. Need more people on these problems. I promise to care more after
we cut latencies under load on wifi by 2 orders of magnitude on 3
chipsets.

>> 2) A protocol that needs "always listening" capability could signal
>> the underlying stack to "make sure" these packets hit the air, and one
>> that also wants "please be lossy" capabl
>> I leave the actual implementation of that request to the fantasies of
>> the authors - a new dscp codepoint or three?
>> /me ducks
>
> No need to duck, Dave, it's very similar to what was done with UDP-Lite,
> where the use of a specific value in the protocol field signals the link
> layer not to discard corrupted frames.  I've never seen it in the wild,
> I wonder why.

Hmm.. In babel's case, switching it to udp-lite would be like 1 line
of code. Not that it would help (unless the "don't multicast this"
code is explicitly filtering out normal udp only), and the flag day
would be no fun, but certainly the basic properties of udplite aren't
entirely unaligned... I have done tests of udplite (I have a patch
available for it for netperf if anyone wants it) and over ipv6, at
least, it did seem to be quite routable over multiple hops.
*link-local* udp-lite should "just work".

>
>> 4) And ya know - it might merely be a (sadly common) bug. Everybody's
>> supposed to wake up for the multicast beacons and get a notification
>> there's more data to come.
>
> Yes, it's obviously a bug.  Just like you, I'm not suprised -- ad-hoc mode
> and power save is the kind of thing that's never tested.  I suggest you
> disable power saving on all your nodes and be done with it.

That does not bode well for normal homenet users in the long run.

> -- Juliusz



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Cerowrt-devel] [Babel-users] Multicast IHUs [was: perverse powersave bug with sta/ap mode]

[Henning Rogge]

On Thu, Apr 28, 2016 at 7:46 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>> 4) And ya know - it might merely be a (sadly common) bug. Everybody's
>>> supposed to wake up for the multicast beacons and get a notification
>>> there's more data to come.
>>
>> Yes, it's obviously a bug.  Just like you, I'm not suprised -- ad-hoc mode
>> and power save is the kind of thing that's never tested.  I suggest you
>> disable power saving on all your nodes and be done with it.
>
> That does not bode well for normal homenet users in the long run.

Does someone even have a viable idea for power saving in adhoc or
802.11s (without forwarding) networks?

Henning

Re: [Cerowrt-devel] Multicast IHUs [was: perverse powersave bug with sta/ap mode]

[Dave Taht]

On Thu, Apr 28, 2016 at 10:10 AM, Juliusz Chroboczek
<jch@pps.univ-paris-diderot.fr> wrote:

>> 4) And ya know - it might merely be a (sadly common) bug. Everybody's
>> supposed to wake up for the multicast beacons and get a notification
>> there's more data to come.
>
> Yes, it's obviously a bug.  Just like you, I'm not suprised -- ad-hoc mode
> and power save is the kind of thing that's never tested.


No, this is the kind of thing that normal users of wifi use -
AP/station mode being the most common mode of operation.

adhoc - rarely functional or tested
power save - VERY tested for people that want to save major power,
which is everybody running on battery, pulling out every trick (even
dubious ones) to meet consumption goals (rather than network
connectivity goals).

I do not know to what extent or where the problem I am seeing is
actually happening, I can look at the multicast beacons harder to see
what's going on.

Wifi powersave is not "go to sleep entirely", it is "please wake up on
this schedule (250ms) so I can poke you with more unicast data if I
have any, it also requires (in the spec) that buffering the
accumulated packets be done til that beacon, and multicast packets are
supposed to be sent as CAB ("crap after beacon" in ath9k's
documentation, content after beacon, elsewhere).

The "buffering til you wake up" requirement is hell on trying to roll
a airtime fairness scheduler, or codel, in stack portions....

Certainly many devices simply disassociate when they go to sleep nowadays.

-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org

Re: [Cerowrt-devel] Multicast IHUs

[Juliusz Chroboczek]

>> No need to duck, Dave, it's very similar to what was done with UDP-Lite,

> Hmm.. In babel's case, switching it to udp-lite would be like 1 line
> of code.

I'm not suggesting that Babel should use UDP-Lite, it would be a silly
idea.  I'm just saying that your « don't buffer this » DSCP codepoint is
similar to the UDP-Lite « don't checksum this » codepoint, so I don't see
why it couldn't be made into a standard.  In fact, I wouldn't be surprised
if .11e already mandated something like that.

-- Juliusz

Re: [Cerowrt-devel] [Make-wifi-fast] [Babel-users] perverse powersave bug with sta/ap mode

[Juliusz Chroboczek]

> I believe the openwrt developers are thinking a long similar lines, see e.g. https://lists.openwrt.org/pipermail/openwrt-devel/2015-June/033398.html

If I read this message right, what Luessing is doing are some special-case
hacks to reduce the number of ND multicasts.  He's not attempting a general
solution for multicast.

Which, unfortunately, is quite typical.  (BATMAN, the routing protocol
that Luessing et al develop, has a number of special-case hacks for ARP
and ND, probably also DHCPv4.)

-- Juliusz

Re: [Cerowrt-devel] [Babel-users] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[Henning Rogge]

I must admit I have been thinking about switching off Duplicate
Address Detection for mesh interfaces automatically... it makes not
much sense on non-transitive interfaces anyways.

Henning

On Thu, Apr 28, 2016 at 8:54 PM, Juliusz Chroboczek
<jch@pps.univ-paris-diderot.fr> wrote:
>> I believe the openwrt developers are thinking a long similar lines, see e.g. https://lists.openwrt.org/pipermail/openwrt-devel/2015-June/033398.html
>
> If I read this message right, what Luessing is doing are some special-case
> hacks to reduce the number of ND multicasts.  He's not attempting a general
> solution for multicast.
>
> Which, unfortunately, is quite typical.  (BATMAN, the routing protocol
> that Luessing et al develop, has a number of special-case hacks for ARP
> and ND, probably also DHCPv4.)
>
> -- Juliusz
>
> _______________________________________________
> Babel-users mailing list
> Babel-users@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users

Re: [Cerowrt-devel] [Babel-users] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[Juliusz Chroboczek]

> I must admit I have been thinking about switching off Duplicate
> Address Detection for mesh interfaces automatically...

What problem are you trying to solve?

-- Juliusz

Re: [Cerowrt-devel] [Babel-users] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[Henning Rogge]

On Thu, Apr 28, 2016 at 9:29 PM, Juliusz Chroboczek
<jch@pps.univ-paris-diderot.fr> wrote:
>> I must admit I have been thinking about switching off Duplicate
>> Address Detection for mesh interfaces automatically...
>
> What problem are you trying to solve?

Less useless overhead on slow-speed networks (think VHF/UHF radio).

DAD works by pretending that the colliding address should be in
communication range, which is not true for mesh networks.

Henning

Re: [Cerowrt-devel] [Babel-users] [Make-wifi-fast] perverse powersave bug with sta/ap mode

[Juliusz Chroboczek]

>> What problem are you trying to solve?

> Less useless overhead on slow-speed networks (think VHF/UHF radio).

> DAD works by pretending that the colliding address should be in
> communication range, which is not true for mesh networks.

I understand that DAD is pretty useless in sparse mesh networks, but is it
worth adding a special case just to avoid three packets every time you
reboot?

If you've got links so slow that DAD is a significant overhead, perhaps
you should be looking into some generic form of header compression rather
than trying to special-case DAD.

-- Juliusz

Re: [Cerowrt-devel] perverse powersave bug with sta/ap mode

[Dave Taht]

I am going to try this again this weekend. I'd written up some of the
contents of the thread
here:http://blog.cerowrt.org/post/failing_over_faster

And I just posted the follow-on test using the "chip" - which garnered
a bit more detail as to how the routes flapped:
http://blog.cerowrt.org/post/babel_half_fail/

Any ideas for what variables to better track appreciated.