[Cerowrt-devel] Fwd: [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Cerowrt-devel] Fwd: [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Dave Taht]

---------- Forwarded message ----------
From: Cullen Jennings <fluffy@iii.ca>
Date: Thu, Jul 16, 2015 at 12:15 AM
Subject: [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e
To: Tim Szigeti <szigeti@cisco.com>, tsvwg@ietf.org



I'm glad to see this draft. This does need to be standardized - just
having unpredictable behavior of equipment in the field is very bad. I
do think this should be standards track and jointly developed by IEEE
and IETF.

I think the metric for how we choose needs to be based on finding a
sane set of defaults that is going to work for the majority of
situations and allow many reconfiguration cases for use cases that
don't work. There is no mapping that will will work for 100% of use
cases.

I'd restructure the draft to simply have figure 4 be an IANA tables
that showed the mappings and then have text to explain why the mapping
is made the way it is. For me the focus of this draft is the default
configuration of devices, I'm assuming that devices can be configured
to override the defaults in this specification.

On  the applicability, it is very hard for an manufacture to know if
an access point is going to be on the edge of the middle of the
network when they develop the code for it. Even when it is deployed,
it is very hard to to know if it is near the edge or not. I would
remove the limitations of applicability and make the applicability be
the internet.

I'd move all the material about how 802.11e works to an appendix.

I don't think the network norm has been to move from DSCP to marking
with smaller number of bits by taking the high order bits (and visa
versa in other direction). I agree some equipment has done this. But
this does not really matter this draft, I would remove this type of
stuff that is hard to a get agreement on and just focus on the
mappings. Similarly I question that multimedia class was intended to
necessarily be video.

The vast majority of the devices that need to implement this will be
deployed in situation where they run the default configuration that
came from the manufacture so ideas like where the edge of the diff
serv domain are can't be used very easily.

On the topic of the actual mappings....

Based on my view of the changed applicability, I think things like CS6
and 7 need to map to UP 6 or 7 not 0

Agree that EF -> 6

We should consider CS5 mapping to same things as EF (6) but should map
to at least 5 so that it is not less than the video flows. Note that
the voice and signaling (EF and CS5) both have the same top 3 bits for
DSCP value and that was by design. Slight rat whole on this topic ...
Cisco's first QoS demo of video over sprints network set the QoS for
video to be above signaling and you once the video started you could
never stop the video or even hang up on the call. Due to a bunch of
bugs on that prototype, you basically you had to pull the battery on
the phone to get your phone so you could control it again.

The practical use of AF4 vs CS4 for video phone calls has always been
confusing.  Over the past 4 years we have spent a huge amount of time
getting the direction to be AF4. If this spec put CS4 above AF4, that
would be a cause multiple manufactures to re-examin all of that and
likely move to CS4 completely reseting the work we have done on this.
The one thing I feel really strongly about is CS4 can't map higher
than AF4.

In the actual use of interactive video, some bits are far more
important than others and we want to be able to indicate this inside
the AF4 class so some should map to 4 and some should map to 5. I
would suggest  something like
AF41 and AF42 map to 5 and AF43 and AF44 map to 4.

Map CS4 to 4

CS3 is particularly tricking. The thing that is tricky about it is it
got labeled Broadcast video.  On unmanaged networks running default
configurations, it is very unlikely to be broadcast video and is much
more likely to be surveillance video. This is probably best mapped to
3 with configuration that allows it to be mapped to 4 instead. Of
course there are networks where the surveillance traffic is more
important than everything else (same as broadcast video) but if the
consumer WIFI camera puts in wireless traffic at UP 5, users will not
be happy. As a result, I would make the default mapping for CS3 and
AF3 be 3. I know it would be appealing to look at what CS3 was meant
to be used for but I think it is more important to think about how it
did get used. The case I am thinking of is a wired surveillance camera
plugged into a 802.11  bridges that does wireless to another AP than
connects to the rest of the internet.

AF2 to 3 or 0. Sure be nice if it went somewhere else but when mapping
64 things to 8, somethings not going to be pretty.  I don't work with
this enough to know when to use 3 and when 0.

My experience is mostly around the audio and video qos so don't really
have any comments on CS2 but mapping to 0 seems OK.

Having AF1 maps to something less that DF seems like it will cause
lots of unhappiness and surprises. I'd map it  3 or 0.

CS1 to 1 makes sense.

As Ken mentioned, adding the the VOICE-ADMIT admit to 6.

On dropping things not in use. That's fine as advice to people
deploying things but probably not much use to equipment vendors.  It
would be nice if all the deployment advice got separated to one
section.





-- 
Dave Täht
worldwide bufferbloat report:
http://www.dslreports.com/speedtest/results/bufferbloat
And:
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast

Re: [Cerowrt-devel] Fwd: [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 37 bytes --]

Link to the spec?

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 76 bytes --]

Re: [Cerowrt-devel] Fwd: [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Alan Jenkins]

On 23/07/15 08:44, Jonathan Morton wrote:
>
> Link to the spec?
>
> - Jonathan Morton
>
>

https://datatracker.ietf.org/doc/draft-szigeti-tsvwg-ieee-802-11e/

Re: [Cerowrt-devel] [Make-wifi-fast] Fwd: [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Sebastian Moeller]

Oh, boy,


On Jul 23, 2015, at 09:49 , Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:

> On 23/07/15 08:44, Jonathan Morton wrote:
>> 
>> Link to the spec?
>> 
>> - Jonathan Morton
>> 
>> 
> 
> https://datatracker.ietf.org/doc/draft-szigeti-tsvwg-ieee-802-11e/


	Not that I am a domain expert, but the whole web of slightly different versions of the same madness are quite amusing ;) What seems clear to me is that the wealth of 64 possible combinations looks a lot like a slippery slope (or better like a full blown slip and slide). 
	As far as I can see, hardware folks and MPLS opted for 3 bits maximum (I take it that wifi is actually 2 bits only), so what real use is in schemes using more than 8 different states beyond that mapping 12/16/64 to 8 is a fun exercise in bike-shedding… 

But since bike shedding is fun here is my “I am not even confused by partial knowledge” proposal for a 3+1bit marking scheme:

1) take the most significant 3 bits to deduce the CS equivalent (willfully ignoring the lower 3 bits) which will be treated as priority levels
2) decide were in the 3 bit range the “normal should be”, say 3 for example (to allow simplistic mapping to 2bit patterns)
3) extend by 1 bit at the end to get to 4 bits (I guess that would be shift by 1 bit?)
4) remark/remap the CS0 equivalent as “normal” + 1 (alternatively remap CS0 to “normal” and the CS that used to live there to “normal”+1)

Heck even just staying at 3 bits and just remarking CS0 to CS3 should do the trick, of putting in a lower priority class below the default best-effort traffic and still keep 802.11 mappings semi-working

And then just treat these as 8 different priority levels with the number coding the priority order using Jonathan’s approach of pairing higher priorities with lower guaranteed bandwidths allotments.
Blissfully ignore the proposed differentiation of the lower 3 bits until there is proof that they are actually helpful...


	I really wonder what the whole brouhaha is all about; wikipedia tells me ( https://en.wikipedia.org/wiki/IEEE_802.11e-2005 ) that Enhanced distributed channel access (EDCA) is sort of a best effort QoS system (what?) and that HCF Controlled Channel Access (HCCA) is where things should be going. Because then the AP manages all air time and all stations and the AP can do “normal”/high level priority queueing into the wifi adapters best-effort queue without the need for the “crazy" that is implementing media access as a race (especially with many clients that just does not sound like an efficient approach). But then I do not claim to be an expert in these matters.

Best Regards & sorry for the gantlet above
	Sebastian

> 
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Cerowrt-devel] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

> https://datatracker.ietf.org/doc/draft-szigeti-tsvwg-ieee-802-11e/

> On the topic of the actual mappings....

Here’s a handy comparison table to show how the draft, CJ’s suggestions, and cake’s current implementation map DSCPs to traffic classes:

DSCP | SZ | CJ | Cake
---------------------
CS7  | XX | VO | VO
CS6  | ?? | VO | VO
EF   | VO | VO | VO
VA   | VO | VO | VO
CS5  | VI | VO | VO
AF4x | VI | VI | VI
CS4  | VI | VI | VO
AF3x | VI | BE | VI
CS3  | VI | BE | VI
AF2x | BE | BE | VI
CS2  | BE | BE | VI
AF1x | BK | BE | BE
DF   | BE | BE | BE
CS1  | BK | BK | BK
TOS4 | BE | BE | VI
TOS2 | BE | BE | BE
TOS1 | BE | BE | VI

Interesting to note that cake puts a lot more traffic in “high” classes than either of these suggestions.  I also note that cake does invert CS4 vs AF4x in a way that CJ doesn’t like - but perhaps this is mitigated by the fact that cake thresholds VI at three times the bandwidth as VO, which I think is appropriate since video consumes more bandwidth than voice (or games) traffic.

None of these suggestions make any practical distinction between the “drop probability” divisions within the AFxx classes - even though in some cases they are mapped to distinct UP values, these always fall into the same major class.  I suppose that distinction would be better left to an AQM algorithm that was aware of them, which cake is not since I can’t immediately see a way to make Codel respond reasonably to it.

The draft doesn’t address the “legacy” codepoints associated with the old TOS bits, but cake does, so I extended the table accordingly.

I couldn’t quite decipher Sebastian’s suggestions into table form, so I omitted those.

The major “dangerous” feature I see in the draft is the treatment of CS6 and CS7 traffic - long on “drop or remark” and short on “if you must, just stick it in VO".  CS6 in particular is used by common NTP implementations, and for good reason.  As far as I’m concerned, equipment should *not* remark or drop traffic by default based solely on its DSCP.  Indeed, equipment should probably assume they are not acting as the edge of a network domain unless specifically configured otherwise.

 - Jonathan Morton

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Sebastian Moeller]

Hi Jonathan,

nice information about the competing schemes (and I do not consider my rant a proposed scheme ;) )

But looking at https://en.wikipedia.org/wiki/IEEE_802.11e-2005 I learn that AC_VO gets 1.5ms guaranteed TXOP (roughly air-time), AC_VI gets 3ms and BE and BK only get  a single MSDU. Even assuming this means an A-MSDU which as far as I can tell reaches up to 8K (in 802.11n, no idea about ac). If I do the math I see that from roughly 42 Mbps on AC_VO actually gets more airtime than BE and BK, AC_VI by the same logic reaches parity with BK and BE from 21 Mbps on. So for a well set up wifi net both VI and VO will get preferred media access and a larger bandwidth share, I hope I misunderstood something, but basically this means that the incentives to choose the correct AC shift with available bandwidth. So from speeds > 21 Mbp any client or AP should use AC_VI exclusively and above 42 AC_VO, or do I miss something here (can clients/STAs actually enforce their marking or does the AP call the shots)? Anyway, for me it seems that the whole IEEE 802.11e thing might contain dragons… (but most likely I simple do not grasp the beauty of 802.11e, for all I know it might actually be well-designed)


Best Regards
	Sebastian

On Jul 30, 2015, at 22:29 , Jonathan Morton <chromatix99@gmail.com> wrote:

>> https://datatracker.ietf.org/doc/draft-szigeti-tsvwg-ieee-802-11e/
> 
>> On the topic of the actual mappings....
> 
> Here’s a handy comparison table to show how the draft, CJ’s suggestions, and cake’s current implementation map DSCPs to traffic classes:
> 
> DSCP | SZ | CJ | Cake
> ---------------------
> CS7  | XX | VO | VO
> CS6  | ?? | VO | VO
> EF   | VO | VO | VO
> VA   | VO | VO | VO
> CS5  | VI | VO | VO
> AF4x | VI | VI | VI
> CS4  | VI | VI | VO
> AF3x | VI | BE | VI
> CS3  | VI | BE | VI
> AF2x | BE | BE | VI
> CS2  | BE | BE | VI
> AF1x | BK | BE | BE
> DF   | BE | BE | BE
> CS1  | BK | BK | BK
> TOS4 | BE | BE | VI
> TOS2 | BE | BE | BE
> TOS1 | BE | BE | VI
> 
> Interesting to note that cake puts a lot more traffic in “high” classes than either of these suggestions.  I also note that cake does invert CS4 vs AF4x in a way that CJ doesn’t like - but perhaps this is mitigated by the fact that cake thresholds VI at three times the bandwidth as VO, which I think is appropriate since video consumes more bandwidth than voice (or games) traffic.
> 
> None of these suggestions make any practical distinction between the “drop probability” divisions within the AFxx classes - even though in some cases they are mapped to distinct UP values, these always fall into the same major class.  I suppose that distinction would be better left to an AQM algorithm that was aware of them, which cake is not since I can’t immediately see a way to make Codel respond reasonably to it.
> 
> The draft doesn’t address the “legacy” codepoints associated with the old TOS bits, but cake does, so I extended the table accordingly.
> 
> I couldn’t quite decipher Sebastian’s suggestions into table form, so I omitted those.
> 
> The major “dangerous” feature I see in the draft is the treatment of CS6 and CS7 traffic - long on “drop or remark” and short on “if you must, just stick it in VO".  CS6 in particular is used by common NTP implementations, and for good reason.  As far as I’m concerned, equipment should *not* remark or drop traffic by default based solely on its DSCP.  Indeed, equipment should probably assume they are not acting as the edge of a network domain unless specifically configured otherwise.
> 
> - Jonathan Morton
> 
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 756 bytes --]

Hardware people tend to think in terms of simple priority queues, much like
old fashioned military communications (see the original IP precedence
spec). Higher priority thus gets higher throughput as well as lower latency.

I note also that in 802.11e, leftover space in a TXOP can't be (or at least
generally isn't) used opportunistically for traffic from another class,
because the four queues are so rigidly separated.

I think the hardware people are shortsighted in this respect. It's so easy
to game simple priority queues when there's no filter on the field
controlling it. That's why cake's Diffserv layer works the way it does. And
if I ever get the chance to do a Wi-Fi specific version, I'll avoid both of
the above problems.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 860 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Sebastian Moeller]

Hi Jonathan, 


On July 30, 2015 11:56:23 PM GMT+02:00, Jonathan Morton <chromatix99@gmail.com> wrote:
>Hardware people tend to think in terms of simple priority queues, much
>like
>old fashioned military communications (see the original IP precedence
>spec). Higher priority thus gets higher throughput as well as lower
>latency.
>
>I note also that in 802.11e, leftover space in a TXOP can't be (or at
>least
>generally isn't) used opportunistically for traffic from another class,
>because the four queues are so rigidly separated.
>
>I think the hardware people are shortsighted in this respect. It's so
>easy
>to game simple priority queues when there's no filter on the field
>controlling it. That's why cake's Diffserv layer works the way it does.
>And
>if I ever get the chance to do a Wi-Fi specific version, I'll avoid
>both of
>the above problems.
>
>- Jonathan Morton

Thanks for the insight. Now I Start to realize why my jome network behaves AS it does. When I run RRUL locally from my macbook over WiFi with cerowrt as AP (which if I recall correctly only uses AC_BE) the macbook's send starves the AP and hence the macbook's receive tanks. Since macos seems to exercise the AC_v[I|o] queues, it hogs airtime and and all systems using lower AC classes see less airtime, less bandwidth and higher latency. I guess my gut feeling would be to run the AP always at AC_VO so it does not get starved. But really calling such a system where any station can inflict that much pain/badness on others 'quality of service' makes me wonder. Then again it certainly affects quality of service just not deterministic or overall positive ;)

Best Regards
       Sebastian
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 4728 bytes --]


Hardware people tend to think about queues way too much in general.  Queues should be almost never occupied.  That causes the highest throughput possible.  And getting there is simple: push queueing back to the source.
 
The average queue length into a shared medium should be as close to zero as possible, and the variance should be as close to zero as possible.  This is why smaller packets are generally better (modulo switching overhead).
 
The ideal network is a network that maintains what I call a "ballistic" phase.  (like a perfect metallic phase in a conductive material).
 
It's easy to prove (as Kleinrock recently did with a student) that a network working optimally will have an average queue length everywhere that is less than 1 packet.
 
I think that is achievable, *even if there is a WiFi network in the middle*, by thinking about the fact that the shared airwaves in a WiFi network behaves like a single link, so all the queues on individual stations are really *one queue*, and that the optimal behavior of that link will be achieved if there is at most one packet queued at a time.
 
The problem with hardware folks and link folks is that they conflate the link with the network - two very different things.  The priority (if there is any) should be resolved by pushing back at the source, NOT by queueing low priority traffic inside the network!

If you think deeply about this, it amounts to a distributed priority-managed source-endpoint-located queuing strategy.  That is not actually hard to think about - when packets are dropped/ECN'd, the node that does the dropping knows a lot about the other competing traffic - in particular, it implicitly reflects some information about the existence of competing traffic to the source/dest pair (and in ECN, that can be rich information, like "the stated urgency of the competing traffic").  Then the decision about retransmitting can be pushed to the sources, with a lot of information about what's competing in the congested situation.
 
This is *far* better than leaving a lot of low priority stuff clogging the intermediate nodes.

So ignore the hardware folks who can't think about the fact that their link is embedded in a context that the link doesn't understand at all!   Don't let them convince you to queue things, especially lower priority things....  instead push congestion back to the source!!!
 
I know it is really, really productive of *research papers* to try to make a DSCP-based switching decision inside the network.  But it is totally ass-backwards in the big picture of an Internet.


On Thursday, July 30, 2015 11:27pm, "Sebastian Moeller" <moeller0@gmx.de> said:



> Hi Jonathan,
> 
> 
> On July 30, 2015 11:56:23 PM GMT+02:00, Jonathan Morton
> <chromatix99@gmail.com> wrote:
> >Hardware people tend to think in terms of simple priority queues, much
> >like
> >old fashioned military communications (see the original IP precedence
> >spec). Higher priority thus gets higher throughput as well as lower
> >latency.
> >
> >I note also that in 802.11e, leftover space in a TXOP can't be (or at
> >least
> >generally isn't) used opportunistically for traffic from another class,
> >because the four queues are so rigidly separated.
> >
> >I think the hardware people are shortsighted in this respect. It's so
> >easy
> >to game simple priority queues when there's no filter on the field
> >controlling it. That's why cake's Diffserv layer works the way it does.
> >And
> >if I ever get the chance to do a Wi-Fi specific version, I'll avoid
> >both of
> >the above problems.
> >
> >- Jonathan Morton
> 
> Thanks for the insight. Now I Start to realize why my jome network behaves AS it
> does. When I run RRUL locally from my macbook over WiFi with cerowrt as AP (which
> if I recall correctly only uses AC_BE) the macbook's send starves the AP and hence
> the macbook's receive tanks. Since macos seems to exercise the AC_v[I|o] queues,
> it hogs airtime and and all systems using lower AC classes see less airtime, less
> bandwidth and higher latency. I guess my gut feeling would be to run the AP always
> at AC_VO so it does not get starved. But really calling such a system where any
> station can inflict that much pain/badness on others 'quality of service' makes me
> wonder. Then again it certainly affects quality of service just not deterministic
> or overall positive ;)
> 
> Best Regards
> Sebastian
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

[-- Attachment #2: Type: text/html, Size: 7205 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]

> I think that is achievable, *even if there is a WiFi network in the
middle*, by thinking about the fact that the shared airwaves in a WiFi
network behaves like a single link, so all the queues on individual
stations are really *one queue*, and that the optimal behavior of that link
will be achieved if there is at most one packet queued at a time.

I agree that queues should be kept short in general. However I don't think
single packet queues are achievable in the general case.

The general case includes Wi-Fi networks, whose TXOP overhead is so
ruinously heavy that sending single MTU sized packets is inefficient.
Aggregating multiple packets into one TXOP requires those several packets
to be present in the buffer at that moment.

The general case includes links which vary in throughput frequently,
perhaps on shorter timescales than an RTT, so either packets must be
buffered or capacity is left unused. This also happens to include Wi-Fi,
but could easily include a standard wired link whose competing load varies.

The endpoints do not have and do not receive sufficient information in
sufficient time to reliably make packets arrive at nodes just in time to be
transmitted. Not even with ECN, not even with the wet dreams of the DCTCP
folks, and not even with ELR (though ELR should be able to make it happen
under steady conditions, there are still transient conditions in the
general case).

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 1545 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Michael Richardson]

Jonathan Morton <chromatix99@gmail.com> wrote:
    >> I think that is achievable, *even if there is a WiFi network in the
    > middle*, by thinking about the fact that the shared airwaves in a WiFi
    > network behaves like a single link, so all the queues on individual
    > stations are really *one queue*, and that the optimal behavior of that
    > link will be achieved if there is at most one packet queued at a time.

    > I agree that queues should be kept short in general. However I don't
    > think single packet queues are achievable in the general case.

    > The general case includes Wi-Fi networks, whose TXOP overhead is so
    > ruinously heavy that sending single MTU sized packets is
    > inefficient. Aggregating multiple packets into one TXOP requires those
    > several packets to be present in the buffer at that moment.

Agreed... if one has a minimum of two queues (and *fq-codel has at least that
many), why isn't the naive method of:
       1) send everything from the low-latency queue
       2) send the other stuff until the TXOP is full

enough?  codel will keep the queues short.
The key thing I think, is that the markings (1)/(2) above be passed along
through all "subqueues"...

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 1017 bytes --]

There is some subtlety of terminology here...

Fq_codel has many queues, but they are not a priori assigned as "low
latency" or otherwise. There is a dynamic sense of whether each queue is
handling sparse or bulk traffic, with the sparse queues being serviced
first. This is a type of priority queuing which is difficult to game (and
completely ignores Diffserv).

Cake does the same thing, and additionally divides the queue using
Diffserv. Etcetera.

Both fq_codel and cake deliver one packet at a time. This is a feature of
the qdisc API within the kernel. They have no visibility of what
aggregation might be happening at the hardware level, only that the driver
is requesting a packet.

Wi-Fi hardware implementing 802.11e tends to have four queues, each of
which is a dumb FIFO, and services only one of them per TXOP (probably the
highest priority one). It doesn't even do the "naive" thing you mention -
which goes some way to illustrate the point of view of the typical hardware
engineer.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 1151 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 2863 bytes --]


It's not infeasible to make queues shorter.  In any case, the throughput of a link does not increase above the point where there is always one packet ready to go by the time the currently outgoing packet is completed.  It physically cannot do better than that.

If hardware designers can't create an interface that achieves that bound I'd be suspicious that they understand how to design hardware.  In the case of WiFi, this also includes the MAC protocol being designed so that when the current packet on the air terminates, the next packet can be immediately begun - that's a little more subtle.

But my point here is that one needs to look at the efficiency of the system as a whole (in context), and paradoxically to the hardware designer mindset, the proper way to think about that efficiency is NOT about link throughput maximization - instead it is an end-to-end property.  One has very little to do with the other.  Queueing doesn't affect link throughput beyond the "double buffering" effect noted above: at most one packet queued behind the currently transmitting packet.
 
Regarding TXOP overhead - rather than complicated queueing, just allow packets to be placed in line *while the currently transmitting packet is going out*, and changed up to the point in time when they begin transmitting. This is trivial in hardware.


On Friday, July 31, 2015 1:04pm, "Jonathan Morton" <chromatix99@gmail.com> said:



> I think that is achievable, *even if there is a WiFi network in the middle*, by thinking about the fact that the shared airwaves in a WiFi network behaves like a single link, so all the queues on individual stations are really *one queue*, and that the optimal behavior of that link will be achieved if there is at most one packet queued at a time.
I agree that queues should be kept short in general. However I don't think single packet queues are achievable in the general case.
The general case includes Wi-Fi networks, whose TXOP overhead is so ruinously heavy that sending single MTU sized packets is inefficient. Aggregating multiple packets into one TXOP requires those several packets to be present in the buffer at that moment.
The general case includes links which vary in throughput frequently, perhaps on shorter timescales than an RTT, so either packets must be buffered or capacity is left unused. This also happens to include Wi-Fi, but could easily include a standard wired link whose competing load varies.
The endpoints do not have and do not receive sufficient information in sufficient time to reliably make packets arrive at nodes just in time to be transmitted. Not even with ECN, not even with the wet dreams of the DCTCP folks, and not even with ELR (though ELR should be able to make it happen under steady conditions, there are still transient conditions in the general case).
- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 4207 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[David Lang]

[-- Attachment #1: Type: TEXT/Plain, Size: 5157 bytes --]

On Mon, 3 Aug 2015, dpreed@reed.com wrote:

> It's not infeasible to make queues shorter.  In any case, the throughput of a 
> link does not increase above the point where there is always one packet ready 
> to go by the time the currently outgoing packet is completed.  It physically 
> cannot do better than that.

change 'one packet' to 'one transmissions worth of packets' and I'll agree

> If hardware designers can't create an interface that achieves that bound I'd 
> be suspicious that they understand how to design hardware.  In the case of 
> WiFi, this also includes the MAC protocol being designed so that when the 
> current packet on the air terminates, the next packet can be immediately begun 
> - that's a little more subtle.

on a shared medium (like radio) things are a bit messier.

There are two issues

1. You shouldn't just transmit to a new station once you finish sending to the 
first. Fairness requires that you pause and give other stations a chance to 
transmit as well.

1. There is per-transmission overhead (including the pause mentioned above) that 
can be very significant for small packets, so there is considerable value in 
sending multiple packets at once. It's a lighter version of what you run into 
inserting into reliable databases. You can insert 1000 records in about the same 
time you can insert 2 records sperately.

The "stock" answer to this is for hardware and software folks to hold off on 
sending anything in case there is more to send later that it can be batched 
with. This maximizes throughput at the cost of latency.

What should be done instead is to send what you have immediatly, and while it's 
sending, queue whatever continues to arrive and the next chance you have to 
send, you will have more to send. This scales the batch size with congestion, 
minimizing latency at the cost of keeping the channel continually busy, but 
inefficiently busy if you aren't at capacity.

> But my point here is that one needs to look at the efficiency of the system as 
> a whole (in context), and paradoxically to the hardware designer mindset, the 
> proper way to think about that efficiency is NOT about link throughput 
> maximization - instead it is an end-to-end property.  One has very little to 
> do with the other.  Queueing doesn't affect link throughput beyond the "double 
> buffering" effect noted above: at most one packet queued behind the currently 
> transmitting packet.
> 
> Regarding TXOP overhead - rather than complicated queueing, just allow packets 
> to be placed in line *while the currently transmitting packet is going out*, 
> and changed up to the point in time when they begin transmitting. This is 
> trivial in hardware.

This is a key thing that a lot of hardware and software folks get wrong. All the 
complexity and bugs that you see around 'blocking/plugging' flows are the result 
of this mistake. As you say, send something as soon as you have it to send. If 
there's more arriving, let it accumulate while the first bit is being sent and 
the next chance you get to send, send everything that's accumulated. This 
minimizes latency, greatly simplifies the code (no need for timers to 
unblock/release the data if more doesn't arrive), and results in the exact same 
throughput under load.

It does have some interesting changes to the utilization curve at part load. 
These could be a problem with wifi under some conditions, but I think the 
trade-off is worth it since the wifi is going to end up running up to it's limit 
sometime anyway, and the part load problems are just previews of what you would 
run into at full load.

David Lang

>
> On Friday, July 31, 2015 1:04pm, "Jonathan Morton" <chromatix99@gmail.com> said:
>
>
>
>> I think that is achievable, *even if there is a WiFi network in the middle*, by thinking about the fact that the shared airwaves in a WiFi network behaves like a single link, so all the queues on individual stations are really *one queue*, and that the optimal behavior of that link will be achieved if there is at most one packet queued at a time.
> I agree that queues should be kept short in general. However I don't think single packet queues are achievable in the general case.
> The general case includes Wi-Fi networks, whose TXOP overhead is so ruinously heavy that sending single MTU sized packets is inefficient. Aggregating multiple packets into one TXOP requires those several packets to be present in the buffer at that moment.
> The general case includes links which vary in throughput frequently, perhaps on shorter timescales than an RTT, so either packets must be buffered or capacity is left unused. This also happens to include Wi-Fi, but could easily include a standard wired link whose competing load varies.
> The endpoints do not have and do not receive sufficient information in sufficient time to reliably make packets arrive at nodes just in time to be transmitted. Not even with ECN, not even with the wet dreams of the DCTCP folks, and not even with ELR (though ELR should be able to make it happen under steady conditions, there are still transient conditions in the general case).
> - Jonathan Morton

[-- Attachment #2: Type: TEXT/PLAIN, Size: 164 bytes --]

_______________________________________________
Cerowrt-devel mailing list
Cerowrt-devel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 7082 bytes --]


I design and build physical layer radio hardware (using SDR reception and transmission in the 5 GHz and 10 GHz Amateur radio bands).
 
Fairness is easy in a MAC. 1 usec. is 1,000 linear feet.  If the next station knows when its turn is, it can start transmitting within a couple of microseconds of seeing the tail of the last packet, and if there is adequate "sounding" of the physical environment, you can calculate tighter bounds than that.  Even if the transmission is 1 Gb/sec, 1 usec. is only 1,000 bits at most.
 
But at the end-to-end layer, today's networks are only at most 20 msec. end-to-end across a continent.  The most buffering you want to see on an end-to-end basis is 10 msec.
 
I disagree strongly that "mice" - small packets - need to be compressed.  Most small packets are very, very latency sensitive (acks, etc.). As long as they are a relatively small portion of capacity, they aren't the place to trade latency degradation for throughput.  That's another example of focusing on the link rather than the end-to-end network context.
(local link acks can be piggy-backed in various ways, so when the local Wireless Ethernet domain is congested, there should be no naked ack packets)
 
Why does anyone measure a link in terms of a measurement of small-packet efficiency?  The end-to-end protocols shouldn't be sending small packets once a queue builds up at the source endpoint.
 


On Monday, August 3, 2015 12:14pm, "David Lang" <david@lang.hm> said:



> On Mon, 3 Aug 2015, dpreed@reed.com wrote:
> 
> > It's not infeasible to make queues shorter. In any case, the throughput of a
> > link does not increase above the point where there is always one packet ready
> > to go by the time the currently outgoing packet is completed. It physically
> > cannot do better than that.
> 
> change 'one packet' to 'one transmissions worth of packets' and I'll agree
> 
> > If hardware designers can't create an interface that achieves that bound I'd
> > be suspicious that they understand how to design hardware. In the case of
> > WiFi, this also includes the MAC protocol being designed so that when the
> > current packet on the air terminates, the next packet can be immediately
> begun
> > - that's a little more subtle.
> 
> on a shared medium (like radio) things are a bit messier.
> 
> There are two issues
> 
> 1. You shouldn't just transmit to a new station once you finish sending to the
> first. Fairness requires that you pause and give other stations a chance to
> transmit as well.
> 
> 1. There is per-transmission overhead (including the pause mentioned above) that
> can be very significant for small packets, so there is considerable value in
> sending multiple packets at once. It's a lighter version of what you run into
> inserting into reliable databases. You can insert 1000 records in about the same
> time you can insert 2 records sperately.
> 
> The "stock" answer to this is for hardware and software folks to hold off on
> sending anything in case there is more to send later that it can be batched
> with. This maximizes throughput at the cost of latency.
> 
> What should be done instead is to send what you have immediatly, and while it's
> sending, queue whatever continues to arrive and the next chance you have to
> send, you will have more to send. This scales the batch size with congestion,
> minimizing latency at the cost of keeping the channel continually busy, but
> inefficiently busy if you aren't at capacity.
> 
> > But my point here is that one needs to look at the efficiency of the system
> as
> > a whole (in context), and paradoxically to the hardware designer mindset, the
> > proper way to think about that efficiency is NOT about link throughput
> > maximization - instead it is an end-to-end property. One has very little to
> > do with the other. Queueing doesn't affect link throughput beyond the
> "double
> > buffering" effect noted above: at most one packet queued behind the currently
> > transmitting packet.
> >
> > Regarding TXOP overhead - rather than complicated queueing, just allow
> packets
> > to be placed in line *while the currently transmitting packet is going out*,
> > and changed up to the point in time when they begin transmitting. This is
> > trivial in hardware.
> 
> This is a key thing that a lot of hardware and software folks get wrong. All the
> complexity and bugs that you see around 'blocking/plugging' flows are the result
> of this mistake. As you say, send something as soon as you have it to send. If
> there's more arriving, let it accumulate while the first bit is being sent and
> the next chance you get to send, send everything that's accumulated. This
> minimizes latency, greatly simplifies the code (no need for timers to
> unblock/release the data if more doesn't arrive), and results in the exact same
> throughput under load.
> 
> It does have some interesting changes to the utilization curve at part load.
> These could be a problem with wifi under some conditions, but I think the
> trade-off is worth it since the wifi is going to end up running up to it's limit
> sometime anyway, and the part load problems are just previews of what you would
> run into at full load.
> 
> David Lang
> 
> >
> > On Friday, July 31, 2015 1:04pm, "Jonathan Morton"
> <chromatix99@gmail.com> said:
> >
> >
> >
> >> I think that is achievable, *even if there is a WiFi network in the
> middle*, by thinking about the fact that the shared airwaves in a WiFi network
> behaves like a single link, so all the queues on individual stations are really
> *one queue*, and that the optimal behavior of that link will be achieved if there
> is at most one packet queued at a time.
> > I agree that queues should be kept short in general. However I don't think
> single packet queues are achievable in the general case.
> > The general case includes Wi-Fi networks, whose TXOP overhead is so ruinously
> heavy that sending single MTU sized packets is inefficient. Aggregating multiple
> packets into one TXOP requires those several packets to be present in the buffer
> at that moment.
> > The general case includes links which vary in throughput frequently, perhaps
> on shorter timescales than an RTT, so either packets must be buffered or capacity
> is left unused. This also happens to include Wi-Fi, but could easily include a
> standard wired link whose competing load varies.
> > The endpoints do not have and do not receive sufficient information in
> sufficient time to reliably make packets arrive at nodes just in time to be
> transmitted. Not even with ECN, not even with the wet dreams of the DCTCP folks,
> and not even with ELR (though ELR should be able to make it happen under steady
> conditions, there are still transient conditions in the general case).
> > - Jonathan Morton_______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
> 

[-- Attachment #2: Type: text/html, Size: 9519 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 89 bytes --]

Because at today's link rates, a full 1500 byte  MTU packet is small.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 136 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[David Lang]

On Mon, 3 Aug 2015, dpreed@reed.com wrote:

> I design and build physical layer radio hardware (using SDR reception and 
> transmission in the 5 GHz and 10 GHz Amateur radio bands).
> 
> Fairness is easy in a MAC. 1 usec. is 1,000 linear feet.  If the next station 
> knows when its turn is, it can start transmitting within a couple of 
> microseconds of seeing the tail of the last packet, and if there is adequate 
> "sounding" of the physical environment, you can calculate tighter bounds than 
> that.  Even if the transmission is 1 Gb/sec, 1 usec. is only 1,000 bits at 
> most.

That requires central coordination of the stations. Something we don't have in 
wifi. Wifi lives and dies with 'listen for a gap, try transmitting, and if you 
collide, backoff a random period'

> But at the end-to-end layer, today's networks are only at most 20 msec. 
> end-to-end across a continent.  The most buffering you want to see on an 
> end-to-end basis is 10 msec.
> 
> I disagree strongly that "mice" - small packets - need to be compressed. 
> Most small packets are very, very latency sensitive (acks, etc.). As long as 
> they are a relatively small portion of capacity, they aren't the place to 
> trade latency degradation for throughput.  That's another example of focusing 
> on the link rather than the end-to-end network context. (local link acks can 
> be piggy-backed in various ways, so when the local Wireless Ethernet domain is 
> congested, there should be no naked ack packets)

umm, you misread what I was saying. I didn't say that we should hold up packets 
in search of throughput. I said the opposite. Instead of holding up small 
packets to maximize throughput, send the first small packet (and pay the 
overhead of doing so that makes it take a long time to do so), and while this is 
happening, additional packets will accumulate. The next transmission slot you 
have available, transmit all (up to a cap) the packets you have pending.

this produces the best possible latency, but uses more air-time than the 
'normal' approach where they hold up packets for a short time to see if they can 
be combined with others.

> Why does anyone measure a link in terms of a measurement of small-packet 
> efficiency?  The end-to-end protocols shouldn't be sending small packets once 
> a queue builds up at the source endpoint.

what if the small packets are not all from the same source? or even if they are 
from the same source IP, are from different ports? they can't just be combined 
at the IP layer.

David Lang

>
>
> On Monday, August 3, 2015 12:14pm, "David Lang" <david@lang.hm> said:
>
>
>
>> On Mon, 3 Aug 2015, dpreed@reed.com wrote:
>> 
>> > It's not infeasible to make queues shorter. In any case, the throughput of a
>> > link does not increase above the point where there is always one packet ready
>> > to go by the time the currently outgoing packet is completed. It physically
>> > cannot do better than that.
>> 
>> change 'one packet' to 'one transmissions worth of packets' and I'll agree
>> 
>> > If hardware designers can't create an interface that achieves that bound I'd
>> > be suspicious that they understand how to design hardware. In the case of
>> > WiFi, this also includes the MAC protocol being designed so that when the
>> > current packet on the air terminates, the next packet can be immediately
>> begun
>> > - that's a little more subtle.
>> 
>> on a shared medium (like radio) things are a bit messier.
>> 
>> There are two issues
>> 
>> 1. You shouldn't just transmit to a new station once you finish sending to the
>> first. Fairness requires that you pause and give other stations a chance to
>> transmit as well.
>> 
>> 1. There is per-transmission overhead (including the pause mentioned above) that
>> can be very significant for small packets, so there is considerable value in
>> sending multiple packets at once. It's a lighter version of what you run into
>> inserting into reliable databases. You can insert 1000 records in about the same
>> time you can insert 2 records sperately.
>> 
>> The "stock" answer to this is for hardware and software folks to hold off on
>> sending anything in case there is more to send later that it can be batched
>> with. This maximizes throughput at the cost of latency.
>> 
>> What should be done instead is to send what you have immediatly, and while it's
>> sending, queue whatever continues to arrive and the next chance you have to
>> send, you will have more to send. This scales the batch size with congestion,
>> minimizing latency at the cost of keeping the channel continually busy, but
>> inefficiently busy if you aren't at capacity.
>> 
>> > But my point here is that one needs to look at the efficiency of the system
>> as
>> > a whole (in context), and paradoxically to the hardware designer mindset, the
>> > proper way to think about that efficiency is NOT about link throughput
>> > maximization - instead it is an end-to-end property. One has very little to
>> > do with the other. Queueing doesn't affect link throughput beyond the
>> "double
>> > buffering" effect noted above: at most one packet queued behind the currently
>> > transmitting packet.
>> >
>> > Regarding TXOP overhead - rather than complicated queueing, just allow
>> packets
>> > to be placed in line *while the currently transmitting packet is going out*,
>> > and changed up to the point in time when they begin transmitting. This is
>> > trivial in hardware.
>> 
>> This is a key thing that a lot of hardware and software folks get wrong. All the
>> complexity and bugs that you see around 'blocking/plugging' flows are the result
>> of this mistake. As you say, send something as soon as you have it to send. If
>> there's more arriving, let it accumulate while the first bit is being sent and
>> the next chance you get to send, send everything that's accumulated. This
>> minimizes latency, greatly simplifies the code (no need for timers to
>> unblock/release the data if more doesn't arrive), and results in the exact same
>> throughput under load.
>> 
>> It does have some interesting changes to the utilization curve at part load.
>> These could be a problem with wifi under some conditions, but I think the
>> trade-off is worth it since the wifi is going to end up running up to it's limit
>> sometime anyway, and the part load problems are just previews of what you would
>> run into at full load.
>> 
>> David Lang
>> 
>> >
>> > On Friday, July 31, 2015 1:04pm, "Jonathan Morton"
>> <chromatix99@gmail.com> said:
>> >
>> >
>> >
>> >> I think that is achievable, *even if there is a WiFi network in the
>> middle*, by thinking about the fact that the shared airwaves in a WiFi network
>> behaves like a single link, so all the queues on individual stations are really
>> *one queue*, and that the optimal behavior of that link will be achieved if there
>> is at most one packet queued at a time.
>> > I agree that queues should be kept short in general. However I don't think
>> single packet queues are achievable in the general case.
>> > The general case includes Wi-Fi networks, whose TXOP overhead is so ruinously
>> heavy that sending single MTU sized packets is inefficient. Aggregating multiple
>> packets into one TXOP requires those several packets to be present in the buffer
>> at that moment.
>> > The general case includes links which vary in throughput frequently, perhaps
>> on shorter timescales than an RTT, so either packets must be buffered or capacity
>> is left unused. This also happens to include Wi-Fi, but could easily include a
>> standard wired link whose competing load varies.
>> > The endpoints do not have and do not receive sufficient information in
>> sufficient time to reliably make packets arrive at nodes just in time to be
>> transmitted. Not even with ECN, not even with the wet dreams of the DCTCP folks,
>> and not even with ELR (though ELR should be able to make it happen under steady
>> conditions, there are still transient conditions in the general case).
>> > - Jonathan Morton_______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>>

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 1504 bytes --]


On Monday, August 3, 2015 8:13pm, "David Lang" <david@lang.hm> said:
> 

> That requires central coordination of the stations. Something we don't have in
> wifi. Wifi lives and dies with 'listen for a gap, try transmitting, and if you
> collide, backoff a random period'


Central coordination is not the only form of coordination... there are perfectly fine decentralized coordination schemes that do better than LBT. Depends on your definition of 802.11, but I did point out that the MAC layer could be a lot better, and internode coordination can be both decentralized and far more power efficient, in principle. It's important to realize that the preparation of an OFDM modulated waveform can be pipelined, so that a transmitter can have the physical waveform "built" (via DFT, etc.) while waiting for its time to go.  And the "collision resolution" can and should be an arbitration process that starts before the current packet in the air is finished.

What prevents this is unnecessary "legacy compatibility" - making high speed modulated packets suffer because there are still stupid 2 Mb/sec. 802.11b devices on the 2.4 GHz band.  There are ways to coexist with legacy systems that are better than transmitting the prefix on the front of every packet (you can transmit a fake 802.11b prefix that will lock out the 2.4 GHz competitors for a period of time when many "turbo" stations occupy the air using better cooperating physical layer methods, as a conceptually trivial example).
 

[-- Attachment #2: Type: text/html, Size: 2249 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Mikael Abrahamsson]

On Fri, 31 Jul 2015, dpreed@reed.com wrote:

> So ignore the hardware folks who can't think about the fact that their 
> link is embedded in a context that the link doesn't understand at all! 
> Don't let them convince you to queue things, especially lower priority 
> things....  instead push congestion back to the source!!!

So while I think you have a point, I don't see how this can be achieved 
(at most 1 packet in the queue) on something like wifi where there are 
retransmits and an onloaded link can have between a few ms and all of a 
sudden have 50-100ms of delay, and then get back to a few ms again). If 
you screetch to a halt when you get this "congestion" (that isn't even 
caused by traffic but by RF environment), if you have packets in the 
buffer and feedback the sender to stop, there after the RF problem has 
past, buffer is emptied, but now basically all traffic has screetched to a 
halt.

So a compromise must be achieved somewhere, so that 300ms RTT flows get 
decent performance without affecting realtime flows. I don't understand 
how both goals of low delay/no buffering and decent high-RTT flow speed, 
can work without some kind of scheme where different flows are put in 
different queues.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Rich Brown]

Here's where I display my massive ignorance of how Linux networking/drivers work. This feels like a place to apply the Nagle algorithm... Maybe everyone already understands this, but, isn't this scheme something like what we're looking for? 

- Initial state: all queues (fq_codel and wifi driver) are empty.

- A packet arrives at fq_codel, it's placed in the proper queue for its flow, and the wifi driver gets tapped on the shoulder that there's something to send.

- Sometime later, the wifi driver has bid for and received an opportunity to transmit. 

- At that time, the wifi driver requests packets from fq_codel until a) the the fq_codel queues are empty, or b) the wifi frame is full. In either case, the wifi driver sends what it has.

- If more packets remain in the fq_codel queue(s), the wifi driver gets tapped on the shoulder again to start another transmission. 

Good Attributes:

- The wifi driver has no "queued packets" per se - only those it pulled from fq_codel for immediate transmission.

- Once the transmit opportunity has come around, it's a matter of microseconds (I assume) to pull in a wifi frame's worth of packets from fq_codel

- A singleton packet (e.g., one widely separated in time from all the other traffic) gets sent as soon as it can, without waiting in hopes "that more traffic will arrive" This leaves the wifi utilization unchanged (since the singleton packet would have to be sent anyway), but avoids the delay for that particular flow.

Downsides:

- Most likely, "It doesn't work that way". :-)

- The wifi driver would probably have to queue the single last-retrieved packet from fq_codel when it doesn't fit in the wifi frame. But this would be an immediate signal to bid for another transmit opportunity.

Rich


On Aug 7, 2015, at 4:28 AM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:

> On Fri, 31 Jul 2015, dpreed@reed.com wrote:
> 
>> So ignore the hardware folks who can't think about the fact that their link is embedded in a context that the link doesn't understand at all! Don't let them convince you to queue things, especially lower priority things....  instead push congestion back to the source!!!
> 
> So while I think you have a point, I don't see how this can be achieved (at most 1 packet in the queue) on something like wifi where there are retransmits and an onloaded link can have between a few ms and all of a sudden have 50-100ms of delay, and then get back to a few ms again). If you screetch to a halt when you get this "congestion" (that isn't even caused by traffic but by RF environment), if you have packets in the buffer and feedback the sender to stop, there after the RF problem has past, buffer is emptied, but now basically all traffic has screetched to a halt.
> 
> So a compromise must be achieved somewhere, so that 300ms RTT flows get decent performance without affecting realtime flows. I don't understand how both goals of low delay/no buffering and decent high-RTT flow speed, can work without some kind of scheme where different flows are put in different queues.
> 
> -- 
> Mikael Abrahamsson    email: swmike@swm.pp.se
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

> On 7 Aug, 2015, at 15:22, Rich Brown <richb.hanover@gmail.com> wrote:
> 
> - At that time, the wifi driver requests packets from fq_codel until a) the the fq_codel queues are empty, or b) the wifi frame is full. In either case, the wifi driver sends what it has.

There’s one big flaw with this: if packets are available for multiple destinations, fq_codel will generally give you a variety pack of packets for each of them.  But a wifi TXOP is for a single destination, so only some of the packets would be eligible for the same aggregate frame.

So what’s needed is a way for the wifi driver to tell the queue that it wants packets for the *same* destination as it’s transmitting to.

> - Once the transmit opportunity has come around, it's a matter of microseconds (I assume) to pull in a wifi frame's worth of packets from fq_codel

This is hard to guarantee in software in a general-purpose OS.

 - Jonathan Morton

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Rich Brown]

Ah..., the bitter bit of the reality sandwich. Yum :-(

But Dave's note about Felix's hook for per-station queueing makes it seem feasible, if only a lot of work.

Thanks, all, for the enlightenment!

Rich

On Aug 7, 2015, at 9:28 AM, Jonathan Morton <chromatix99@gmail.com> wrote:

> 
>> On 7 Aug, 2015, at 15:22, Rich Brown <richb.hanover@gmail.com> wrote:
>> 
>> - At that time, the wifi driver requests packets from fq_codel until a) the the fq_codel queues are empty, or b) the wifi frame is full. In either case, the wifi driver sends what it has.
> 
> There’s one big flaw with this: if packets are available for multiple destinations, fq_codel will generally give you a variety pack of packets for each of them.  But a wifi TXOP is for a single destination, so only some of the packets would be eligible for the same aggregate frame.
> 
> So what’s needed is a way for the wifi driver to tell the queue that it wants packets for the *same* destination as it’s transmitting to.
> 
>> - Once the transmit opportunity has come around, it's a matter of microseconds (I assume) to pull in a wifi frame's worth of packets from fq_codel
> 
> This is hard to guarantee in software in a general-purpose OS.
> 
> - Jonathan Morton
> 

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Simon Barber]

Some level of hardware queueing is necessary to meet the response time. 
It's further complicated by the need to include retries at the start of the 
next aggregate, but you only learn which ones from the block-ACK received 
after the last one. Given the hardware's need to have frame length well 
before the data you may have only a few microseconds to make some 
decisions. This is why all recent chipsets have a fairly fast CPU on board 
dedicated to implementing some of these functions. Of course the quality of 
the design and implementation varies hugely from vendor to vendor, and the 
API in terms of queueing varies.

Simon

Sent with AquaMail for Android
http://www.aqua-mail.com


On August 7, 2015 10:36:04 AM Rich Brown <richb.hanover@gmail.com> wrote:

> Ah..., the bitter bit of the reality sandwich. Yum :-(
>
> But Dave's note about Felix's hook for per-station queueing makes it seem 
> feasible, if only a lot of work.
>
> Thanks, all, for the enlightenment!
>
> Rich
>
> On Aug 7, 2015, at 9:28 AM, Jonathan Morton <chromatix99@gmail.com> wrote:
>
> >
> >> On 7 Aug, 2015, at 15:22, Rich Brown <richb.hanover@gmail.com> wrote:
> >>
> >> - At that time, the wifi driver requests packets from fq_codel until a) 
> the the fq_codel queues are empty, or b) the wifi frame is full. In either 
> case, the wifi driver sends what it has.
> >
> > There’s one big flaw with this: if packets are available for multiple 
> destinations, fq_codel will generally give you a variety pack of packets 
> for each of them.  But a wifi TXOP is for a single destination, so only 
> some of the packets would be eligible for the same aggregate frame.
> >
> > So what’s needed is a way for the wifi driver to tell the queue that it 
> wants packets for the *same* destination as it’s transmitting to.
> >
> >> - Once the transmit opportunity has come around, it's a matter of 
> microseconds (I assume) to pull in a wifi frame's worth of packets from 
> fq_codel
> >
> > This is hard to guarantee in software in a general-purpose OS.
> >
> > - Jonathan Morton
> >
>
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3553 bytes --]

On Fri, 7 Aug 2015, Jonathan Morton wrote:

>> On 7 Aug, 2015, at 15:22, Rich Brown <richb.hanover@gmail.com> wrote:
>> 
>> - At that time, the wifi driver requests packets from fq_codel until a) the 
>> the fq_codel queues are empty, or b) the wifi frame is full. In either case, 
>> the wifi driver sends what it has.
>
> There’s one big flaw with this: if packets are available for multiple 
> destinations, fq_codel will generally give you a variety pack of packets for 
> each of them.  But a wifi TXOP is for a single destination, so only some of 
> the packets would be eligible for the same aggregate frame.
>
> So what’s needed is a way for the wifi driver to tell the queue that it wants 
> packets for the *same* destination as it’s transmitting to.

how about when the queue hands packets to the wifi driver, it hands all packets 
to that same destination, no matter where they are in the queue (up to a max 
size, and the queue may adjust that max size within a range for fairness)

>> - Once the transmit opportunity has come around, it's a matter of 
>> microseconds (I assume) to pull in a wifi frame's worth of packets from 
>> fq_codel
>
> This is hard to guarantee in software in a general-purpose OS.

you really want to have the pckets assembled and ready to go rather than trying 
to pull them at that point.

But what happens right now is that the queue hands packets to the wifi driver, 
then the wifi driver has it's own queues that it uses to gather the packets for 
each destination.

If we can find a way to make it reasonable to short-circuit the wifi driver 
queues by making it efficient to work from the main network queues, we can work 
to eliminate the second layer of queues.



so thinking about the requierments from the driver point of view

It needs to be able to pull a chunk of data to transmit (multple packets), it 
isn't going to know very much ahead of time what speed it's going to use to talk 
to this destination, and this is going to drastically affect how long it takes 
to transmit the bits. so when it grabs data from the queue, it needs to feed 
back to the queue the transmit time for those bits, and the queue uses that 
instead of the count of bits to determine fairness.

The queue will be deciding fairness based on whois behind in their 'fair share' 
of transmit time. So the wifi driver isn't going to know when it asks for the 
net chunk of data to transmit, who it will be going to. So it will need to get 
the destination, see the speed to use to that destination, pass the speed to 
a calculation for how much data to send, then grab that much data (rounded to a 
packet boundry)



Is this sort of logic useful anywhere other than in wifi?

Wifi is the only place I know of where the transmit bit rate is going to vary 
depending on the next hop address.

I know that the inter-packet gap defined for ethernet can amount to a large 
percentage of bandwidth at high speeds. Can multiple packets to the same 
destination be combined with a smaller gap between packets? (the gap timing was 
based on the speed-of-light time needed for the entire shared bus to quiece back 
in the 10mb half-duplex hub days). If so, then there's value in bundling packets 
to the same destination together.


If this sort of logic is not useful anywhere other than in wifi, maby the right 
answer is to have a way of short-circuiting the main OS queues and have a wifi 
specific queue that can directly look at the per-client speeds/etc when deciding 
who goes next and how much to dispatch?

David Lang

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 2995 bytes --]


 
On Friday, August 7, 2015 4:03pm, "David Lang" <david@lang.hm> said:
> 

> Wifi is the only place I know of where the transmit bit rate is going to vary
> depending on the next hop address.


This is an interesting core issue.  The question is whether additional queueing helps or hurts this, and whether the MAC protocol of WiFi deals well or poorly with this issue.  It is clear that this is a peculiarly WiFi'ish issue.
 
It's not clear that the best transmit rate remains stable for very long, or even how to predict the "best rate" for the next station since the next station is one you may not have transmitted to for a long time, so your "best rate" information is old.  Queueing makes information about the channel older, by binding it too early.  Sending longer frames means retransmitting longer frames when they don't get through, rather than agilely picking a better rate after a few bits.
 
The MAC protocol really should give the receiver some opportunity to control the rate of the next packet it gets (which it can do because it can measure the channel from the transmitter to itself, by listening to prior transmissions).  Or at least to signal channel changes that might require a new signalling rate.
 
This suggests that a transmitter might want to "warn" a receiver that some packets will be coming its way, so the receiver can preemptively change the desired rate.  Thus, perhaps an RTS-CTS like mechanism can be embedded in the MAC protocol, which requires that the device "look ahead" at the packets it might be sending.
 
On the other hand, that only works if the transmitter deliberately congests itself so that it has a queue built up to look at.
 
The tradeoffs are not obvious here at all.  On the other hand, one could do something much simpler - just have the transmitter slow down to the worst-case rate required by any receiving system.
 
As the number of stations in range gets larger, though, it seems unlikely that "batching" multiple packets to the same destination is a good idea at all - because to achieve that, one must have n_destinations * batch_size chunks of data queued in the system as a whole, and that gets quite large.  I suspect it would be better to find a lower level way to just keep the packets going out as fast as they arrive, so no clog occurs, and to slow down the stuff at the source as quickly as possible.
 
[one should also dive into the reason for maintaining variable rates - multipath to a particular destination may require longer symbols for decoding without ISI.  And when multipath is involved, you may have to retransmit at a slower rate. There's usually not much "noise" at the receiver compared to the multipath environment. (one of the reasons why mesh can be a lot better is that shorter distances have much less multipath effect, so you can get higher symbol rates by going multi-hop, and of course higher symbol rates compensate for more airtime occupied by a packet due to repeating).]
 
 

[-- Attachment #2: Type: text/html, Size: 5343 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[David Lang]

On Fri, 7 Aug 2015, dpreed@reed.com wrote:

> On Friday, August 7, 2015 4:03pm, "David Lang" <david@lang.hm> said:
>> 
>
>> Wifi is the only place I know of where the transmit bit rate is going to vary
>> depending on the next hop address.
>
>
> This is an interesting core issue.  The question is whether additional 
> queueing helps or hurts this, and whether the MAC protocol of WiFi deals well 
> or poorly with this issue.  It is clear that this is a peculiarly WiFi'ish 
> issue.
> 
> It's not clear that the best transmit rate remains stable for very long, or 
> even how to predict the "best rate" for the next station since the next 
> station is one you may not have transmitted to for a long time, so your "best 
> rate" information is old.

I wasn't even talking about the stability of the data rate to one destination. I 
was talking about the fact that you may have a 1.3Gb connection to system A (a 
desktop with a -ac 3x3 radio) and a 1Mb connection to machine B (an IoT 802.11b 
thermostat)

trying to do BQL across 3+ orders of magnatude in speed isn't going to work 
wihtout taking the speed into account.

Even if all you do is estimate with the last known speed, you will do better 
than ignorming the speed entirely.

If the wifi can 'return' data to the queue when the transmission fails, it can 
then fetch less data when it 're-transmits' the data at a lower speed.

>  Queueing makes information about the channel older, 
> by binding it too early.  Sending longer frames means retransmitting longer 
> frames when they don't get through, rather than agilely picking a better rate 
> after a few bits.

As I understand wifi, once a transmission starts, it must continue at that same 
data rate, it can't change mid-transmission (and tehre would be no way of 
getting feedback in the middle of a transmission to know that it would need to 
change)

> The MAC protocol really should give the receiver some opportunity to control 
> the rate of the next packet it gets (which it can do because it can measure 
> the channel from the transmitter to itself, by listening to prior 
> transmissions).  Or at least to signal channel changes that might require a 
> new signalling rate.
> 
> This suggests that a transmitter might want to "warn" a receiver that some 
> packets will be coming its way, so the receiver can preemptively change the 
> desired rate.  Thus, perhaps an RTS-CTS like mechanism can be embedded in the 
> MAC protocol, which requires that the device "look ahead" at the packets it 
> might be sending.

the recipient will receive a signal at any data rate, you don't have to tell it 
ahead of time what rate is going to be sent. If it's being sent with a known 
encoding, it will be decoded.

The sender picks the rate based on a number of things

1. what the other end said they could do based on the mode that they are 
connected with (b vs g vs n vs bonded n vs ac vs 2x2 ac etc)

2. what has worked in the past. (with failed transmissions resulting in dropping 
the rate)

there may be other data like last known signal strength in the mix as well.


> On the other hand, that only works if the transmitter deliberately congests 
> itself so that it has a queue built up to look at.

no, the table of associated devices keeps track of things like the last known 
signal strength, connection mode, etc. no congestion needed.

> The tradeoffs are not obvious here at all.  On the other hand, one could do 
> something much simpler - just have the transmitter slow down to the worst-case 
> rate required by any receiving system.

that's 1Mb/sec. This is the rate used for things like SSID broadcasts.

Once a system connects, you know from the connection handshake what speeds could 
work. no need to limit yourself the the minimum that they all can know at that 
point.

> As the number of stations in range gets larger, though, it seems unlikely that 
> "batching" multiple packets to the same destination is a good idea at all - 
> because to achieve that, one must have n_destinations * batch_size chunks of 
> data queued in the system as a whole, and that gets quite large.  I suspect it 
> would be better to find a lower level way to just keep the packets going out 
> as fast as they arrive, so no clog occurs, and to slow down the stuff at the 
> source as quickly as possible.

no, no, no

you are falling into the hardware designer trap that we just talked about :-)

you don't wait for the buffers to fill and always send full buffers, you 
oppertunisticaly send data up to the max size.

you do want to send multiple packets if you have them waiting. Because if you 
can send 10 packets to machine A and 10 packets to machine B in the time that it 
would take to send one packet to A, one packet to B, a second packet to A and a 
second packet to B, you have a substantial win for both A and B at the cost of 
very little latency for either.

If there is so little traffic that sending the packets out one at a time doesn't 
generate any congeston, then good, do that [1]. but when you max out the 
airtime, getting more data through in the same amount of airtime by sending 
larger batches is a win

[1] if you are trying to share the same channel with others, this may be a 
problem as it uses more airtime to send the same amount of data than always 
batching. But this is a case of less than optimal network design ;-)

> [one should also dive into the reason for maintaining variable rates - 
> multipath to a particular destination may require longer symbols for decoding 
> without ISI.  And when multipath is involved, you may have to retransmit at a 
> slower rate. There's usually not much "noise" at the receiver compared to the 
> multipath environment. (one of the reasons why mesh can be a lot better is 
> that shorter distances have much less multipath effect, so you can get higher 
> symbol rates by going multi-hop, and of course higher symbol rates compensate 
> for more airtime occupied by a packet due to repeating).]

distance, interference, noise, etc are all variable in wifi. As a result, you 
need to adapt.

The problem is that the adaptation is sometimes doing the wrong thing.

simlifying things a bit:

If your data doesn't get through at rate A, is the right thing to drop to rate 
A/2 and re-transmit?

If the reason it didn't go through is that the signal is too weak for the rateA 
encoding, then yes.

If the reason it didn't go through is that your transmission was stepped on by 
something you can't hear (and can't hear you), but the recipient can here, then 
slowing down means that you take twice the airtime to get the message through, 
and you now have twice the chance of being stepped on again. Repeat and you 
quickly get to everyone broadcasting at low rates and nothing getting through.


This is the key reason that dense wifi networks 'fall off the cliff' when they 
hit saturation, the backoff that is entirely correct for a weak-signal, low 
usage situations is entirely wrong in dense environments.

David Lang

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 11874 bytes --]


David - I find it interesting that you think I am an idiot.  I design waveforms for radios, and am, among other things, a fully trained electrical engineer with deep understanding of information theory, EM waves, propagation, etc. as well as an Amateur Radio builder focused on building experimental radio network systems in the 5 GHz and 10 GHz Amateur Radio bands.
 
I know a heck of a lot about 802.11 PHY layer and modulation, propagation, etc., and have been measuring the signals in my personal lab, as well as having done so when I was teaching at MIT, working on cooperative network diversity protocols (physical layers for mesh cooperation in digital networks).
 
And I was there with Metcalfe and Boggs when they designed Ethernet's PHY and MAC, and personally worked on the protocol layers in what became the Token Ring standard as well - so I understand the backoff and other issues associated with LANs.  (I wrote an invited paper in IEEE Proceedings "An Introduction to Local Area Networks" that appeared in the same special issue as the Cerf and Kahn paper entitled "A Transmission Control Protocol" that described the first Internet protocol concept..)
 
I guess what I'm saying is not that I'm always correct - no one is, but I would suggest that it's worth considering that I might know a little more than most people about some things - especially the physical and MAC layers of 802.11, but also about the internal electronic design of radio transceivers and digital interfaces to them. From some of your comments below, I think you either misunderstood my point (my fault for not explaining it better) or are misinformed.

There's a lot of "folklore" out there about radio systems and WiFi that is quite wrong, and you seem to be quoting some of it - e.g. the idea that the 1 Mb/s waveform of 802.11b DSSS is somehow more reliable than the lowest-rate OFDM modulations, which is often false.  The 20 MHz-wide M0 modulation with 800ns GI gives 6.2 Mb/s and typically much more reliable than than the 802.11b standard 1 Mb/sec DSSS signals in normal environments, with typical receiver designs. It's not the case that beacon frames are transmitted at 1 Mb/sec. - that is only true when there are 802.11b stations *associated* with the access point (which cannot happen at 5 GHz). Nor is it true that the preamble for ERP frames is wastefully long. The preamble for an ERP (OFDM operation) frame is about 6 microseconds long, except in the odd case on 2.4GHz of compatibility-mode (OFDM-DSSS) operation, where the DSSS preamble is used.   The DSSS preamble is 72 usec. long, because 72 bits at 1 Mb/sec takes that long, but the ERP frame's preamble is much shorter.
 
In any case, my main points were about the fact that "channel estimation" is the key issue in deciding on a modulation to use (and MIMO settings to use), and the problem with that is that channels change characteristics quite quickly indoors! A spinning fan blade can create significant variation in the impulse response over a period of a couple milliseconds.  To do well on channel estimation to pick a high data rate, you need to avoid a backlog in the collection of outbound packets on all stations - which means minimizing queue buildup (even if that means sending shorter packets, getting a higher data rate will minimize channel occupancy).
 
Long frames make congested networks work badly - ideally there would only be one frame ready to go when the current frame is transmitted, but the longer the frame, the more likely more than one station will be ready, and the longer the frames will be (if they are being combined).  That means that the penalty due to, and frequency of, collisions where more than one frame are being sent at the same time grows, wasting airtime with collisions.  That's why CTS/RTS is often a good approach (the CTS/RTS frames are short, so a collision will be less wasteful of airtime).  But due to preamble size, etc., CTS/RTS can't be very short, so an alternative hybrid approach is useful (assume that all stations transmit CTS frames at the same time, you can use the synchronization acquired during the CTS to mitigate the need for a preamble on the packet sent after the RTS).   (One of the papers I did with my student Aggelos Bletsas on Cooperative Diversity uses CTS/RTS in this clever way - to measure the channel while acquiring it).
 
 
 

On Friday, August 7, 2015 6:31pm, "David Lang" <david@lang.hm> said:



> On Fri, 7 Aug 2015, dpreed@reed.com wrote:
> 
> > On Friday, August 7, 2015 4:03pm, "David Lang" <david@lang.hm> said:
> >>
> >
> >> Wifi is the only place I know of where the transmit bit rate is going to
> vary
> >> depending on the next hop address.
> >
> >
> > This is an interesting core issue. The question is whether additional
> > queueing helps or hurts this, and whether the MAC protocol of WiFi deals well
> > or poorly with this issue. It is clear that this is a peculiarly WiFi'ish
> > issue.
> >
> > It's not clear that the best transmit rate remains stable for very long, or
> > even how to predict the "best rate" for the next station since the next
> > station is one you may not have transmitted to for a long time, so your "best
> > rate" information is old.
> 
> I wasn't even talking about the stability of the data rate to one destination. I
> was talking about the fact that you may have a 1.3Gb connection to system A (a
> desktop with a -ac 3x3 radio) and a 1Mb connection to machine B (an IoT 802.11b
> thermostat)
> 
> trying to do BQL across 3+ orders of magnatude in speed isn't going to work
> wihtout taking the speed into account.
> 
> Even if all you do is estimate with the last known speed, you will do better
> than ignorming the speed entirely.
> 
> If the wifi can 'return' data to the queue when the transmission fails, it can
> then fetch less data when it 're-transmits' the data at a lower speed.
> 
> > Queueing makes information about the channel older,
> > by binding it too early. Sending longer frames means retransmitting longer
> > frames when they don't get through, rather than agilely picking a better rate
> > after a few bits.
> 
> As I understand wifi, once a transmission starts, it must continue at that same
> data rate, it can't change mid-transmission (and tehre would be no way of
> getting feedback in the middle of a transmission to know that it would need to
> change)
> 
> > The MAC protocol really should give the receiver some opportunity to control
> > the rate of the next packet it gets (which it can do because it can measure
> > the channel from the transmitter to itself, by listening to prior
> > transmissions). Or at least to signal channel changes that might require a
> > new signalling rate.
> >
> > This suggests that a transmitter might want to "warn" a receiver that some
> > packets will be coming its way, so the receiver can preemptively change the
> > desired rate. Thus, perhaps an RTS-CTS like mechanism can be embedded in the
> > MAC protocol, which requires that the device "look ahead" at the packets it
> > might be sending.
> 
> the recipient will receive a signal at any data rate, you don't have to tell it
> ahead of time what rate is going to be sent. If it's being sent with a known
> encoding, it will be decoded.
> 
> The sender picks the rate based on a number of things
> 
> 1. what the other end said they could do based on the mode that they are
> connected with (b vs g vs n vs bonded n vs ac vs 2x2 ac etc)
> 
> 2. what has worked in the past. (with failed transmissions resulting in dropping
> the rate)
> 
> there may be other data like last known signal strength in the mix as well.
> 
> 
> > On the other hand, that only works if the transmitter deliberately congests
> > itself so that it has a queue built up to look at.
> 
> no, the table of associated devices keeps track of things like the last known
> signal strength, connection mode, etc. no congestion needed.
> 
> > The tradeoffs are not obvious here at all. On the other hand, one could do
> > something much simpler - just have the transmitter slow down to the
> worst-case
> > rate required by any receiving system.
> 
> that's 1Mb/sec. This is the rate used for things like SSID broadcasts.
> 
> Once a system connects, you know from the connection handshake what speeds could
> work. no need to limit yourself the the minimum that they all can know at that
> point.
> 
> > As the number of stations in range gets larger, though, it seems unlikely
> that
> > "batching" multiple packets to the same destination is a good idea at all -
> > because to achieve that, one must have n_destinations * batch_size chunks of
> > data queued in the system as a whole, and that gets quite large. I suspect
> it
> > would be better to find a lower level way to just keep the packets going out
> > as fast as they arrive, so no clog occurs, and to slow down the stuff at the
> > source as quickly as possible.
> 
> no, no, no
> 
> you are falling into the hardware designer trap that we just talked about :-)
> 
> you don't wait for the buffers to fill and always send full buffers, you
> oppertunisticaly send data up to the max size.
> 
> you do want to send multiple packets if you have them waiting. Because if you
> can send 10 packets to machine A and 10 packets to machine B in the time that it
> would take to send one packet to A, one packet to B, a second packet to A and a
> second packet to B, you have a substantial win for both A and B at the cost of
> very little latency for either.
> 
> If there is so little traffic that sending the packets out one at a time doesn't
> generate any congeston, then good, do that [1]. but when you max out the
> airtime, getting more data through in the same amount of airtime by sending
> larger batches is a win
> 
> [1] if you are trying to share the same channel with others, this may be a
> problem as it uses more airtime to send the same amount of data than always
> batching. But this is a case of less than optimal network design ;-)
> 
> > [one should also dive into the reason for maintaining variable rates -
> > multipath to a particular destination may require longer symbols for decoding
> > without ISI. And when multipath is involved, you may have to retransmit at a
> > slower rate. There's usually not much "noise" at the receiver compared to the
> > multipath environment. (one of the reasons why mesh can be a lot better is
> > that shorter distances have much less multipath effect, so you can get higher
> > symbol rates by going multi-hop, and of course higher symbol rates compensate
> > for more airtime occupied by a packet due to repeating).]
> 
> distance, interference, noise, etc are all variable in wifi. As a result, you
> need to adapt.
> 
> The problem is that the adaptation is sometimes doing the wrong thing.
> 
> simlifying things a bit:
> 
> If your data doesn't get through at rate A, is the right thing to drop to rate
> A/2 and re-transmit?
> 
> If the reason it didn't go through is that the signal is too weak for the rateA
> encoding, then yes.
> 
> If the reason it didn't go through is that your transmission was stepped on by
> something you can't hear (and can't hear you), but the recipient can here, then
> slowing down means that you take twice the airtime to get the message through,
> and you now have twice the chance of being stepped on again. Repeat and you
> quickly get to everyone broadcasting at low rates and nothing getting through.
> 
> 
> This is the key reason that dense wifi networks 'fall off the cliff' when they
> hit saturation, the backoff that is entirely correct for a weak-signal, low
> usage situations is entirely wrong in dense environments.
> 
> David Lang
> 

[-- Attachment #2: Type: text/html, Size: 15215 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 13152 bytes --]

I'll reply more later. I don't think you are an idiot, I think you are too 
caught up in the no-olds-barred world of your ham tinkering to take into account 
the realities of the existing wifi reality.

for make-wifi-fast we don't get to start from scratch, there is a huge installed 
base that we can't change and have to interact with. Like the bufferbloat 
effort, we cn find better ways of doing things and try to get them ot there, but 
they have to work seamlessly with the existing equipment and protocols.

also, I tend to give a lot of background to justify my conclusions, not because 
I assume yuo dont know any of it, but for the two reasons that if I have a logic 
flaw or am basing my results on faulty info I can be corrected, and so that 
others in or watching the discussion who don't know these details can be brought 
up to speed and contribute.

David Lang


  On Sat, 8 Aug 2015, dpreed@reed.com wrote:

> Date: Sat, 8 Aug 2015 16:46:05 -0400 (EDT)
> From: dpreed@reed.com
> To: David Lang <david@lang.hm>
> Cc: Jonathan Morton <chromatix99@gmail.com>,
>     cerowrt-devel@lists.bufferbloat.net, make-wifi-fast@lists.bufferbloat.net
> Subject: Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on
>     draft-szigeti-tsvwg-ieee-802-11e
> 
>
> David - I find it interesting that you think I am an idiot.  I design waveforms for radios, and am, among other things, a fully trained electrical engineer with deep understanding of information theory, EM waves, propagation, etc. as well as an Amateur Radio builder focused on building experimental radio network systems in the 5 GHz and 10 GHz Amateur Radio bands.
> 
> I know a heck of a lot about 802.11 PHY layer and modulation, propagation, etc., and have been measuring the signals in my personal lab, as well as having done so when I was teaching at MIT, working on cooperative network diversity protocols (physical layers for mesh cooperation in digital networks).
> 
> And I was there with Metcalfe and Boggs when they designed Ethernet's PHY and MAC, and personally worked on the protocol layers in what became the Token Ring standard as well - so I understand the backoff and other issues associated with LANs.  (I wrote an invited paper in IEEE Proceedings "An Introduction to Local Area Networks" that appeared in the same special issue as the Cerf and Kahn paper entitled "A Transmission Control Protocol" that described the first Internet protocol concept..)
> 
> I guess what I'm saying is not that I'm always correct - no one is, but I would suggest that it's worth considering that I might know a little more than most people about some things - especially the physical and MAC layers of 802.11, but also about the internal electronic design of radio transceivers and digital interfaces to them. From some of your comments below, I think you either misunderstood my point (my fault for not explaining it better) or are misinformed.
>
> There's a lot of "folklore" out there about radio systems and WiFi that is quite wrong, and you seem to be quoting some of it - e.g. the idea that the 1 Mb/s waveform of 802.11b DSSS is somehow more reliable than the lowest-rate OFDM modulations, which is often false.  The 20 MHz-wide M0 modulation with 800ns GI gives 6.2 Mb/s and typically much more reliable than than the 802.11b standard 1 Mb/sec DSSS signals in normal environments, with typical receiver designs. It's not the case that beacon frames are transmitted at 1 Mb/sec. - that is only true when there are 802.11b stations *associated* with the access point (which cannot happen at 5 GHz). Nor is it true that the preamble for ERP frames is wastefully long. The preamble for an ERP (OFDM operation) frame is about 6 microseconds long, except in the odd case on 2.4GHz of compatibility-mode (OFDM-DSSS) operation, where the DSSS preamble is used.   The DSSS preamble is 72 usec. long, because 72 bits at 1 Mb/sec takes that!
  long, but the ERP frame's preamble is much shorter.
> 
> In any case, my main points were about the fact that "channel estimation" is the key issue in deciding on a modulation to use (and MIMO settings to use), and the problem with that is that channels change characteristics quite quickly indoors! A spinning fan blade can create significant variation in the impulse response over a period of a couple milliseconds.  To do well on channel estimation to pick a high data rate, you need to avoid a backlog in the collection of outbound packets on all stations - which means minimizing queue buildup (even if that means sending shorter packets, getting a higher data rate will minimize channel occupancy).
> 
> Long frames make congested networks work badly - ideally there would only be one frame ready to go when the current frame is transmitted, but the longer the frame, the more likely more than one station will be ready, and the longer the frames will be (if they are being combined).  That means that the penalty due to, and frequency of, collisions where more than one frame are being sent at the same time grows, wasting airtime with collisions.  That's why CTS/RTS is often a good approach (the CTS/RTS frames are short, so a collision will be less wasteful of airtime).  But due to preamble size, etc., CTS/RTS can't be very short, so an alternative hybrid approach is useful (assume that all stations transmit CTS frames at the same time, you can use the synchronization acquired during the CTS to mitigate the need for a preamble on the packet sent after the RTS).   (One of the papers I did with my student Aggelos Bletsas on Cooperative Diversity uses CTS/RTS in this clever way - t!
 o measure the channel while acquiring it).
> 
> 
> 
>
> On Friday, August 7, 2015 6:31pm, "David Lang" <david@lang.hm> said:
>
>
>
>> On Fri, 7 Aug 2015, dpreed@reed.com wrote:
>> 
>> > On Friday, August 7, 2015 4:03pm, "David Lang" <david@lang.hm> said:
>> >>
>> >
>> >> Wifi is the only place I know of where the transmit bit rate is going to
>> vary
>> >> depending on the next hop address.
>> >
>> >
>> > This is an interesting core issue. The question is whether additional
>> > queueing helps or hurts this, and whether the MAC protocol of WiFi deals well
>> > or poorly with this issue. It is clear that this is a peculiarly WiFi'ish
>> > issue.
>> >
>> > It's not clear that the best transmit rate remains stable for very long, or
>> > even how to predict the "best rate" for the next station since the next
>> > station is one you may not have transmitted to for a long time, so your "best
>> > rate" information is old.
>> 
>> I wasn't even talking about the stability of the data rate to one destination. I
>> was talking about the fact that you may have a 1.3Gb connection to system A (a
>> desktop with a -ac 3x3 radio) and a 1Mb connection to machine B (an IoT 802.11b
>> thermostat)
>> 
>> trying to do BQL across 3+ orders of magnatude in speed isn't going to work
>> wihtout taking the speed into account.
>> 
>> Even if all you do is estimate with the last known speed, you will do better
>> than ignorming the speed entirely.
>> 
>> If the wifi can 'return' data to the queue when the transmission fails, it can
>> then fetch less data when it 're-transmits' the data at a lower speed.
>> 
>> > Queueing makes information about the channel older,
>> > by binding it too early. Sending longer frames means retransmitting longer
>> > frames when they don't get through, rather than agilely picking a better rate
>> > after a few bits.
>> 
>> As I understand wifi, once a transmission starts, it must continue at that same
>> data rate, it can't change mid-transmission (and tehre would be no way of
>> getting feedback in the middle of a transmission to know that it would need to
>> change)
>> 
>> > The MAC protocol really should give the receiver some opportunity to control
>> > the rate of the next packet it gets (which it can do because it can measure
>> > the channel from the transmitter to itself, by listening to prior
>> > transmissions). Or at least to signal channel changes that might require a
>> > new signalling rate.
>> >
>> > This suggests that a transmitter might want to "warn" a receiver that some
>> > packets will be coming its way, so the receiver can preemptively change the
>> > desired rate. Thus, perhaps an RTS-CTS like mechanism can be embedded in the
>> > MAC protocol, which requires that the device "look ahead" at the packets it
>> > might be sending.
>> 
>> the recipient will receive a signal at any data rate, you don't have to tell it
>> ahead of time what rate is going to be sent. If it's being sent with a known
>> encoding, it will be decoded.
>> 
>> The sender picks the rate based on a number of things
>> 
>> 1. what the other end said they could do based on the mode that they are
>> connected with (b vs g vs n vs bonded n vs ac vs 2x2 ac etc)
>> 
>> 2. what has worked in the past. (with failed transmissions resulting in dropping
>> the rate)
>> 
>> there may be other data like last known signal strength in the mix as well.
>> 
>> 
>> > On the other hand, that only works if the transmitter deliberately congests
>> > itself so that it has a queue built up to look at.
>> 
>> no, the table of associated devices keeps track of things like the last known
>> signal strength, connection mode, etc. no congestion needed.
>> 
>> > The tradeoffs are not obvious here at all. On the other hand, one could do
>> > something much simpler - just have the transmitter slow down to the
>> worst-case
>> > rate required by any receiving system.
>> 
>> that's 1Mb/sec. This is the rate used for things like SSID broadcasts.
>> 
>> Once a system connects, you know from the connection handshake what speeds could
>> work. no need to limit yourself the the minimum that they all can know at that
>> point.
>> 
>> > As the number of stations in range gets larger, though, it seems unlikely
>> that
>> > "batching" multiple packets to the same destination is a good idea at all -
>> > because to achieve that, one must have n_destinations * batch_size chunks of
>> > data queued in the system as a whole, and that gets quite large. I suspect
>> it
>> > would be better to find a lower level way to just keep the packets going out
>> > as fast as they arrive, so no clog occurs, and to slow down the stuff at the
>> > source as quickly as possible.
>> 
>> no, no, no
>> 
>> you are falling into the hardware designer trap that we just talked about :-)
>> 
>> you don't wait for the buffers to fill and always send full buffers, you
>> oppertunisticaly send data up to the max size.
>> 
>> you do want to send multiple packets if you have them waiting. Because if you
>> can send 10 packets to machine A and 10 packets to machine B in the time that it
>> would take to send one packet to A, one packet to B, a second packet to A and a
>> second packet to B, you have a substantial win for both A and B at the cost of
>> very little latency for either.
>> 
>> If there is so little traffic that sending the packets out one at a time doesn't
>> generate any congeston, then good, do that [1]. but when you max out the
>> airtime, getting more data through in the same amount of airtime by sending
>> larger batches is a win
>> 
>> [1] if you are trying to share the same channel with others, this may be a
>> problem as it uses more airtime to send the same amount of data than always
>> batching. But this is a case of less than optimal network design ;-)
>> 
>> > [one should also dive into the reason for maintaining variable rates -
>> > multipath to a particular destination may require longer symbols for decoding
>> > without ISI. And when multipath is involved, you may have to retransmit at a
>> > slower rate. There's usually not much "noise" at the receiver compared to the
>> > multipath environment. (one of the reasons why mesh can be a lot better is
>> > that shorter distances have much less multipath effect, so you can get higher
>> > symbol rates by going multi-hop, and of course higher symbol rates compensate
>> > for more airtime occupied by a packet due to repeating).]
>> 
>> distance, interference, noise, etc are all variable in wifi. As a result, you
>> need to adapt.
>> 
>> The problem is that the adaptation is sometimes doing the wrong thing.
>> 
>> simlifying things a bit:
>> 
>> If your data doesn't get through at rate A, is the right thing to drop to rate
>> A/2 and re-transmit?
>> 
>> If the reason it didn't go through is that the signal is too weak for the rateA
>> encoding, then yes.
>> 
>> If the reason it didn't go through is that your transmission was stepped on by
>> something you can't hear (and can't hear you), but the recipient can here, then
>> slowing down means that you take twice the airtime to get the message through,
>> and you now have twice the chance of being stepped on again. Repeat and you
>> quickly get to everyone broadcasting at low rates and nothing getting through.
>> 
>> 
>> This is the key reason that dense wifi networks 'fall off the cliff' when they
>> hit saturation, the backoff that is entirely correct for a weak-signal, low
>> usage situations is entirely wrong in dense environments.
>> 
>> David Lang
>>

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 1414 bytes --]

The question of whether to aggregate under congested conditions is
controversial, probably because it depends on complex conditions.  There
are arguments both for and against.

It may be worth considering it as a risk/reward tradeoff.  Given N packets
(which for brevity I'll assume are equal MTU sized), the reward is
obviously proportional to N.  Risk however is calculated as probability *
consequence.

Assuming all packets in the aggregate are lost on collision, the risk of
collision scales with L*N, where L is N plus the overhead of the TXOP.
Under that argument, usually you should not aggregate if the probability of
collision is high.

However, if only one packet is lost due to collision with, for example, a
small RTS probe which is not answered, the risk scales with L, which is
sublinear compared to the reward relative to the amount of aggregation
(especially at high data rates where the TXOP overhead is substantial).
Under this assumption, aggregation is usually profitable even with a high
collision probability, and results in overall higher efficiency whether or
not collisions are likely.

This is the difference between the typical 802.11n situation (one checksum
per aggregate) and the mandatory 802.11ac capability of a checksum per
packet.  As long as you also employ RTS/CTS when appropriate, the
possibility of collisions is no longer a reason to avoid aggregating.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 1535 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[David Lang]

On Sun, 9 Aug 2015, Jonathan Morton wrote:

> The question of whether to aggregate under congested conditions is
> controversial, probably because it depends on complex conditions.  There
> are arguments both for and against.
>
> It may be worth considering it as a risk/reward tradeoff.  Given N packets
> (which for brevity I'll assume are equal MTU sized), the reward is
> obviously proportional to N.  Risk however is calculated as probability *
> consequence.
>
> Assuming all packets in the aggregate are lost on collision, the risk of
> collision scales with L*N, where L is N plus the overhead of the TXOP.
> Under that argument, usually you should not aggregate if the probability of
> collision is high.
>
> However, if only one packet is lost due to collision with, for example, a
> small RTS probe which is not answered, the risk scales with L, which is
> sublinear compared to the reward relative to the amount of aggregation
> (especially at high data rates where the TXOP overhead is substantial).
> Under this assumption, aggregation is usually profitable even with a high
> collision probability, and results in overall higher efficiency whether or
> not collisions are likely.
>
> This is the difference between the typical 802.11n situation (one checksum
> per aggregate) and the mandatory 802.11ac capability of a checksum per
> packet.  As long as you also employ RTS/CTS when appropriate, the
> possibility of collisions is no longer a reason to avoid aggregating.

remember that there are stations out there that aren't going to hear your 
RTS/CTS, especially in dense layouts.

Just like wired networks benefit greatly from time-based queues rather than 
packet count based queues, I think that wifi aggregation should not be based on 
packet count (or even aggregate size) but rather the amont of airtime that's 
going to be used (aggregate size * bit rate + overhead)

If the AP can keep track of how many collions it's had/seen over the last X 
time, that can factor in as well. I agree that the 802.11ac ability to only 
loose a packet instead of the entire transmission is a big step forwards, 
unfortunantly there's not that much equipment out there yet that will take 
advantage of it. But it does mean that it's probably worth having two different 
algorithms for the -ac and non -ac endpoints.

Which makes it even more important that the queue logic get information about 
the particular endpoints when deciding what data should be transmitted next.

David Lang

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Mikael Abrahamsson]

On Sun, 9 Aug 2015, David Lang wrote:

> Just like wired networks benefit greatly from time-based queues rather 
> than packet count based queues, I think that wifi aggregation should not 
> be based on packet count (or even aggregate size) but rather the amont 
> of airtime that's going to be used (aggregate size * bit rate + 
> overhead)

I have been involved in 3GPP networking. In for instnace LTE, you can tune 
the scheduler to allocate resources in multiple ways, for instance so that 
each user gets similar amount of transfered data/second, or they get 
access to equal amount of "airtime resources" (which is called TTI 
(https://en.wikipedia.org/wiki/Transmission_Time_Interval), which is time 
slot and frequency divided in LTE (LTE has a lot of subcarriers (OFDM) and 
each subcarrier has 1ms TTIs)).

Personally I favor the "airtime resource fairness", becuase that means a 
station with bad connectivity doesn't harm a station with good 
connectivity. I think it's also intuitive to people that if they have bad 
radio conditions, their network performance goes down. If you give 
everybody the same speed even though some needs a lot more airtime 
resource to attain that speed, that person will never know they're hogging 
resources and will never try to improve the situation.

So if I understood you correctly above, my opinion is in agreement with 
what you wrote.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[David Lang]

On Sun, 9 Aug 2015, Jonathan Morton wrote:

> This is the difference between the typical 802.11n situation (one checksum
> per aggregate) and the mandatory 802.11ac capability of a checksum per
> packet.  As long as you also employ RTS/CTS when appropriate, the
> possibility of collisions is no longer a reason to avoid aggregating.

you say the 'typical' 802.11n situation is one checksum per transmission. Is 
this configurable in OpenWRT? or is it a driver/hardware issue? Does it require 
special client support?

David Lang

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 868 bytes --]

The only mandatory form of aggregation in 'n' is of the one-checksum type,
even though other types are permitted.  The overhead at the data layer is
slightly less, and checksum failure handling on receive is simpler (just
throw the whole thing out and nak it), as is handling the nak at the
transmitter (just retransmit the whole aggregate at the next opportunity).
Most 'n' hardware thus caters only to this lowest common denominator.

I'm not sure whether soft-MAC type hardware (like ath9k) can also support
the more flexible type via driver support - I would hope so - but hard-MAC
hardware almost certainly can't be retrofitted in this way.

However, since 'ac' hardware is required to support individual-checksum
aggregation, a dual-band 'ac' card running on 2.4 GHz will effectively be
an 'n' card with such support, even if it's a hard-MAC.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 999 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[David Lang]

On Fri, 14 Aug 2015, Jonathan Morton wrote:

> The only mandatory form of aggregation in 'n' is of the one-checksum type,
> even though other types are permitted.  The overhead at the data layer is
> slightly less, and checksum failure handling on receive is simpler (just
> throw the whole thing out and nak it), as is handling the nak at the
> transmitter (just retransmit the whole aggregate at the next opportunity).
> Most 'n' hardware thus caters only to this lowest common denominator.
>
> I'm not sure whether soft-MAC type hardware (like ath9k) can also support
> the more flexible type via driver support - I would hope so - but hard-MAC
> hardware almost certainly can't be retrofitted in this way.

if the ath9k driver could support this, would this cause a problem on stock 
clients?

> However, since 'ac' hardware is required to support individual-checksum
> aggregation, a dual-band 'ac' card running on 2.4 GHz will effectively be
> an 'n' card with such support, even if it's a hard-MAC.

right, I'm looking at what I can do to improve things even on non-ac stuff.

David Lang

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

There's a capability handshake during association, so there's no
compatibility problem.  Clients supporting only the single checksum won't
understand multiple-checksum aggregates at all, but the transmitter will
know not to send that type to them.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 322 bytes --]

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[David Lang]

On Sat, 8 Aug 2015, dpreed@reed.com wrote:

> There's a lot of "folklore" out there about radio systems and WiFi that is 
> quite wrong, and you seem to be quoting some of it - e.g. the idea that the 1 
> Mb/s waveform of 802.11b DSSS is somehow more reliable than the lowest-rate 
> OFDM modulations, which is often false.

I agree with you, but my understanding is that the current algorithms always 
assume that slower == more robust transmissions. My point was that in a weak 
signal environement where you have troble decoding individual bits this is true 
(or close enough to true for "failed transmission" -> "retransmit at a slower 
rate" to be a very useful algorithm, but in a congested environment where your 
biggest problem is being stepped on by other tranmissions, this is close to 
suicide instead.

> The 20 MHz-wide M0 modulation with 800ns GI gives 6.2 Mb/s and typically much 
> more reliable than than the 802.11b standard 1 Mb/sec DSSS signals in normal 
> environments, with typical receiver designs.

Interesting and good to know.

> It's not the case that beacon frames are transmitted at 1 Mb/sec. - 
> that is only true when there are 802.11b stations *associated* with the access 
> point (which cannot happen at 5 GHz).

Also interesting. I wish I knew of a way to disable the 802.11b modes on teh 
wndr3800 or wrt1200 series APs. I've seen some documentation online talking 
about it, but it's never worked when I've tried it.

Dave Taht did some experimentation with cerowrt in increasing the broadcase 
rate, but my understanding is that he had to back out those changes because they 
didnt' work well in the real world.

> Nor is it true that the preamble for ERP 
> frames is wastefully long. The preamble for an ERP (OFDM operation) frame is 
> about 6 microseconds long, except in the odd case on 2.4GHz of 
> compatibility-mode (OFDM-DSSS) operation, where the DSSS preamble is used. 
> The DSSS preamble is 72 usec. long, because 72 bits at 1 Mb/sec takes that 
> long, but the ERP frame's preamble is much shorter.

Is compatibility mode needed for 802.11g or 802.11b compatibility?

> In any case, my main points were about the fact that "channel estimation" is 
> the key issue in deciding on a modulation to use (and MIMO settings to use), 
> and the problem with that is that channels change characteristics quite 
> quickly indoors! A spinning fan blade can create significant variation in the 
> impulse response over a period of a couple milliseconds.  To do well on 
> channel estimation to pick a high data rate, you need to avoid a backlog in 
> the collection of outbound packets on all stations - which means minimizing 
> queue buildup (even if that means sending shorter packets, getting a higher 
> data rate will minimize channel occupancy).
> 
> Long frames make congested networks work badly - ideally there would only be 
> one frame ready to go when the current frame is transmitted, but the longer 
> the frame, the more likely more than one station will be ready, and the longer 
> the frames will be (if they are being combined).  That means that the penalty 
> due to, and frequency of, collisions where more than one frame are being sent 
> at the same time grows, wasting airtime with collisions.  That's why CTS/RTS 
> is often a good approach (the CTS/RTS frames are short, so a collision will be 
> less wasteful of airtime).

I run the wireless network for the Scale conference where we get a couple 
thousand people showing up with their equipment. I'm gearing up for next year's 
conference (decideing what I'm going to try, what equipment I'm going to need, 
etc). I would love to get any help you can offer on this, and I'm willing to do 
a fair bit of experimentation and a lot of measurements to see what's happening 
in the real world. I haven't been setting anything to specifically enable 
RTS/CTS in the past.

> But due to preamble size, etc., CTS/RTS can't be 
> very short, so an alternative hybrid approach is useful (assume that all 
> stations transmit CTS frames at the same time, you can use the synchronization 
> acquired during the CTS to mitigate the need for a preamble on the packet sent 
> after the RTS).  (One of the papers I did with my student Aggelos Bletsas on 
> Cooperative Diversity uses CTS/RTS in this clever way - to measure the channel 
> while acquiring it).

how do you get the stations synchronized?

David Lang

> 
> 
>
> On Friday, August 7, 2015 6:31pm, "David Lang" <david@lang.hm> said:
>
>
>
>> On Fri, 7 Aug 2015, dpreed@reed.com wrote:
>> 
>> > On Friday, August 7, 2015 4:03pm, "David Lang" <david@lang.hm> said:
>> >>
>> >
>> >> Wifi is the only place I know of where the transmit bit rate is going to
>> vary
>> >> depending on the next hop address.
>> >
>> >
>> > This is an interesting core issue. The question is whether additional
>> > queueing helps or hurts this, and whether the MAC protocol of WiFi deals well
>> > or poorly with this issue. It is clear that this is a peculiarly WiFi'ish
>> > issue.
>> >
>> > It's not clear that the best transmit rate remains stable for very long, or
>> > even how to predict the "best rate" for the next station since the next
>> > station is one you may not have transmitted to for a long time, so your "best
>> > rate" information is old.
>> 
>> I wasn't even talking about the stability of the data rate to one destination. I
>> was talking about the fact that you may have a 1.3Gb connection to system A (a
>> desktop with a -ac 3x3 radio) and a 1Mb connection to machine B (an IoT 802.11b
>> thermostat)
>> 
>> trying to do BQL across 3+ orders of magnatude in speed isn't going to work
>> wihtout taking the speed into account.
>> 
>> Even if all you do is estimate with the last known speed, you will do better
>> than ignorming the speed entirely.
>> 
>> If the wifi can 'return' data to the queue when the transmission fails, it can
>> then fetch less data when it 're-transmits' the data at a lower speed.
>> 
>> > Queueing makes information about the channel older,
>> > by binding it too early. Sending longer frames means retransmitting longer
>> > frames when they don't get through, rather than agilely picking a better rate
>> > after a few bits.
>> 
>> As I understand wifi, once a transmission starts, it must continue at that same
>> data rate, it can't change mid-transmission (and tehre would be no way of
>> getting feedback in the middle of a transmission to know that it would need to
>> change)
>> 
>> > The MAC protocol really should give the receiver some opportunity to control
>> > the rate of the next packet it gets (which it can do because it can measure
>> > the channel from the transmitter to itself, by listening to prior
>> > transmissions). Or at least to signal channel changes that might require a
>> > new signalling rate.
>> >
>> > This suggests that a transmitter might want to "warn" a receiver that some
>> > packets will be coming its way, so the receiver can preemptively change the
>> > desired rate. Thus, perhaps an RTS-CTS like mechanism can be embedded in the
>> > MAC protocol, which requires that the device "look ahead" at the packets it
>> > might be sending.
>> 
>> the recipient will receive a signal at any data rate, you don't have to tell it
>> ahead of time what rate is going to be sent. If it's being sent with a known
>> encoding, it will be decoded.
>> 
>> The sender picks the rate based on a number of things
>> 
>> 1. what the other end said they could do based on the mode that they are
>> connected with (b vs g vs n vs bonded n vs ac vs 2x2 ac etc)
>> 
>> 2. what has worked in the past. (with failed transmissions resulting in dropping
>> the rate)
>> 
>> there may be other data like last known signal strength in the mix as well.
>> 
>> 
>> > On the other hand, that only works if the transmitter deliberately congests
>> > itself so that it has a queue built up to look at.
>> 
>> no, the table of associated devices keeps track of things like the last known
>> signal strength, connection mode, etc. no congestion needed.
>> 
>> > The tradeoffs are not obvious here at all. On the other hand, one could do
>> > something much simpler - just have the transmitter slow down to the
>> worst-case
>> > rate required by any receiving system.
>> 
>> that's 1Mb/sec. This is the rate used for things like SSID broadcasts.
>> 
>> Once a system connects, you know from the connection handshake what speeds could
>> work. no need to limit yourself the the minimum that they all can know at that
>> point.
>> 
>> > As the number of stations in range gets larger, though, it seems unlikely
>> that
>> > "batching" multiple packets to the same destination is a good idea at all -
>> > because to achieve that, one must have n_destinations * batch_size chunks of
>> > data queued in the system as a whole, and that gets quite large. I suspect
>> it
>> > would be better to find a lower level way to just keep the packets going out
>> > as fast as they arrive, so no clog occurs, and to slow down the stuff at the
>> > source as quickly as possible.
>> 
>> no, no, no
>> 
>> you are falling into the hardware designer trap that we just talked about :-)
>> 
>> you don't wait for the buffers to fill and always send full buffers, you
>> oppertunisticaly send data up to the max size.
>> 
>> you do want to send multiple packets if you have them waiting. Because if you
>> can send 10 packets to machine A and 10 packets to machine B in the time that it
>> would take to send one packet to A, one packet to B, a second packet to A and a
>> second packet to B, you have a substantial win for both A and B at the cost of
>> very little latency for either.
>> 
>> If there is so little traffic that sending the packets out one at a time doesn't
>> generate any congeston, then good, do that [1]. but when you max out the
>> airtime, getting more data through in the same amount of airtime by sending
>> larger batches is a win
>> 
>> [1] if you are trying to share the same channel with others, this may be a
>> problem as it uses more airtime to send the same amount of data than always
>> batching. But this is a case of less than optimal network design ;-)
>> 
>> > [one should also dive into the reason for maintaining variable rates -
>> > multipath to a particular destination may require longer symbols for decoding
>> > without ISI. And when multipath is involved, you may have to retransmit at a
>> > slower rate. There's usually not much "noise" at the receiver compared to the
>> > multipath environment. (one of the reasons why mesh can be a lot better is
>> > that shorter distances have much less multipath effect, so you can get higher
>> > symbol rates by going multi-hop, and of course higher symbol rates compensate
>> > for more airtime occupied by a packet due to repeating).]
>> 
>> distance, interference, noise, etc are all variable in wifi. As a result, you
>> need to adapt.
>> 
>> The problem is that the adaptation is sometimes doing the wrong thing.
>> 
>> simlifying things a bit:
>> 
>> If your data doesn't get through at rate A, is the right thing to drop to rate
>> A/2 and re-transmit?
>> 
>> If the reason it didn't go through is that the signal is too weak for the rateA
>> encoding, then yes.
>> 
>> If the reason it didn't go through is that your transmission was stepped on by
>> something you can't hear (and can't hear you), but the recipient can here, then
>> slowing down means that you take twice the airtime to get the message through,
>> and you now have twice the chance of being stepped on again. Repeat and you
>> quickly get to everyone broadcasting at low rates and nothing getting through.
>> 
>> 
>> This is the key reason that dense wifi networks 'fall off the cliff' when they
>> hit saturation, the backoff that is entirely correct for a weak-signal, low
>> usage situations is entirely wrong in dense environments.
>> 
>> David Lang
>>

Re: [Cerowrt-devel] [Make-wifi-fast] [tsvwg] Comments on draft-szigeti-tsvwg-ieee-802-11e

[Simon Barber]

On 8/7/2015 3:31 PM, David Lang wrote:
> On Fri, 7 Aug 2015, dpreed@reed.com wrote:
>
>> On Friday, August 7, 2015 4:03pm, "David Lang" <david@lang.hm> said:
>>>
>>
>>> Wifi is the only place I know of where the transmit bit rate is 
>>> going to vary
>>> depending on the next hop address.
>>
>>
>> This is an interesting core issue.  The question is whether 
>> additional queueing helps or hurts this, and whether the MAC protocol 
>> of WiFi deals well or poorly with this issue.  It is clear that this 
>> is a peculiarly WiFi'ish issue.
>>
>> It's not clear that the best transmit rate remains stable for very 
>> long, or even how to predict the "best rate" for the next station 
>> since the next station is one you may not have transmitted to for a 
>> long time, so your "best rate" information is old.
>
> I wasn't even talking about the stability of the data rate to one 
> destination. I was talking about the fact that you may have a 1.3Gb 
> connection to system A (a desktop with a -ac 3x3 radio) and a 1Mb 
> connection to machine B (an IoT 802.11b thermostat)
>
> trying to do BQL across 3+ orders of magnatude in speed isn't going to 
> work wihtout taking the speed into account.
>
> Even if all you do is estimate with the last known speed, you will do 
> better than ignorming the speed entirely.
I have been a proponent of Time Queue Limits (TQL) for wifi for a long time!

Simon

>
> If the wifi can 'return' data to the queue when the transmission 
> fails, it can then fetch less data when it 're-transmits' the data at 
> a lower speed.
>
>>  Queueing makes information about the channel older, by binding it 
>> too early.  Sending longer frames means retransmitting longer frames 
>> when they don't get through, rather than agilely picking a better 
>> rate after a few bits.
>
> As I understand wifi, once a transmission starts, it must continue at 
> that same data rate, it can't change mid-transmission (and tehre would 
> be no way of getting feedback in the middle of a transmission to know 
> that it would need to change)
>
>> The MAC protocol really should give the receiver some opportunity to 
>> control the rate of the next packet it gets (which it can do because 
>> it can measure the channel from the transmitter to itself, by 
>> listening to prior transmissions).  Or at least to signal channel 
>> changes that might require a new signalling rate.
>>
>> This suggests that a transmitter might want to "warn" a receiver that 
>> some packets will be coming its way, so the receiver can preemptively 
>> change the desired rate.  Thus, perhaps an RTS-CTS like mechanism can 
>> be embedded in the MAC protocol, which requires that the device "look 
>> ahead" at the packets it might be sending.
>
> the recipient will receive a signal at any data rate, you don't have 
> to tell it ahead of time what rate is going to be sent. If it's being 
> sent with a known encoding, it will be decoded.
>
> The sender picks the rate based on a number of things
>
> 1. what the other end said they could do based on the mode that they 
> are connected with (b vs g vs n vs bonded n vs ac vs 2x2 ac etc)
>
> 2. what has worked in the past. (with failed transmissions resulting 
> in dropping the rate)
>
> there may be other data like last known signal strength in the mix as 
> well.
>
>
>> On the other hand, that only works if the transmitter deliberately 
>> congests itself so that it has a queue built up to look at.
>
> no, the table of associated devices keeps track of things like the 
> last known signal strength, connection mode, etc. no congestion needed.
>
>> The tradeoffs are not obvious here at all.  On the other hand, one 
>> could do something much simpler - just have the transmitter slow down 
>> to the worst-case rate required by any receiving system.
>
> that's 1Mb/sec. This is the rate used for things like SSID broadcasts.
>
> Once a system connects, you know from the connection handshake what 
> speeds could work. no need to limit yourself the the minimum that they 
> all can know at that point.
>
>> As the number of stations in range gets larger, though, it seems 
>> unlikely that "batching" multiple packets to the same destination is 
>> a good idea at all - because to achieve that, one must have 
>> n_destinations * batch_size chunks of data queued in the system as a 
>> whole, and that gets quite large.  I suspect it would be better to 
>> find a lower level way to just keep the packets going out as fast as 
>> they arrive, so no clog occurs, and to slow down the stuff at the 
>> source as quickly as possible.
>
> no, no, no
>
> you are falling into the hardware designer trap that we just talked 
> about :-)
>
> you don't wait for the buffers to fill and always send full buffers, 
> you oppertunisticaly send data up to the max size.
>
> you do want to send multiple packets if you have them waiting. Because 
> if you can send 10 packets to machine A and 10 packets to machine B in 
> the time that it would take to send one packet to A, one packet to B, 
> a second packet to A and a second packet to B, you have a substantial 
> win for both A and B at the cost of very little latency for either.
>
> If there is so little traffic that sending the packets out one at a 
> time doesn't generate any congeston, then good, do that [1]. but when 
> you max out the airtime, getting more data through in the same amount 
> of airtime by sending larger batches is a win
>
> [1] if you are trying to share the same channel with others, this may 
> be a problem as it uses more airtime to send the same amount of data 
> than always batching. But this is a case of less than optimal network 
> design ;-)
>
>> [one should also dive into the reason for maintaining variable rates 
>> - multipath to a particular destination may require longer symbols 
>> for decoding without ISI.  And when multipath is involved, you may 
>> have to retransmit at a slower rate. There's usually not much "noise" 
>> at the receiver compared to the multipath environment. (one of the 
>> reasons why mesh can be a lot better is that shorter distances have 
>> much less multipath effect, so you can get higher symbol rates by 
>> going multi-hop, and of course higher symbol rates compensate for 
>> more airtime occupied by a packet due to repeating).]
>
> distance, interference, noise, etc are all variable in wifi. As a 
> result, you need to adapt.
>
> The problem is that the adaptation is sometimes doing the wrong thing.
>
> simlifying things a bit:
>
> If your data doesn't get through at rate A, is the right thing to drop 
> to rate A/2 and re-transmit?
>
> If the reason it didn't go through is that the signal is too weak for 
> the rateA encoding, then yes.
>
> If the reason it didn't go through is that your transmission was 
> stepped on by something you can't hear (and can't hear you), but the 
> recipient can here, then slowing down means that you take twice the 
> airtime to get the message through, and you now have twice the chance 
> of being stepped on again. Repeat and you quickly get to everyone 
> broadcasting at low rates and nothing getting through.
>
>
> This is the key reason that dense wifi networks 'fall off the cliff' 
> when they hit saturation, the backoff that is entirely correct for a 
> weak-signal, low usage situations is entirely wrong in dense 
> environments.
>
> David Lang
> _______________________________________________
> Make-wifi-fast mailing list
> Make-wifi-fast@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/make-wifi-fast