[Starlink] RFC: Latency test case text and example report.

[Ben Greear]

I have been working on a latency test that I hope can be included in the TR398 issue 3
document.  It is based somewhat on Toke's paper on buffer bloat and latency testing,
with a notable change that I'm doing this on 32 stations in part of the test.

I implemented this test case, and an example run against an enterprise grade AX AP
is here.  There could still be bugs in my implementation, but I think it is at least
close to correct:

http://www.candelatech.com/examples/tr398v3-latency-report.pdf

TLDR:  Runs OK with single station, but sees 1+second one-way latency with 32 stations and high load, and UDP often
   is not able to see any throughput at all, I guess due to too many packets being lost
   or something.  I hope to run against some cutting-edge OpenWRT APs soon.

One note on TCP Latency:  This is time to transmit a 64k chunk of data over TCP, not a single
frame.

My testbed used 32 Intel ax210 radios as stations in this test.

I am interested in feedback from this list if anyone has opinions.

Here is text of the test case:

The Latency test intends to verify latency under low, high, and maximum AP traffic load, with
1 and 32 stations. Traffic load is 4 bi-directional TCP streams for each station, plus a
low speed UDP connection to probe latency.

Test Procedure

DUT should be configured for 20Mhz on 2.4Ghz and 80Mhz on 5Ghz and stations should use
two spatial streams.

1: For each combination of:  2.4Ghz N, 5Ghz AC, 2.4Ghz AX, 5Ghz AX:

2: Configure attenuators to emulate 2-meter distance between stations and AP.

3: Create 32 stations and allow one to associate with the DUT.  The other 31 are admin-down.

4: Create AP to Station (download) TCP stream, and run for 120 seconds, recoard
    throughput as 'maximum_load'.  Stop this connection.

5: Calculate offered_load as 1% of maximum_load.

6: Create 4 TCP streams on each active station, each configured for Upload and Download rate of
    offered_load / (4 * active_station_count * 2).

6: Create 1 UDP stream on each active station, configured for 56kbps traffic Upload and 56kbps traffic Download.

7: Start all TCP and UDP connections.  Wait 30 seconds to let traffic settle.

8: Every 10 seconds for 120 seconds, record one-way download latency over the last 10 seconds for each UDP connection.  Depending on test
    equipment features, this may mean you need to start/stop the UDP every 10 seconds or clear the UDP connection
    counters.

9: Calculate offered_load as 70% of maximum_load, and repeat steps 6 - 9 inclusive.

10: Calculate offered_load as 125% of maximum_load, and repeat steps 6 - 9 inclusive.

11: Allow the other 31 stations to associate, and repeat steps 5 - 11 inclusive with all 32 stations active.


Pass/Fail Criteria

1: For each test configuration running at 1% of maximum load:  Average of all UDP latency samples must be less than 10ms.
2: For each test configuration running at 1% of maximum load:  Maximum of all UDP latency samples must be less than 20ms.
3: For each test configuration running at 70% of maximum load:  Average of all UDP latency samples must be less than 20ms.
4: For each test configuration running at 70% of maximum load:  Maximum of all UDP latency samples must be less than 40ms.
5: For each test configuration running at 125% of maximum load:  Average of all UDP latency samples must be less than 50ms.
6: For each test configuration running at 125% of maximum load:  Maximum of all UDP latency samples must be less than 100ms.
7: For each test configuration: Each UDP connection upload throughput must be at least 1/2 of requested UDP speed for final 10-second test interval.
8: For each test configuration: Each UDP connection download throughput must be at least 1/2 of requested UDP speed for final 10-second test interval.


-- 
Ben Greear <greearb at candelatech.com>
Candela Technologies Inc  http://www.candelatech.com