From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [67.231.154.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 7EF633B2A4 for ; Sun, 26 Sep 2021 17:59:56 -0400 (EDT) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.110.50.13]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 44AAE2005E for ; Sun, 26 Sep 2021 21:59:55 +0000 (UTC) Received: from mail3.candelatech.com (mail2.candelatech.com [208.74.158.173]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 0A22C44006D for ; Sun, 26 Sep 2021 21:59:54 +0000 (UTC) Received: from [192.168.1.115] (unknown [206.214.234.210]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail3.candelatech.com (Postfix) with ESMTPSA id 56C5A13C2B0 for ; Sun, 26 Sep 2021 14:59:54 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 mail3.candelatech.com 56C5A13C2B0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=candelatech.com; s=default; t=1632693594; bh=hApE9/+VHg8Hg1qhGcZdjqaIFgilM473jLEUD+06FGE=; h=To:From:Subject:Date:From; b=VnnKwSO9KQhxTsrV2lZWqys6rF42UmgbRSB1U1vd2kZU3H4gvYExxWvTN00IcST1p 9gw3Rn84PQSamKZ2Xn3kAx0qCAMq7dIz1rs6DCrxBvKA28VXlon/DEYNTfqRoJcGfG 1vH5QEFYeqLIjGsTqTNlw80U3xh2De8KOboetA3U= To: starlink@lists.bufferbloat.net From: Ben Greear Organization: Candela Technologies Message-ID: Date: Sun, 26 Sep 2021 14:59:53 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-MW Content-Transfer-Encoding: 7bit X-MDID: 1632693595-0eiQFezhJU4k Subject: [Starlink] RFC: Latency test case text and example report. X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Sep 2021 21:59:56 -0000 I have been working on a latency test that I hope can be included in the TR398 issue 3 document. It is based somewhat on Toke's paper on buffer bloat and latency testing, with a notable change that I'm doing this on 32 stations in part of the test. I implemented this test case, and an example run against an enterprise grade AX AP is here. There could still be bugs in my implementation, but I think it is at least close to correct: http://www.candelatech.com/examples/tr398v3-latency-report.pdf TLDR: Runs OK with single station, but sees 1+second one-way latency with 32 stations and high load, and UDP often is not able to see any throughput at all, I guess due to too many packets being lost or something. I hope to run against some cutting-edge OpenWRT APs soon. One note on TCP Latency: This is time to transmit a 64k chunk of data over TCP, not a single frame. My testbed used 32 Intel ax210 radios as stations in this test. I am interested in feedback from this list if anyone has opinions. Here is text of the test case: The Latency test intends to verify latency under low, high, and maximum AP traffic load, with 1 and 32 stations. Traffic load is 4 bi-directional TCP streams for each station, plus a low speed UDP connection to probe latency. Test Procedure DUT should be configured for 20Mhz on 2.4Ghz and 80Mhz on 5Ghz and stations should use two spatial streams. 1: For each combination of: 2.4Ghz N, 5Ghz AC, 2.4Ghz AX, 5Ghz AX: 2: Configure attenuators to emulate 2-meter distance between stations and AP. 3: Create 32 stations and allow one to associate with the DUT. The other 31 are admin-down. 4: Create AP to Station (download) TCP stream, and run for 120 seconds, recoard throughput as 'maximum_load'. Stop this connection. 5: Calculate offered_load as 1% of maximum_load. 6: Create 4 TCP streams on each active station, each configured for Upload and Download rate of offered_load / (4 * active_station_count * 2). 6: Create 1 UDP stream on each active station, configured for 56kbps traffic Upload and 56kbps traffic Download. 7: Start all TCP and UDP connections. Wait 30 seconds to let traffic settle. 8: Every 10 seconds for 120 seconds, record one-way download latency over the last 10 seconds for each UDP connection. Depending on test equipment features, this may mean you need to start/stop the UDP every 10 seconds or clear the UDP connection counters. 9: Calculate offered_load as 70% of maximum_load, and repeat steps 6 - 9 inclusive. 10: Calculate offered_load as 125% of maximum_load, and repeat steps 6 - 9 inclusive. 11: Allow the other 31 stations to associate, and repeat steps 5 - 11 inclusive with all 32 stations active. Pass/Fail Criteria 1: For each test configuration running at 1% of maximum load: Average of all UDP latency samples must be less than 10ms. 2: For each test configuration running at 1% of maximum load: Maximum of all UDP latency samples must be less than 20ms. 3: For each test configuration running at 70% of maximum load: Average of all UDP latency samples must be less than 20ms. 4: For each test configuration running at 70% of maximum load: Maximum of all UDP latency samples must be less than 40ms. 5: For each test configuration running at 125% of maximum load: Average of all UDP latency samples must be less than 50ms. 6: For each test configuration running at 125% of maximum load: Maximum of all UDP latency samples must be less than 100ms. 7: For each test configuration: Each UDP connection upload throughput must be at least 1/2 of requested UDP speed for final 10-second test interval. 8: For each test configuration: Each UDP connection download throughput must be at least 1/2 of requested UDP speed for final 10-second test interval. -- Ben Greear Candela Technologies Inc http://www.candelatech.com