From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [148.163.129.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B2A0B3B29D for ; Tue, 13 Sep 2022 11:58:17 -0400 (EDT) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.7.67.131]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id AEBEE2007B; Tue, 13 Sep 2022 15:58:15 +0000 (UTC) Received: from mail3.candelatech.com (mail2.candelatech.com [208.74.158.173]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 5E945980068; Tue, 13 Sep 2022 15:58:15 +0000 (UTC) Received: from [192.168.100.195] (50-251-239-81-static.hfc.comcastbusiness.net [50.251.239.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail3.candelatech.com (Postfix) with ESMTPSA id F2E4B13C2B0; Tue, 13 Sep 2022 08:58:14 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 mail3.candelatech.com F2E4B13C2B0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=candelatech.com; s=default; t=1663084695; bh=+MGHXd+bzUcv0gSW/J8+K62dr6aYFXin2V13V5mZQ1c=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=qrDH7wz53L90+pacwqUIZBw2mmfctgt+HNzQowugVFjVhXZ7V+UDRWLucMAHX9rua +Ioe7QzB93ggp8IMPNX7n9DAV0GoylOKf/525LPfOW49Er42nUp1j8plKVLU51x1by YVNVvBB59pIaPCfrgDxzs79pjjhX8JV/J6PAldVQ= To: Dave Taht Cc: Dave Taht via Starlink References: From: Ben Greear Organization: Candela Technologies Message-ID: Date: Tue, 13 Sep 2022 08:58:14 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-MDID: 1663084696-dd6L938RXBgf Subject: Re: [Starlink] RFC: Latency test case text and example report. X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2022 15:58:17 -0000 On 9/13/22 8:39 AM, Dave Taht wrote: > hey, ben, I'm curious if this test made it into TR398? Is it possible > to setup some of this or parts of TR398 to run over starlink? > > I'm also curious as to if any commercial ax APs were testing out > better than when you tested about this time last year. I've just gone > through 9 months of pure hell getting openwrt's implementation of the > mt76 and ath10k to multiplex a lot better, and making some forward > progress again ( > https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/830 ) > and along the way ran into new problems with location scanning and > apple's airdrop.... > > but I just got a batch of dismal results back from the ax210 and > mt79... tell me that there's an AP shipping from someone that scales a > bit better? Lie if you must... An mtk7915 based AP that is running recent owrt did better than others. http://www.candelatech.com/examples/TR-398v2-2022-06-05-08-28-57-6.2.6-latency-virt-sta-new-atf-c/ The test was at least tentatively accepted into tr398v3, but I don't think anyone other than ourselves has implemented or tested it. I think the pass/fail will need to be adjusted to make it easier to pass. Some APs were showing multiple seconds of latency, so maybe a few hundred MS is really OK. The test should be able to run over WAN if desired, though it would take a bit of extra setup to place an upstream LANforge endpoint on a cloud VM. If someone@spacex wants to run this test, please contact me off list and we can help make it happen. Thanks, Ben > > On Sun, Sep 26, 2021 at 2:59 PM Ben Greear wrote: >> >> I have been working on a latency test that I hope can be included in the TR398 issue 3 >> document. It is based somewhat on Toke's paper on buffer bloat and latency testing, >> with a notable change that I'm doing this on 32 stations in part of the test. >> >> I implemented this test case, and an example run against an enterprise grade AX AP >> is here. There could still be bugs in my implementation, but I think it is at least >> close to correct: >> >> http://www.candelatech.com/examples/tr398v3-latency-report.pdf >> >> TLDR: Runs OK with single station, but sees 1+second one-way latency with 32 stations and high load, and UDP often >> is not able to see any throughput at all, I guess due to too many packets being lost >> or something. I hope to run against some cutting-edge OpenWRT APs soon. >> >> One note on TCP Latency: This is time to transmit a 64k chunk of data over TCP, not a single >> frame. >> >> My testbed used 32 Intel ax210 radios as stations in this test. >> >> I am interested in feedback from this list if anyone has opinions. >> >> Here is text of the test case: >> >> The Latency test intends to verify latency under low, high, and maximum AP traffic load, with >> 1 and 32 stations. Traffic load is 4 bi-directional TCP streams for each station, plus a >> low speed UDP connection to probe latency. >> >> Test Procedure >> >> DUT should be configured for 20Mhz on 2.4Ghz and 80Mhz on 5Ghz and stations should use >> two spatial streams. >> >> 1: For each combination of: 2.4Ghz N, 5Ghz AC, 2.4Ghz AX, 5Ghz AX: >> >> 2: Configure attenuators to emulate 2-meter distance between stations and AP. >> >> 3: Create 32 stations and allow one to associate with the DUT. The other 31 are admin-down. >> >> 4: Create AP to Station (download) TCP stream, and run for 120 seconds, recoard >> throughput as 'maximum_load'. Stop this connection. >> >> 5: Calculate offered_load as 1% of maximum_load. >> >> 6: Create 4 TCP streams on each active station, each configured for Upload and Download rate of >> offered_load / (4 * active_station_count * 2). >> >> 6: Create 1 UDP stream on each active station, configured for 56kbps traffic Upload and 56kbps traffic Download. >> >> 7: Start all TCP and UDP connections. Wait 30 seconds to let traffic settle. >> >> 8: Every 10 seconds for 120 seconds, record one-way download latency over the last 10 seconds for each UDP connection. Depending on test >> equipment features, this may mean you need to start/stop the UDP every 10 seconds or clear the UDP connection >> counters. >> >> 9: Calculate offered_load as 70% of maximum_load, and repeat steps 6 - 9 inclusive. >> >> 10: Calculate offered_load as 125% of maximum_load, and repeat steps 6 - 9 inclusive. >> >> 11: Allow the other 31 stations to associate, and repeat steps 5 - 11 inclusive with all 32 stations active. >> >> >> Pass/Fail Criteria >> >> 1: For each test configuration running at 1% of maximum load: Average of all UDP latency samples must be less than 10ms. >> 2: For each test configuration running at 1% of maximum load: Maximum of all UDP latency samples must be less than 20ms. >> 3: For each test configuration running at 70% of maximum load: Average of all UDP latency samples must be less than 20ms. >> 4: For each test configuration running at 70% of maximum load: Maximum of all UDP latency samples must be less than 40ms. >> 5: For each test configuration running at 125% of maximum load: Average of all UDP latency samples must be less than 50ms. >> 6: For each test configuration running at 125% of maximum load: Maximum of all UDP latency samples must be less than 100ms. >> 7: For each test configuration: Each UDP connection upload throughput must be at least 1/2 of requested UDP speed for final 10-second test interval. >> 8: For each test configuration: Each UDP connection download throughput must be at least 1/2 of requested UDP speed for final 10-second test interval. >> >> >> -- >> Ben Greear >> Candela Technologies Inc http://www.candelatech.com >> _______________________________________________ >> Starlink mailing list >> Starlink@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/starlink > > > -- Ben Greear Candela Technologies Inc http://www.candelatech.com