From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ej1-x636.google.com (mail-ej1-x636.google.com [IPv6:2a00:1450:4864:20::636]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 48A0C3B29D for ; Tue, 13 Sep 2022 12:12:39 -0400 (EDT) Received: by mail-ej1-x636.google.com with SMTP id y3so28756210ejc.1 for ; Tue, 13 Sep 2022 09:12:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date; bh=2NebpaVEwgzMXrcbWZqFt/0RDKHgN8hu42nc6olePgw=; b=phz/PIPjx84ckFTDmhs85LX3nMdhbg4sbitvh6A5x5ugfxY5Y8+exYRTBHras2Qk3p fnSvyNPnvtsUFwaNCSGJ6t/radwV/9H188D9kZ/7y8PTC+G8FzAz/logi7q+t1bIFaol XSchT37b7Wx0bpm/MZdVl0qeao9zAVoTY1H+KN3QtkxChUa7fHZPWtGCtZm1YMIs1V6c FBBMf9hJojDzIIPqzC1BEkDmsVtIs6aYcBEHWnxDvToKmZg5l63kkkRrIqEMz781SvX7 pw7qkQZI7ezu1dla6yqfNxY8n77/chJkkwbXI7ngVSk7mDGdJUld0EYH2tDTr26LB2CU maXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date; bh=2NebpaVEwgzMXrcbWZqFt/0RDKHgN8hu42nc6olePgw=; b=ulYM0ETATs2o1dr8+DbPBt300LIUTc/L2MLrtTNZ6TfiVLrzgbnvQv6ZDpsKiTJjsJ ggK6RaHnfbE+KsnAunYfKkAIHmD5oGG2zRtMzSGIFHti8BrQ2xlNTEOYazZi2i4HRi1c 1XSo8WAgX1/Kgjarl53qq/QXfrk8U4YkavD3MNgzNyWNXw/jnC+NToYEvSnSsZ6qEqhW JnD0kB6VrD3a3HTMfsxkYPFnG7UTjBmfBC0i9wxjEsTpkPfjrLLfAcNLwhXkare18+qf Xmj5fsi64Sd+RC/miW/zCM555NwdphG5PlFTuGInGSBUDayNFvNsWWpcsOtZxKd22g/G kvsg== X-Gm-Message-State: ACgBeo2FqbeVT5gzaJkvQTsx0yritSdzRM4mGDItzu6GpGnizMIovdir K9RVKKYYnmCVK1Ero7TsRCab5P1514i0AvTGllNbNIf0vfk= X-Google-Smtp-Source: AA6agR6UdHZZDVOWznKJW/zDezOqQI6qSw+KqDyrP2hKhTeOUnpnYnzss/fry4zDjccAYyOpC5VkVYFFPHfFDYAEfsI= X-Received: by 2002:a17:907:2bf8:b0:770:837a:e3b8 with SMTP id gv56-20020a1709072bf800b00770837ae3b8mr21356573ejc.562.1663085558158; Tue, 13 Sep 2022 09:12:38 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Tue, 13 Sep 2022 09:12:26 -0700 Message-ID: To: Ben Greear Cc: Dave Taht via Starlink Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Starlink] RFC: Latency test case text and example report. X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Sep 2022 16:12:39 -0000 On Tue, Sep 13, 2022 at 8:58 AM Ben Greear wrote: > > On 9/13/22 8:39 AM, Dave Taht wrote: > > hey, ben, I'm curious if this test made it into TR398? Is it possible > > to setup some of this or parts of TR398 to run over starlink? > > > > I'm also curious as to if any commercial ax APs were testing out > > better than when you tested about this time last year. I've just gone > > through 9 months of pure hell getting openwrt's implementation of the > > mt76 and ath10k to multiplex a lot better, and making some forward > > progress again ( > > https://forum.openwrt.org/t/aql-and-the-ath10k-is-lovely/59002/830 ) > > and along the way ran into new problems with location scanning and > > apple's airdrop.... > > > > but I just got a batch of dismal results back from the ax210 and > > mt79... tell me that there's an AP shipping from someone that scales a > > bit better? Lie if you must... > > An mtk7915 based AP that is running recent owrt did better than others. > > http://www.candelatech.com/examples/TR-398v2-2022-06-05-08-28-57-6.2.6-la= tency-virt-sta-new-atf-c/ I wanted to be happy, but... tcp... http://www.candelatech.com/examples/TR-398v2-2022-06-05-08-28-57-6.2.6-late= ncy-virt-sta-new-atf-c/chart-31.png what's the chipset driving these tests nowadays? > The test was at least tentatively accepted into tr398v3, but I don't thin= k anyone other than ourselves has implemented > or tested it. I think the pass/fail will need to be adjusted to make it = easier to pass. Some APs were showing > multiple seconds of latency, so maybe a few hundred MS is really OK. > > The test should be able to run over WAN if desired, though it would take = a bit > of extra setup to place an upstream LANforge endpoint on a cloud VM. > > If someone@spacex wants to run this test, please contact me off list and = we can help > make it happen. > > Thanks, > Ben > > > > > On Sun, Sep 26, 2021 at 2:59 PM Ben Greear wr= ote: > >> > >> I have been working on a latency test that I hope can be included in t= he TR398 issue 3 > >> document. It is based somewhat on Toke's paper on buffer bloat and la= tency testing, > >> with a notable change that I'm doing this on 32 stations in part of th= e test. > >> > >> I implemented this test case, and an example run against an enterprise= grade AX AP > >> is here. There could still be bugs in my implementation, but I think = it is at least > >> close to correct: > >> > >> http://www.candelatech.com/examples/tr398v3-latency-report.pdf > >> > >> TLDR: Runs OK with single station, but sees 1+second one-way latency = with 32 stations and high load, and UDP often > >> is not able to see any throughput at all, I guess due to too many = packets being lost > >> or something. I hope to run against some cutting-edge OpenWRT APs= soon. > >> > >> One note on TCP Latency: This is time to transmit a 64k chunk of data= over TCP, not a single > >> frame. > >> > >> My testbed used 32 Intel ax210 radios as stations in this test. > >> > >> I am interested in feedback from this list if anyone has opinions. > >> > >> Here is text of the test case: > >> > >> The Latency test intends to verify latency under low, high, and maximu= m AP traffic load, with > >> 1 and 32 stations. Traffic load is 4 bi-directional TCP streams for ea= ch station, plus a > >> low speed UDP connection to probe latency. > >> > >> Test Procedure > >> > >> DUT should be configured for 20Mhz on 2.4Ghz and 80Mhz on 5Ghz and sta= tions should use > >> two spatial streams. > >> > >> 1: For each combination of: 2.4Ghz N, 5Ghz AC, 2.4Ghz AX, 5Ghz AX: > >> > >> 2: Configure attenuators to emulate 2-meter distance between stations = and AP. > >> > >> 3: Create 32 stations and allow one to associate with the DUT. The ot= her 31 are admin-down. > >> > >> 4: Create AP to Station (download) TCP stream, and run for 120 seconds= , recoard > >> throughput as 'maximum_load'. Stop this connection. > >> > >> 5: Calculate offered_load as 1% of maximum_load. > >> > >> 6: Create 4 TCP streams on each active station, each configured for Up= load and Download rate of > >> offered_load / (4 * active_station_count * 2). > >> > >> 6: Create 1 UDP stream on each active station, configured for 56kbps t= raffic Upload and 56kbps traffic Download. > >> > >> 7: Start all TCP and UDP connections. Wait 30 seconds to let traffic = settle. > >> > >> 8: Every 10 seconds for 120 seconds, record one-way download latency o= ver the last 10 seconds for each UDP connection. Depending on test > >> equipment features, this may mean you need to start/stop the UDP = every 10 seconds or clear the UDP connection > >> counters. > >> > >> 9: Calculate offered_load as 70% of maximum_load, and repeat steps 6 -= 9 inclusive. > >> > >> 10: Calculate offered_load as 125% of maximum_load, and repeat steps 6= - 9 inclusive. > >> > >> 11: Allow the other 31 stations to associate, and repeat steps 5 - 11 = inclusive with all 32 stations active. > >> > >> > >> Pass/Fail Criteria > >> > >> 1: For each test configuration running at 1% of maximum load: Average= of all UDP latency samples must be less than 10ms. > >> 2: For each test configuration running at 1% of maximum load: Maximum= of all UDP latency samples must be less than 20ms. > >> 3: For each test configuration running at 70% of maximum load: Averag= e of all UDP latency samples must be less than 20ms. > >> 4: For each test configuration running at 70% of maximum load: Maximu= m of all UDP latency samples must be less than 40ms. > >> 5: For each test configuration running at 125% of maximum load: Avera= ge of all UDP latency samples must be less than 50ms. > >> 6: For each test configuration running at 125% of maximum load: Maxim= um of all UDP latency samples must be less than 100ms. > >> 7: For each test configuration: Each UDP connection upload throughput = must be at least 1/2 of requested UDP speed for final 10-second test interv= al. > >> 8: For each test configuration: Each UDP connection download throughpu= t must be at least 1/2 of requested UDP speed for final 10-second test inte= rval. > >> > >> > >> -- > >> Ben Greear > >> Candela Technologies Inc http://www.candelatech.com > >> _______________________________________________ > >> Starlink mailing list > >> Starlink@lists.bufferbloat.net > >> https://lists.bufferbloat.net/listinfo/starlink > > > > > > > > > -- > Ben Greear > Candela Technologies Inc http://www.candelatech.com > --=20 FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_code= l/ Dave T=C3=A4ht CEO, TekLibre, LLC