From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-x232.google.com (mail-lj1-x232.google.com [IPv6:2a00:1450:4864:20::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id EB2CB3CB39 for ; Thu, 7 Dec 2023 11:14:41 -0500 (EST) Received: by mail-lj1-x232.google.com with SMTP id 38308e7fff4ca-2c9f572c4c5so13976871fa.2 for ; Thu, 07 Dec 2023 08:14:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701965680; x=1702570480; darn=lists.bufferbloat.net; h=content-transfer-encoding:to:subject:message-id:date:from :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=HDx58ntbVeCa9GWKdrj+e68QUWs6uF1cev715X4SR2I=; b=JMFEVAe8ZSXBNBTtOv1nBC5Y4lhEEcXC+/G+Mob2aNsDbrrygCvbt06M8dnldjjbxP Ow6ZXLe8EkHCFtl2uJWVGBpRgniOR/7PLkLP1NoMbpHVrcDGOg0hWbHF6yUf/vQd1O/d JdrK/HNMBC9tUEQWXZOGWI+wrtko1QYO/l3xKm6mFR4f+wp9ec61TdbJgEKqBLHWPY47 4/BzNBGmwJpD+6U+0z04s8aYPLaG7VMNvzDEXFt7Oqh8SoScmqDJ3kUXgMMxQsMGhQIq PvHhaSbxfcwI0URHsjIypkGJOkJDV6BArJ/egZgc5zMS+QMwMgtHfYypI3SS7hF1U8g0 q0ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701965680; x=1702570480; h=content-transfer-encoding:to:subject:message-id:date:from :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HDx58ntbVeCa9GWKdrj+e68QUWs6uF1cev715X4SR2I=; b=e5S8REQSTL4ght4X22jfPMhF4myDa744icLPDHN5742aRrAbOMd/3cu5CGJh+c26XG nuZsQpVEl3ec+1M2o4Nt38r9ZSsDLCt3TbBjdUxfthU3Fran9eCRvxDURO1v4DVUWkm4 LMdG/lMzBX0lkdPihYiWUI0iC1ba12NmUxLocurZGhzqANdtk1I9ZeW5vIm6c/uKrP99 OnRoPHwmFUwchGDcORt/KFhVUzMmg5JrLIrbTAQFXA8+cVR/vBihmxX8JCWY2h+q/scy 1du9UAO+2sSpnHpbtW2XbCCvVKihvMEfC8y8PWZH2x8cZ9Y0CALei2ZBGlDYq5jc3B8i o+lw== X-Gm-Message-State: AOJu0Yws6che3qTlO5FPHc+Vu9G6PWhWu3GR1/YvbqtPhO6MJtJFmKX5 tuy3R72JVVmCoOZgYFWCFj0+nIfLjpNTdifwQU3A1pakrys= X-Google-Smtp-Source: AGHT+IG2E3sdvRfmXIj7kZjurQpV8+yzlcSbY7KxcPKvSIFlobEHF59dGQNpJRBwxTyfRNflE7dK577phZytSdsG9R0= X-Received: by 2002:a2e:92d7:0:b0:2ca:14ef:3837 with SMTP id k23-20020a2e92d7000000b002ca14ef3837mr1688293ljh.89.1701965679897; Thu, 07 Dec 2023 08:14:39 -0800 (PST) MIME-Version: 1.0 Received: by 2002:a2e:c49:0:b0:2c8:9257:bcbc with HTTP; Thu, 7 Dec 2023 08:14:39 -0800 (PST) From: =?UTF-8?Q?David_Fern=C3=A1ndez?= Date: Thu, 7 Dec 2023 17:14:39 +0100 Message-ID: To: starlink@lists.bufferbloat.net Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Starlink] [NNagain] CFP march 1 - network measurement conference X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Dec 2023 16:14:42 -0000 How about media over QUIC vs. DASH latency performance? https://github.com/Dash-Industry-Forum/Dash-Industry-Forum.github.io/files/= 13530147/Media-over-QUIC-Will-Law.pdf https://github.com/Dash-Industry-Forum/Dash-Industry-Forum.github.io/files/= 13531331/MOQ_2023_12.pdf Then, there is the question of video codecs, because sending random bytes is one thing, to measure performance, but in the end we want to measure the capability of networks to convey meaningful information, e.g. a video. When measuring packet losses, I think it is important to count separately the packet losses in buffers from packet errors, at least for wireless links that could be experiencing high bit error rates in the link. Then, if ECN is in use, the number of packets marked as CE (Congestion Experienced) can be also an interesting metric to collect. Besides the energy efficiency, in terms of Joules consumed per bit correctly received, which maybe can just be estimated, not accurately measured end-to-end, just at the receiver, maybe, I think it is also important to consider the spectrum efficiency (bit/s/Hz) of the communication system or network. Regards, David Date: Thu, 7 Dec 2023 12:43:53 +0100 From: Nitinder Mohan To: Ricky Mok , starlink@lists.bufferbloat.net Subject: Re: [Starlink] [NNagain] CFP march 1 - network measurement conference Message-ID: Content-Type: text/plain; charset=3D"utf-8" Hi Dave (Ricky, all), Thanks for sharing the conference call. I am one of the TPC chair of the conference and we are really looking forward to cool submissions. So please get the idea mill going :) [Putting my TPC chair hat aside] Application/CDN measurements would be cool! Most CDNs would probably map Starlink user to CDN server based on anycast but I am not sure if the server selection would be close to teh PoP or to the actual user location. Especially for EU users who are mapped to PoPs in other countries, would you get content of the PoP location country or of your own? What about offshore places that are connecting to GSes via long ISL chains (e.g. islands in south of Africa)? Like many folks on this mailing list can already attest, performing real application workload experiments will be key here =E2=80=94 performanc= e under load is very different from ping/traceroutes based results that majority of publications in this space have relied on. Thanks and Regards Nitinder Mohan Technical University Munich (TUM) https://www.nitindermohan.com/ From: Ricky Mok via Starlink Reply: Ricky Mok Date: 7. December 2023 at 03:49:54 To: starlink@lists.bufferbloat.net Subject: Re: [Starlink] [NNagain] CFP march 1 - network measurement confer= ence How about applications? youtube and netflix? (TPC of this conference this year) Ricky On 12/6/23 18:22, Bill Woodcock via Starlink wrote: > >> On Dec 6, 2023, at 22:46, Sauli Kiviranta via Nnagain wrote: >> What would be a comprehensive measurement? Should cover all/most relevan= t areas? > It=E2=80=99s easy to specify a suite of measurements which is too heavy t= o be easily implemented or supported on the network. Also, as you point out= , many things can be derived from raw data, so don=E2=80=99t necessarily re= quire additional specific measurements. > >> Payload Size: The size of data being transmitted. >> Event Rate: The frequency at which payloads are transmitted. >> Bitrate: The combination of rate and size transferred in a given test. >> Throughput: The data transfer capability achieved on the test path. > All of that can probably be derived from sufficiently finely-grained TCP = data. i.e. if you had a PCAP of a TCP flow that constituted the measurement= , you=E2=80=99d be able to derive all of the above. > >> Bandwidth: The data transfer capacity available on the test path. > Presumably the goal of a TCP transaction measurement would be to enable t= his calculation. > >> Transfer Efficiency: The ratio of useful payload data to the overhead da= ta. > This is a how-its-used rather than a property-of-the-network. If there ar= e network-inherent overheads, they=E2=80=99re likely to be not directly vis= ible to endpoints, only inferable, and might require external knowledge of = the network. So, I=E2=80=99d put this out-of-scope. > >> Round-Trip Time (RTT): The ping delay time to the target server and back= . >> RTT Jitter: The variation in the delay of round-trip time. >> Latency: The transmission delay time to the target server and back. >> Latency Jitter: The variation in delay of latency. > RTT is measurable. If Latency is RTT minus processing delay on the remote= end, I=E2=80=99m not sure it=E2=80=99s really measurable, per se, without = the remote end being able to accurately clock itself, or an independent van= tage point adjacent to the remote end. This is the old one-way-delay measur= ement problem in different guise, I think. Anyway, I think RTT is easy and = necessary, and I think latency is difficult and probably an anchor not wort= h attaching to anything we want to see done in the near term. Latency jitte= r likewise. > >> Bit Error Rate: The corrupted bits as a percentage of the total >> transmitted data. > This seems like it can be derived from a PCAP, but doesn=E2=80=99t really= constitute an independent measurement. > >> Packet Loss: The percentage of packets lost that needed to be recovered. > Yep. > >> Energy Efficiency: The amount of energy consumed to achieve the test res= ult. > Not measurable. > >> Did I overlook something? > Out-of-order delivery is the fourth classical quality criterion. There ar= e folks who argue that it doesn=E2=80=99t matter anymore, and others who (m= ore compellingly, to my mind) argue that it=E2=80=99s at least as relevant = as ever. > > Thus, for an actual measurement suite: > > - A TCP transaction > > =E2=80=A6from which we can observe: > > - Loss > - RTT (which I=E2=80=99ll just call =E2=80=9CLatency=E2=80=9D because tha= t=E2=80=99s what people have called it in the past) > - out-of-order delivery > - Jitter in the above three, if the transaction continues long enough > > =E2=80=A6and we can calculate: > > - Goodput > > In addition to these, I think it=E2=80=99s necessary to also associate a = traceroute (and, if available and reliable, a reverse-path traceroute) in o= rder that it be clear what was measured, and a timestamp, and a digital sig= nature over the whole thing, so we can know who=E2=80=99s attesting to the = measurement. > > -Bill >