From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout1.rbg.tum.de (mailout1.rbg.tum.de [131.159.0.201]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 193EB3CB39 for ; Thu, 7 Dec 2023 06:44:22 -0500 (EST) Received: from mailrelay1.rbg.tum.de (mailrelay1.in.tum.de [131.159.254.14]) by mailout1.rbg.tum.de (Postfix) with ESMTPS id 25E6729B; Thu, 7 Dec 2023 12:44:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=in.tum.de; s=20220209; t=1701949460; bh=VdbUO+ciF1rFR9koMK6UxpzolA5d1jmpu1D+YYWFwQY=; h=Date:From:To:In-Reply-To:References:Subject:From; b=QKAnhAj3EOzx3ssj7Y6dnPe/cS1MfTeW+4mhqvpoSgpm5d6eG0iZJdyqgGtStw46d aviZ9Quy4p4Zc0O9DeU0Tz4yHS3ywofaTwYN/NfXaRYJmqclbd6kfuI+i4bmX8ahOE k2+DwGnNaz254raqvYisDjTtJhYeqmb3dyZWzL4aDCAHyOj7gF857P5uaJHXnNGZ4O QtfQejczXVqySvsO9vYaMKi1HsvHXXPLH7vme2+xJF4zxYXkYzB4zal8bKJaQ++OG2 KFk/Ylq0b2ouRN6pcvP44ea/wixKJDUj8Lw68GQH/lbjFtCiTrJWAtx77B2zLbvomP AVdfDMud/bW1g== Received: by mailrelay1.rbg.tum.de (Postfix, from userid 112) id 22B3328B; Thu, 7 Dec 2023 12:44:20 +0100 (CET) Received: from mailrelay1.rbg.tum.de (localhost [127.0.0.1]) by mailrelay1.rbg.tum.de (Postfix) with ESMTP id E75BDCB; Thu, 7 Dec 2023 12:44:19 +0100 (CET) Received: from mail.in.tum.de (vmrbg426.in.tum.de [131.159.0.73]) by mailrelay1.rbg.tum.de (Postfix) with ESMTPS id E524830; Thu, 7 Dec 2023 12:44:19 +0100 (CET) Received: by mail.in.tum.de (Postfix, from userid 112) id E238D4A02B0; Thu, 7 Dec 2023 12:44:19 +0100 (CET) Received: (Authenticated sender: mohan) by mail.in.tum.de (Postfix) with ESMTPSA id 354B54A01A8; Thu, 7 Dec 2023 12:44:09 +0100 (CET) (Extended-Queue-bit xtech_xg@fff.in.tum.de) Date: Thu, 7 Dec 2023 12:43:53 +0100 From: Nitinder Mohan To: Ricky Mok , starlink@lists.bufferbloat.net Message-ID: In-Reply-To: <2dc77c3c-3dc5-47a2-b15a-b643f7ff02bf@caida.org> References: <6CA05409-11BD-4733-9CB1-14400736F050@pch.net> <2dc77c3c-3dc5-47a2-b15a-b643f7ff02bf@caida.org> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="6571b008_170a50bf_1586b" Subject: Re: [Starlink] [NNagain] CFP march 1 - network measurement conference X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Dec 2023 11:44:22 -0000 --6571b008_170a50bf_1586b Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hi Dave (Ricky, all), Thanks for sharing the conference call. I am one of the TPC chair of the = conference and we are really looking forward to cool submissions. So plea= se get the idea mill going :) =5BPutting my TPC chair hat aside=5D Application/CDN measurements would b= e cool=21 Most CDNs would probably map Starlink user to CDN server based = on anycast but I am not sure if the server selection would be close to te= h PoP or to the actual user location. Especially for EU users who are map= ped to PoPs in other countries, would you get content of the PoP location= country or of your own=3F What about offshore places that are connecting= to GSes via long ISL chains (e.g. islands in south of Africa)=3F Like many folks on this mailing list can already attest, performing real = application workload experiments will be key here =E2=80=94 performance u= nder load is very different from ping/traceroutes based results that majo= rity of publications in this space have relied on. Thanks and Regards Nitinder Mohan Technical University Munich (TUM) https://www.nitindermohan.com/ =46rom:=C2=A0Ricky Mok via Starlink Reply:=C2=A0Ricky Mok Date:=C2=A07. December 2023 at 03:49:54 To:=C2=A0starlink=40lists.bufferbloat.net Subject:=C2=A0 Re: =5BStarlink=5D =5BNNagain=5D C=46P march 1 - network m= easurement conference How about applications=3F youtube and netflix=3F (TPC of this conference this year) Ricky On 12/6/23 18:22, Bill Woodcock via Starlink wrote: > >> On Dec 6, 2023, at 22:46, Sauli Kiviranta via Nnagain wrote: >> What would be a comprehensive measurement=3F Should cover all/most rel= evant areas=3F > It=E2=80=99s easy to specify a suite of measurements which is too heavy= to be easily implemented or supported on the network. Also, as you point= out, many things can be derived from raw data, so don=E2=80=99t necessar= ily require additional specific measurements. > >> Payload Size: The size of data being transmitted. >> Event Rate: The frequency at which payloads are transmitted. >> Bitrate: The combination of rate and size transferred in a given test.= >> Throughput: The data transfer capability achieved on the test path. > All of that can probably be derived from sufficiently finely-grained TC= P data. i.e. if you had a PCAP of a TCP flow that constituted the measure= ment, you=E2=80=99d be able to derive all of the above. > >> Bandwidth: The data transfer capacity available on the test path. > Presumably the goal of a TCP transaction measurement would be to enable= this calculation. > >> Transfer Efficiency: The ratio of useful payload data to the overhead = data. > This is a how-its-used rather than a property-of-the-network. If there = are network-inherent overheads, they=E2=80=99re likely to be not directly= visible to endpoints, only inferable, and might require external knowled= ge of the network. So, I=E2=80=99d put this out-of-scope. > >> Round-Trip Time (RTT): The ping delay time to the target server and ba= ck. >> RTT Jitter: The variation in the delay of round-trip time. >> Latency: The transmission delay time to the target server and back. >> Latency Jitter: The variation in delay of latency. > RTT is measurable. If Latency is RTT minus processing delay on the remo= te end, I=E2=80=99m not sure it=E2=80=99s really measurable, per se, with= out the remote end being able to accurately clock itself, or an independe= nt vantage point adjacent to the remote end. This is the old one-way-dela= y measurement problem in different guise, I think. Anyway, I think RTT is= easy and necessary, and I think latency is difficult and probably an anc= hor not worth attaching to anything we want to see done in the near term.= Latency jitter likewise. > >> Bit Error Rate: The corrupted bits as a percentage of the total >> transmitted data. > This seems like it can be derived from a PCAP, but doesn=E2=80=99t real= ly constitute an independent measurement. > >> Packet Loss: The percentage of packets lost that needed to be recovere= d. > Yep. > >> Energy Efficiency: The amount of energy consumed to achieve the test r= esult. > Not measurable. > >> Did I overlook something=3F > Out-of-order delivery is the fourth classical quality criterion. There = are folks who argue that it doesn=E2=80=99t matter anymore, and others wh= o (more compellingly, to my mind) argue that it=E2=80=99s at least as rel= evant as ever. > > Thus, for an actual measurement suite: > > - A TCP transaction > > =E2=80=A6from which we can observe: > > - Loss > - RTT (which I=E2=80=99ll just call =E2=80=9CLatency=E2=80=9D because t= hat=E2=80=99s what people have called it in the past) > - out-of-order delivery > - Jitter in the above three, if the transaction continues long enough > > =E2=80=A6and we can calculate: > > - Goodput > > In addition to these, I think it=E2=80=99s necessary to also associate = a traceroute (and, if available and reliable, a reverse-path traceroute) = in order that it be clear what was measured, and a timestamp, and a digit= al signature over the whole thing, so we can know who=E2=80=99s attesting= to the measurement. > > -Bill > > > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F > Starlink mailing list > Starlink=40lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/starlink =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F Starlink mailing list Starlink=40lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/starlink --6571b008_170a50bf_1586b Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
Hi Dave (Ricky, all),
<= div style=3D=22font-family:CMU Sans Serif,Arial;font-size:14px; =22>
<= /div>
Thanks for sharing the conference call. I am one of the TPC chair of the= conference and we are really looking forward to cool submissions. So ple= ase get the idea mill going :)

=5BPutting my TPC chair hat aside=5D= Application/CDN measurements would be cool=21 Most CDNs would probably m= ap Starlink user to CDN server based on anycast but I am not sure if the = server selection would be close to teh PoP or to the actual user location= . Especially for EU users who are mapped to PoPs in other countries, woul= d you get content of the PoP location country or of your own=3F What abou= t offshore places that are connecting to GSes via long ISL chains (e.g. i= slands in south of Africa)=3F

Like many folks on this mailing list= can already attest, performing real application workload experiments wil= l be key here =E2=80=94 performance under load is very different from pin= g/traceroutes based results that majority of publications in this space h= ave relied on.

Thanks and Regards

Nitinder Mohan
Technical University Munich (TUM)

=46rom: Ricky Mok via Starlink <starlink=40lists.bufferbloat.net>
Reply: = ;Ricky Mok <cskpmok=40caida.org>
Date: 7. December 2023 at 03:49:54
To: = starlink=40lists.bufferbloat.net <= a href=3D=22mailto:starlink=40lists.bufferbloat.net=22><starlink=40lis= ts.bufferbloat.net>
Subject:  Re: =5BStarlink=5D =5BNNagain=5D C=46P march 1 - network measurement co= nference

How about applications=3F youtube= and netflix=3F

(TPC of this conference this year)

Ricky
On 12/6/23 18:22, Bill Woodcock via Starlink wrote:
>
>&= gt; On Dec 6, 2023, at 22:46, Sauli Kiviranta via Nnagain <nnagain=40l= ists.bufferbloat.net> wrote:
>> What would be a comprehensive= measurement=3F Should cover all/most relevant areas=3F
> It=E2=80=99= s easy to specify a suite of measurements which is too heavy to be easily= implemented or supported on the network. Also, as you point out, many t= hings can be derived from raw data, so don=E2=80=99t necessarily require = additional specific measurements.
>
>> Payload Size: The s= ize of data being transmitted.
>> Event Rate: The frequency at w= hich payloads are transmitted.
>> Bitrate: The combination of ra= te and size transferred in a given test.
>> Throughput: The data= transfer capability achieved on the test path.
> All of that can p= robably be derived from sufficiently finely-grained TCP data. i.e. if yo= u had a PCAP of a TCP flow that constituted the measurement, you=E2=80=99= d be able to derive all of the above.
>
>> Bandwidth: The = data transfer capacity available on the test path.
> Presumably the= goal of a TCP transaction measurement would be to enable this calculatio= n.
>
>> Transfer Efficiency: The ratio of useful payload d= ata to the overhead data.
> This is a how-its-used rather than a pr= operty-of-the-network. If there are network-inherent overheads, they=E2=80= =99re likely to be not directly visible to endpoints, only inferable, and= might require external knowledge of the network. So, I=E2=80=99d put th= is out-of-scope.
>
>> Round-Trip Time (RTT): The ping dela= y time to the target server and back.
>> RTT Jitter: The variati= on in the delay of round-trip time.
>> Latency: The transmission= delay time to the target server and back.
>> Latency Jitter: Th= e variation in delay of latency.
> RTT is measurable. If Latency i= s RTT minus processing delay on the remote end, I=E2=80=99m not sure it=E2= =80=99s really measurable, per se, without the remote end being able to a= ccurately clock itself, or an independent vantage point adjacent to the r= emote end. This is the old one-way-delay measurement problem in differen= t guise, I think. Anyway, I think RTT is easy and necessary, and I think= latency is difficult and probably an anchor not worth attaching to anyth= ing we want to see done in the near term. Latency jitter likewise.
&g= t;
>> Bit Error Rate: The corrupted bits as a percentage of the = total
>> transmitted data.
> This seems like it can be der= ived from a PCAP, but doesn=E2=80=99t really constitute an independent me= asurement.
>
>> Packet Loss: The percentage of packets los= t that needed to be recovered.
> Yep.
>
>> Energy Ef= ficiency: The amount of energy consumed to achieve the test result.
&g= t; Not measurable.
>
>> Did I overlook something=3F
>= ; Out-of-order delivery is the fourth classical quality criterion. There= are folks who argue that it doesn=E2=80=99t matter anymore, and others w= ho (more compellingly, to my mind) argue that it=E2=80=99s at least as re= levant as ever.
>
> Thus, for an actual measurement suite:>
> - A TCP transaction
>
> =E2=80=A6from which w= e can observe:
>
> - Loss
> - RTT (which I=E2=80=99= ll just call =E2=80=9CLatency=E2=80=9D because that=E2=80=99s what people= have called it in the past)
> - out-of-order delivery
> = - Jitter in the above three, if the transaction continues long enough
= >
> =E2=80=A6and we can calculate:
>
> - Goodput>
> In addition to these, I think it=E2=80=99s necessary to al= so associate a traceroute (and, if available and reliable, a reverse-path= traceroute) in order that it be clear what was measured, and a timestamp= , and a digital signature over the whole thing, so we can know who=E2=80=99= s attesting to the measurement.
>
> = -Bill
>
>
> =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
> Starlink mailing list
> Star= link=40lists.bufferbloat.net
> https://lists.bufferbloat.net/listin= fo/starlink
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F
Starlink mailing list
Starlink=40lists.bufferbloat.net
ht= tps://lists.bufferbloat.net/listinfo/starlink
 --6571b008_170a50bf_1586b--