From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from bobcat.rjmcmahon.com (bobcat.rjmcmahon.com [45.33.58.123]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 1325A3CB39; Fri, 8 Dec 2023 01:03:07 -0500 (EST) Received: from mail.rjmcmahon.com (bobcat.rjmcmahon.com [45.33.58.123]) by bobcat.rjmcmahon.com (Postfix) with ESMTPA id 25BA31B31F; Thu, 7 Dec 2023 22:03:07 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 bobcat.rjmcmahon.com 25BA31B31F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rjmcmahon.com; s=bobcat; t=1702015387; bh=dJeX6Wg9tX5ChbqAuu9fPxE5CSOlteIjyKkqbLlOYfQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=npQTA75V6JfDH1QnHsRS6735B4eQiykcpL6DIdZY4+EV1FiEs31IeZC7Wim+naKzB 9i2zn7iE8y3TemIew3UhB1BcGZ6x7ALO8KOicpQmFwC+Hh2bUkwubcnDXvLTP1D1qk ZflviVh5GbSZcGRtcddb85Mf113DhH43x4en2RL4= MIME-Version: 1.0 Date: Thu, 07 Dec 2023 22:03:07 -0800 From: rjmcmahon To: =?UTF-8?Q?Network_Neutrality_is_back!_Let=C2=B4s_make_the_technical_a?= =?UTF-8?Q?spects_heard_this_time!?= Cc: Bill Woodcock , Dave Taht via Starlink , bloat In-Reply-To: <6CA05409-11BD-4733-9CB1-14400736F050@pch.net> References: <6CA05409-11BD-4733-9CB1-14400736F050@pch.net> Message-ID: <189c66d8b0b38a3d2de565c6f22de1da@rjmcmahon.com> X-Sender: rjmcmahon@rjmcmahon.com Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: [Starlink] [NNagain] CFP march 1 - network measurement conference X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Dec 2023 06:03:08 -0000 iperf 2 supports OWD in multiple forms. A raspberry pi 5 has a realtime clock and hardware PTP and gpio PPS. The retail cost for a pi5 with GPS atomic clock and active fan is less than $150 [rjmcmahon@fedora iperf2-code]$ src/iperf -c 192.168.1.35 --bounceback --trip-times --bounceback-period 0 -i 1 -t 4 ------------------------------------------------------------ Client connecting to 192.168.1.35, TCP port 5001 with pid 48142 (1/0 flows/load) Bounceback test (req/reply size = 100 Byte/ 100 Byte) (server hold req=0 usecs & tcp_quickack) TCP congestion control using cubic TOS set to 0x0 and nodelay (Nagle off) TCP window size: 85.0 KByte (default) Event based writes (pending queue watermark at 16384 bytes) ------------------------------------------------------------ [ 1] local 192.168.1.103%enp4s0 port 50558 connected with 192.168.1.35 port 5001 (prefetch=16384) (bb w/quickack req/reply/hold=100/100/0) (trip-times) (sock=3) (icwnd/mss/irtt=14/1448/541) (ct=0.59 ms) on 2023-12-07 22:01:39.240 (PST) [ ID] Interval Transfer Bandwidth BB cnt=avg/min/max/stdev Rtry Cwnd/RTT RPS(avg) [ 1] 0.00-1.00 sec 739 KBytes 6.05 Mbits/sec 7566=0.130/0.099/0.627/0.007 ms 0 14K/115 us 7666 rps [ 1] 0.00-1.00 sec OWD (ms) Cnt=7566 TX=0.072/0.038/0.163/0.002 RX=0.058/0.047/0.156/0.004 Asymmetry=0.015/0.001/0.103/0.004 [ 1] 1.00-2.00 sec 745 KBytes 6.10 Mbits/sec 7630=0.130/0.082/0.422/0.005 ms 0 14K/114 us 7722 rps [ 1] 1.00-2.00 sec OWD (ms) Cnt=7630 TX=0.073/0.027/0.364/0.004 RX=0.057/0.048/0.097/0.003 Asymmetry=0.016/0.000/0.306/0.005 [ 1] 2.00-3.00 sec 749 KBytes 6.14 Mbits/sec 7671=0.129/0.085/0.252/0.004 ms 0 14K/113 us 7756 rps [ 1] 2.00-3.00 sec OWD (ms) Cnt=7671 TX=0.073/0.031/0.193/0.003 RX=0.056/0.047/0.102/0.003 Asymmetry=0.017/0.000/0.134/0.004 [ 1] 3.00-4.00 sec 737 KBytes 6.04 Mbits/sec 7546=0.131/0.085/0.290/0.004 ms 0 14K/115 us 7629 rps [ 1] 3.00-4.00 sec OWD (ms) Cnt=7546 TX=0.073/0.030/0.231/0.003 RX=0.058/0.047/0.105/0.003 Asymmetry=0.015/0.000/0.172/0.004 [ 1] 0.00-4.00 sec 2.90 MBytes 6.08 Mbits/sec 30414=0.130/0.082/0.627/0.005 ms 0 14K/376 us 7693 rps [ 1] 0.00-4.00 sec OWD (ms) Cnt=30414 TX=0.073/0.027/0.364/0.003 RX=0.057/0.047/0.156/0.004 Asymmetry=0.016/0.000/0.306/0.004 [ 1] 0.00-4.00 sec OWD-TX(f)-PDF: bin(w=100us):cnt(30414)=1:30393,2:19,3:1,4:1 (5.00/95.00/99.7%=1/1/1,Outliers=0,obl/obu=0/0) [ 1] 0.00-4.00 sec OWD-RX(f)-PDF: bin(w=100us):cnt(30414)=1:30400,2:14 (5.00/95.00/99.7%=1/1/1,Outliers=0,obl/obu=0/0) [ 1] 0.00-4.00 sec BB8(f)-PDF: bin(w=100us):cnt(30414)=1:6,2:30392,3:14,5:1,7:1 (5.00/95.00/99.7%=2/2/2,Outliers=16,obl/obu=0/0) Bob >> On Dec 6, 2023, at 22:46, Sauli Kiviranta via Nnagain >> wrote: >> What would be a comprehensive measurement? Should cover all/most >> relevant areas? > > It’s easy to specify a suite of measurements which is too heavy to be > easily implemented or supported on the network. Also, as you point > out, many things can be derived from raw data, so don’t necessarily > require additional specific measurements. > >> Payload Size: The size of data being transmitted. >> Event Rate: The frequency at which payloads are transmitted. >> Bitrate: The combination of rate and size transferred in a given test. >> Throughput: The data transfer capability achieved on the test path. > > All of that can probably be derived from sufficiently finely-grained > TCP data. i.e. if you had a PCAP of a TCP flow that constituted the > measurement, you’d be able to derive all of the above. > >> Bandwidth: The data transfer capacity available on the test path. > > Presumably the goal of a TCP transaction measurement would be to > enable this calculation. > >> Transfer Efficiency: The ratio of useful payload data to the overhead >> data. > > This is a how-its-used rather than a property-of-the-network. If > there are network-inherent overheads, they’re likely to be not > directly visible to endpoints, only inferable, and might require > external knowledge of the network. So, I’d put this out-of-scope. > >> Round-Trip Time (RTT): The ping delay time to the target server and >> back. >> RTT Jitter: The variation in the delay of round-trip time. >> Latency: The transmission delay time to the target server and back. >> Latency Jitter: The variation in delay of latency. > > RTT is measurable. If Latency is RTT minus processing delay on the > remote end, I’m not sure it’s really measurable, per se, without the > remote end being able to accurately clock itself, or an independent > vantage point adjacent to the remote end. This is the > old[rjmcmahon@fedora iperf2-code]$ src/iperf -c 192.168.1.35 > --bounceback --trip-times --bounceback-period 0 -i 1 -t 4 ------------------------------------------------------------ Client connecting to 192.168.1.35, TCP port 5001 with pid 46358 (1/0 flows/load) Bounceback test (req/reply size = 100 Byte/ 100 Byte) (server hold req=0 usecs & tcp_quickack) TCP congestion control using cubic TOS set to 0x0 and nodelay (Nagle off) TCP window size: 85.0 KByte (default) Event based writes (pending queue watermark at 16384 bytes) ------------------------------------------------------------ [ 1] local 192.168.1.103%enp4s0 port 60788 connected with 192.168.1.35 port 5001 (prefetch=16384) (bb w/quickack req/reply/hold=100/100/0) (trip-times) (sock=3) (icwnd/mss/irtt=14/1448/168) (ct=0.23 ms) on 2023-12-07 21:21:31.417 (PST) [ ID] Interval Transfer Bandwidth BB cnt=avg/min/max/stdev Rtry Cwnd/RTT RPS(avg) [ 1] 0.00-1.00 sec 745 KBytes 6.10 Mbits/sec 7631=0.129/0.096/0.637/0.007 ms 0 14K/114 us 7733 rps [ 1] 0.00-1.00 sec OWD (ms) Cnt=7631 TX=0.068/0.034/0.191/0.003 RX=0.061/0.049/0.118/0.004 Asymmetry=0.009/0.000/0.130/0.004 ** reset [ 1] 1.00-2.00 sec 751 KBytes 6.15 Mbits/sec 7689=0.129/0.092/0.350/0.005 ms 0 14K/115 us 7782 rps [ 1] 1.00-2.00 sec OWD (ms) Cnt=7689 TX=0.069/0.030/0.288/0.004 RX=0.060/0.052/0.116/0.003 Asymmetry=0.009/0.000/0.227/0.004 ** reset [ 1] 2.00-3.00 sec 748 KBytes 6.13 Mbits/sec 7664=0.129/0.085/0.378/0.004 ms 0 14K/115 us 7751 rps [ 1] 2.00-3.00 sec OWD (ms) Cnt=7664 TX=0.069/0.025/0.313/0.003 RX=0.060/0.053/0.098/0.002 Asymmetry=0.008/0.000/0.248/0.004 ** reset [ 1] 3.00-4.00 sec 752 KBytes 6.16 Mbits/sec 7698=0.128/0.087/0.322/0.004 ms 0 14K/114 us 7787 rps [ 1] 3.00-4.00 sec OWD (ms) Cnt=7698 TX=0.068/0.023/0.257/0.003 RX=0.060/0.052/0.091/0.002 Asymmetry=0.009/0.000/0.192/0.004 ** reset ** reset [ 1] 0.00-4.00 sec 2.93 MBytes 6.13 Mbits/sec 30683=0.129/0.085/0.637/0.005 ms 0 14K/408 us 7763 rps [ 1] 0.00-4.00 sec OWD (ms) Cnt=30683 TX=0.068/0.023/0.313/0.003 RX=0.060/0.049/0.118/0.003 Asymmetry=0.009/0.000/0.248/0.004 [ 1] 0.00-4.00 sec OWD-TX(f)-PDF: bin(w=100us):cnt(30683)=1:30663,2:17,3:2,4:1 (5.00/95.00/99.7%=1/1/1,Outliers=0,obl/obu=0/0) [ 1] 0.00-4.00 sec OWD-RX(f)-PDF: bin(w=100us):cnt(30683)=1:30669,2:14 (5.00/95.00/99.7%=1/1/1,Outliers=0,obl/obu=0/0) [ 1] 0.00-4.00 sec BB8(f)-PDF: bin(w=100us):cnt(30683)=1:7,2:30663,3:9,4:3,7:1 (5.00/95.00/99.7%=2/2/2,Outliers=13,obl/obu=0/0) [rjmcmahon@fedora iperf2-code]$ emacs src/Reporter.c [rjmcmahon@fedora iperf2-code]$ make -j make all-recursive make[1]: Entering directory '/home/rjmcmahon/Code/csv/iperf2-code' Making all in compat make[2]: Entering directory '/home/rjmcmahon/Code/csv/iperf2-code/compat' make[2]: Nothing to be done for 'all'. make[2]: Leaving directory '/home/rjmcmahon/Code/csv/iperf2-code/compat' Making all in doc make[2]: Entering directory '/home/rjmcmahon/Code/csv/iperf2-code/doc' make[2]: Nothing to be done for 'all'. make[2]: Leaving directory '/home/rjmcmahon/Code/csv/iperf2-code/doc' Making all in include make[2]: Entering directory '/home/rjmcmahon/Code/csv/iperf2-code/include' make[2]: Nothing to be done for 'all'. make[2]: Leaving directory '/home/rjmcmahon/Code/csv/iperf2-code/include' Making all in src make[2]: Entering directory '/home/rjmcmahon/Code/csv/iperf2-code/src' gcc -DHAVE_CONFIG_H -I. -I.. -I../include -I../include -Wall -O2 -O2 -MT Reporter.o -MD -MP -MF .deps/Reporter.Tpo -c -o Reporter.o Reporter.c mv -f .deps/Reporter.Tpo .deps/Reporter.Po g++ -Wall -O2 -O2 -O2 -pthread -DHAVE_CONFIG_H -o iperf Client.o Extractor.o isochronous.o Launch.o active_hosts.o Listener.o Locale.o PerfSocket.o Reporter.o Reports.o ReportOutputs.o Server.o Settings.o SocketAddr.o gnu_getopt.o gnu_getopt_long.o histogram.o main.o service.o socket_io.o stdio.o packet_ring.o tcp_window_size.o pdfs.o dscp.o iperf_formattime.o iperf_multicast_api.o checksums.o ../compat/libcompat.a -lrt make[2]: Leaving directory '/home/rjmcmahon/Code/csv/iperf2-code/src' Making all in man make[2]: Entering directory '/home/rjmcmahon/Code/csv/iperf2-code/man' make[2]: Nothing to be done for 'all'. make[2]: Leaving directory '/home/rjmcmahon/Code/csv/iperf2-code/man' Making all in flows make[2]: Entering directory '/home/rjmcmahon/Code/csv/iperf2-code/flows' make[2]: Nothing to be done for 'all'. make[2]: Leaving directory '/home/rjmcmahon/Code/csv/iperf2-code/flows' make[2]: Entering directory '/home/rjmcmahon/Code/csv/iperf2-code' make[2]: Leaving directory '/home/rjmcmahon/Code/csv/iperf2-code' make[1]: Leaving directory '/home/rjmcmahon/Code/csv/iperf2-code' [rjmcmahon@fedora iperf2-code]$ src/iperf -c 192.168.1.35 --bounceback --trip-times --bounceback-period 0 -i 1 -t 4 ------------------------------------------------------------ Client connecting to 192.168.1.35, TCP port 5001 with pid 46427 (1/0 flows/load) Bounceback test (req/reply size = 100 Byte/ 100 Byte) (server hold req=0 usecs & tcp_quickack) TCP congestion control using cubic TOS set to 0x0 and nodelay (Nagle off) TCP window size: 85.0 KByte (default) Event based writes (pending queue watermark at 16384 bytes) ------------------------------------------------------------ [ 1] local 192.168.1.103%enp4s0 port 37748 connected with 192.168.1.35 port 5001 (prefetch=16384) (bb w/quickack req/reply/hold=100/100/0) (trip-times) (sock=3) (icwnd/mss/irtt=14/1448/177) (ct=0.23 ms) on 2023-12-07 21:21:46.282 (PST) [ ID] Interval Transfer Bandwidth BB cnt=avg/min/max/stdev Rtry Cwnd/RTT RPS(avg) [ 1] 0.00-1.00 sec 738 KBytes 6.04 Mbits/sec 7552=0.131/0.115/0.642/0.007 ms 0 14K/115 us 7652 rps [ 1] 0.00-1.00 sec OWD (ms) Cnt=7552 TX=0.070/0.053/0.184/0.002 RX=0.061/0.048/0.113/0.004 Asymmetry=0.009/0.000/0.116/0.004 [ 1] 1.00-2.00 sec 739 KBytes 6.05 Mbits/sec 7568=0.131/0.078/0.362/0.004 ms 0 14K/114 us 7657 rps [ 1] 1.00-2.00 sec OWD (ms) Cnt=7568 TX=0.070/0.020/0.298/0.003 RX=0.061/0.051/0.097/0.003 Asymmetry=0.009/0.000/0.235/0.004 [ 1] 2.00-3.00 sec 739 KBytes 6.06 Mbits/sec 7571=0.131/0.089/0.279/0.005 ms 0 14K/115 us 7660 rps [ 1] 2.00-3.00 sec OWD (ms) Cnt=7571 TX=0.070/0.025/0.215/0.003 RX=0.060/0.051/0.181/0.004 Asymmetry=0.010/0.000/0.151/0.004 [ 1] 3.00-4.00 sec 751 KBytes 6.15 Mbits/sec 7693=0.129/0.090/0.284/0.004 ms 0 14K/115 us 7780 rps [ 1] 3.00-4.00 sec OWD (ms) Cnt=7693 TX=0.070/0.032/0.223/0.004 RX=0.058/0.050/0.091/0.002 Asymmetry=0.013/0.000/0.162/0.004 [ 1] 0.00-4.00 sec 2.90 MBytes 6.07 Mbits/sec 30385=0.130/0.078/0.642/0.005 ms 0 14K/404 us 7687 rps [ 1] 0.00-4.00 sec OWD (ms) Cnt=30385 TX=0.070/0.020/0.298/0.003 RX=0.060/0.048/0.181/0.004 Asymmetry=0.010/0.000/0.235/0.004 [ 1] 0.00-4.00 sec OWD-TX(f)-PDF: bin(w=100us):cnt(30385)=1:30366,2:16,3:3 (5.00/95.00/99.7%=1/1/1,Outliers=0,obl/obu=0/0) [ 1] 0.00-4.00 sec OWD-RX(f)-PDF: bin(w=100us):cnt(30385)=1:30360,2:25 (5.00/95.00/99.7%=1/1/1,Outliers=0,obl/obu=0/0) [ 1] 0.00-4.00 sec BB8(f)-PDF: bin(w=100us):cnt(30385)=1:4,2:30366,3:13,4:1,7:1 (5.00/95.00/99.7%=2/2/2,Outliers=15,obl/obu=0/0) [rjmcmahon@fedora iperf2-code]$ git diff > one-way-delay measurement problem in different guise, I think. > Anyway, I think RTT is easy and necessary, and I think latency is > difficult and probably an anchor not worth attaching to anything we > want to see done in the near term. Latency jitter likewise. > >> Bit Error Rate: The corrupted bits as a percentage of the total >> transmitted data. > > This seems like it can be derived from a PCAP, but doesn’t really > constitute an independent measurement. > >> Packet Loss: The percentage of packets lost that needed to be >> recovered. > > Yep. > >> Energy Efficiency: The amount of energy consumed to achieve the test >> result. > > Not measurable. > >> Did I overlook something? > > Out-of-order delivery is the fourth classical quality criterion. > There are folks who argue that it doesn’t matter anymore, and others > who (more compellingly, to my mind) argue that it’s at least as > relevant as ever. > > Thus, for an actual measurement suite: > > - A TCP transaction > > …from which we can observe: > > - Loss > - RTT (which I’ll just call “Latency” because that’s what people have > called it in the past) > - out-of-order delivery > - Jitter in the above three, if the transaction continues long enough > > …and we can calculate: > > - Goodput > > In addition to these, I think it’s necessary to also associate a > traceroute (and, if available and reliable, a reverse-path traceroute) > in order that it be clear what was measured, and a timestamp, and a > digital signature over the whole thing, so we can know who’s attesting > to the measurement. > > -Bill > > > _______________________________________________ > Nnagain mailing list > Nnagain@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/nnagain