From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from atl4mhob17.registeredsite.com (atl4mhob17.registeredsite.com [209.17.115.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 344A83B2A4 for ; Wed, 6 Dec 2023 20:54:53 -0500 (EST) Received: from mymail.myregisteredsite.com (jax4wmnode3b.mymail.myregisteredsite.com [209.237.134.215]) by atl4mhob17.registeredsite.com (8.14.4/8.14.4) with SMTP id 3B71spKP044347 for ; Wed, 6 Dec 2023 20:54:51 -0500 Received: (qmail 20237 invoked by uid 80); 7 Dec 2023 01:54:50 -0000 Received: from unknown (HELO ?192.168.1.100?) (jack@3kitty.org@76.137.180.175) by 209.237.134.154 with ESMTPA; 7 Dec 2023 01:54:50 -0000 Content-Type: multipart/alternative; boundary="------------3ts3aI9OilHOtcVB0NP1Fajl" Message-ID: <6d87504d-294b-457d-8eb8-ff52e26364e7@3kitty.org> Date: Wed, 6 Dec 2023 17:54:23 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: nnagain@lists.bufferbloat.net References: From: Jack Haverty Autocrypt: addr=jack@3kitty.org; keydata= xsDNBGCm2psBDADGOWO8n9wfkDW9ZUEo8o+SZ5MU9us2il+fS4EFM/RaZFIbQ+P72bExzSd3 WnJdPfqO1O7Q+dRnvVO9+G2/9oT/uRZVaE05+SothzKZBv32HcZoUkdNZOTqSkdo3EwNPjid LLxX+dMBxMpR3pBdvGN8Z7lnZe6fV4QO2xtd58y3B33AVZJp+RuNwucby9dY2meyy2BJVKrx mKhYXAucVyg0ALVIchHt9UknVW4aLvQF+oMfzXVvCWeguW+DvbyazVceWGO7FSgUJ8ED3Ii7 xAR5zZJ1LASoMhG1ixg07P9Uy4ohV6c+c0yV9SY4yqhZ3+zN2cm9h/aXpwjSuiVVAJbK7zzb FjI+h89dbnaVQrLx6GikV0OVYqC6TCeMfCFZQAJLs1icxQi3BLL7O1fbTGatEfTgLa5nqfKq K/D/HlOCUeFxqZI8hXvT5dG4e1m3ilpF2/ytcWKSVg3d699UFntPv3sEbAQwwfXsnuD4Hem6 0Ao0/z41n8x1aeZE80FdkpEAEQEAAc0eSmFjayBIYXZlcnR5IDxqYWNrQDNraXR0eS5vcmc+ wsEJBBMBCAAzFiEEZLvMn5vmvTAlFEILdGzDIkA7jlAFAmCm2pwCGwMFCwkIBwIGFQgJCgsC BRYCAwEAAAoJEHRswyJAO45QuX0L/jOluv8fr/BmuEEQsWWGW6oARIbjDQrI93kXIJXuPnfp tGjkx/f1TMIzI2B9s/tejiYE7IZOhWbX1YvKF0UbkSJi50UyV9XtYRnLdD5TcksKB4luDF8S R+nj5WBm17Bp8qwriCMgA1jGL2wQ7J1KUw4Q/gsMcjhn/39PevswkriU2qqVplfCs9yTTMU5 SvtE2U9F3Y1ZINHn3kUysvxhRFd+Oh3PocWHmVE+hkII+qsra6z4eztDgoB+vqxmOJEdtvex GhT8OKu74DacguZVfu/AV+cwpX701sdjJrMyKjcv8uhFLM/E5gf6kSUAFxBVwe6pNDmAgmbS c0fAFrZjgXxNxxndpu/8OAUDVzKg+l5WJ0nWss9Q14BwA+FcoclO3lwzFu7jOiLvkm7jQkFB o+p8Owe4iAED1KK/aocIa/RiD4sZ3KXUJ92kkemZ1Qe2XpFVdzxaQDG0huNkc5Mie9rdt62O Ae+5cYdPeWmBVn+pFNs5H09kQQbVR5pUxe2Aps7AzQRgptqcAQwAzzougHNMFr/O/L8HnNJW 1YyOuX0PEVNUXQPwkxKuD8bAXsPr4Hv1a+840ByesiJSadhQgVSMruRqoQC5tTkbEWkqlfDW waNAdqCJOXl2T6gtK7RpcHNx7+/du/gCAhHOXqH1Qfs0Zi3YEbR/kQFRP3wD4GiCvHSny8zJ X9plIHqQGoE5DePNAtE2KimbFMsjguqJgq5x0tMf3qEaMNd0IGTStGpcC49iss71slotH091 Y1Yo9CpzL6rj8IP0BfssEujAvf3Gbf1oi92JRE3s2humFDfPvSlHmRIfWPQ4qFOw1zmlzsV1 eg83gErKbjaDdkbwQA85RTmMVKNVvonM80WB6jAg8tlJ5VlYlpbzASpJRNj+FL1LLBQxCbPU eFwrzqYgNvtdKR7j5nTgdndCxq+2aws/aAjdL10S8yeH7ZOpNPzjDJfMSt/L1O25zPUhXdQC 9AZNYsfyV7rf+POEgVpIEth1fT9WbmS0rZxRd/+y628n31GicbA+teN890vdABEBAAHCwPYE GAEIACAWIQRku8yfm+a9MCUUQgt0bMMiQDuOUAUCYKbanQIbDAAKCRB0bMMiQDuOUF1LC/4q 4pLtmDt6TIET2H7zGj5ie3ng7kC7YqtFPYwgLQzs9WeqQ/5WowEmHOPonBcqhGbtDj22GebQ 7w0RoUHb+aXsbC85I/C+nWgT1ZcfMBTHGlBcIQvOCNG18g87Ha9jgD0HnW4bRUkZmGMpP0Yd TLM+PBNu41AK6z82VPQrfTuPKqwAAS2FK/RpF2xB7rjpETzIPl9Dj9EAkRbviURIg0BQkmej l02FLzGmlTfBIDHBdEgzvD71Z5H9BP8DAbxBzonSTzx/KZyv7njSUzdVLW+5O/WzPgb4Qt4I jQd66LS9HWS1G7AcLjiSQAIf8v7JkX3NwtN+NGX5cmt2p0e9FOOKWXVgCIgPN3/712EEGAgq UUxuPEBD5DrRCgjZL40eHxQza2BAhoVoWopUCGZdCCZJP3iF7818wIph0U393DELG9NAGLJa qkoA8KBimXp9Rd2QvpA864JRy/REoEOEF9lm3clriLyEqaL/VMIQRhl/VSkUuez4Wr68eHus TFdwePg= In-Reply-To: Subject: Re: [NNagain] CFP march 1 - network measurement conference X-BeenThere: nnagain@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: =?utf-8?q?Network_Neutrality_is_back!_Let=C2=B4s_make_the_technical_aspects_heard_this_time!?= List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Dec 2023 01:54:53 -0000 This is a multi-part message in MIME format. --------------3ts3aI9OilHOtcVB0NP1Fajl Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit IMHO, characterizing performance warrants more measurements, e.g.,: - amount (bytes, datagrams) presumed lost and re-transmitted by the sender - amount (bytes, datagrams) discarded at the receiver because they were already received - amount (bytes, datagrams) discarded at the receiver because they arrived too late to be useful With such data, it would be possible to measure things like "useful throughput", i.e., the data successfully delivered from source to destination which was actually useful for the associated user's application. Some useful measurements were included in RFC 1213, e.g., in the "TCP" and "UDP" sections, such as number of retransmissions.  These measurements likely require instrumentation in the sending and receiving computers, where the TCP and similar algorithms operate. Measurements also may depend strongly on the particular computer type and operating system.   Measurements obtained from various "probe" sources and destinations, such as a "test host", provide only part of a "comprehensive measurement".  A complete characterization of the performance achieved would include measurement data from the users' host computers attached to the Internet, as they are doing whatever the user does. I don't recall if the SNMP MIB definitions were ever extended to include metrics appropriate for uses such as VOIP, video conferencing, remote operation, et al.   Those are the kinds of uses which are most sensitive to latency and variance in latency and throughput, and probably most interesting to measure. Jack Haverty On 12/6/23 13:46, Sauli Kiviranta via Nnagain wrote: > Thank you for sharing! This looks very promising! > > What would be a comprehensive measurement? Should cover all/most relevant areas? > > Payload Size: The size of data being transmitted. > Event Rate: The frequency at which payloads are transmitted. > Bitrate: The combination of rate and size transferred in a given test. > Bandwidth: The data transfer capacity available on the test path. > Throughput: The data transfer capability achieved on the test path. > Transfer Efficiency: The ratio of useful payload data to the overhead data. > Round-Trip Time (RTT): The ping delay time to the target server and back. > RTT Jitter: The variation in the delay of round-trip time. > Latency: The transmission delay time to the target server and back. > Latency Jitter: The variation in delay of latency. > Bit Error Rate: The corrupted bits as a percentage of the total > transmitted data. > Packet Loss: The percentage of packets lost that needed to be recovered. > Energy Efficiency: The amount of energy consumed to achieve the test result. > > Did I overlook something? Too many dimensions to cover? Obviously some > of those are derived, so not part of the whole set as such. > > Maybe then next would be to have different profiles where any of those > parameters may vary over the test run. e.g. profiles that model > congested base stations for mobile data. Different use case specific > payload profiles e.g. gop for video transfer? > > Best regards, > Sauli > > > On 12/6/23, Dave Taht via Nnagain wrote: >> This CFP looks pretty good to me:https://tma.ifip.org/2024/call-for-papers/ >> >> Because: >> >> ¨To further encourage the results’ faithfulness and avoid publication >> bias, the conference will particularly encourage negative results >> revealed by novel measurement methods or vantage points. All regular >> papers are hence encouraged to discuss the limitations of the >> presented approaches and also mention which experiments did not work. >> Additionally, TMA will also be open to accepting papers that >> exclusively deal with negative results, especially when new >> measurement methods or perspectives offer insight into the limitations >> and challenges of network measurement in practice. Negative results >> will be evaluated based on their impact (e.g. revealed in realistic >> production networks) as well as the novelty of the vantage points >> (e.g. scarce data source) or measurement techniques that revealed >> them." >> >> >> -- >> :( My old R&D campus is up for sale:https://tinyurl.com/yurtlab >> Dave Täht CSO, LibreQos >> _______________________________________________ >> Nnagain mailing list >> Nnagain@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/nnagain >> > _______________________________________________ > Nnagain mailing list > Nnagain@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/nnagain --------------3ts3aI9OilHOtcVB0NP1Fajl Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit IMHO, characterizing performance warrants more measurements, e.g.,:

- amount (bytes, datagrams) presumed lost and re-transmitted by the sender
- amount (bytes, datagrams) discarded at the receiver because they were already received
- amount (bytes, datagrams) discarded at the receiver because they arrived too late to be useful

With such data, it would be possible to measure things like "useful throughput", i.e., the data successfully delivered from source to destination which was actually useful for the associated user's application.

Some useful measurements were included in RFC 1213, e.g., in the "TCP" and "UDP" sections, such as number of retransmissions.  These measurements likely require instrumentation in the sending and receiving computers, where the TCP and similar algorithms operate.  

Measurements also may depend strongly on the particular computer type and operating system.   Measurements obtained from various "probe" sources and destinations, such as a "test host", provide only part of a "comprehensive measurement".  A complete characterization of the performance achieved would include measurement data from the users' host computers attached to the Internet, as they are doing whatever the user does.

I don't recall if the SNMP MIB definitions were ever extended to include metrics appropriate for uses such as VOIP, video conferencing, remote operation, et al.   Those are the kinds of uses which are most sensitive to latency and variance in latency and throughput, and probably most interesting to measure.

Jack Haverty


On 12/6/23 13:46, Sauli Kiviranta via Nnagain wrote:
Thank you for sharing! This looks very promising!

What would be a comprehensive measurement? Should cover all/most relevant areas?

Payload Size: The size of data being transmitted.
Event Rate: The frequency at which payloads are transmitted.
Bitrate: The combination of rate and size transferred in a given test.
Bandwidth: The data transfer capacity available on the test path.
Throughput: The data transfer capability achieved on the test path.
Transfer Efficiency: The ratio of useful payload data to the overhead data.
Round-Trip Time (RTT): The ping delay time to the target server and back.
RTT Jitter: The variation in the delay of round-trip time.
Latency: The transmission delay time to the target server and back.
Latency Jitter: The variation in delay of latency.
Bit Error Rate: The corrupted bits as a percentage of the total
transmitted data.
Packet Loss: The percentage of packets lost that needed to be recovered.
Energy Efficiency: The amount of energy consumed to achieve the test result.

Did I overlook something? Too many dimensions to cover? Obviously some
of those are derived, so not part of the whole set as such.

Maybe then next would be to have different profiles where any of those
parameters may vary over the test run. e.g. profiles that model
congested base stations for mobile data. Different use case specific
payload profiles e.g. gop for video transfer?

Best regards,
Sauli


On 12/6/23, Dave Taht via Nnagain <nnagain@lists.bufferbloat.net> wrote:

This CFP looks pretty good to me: https://tma.ifip.org/2024/call-for-papers/

Because:

¨To further encourage the results’ faithfulness and avoid publication
bias, the conference will particularly encourage negative results
revealed by novel measurement methods or vantage points. All regular
papers are hence encouraged to discuss the limitations of the
presented approaches and also mention which experiments did not work.
Additionally, TMA will also be open to accepting papers that
exclusively deal with negative results, especially when new
measurement methods or perspectives offer insight into the limitations
and challenges of network measurement in practice. Negative results
will be evaluated based on their impact (e.g. revealed in realistic
production networks) as well as the novelty of the vantage points
(e.g. scarce data source) or measurement techniques that revealed
them."


--
:( My old R&D campus is up for sale: https://tinyurl.com/yurtlab
Dave Täht CSO, LibreQos
_______________________________________________
Nnagain mailing list
Nnagain@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/nnagain


_______________________________________________
Nnagain mailing list
Nnagain@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/nnagain

--------------3ts3aI9OilHOtcVB0NP1Fajl--