IMHO, characterizing performance warrants more measurements, e.g.,: - amount (bytes, datagrams) presumed lost and re-transmitted by the sender - amount (bytes, datagrams) discarded at the receiver because they were already received - amount (bytes, datagrams) discarded at the receiver because they arrived too late to be useful With such data, it would be possible to measure things like "useful throughput", i.e., the data successfully delivered from source to destination which was actually useful for the associated user's application. Some useful measurements were included in RFC 1213, e.g., in the "TCP" and "UDP" sections, such as number of retransmissions.  These measurements likely require instrumentation in the sending and receiving computers, where the TCP and similar algorithms operate. Measurements also may depend strongly on the particular computer type and operating system.   Measurements obtained from various "probe" sources and destinations, such as a "test host", provide only part of a "comprehensive measurement".  A complete characterization of the performance achieved would include measurement data from the users' host computers attached to the Internet, as they are doing whatever the user does. I don't recall if the SNMP MIB definitions were ever extended to include metrics appropriate for uses such as VOIP, video conferencing, remote operation, et al.   Those are the kinds of uses which are most sensitive to latency and variance in latency and throughput, and probably most interesting to measure. Jack Haverty On 12/6/23 13:46, Sauli Kiviranta via Nnagain wrote: > Thank you for sharing! This looks very promising! > > What would be a comprehensive measurement? Should cover all/most relevant areas? > > Payload Size: The size of data being transmitted. > Event Rate: The frequency at which payloads are transmitted. > Bitrate: The combination of rate and size transferred in a given test. > Bandwidth: The data transfer capacity available on the test path. > Throughput: The data transfer capability achieved on the test path. > Transfer Efficiency: The ratio of useful payload data to the overhead data. > Round-Trip Time (RTT): The ping delay time to the target server and back. > RTT Jitter: The variation in the delay of round-trip time. > Latency: The transmission delay time to the target server and back. > Latency Jitter: The variation in delay of latency. > Bit Error Rate: The corrupted bits as a percentage of the total > transmitted data. > Packet Loss: The percentage of packets lost that needed to be recovered. > Energy Efficiency: The amount of energy consumed to achieve the test result. > > Did I overlook something? Too many dimensions to cover? Obviously some > of those are derived, so not part of the whole set as such. > > Maybe then next would be to have different profiles where any of those > parameters may vary over the test run. e.g. profiles that model > congested base stations for mobile data. Different use case specific > payload profiles e.g. gop for video transfer? > > Best regards, > Sauli > > > On 12/6/23, Dave Taht via Nnagain wrote: >> This CFP looks pretty good to me:https://tma.ifip.org/2024/call-for-papers/ >> >> Because: >> >> ¨To further encourage the results’ faithfulness and avoid publication >> bias, the conference will particularly encourage negative results >> revealed by novel measurement methods or vantage points. All regular >> papers are hence encouraged to discuss the limitations of the >> presented approaches and also mention which experiments did not work. >> Additionally, TMA will also be open to accepting papers that >> exclusively deal with negative results, especially when new >> measurement methods or perspectives offer insight into the limitations >> and challenges of network measurement in practice. Negative results >> will be evaluated based on their impact (e.g. revealed in realistic >> production networks) as well as the novelty of the vantage points >> (e.g. scarce data source) or measurement techniques that revealed >> them." >> >> >> -- >> :( My old R&D campus is up for sale:https://tinyurl.com/yurtlab >> Dave Täht CSO, LibreQos >> _______________________________________________ >> Nnagain mailing list >> Nnagain@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/nnagain >> > _______________________________________________ > Nnagain mailing list > Nnagain@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/nnagain