From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mo4-p01-ob.smtp.rzone.de (mo4-p01-ob.smtp.rzone.de [81.169.146.166]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id C4CCF3CB37; Tue, 13 Jul 2021 03:15:10 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1626160490; s=strato-dkim-0002; d=rizk.com.de; h=Message-ID:Date:Subject:In-Reply-To:References:Cc:To:From:Cc:Date: From:Subject:Sender; bh=SGLzpphi4T5NLxTluvTnpOJ4gbaAD8IE3ekeoe0cLVQ=; b=FTt/FDXdp9l4yXOoQb7g5K2o0Rg+wAj/laFymQFWn+skdCGhc0ikGOZO7CXnGbtapq wUQZVbD280Zqv0OwNzp/aQE4YsQunWILVHhtqsHGxmeYQgxgrDciGZwOUIXcZjgepxu6 cRQkpNn663a53GadZdKxsYqN1hc6FcScVWcA65XvwQeGM5rTe6JGFf6TfR5iu8PMB1iJ c30C13Gu1s+xDu2zW5wHlzbVY+8lLFg+rmghf+uN3RghaCDMdbMRU+S+83hQCqMOU/E1 xY8CnlGyePGOtgqtZHJ7YXgDNjI/9j9aFmqvkRsloiwVDiJTizqnrsHEEPCdEB72WKv4 YlUg== Authentication-Results: strato.com; dkim=none X-RZG-AUTH: ":LWEWVVKhYfTw4zlzfMwJnVq40pjV1GLj8DAaj+soHJhMVuM98IRp3dpH6tSrLp41SAS0X50=" X-RZG-CLASS-ID: mo00 Received: from NCSNB2104a by smtp.strato.de (RZmta 47.28.1 AUTH) with ESMTPSA id j044b8x6D7Eowsz (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256 bits)) (Client did not present a certificate); Tue, 13 Jul 2021 09:14:50 +0200 (CEST) From: "Amr Rizk" To: "'Ben Greear'" , "'Bob McMahon'" Cc: , "'Make-Wifi-fast'" , "'Leonard Kleinrock'" , "'David P. Reed'" , "'Cake List'" , , "'cerowrt-devel'" , "'bloat'" References: <1625188609.32718319@apps.rackspace.com> <989de0c1-e06c-cda9-ebe6-1f33df8a4c24@candelatech.com> <1625773080.94974089@apps.rackspace.com> <1625859083.09751240@apps.rackspace.com> <1626111630.69692379@apps.rackspace.com> <9c3d61c1-7013-414e-964d-9e83f596e69d@candelatech.com> <1e8bdf58-2a21-f543-a248-be58bcbddbcf@candelatech.com> In-Reply-To: <1e8bdf58-2a21-f543-a248-be58bcbddbcf@candelatech.com> Date: Tue, 13 Jul 2021 09:14:49 +0200 Message-ID: <02c601d777b6$c4ce5a10$4e6b0e30$@rizk.com.de> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQIpkeXPK9TTdTsfz6hOSLk27Y5b9wIS9hrDAiraS/kA4qddxAIZGTeqAboI73gBa9qzkAGXC5AfAoD9k9kBh9+syQIw8u76Awr/S6EBiolHiwE0++jfAdd+Q7apzbGaAA== Content-Language: de X-Mailman-Approved-At: Tue, 13 Jul 2021 06:06:12 -0400 Subject: Re: [Cerowrt-devel] [Bloat] Little's Law mea culpa, but not invalidating my main point X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jul 2021 07:15:11 -0000 Ben,=20 it depends on what one tries to measure. Doing a rate scan using UDP (to = measure latency distributions under load) is the best thing that we have = but without actually knowing how resources are shared (fair share as in = WiFi, FIFO as nearly everywhere else) it becomes very difficult to = interpret the results or provide a proper argument on latency. You are = right - TCP stats are a proxy for user experience but I believe they are = difficult to reproduce (we are always talking about very short TCP flows = - the infinite TCP flow that converges to a steady behavior is purely = academic). By the way, Little's law is a strong tool when it comes to averages. To = be able to say more (e.g. 1% of the delays is larger than x) one = requires more information (e.g. the traffic - On-OFF pattern) see [1]. = I am not sure when does such information readily exist.=20 Best Amr=20 [1] https://dl.acm.org/doi/10.1145/3341617.3326146 or if behind a = paywall https://www.dcs.warwick.ac.uk/~florin/lib/sigmet19b.pdf -------------------------------- Amr Rizk (amr.rizk@uni-due.de) University of Duisburg-Essen -----Urspr=C3=BCngliche Nachricht----- Von: Bloat Im Auftrag von Ben = Greear Gesendet: Montag, 12. Juli 2021 22:32 An: Bob McMahon Cc: starlink@lists.bufferbloat.net; Make-Wifi-fast = ; Leonard Kleinrock = ; David P. Reed ; Cake List = ; codel@lists.bufferbloat.net; cerowrt-devel = ; bloat = Betreff: Re: [Bloat] Little's Law mea culpa, but not invalidating my = main point UDP is better for getting actual packet latency, for sure. TCP is = typical-user-experience-latency though, so it is also useful. I'm interested in the test and visualization side of this. If there = were a way to give engineers a good real-time look at a complex = real-world network, then they have something to go on while trying to = tune various knobs in their network to improve it. I'll let others try to figure out how build and tune the knobs, but the = data acquisition and visualization is something we might try to = accomplish. I have a feeling I'm not the first person to think of this, = however....probably someone already has done such a thing. Thanks, Ben On 7/12/21 1:04 PM, Bob McMahon wrote: > I believe end host's TCP stats are insufficient as seen per the=20 > "failed" congested control mechanisms over the last decades. I think=20 > Jaffe pointed this out in > 1979 though he was using what's been deemed on this thread as = "spherical cow queueing theory." >=20 > "Flow control in store-and-forward computer networks is appropriate=20 > for decentralized execution. A formal description of a class of=20 > "decentralized flow control algorithms" is given. The feasibility of=20 > maximizing power with such algorithms is investigated. On the=20 > assumption that communication links behave like M/M/1 servers it is = shown that no "decentralized flow control algorithm" can maximize = network power. Power has been suggested in the literature as a network = performance objective. It is also shown that no objective based only on = the users' throughputs and average delay is decentralizable. Finally, a = restricted class of algorithms cannot even approximate power." >=20 > https://ieeexplore.ieee.org/document/1095152 >=20 > Did Jaffe make a mistake? >=20 > Also, it's been observed that latency is non-parametric in it's=20 > distributions and computing gaussians per the central limit theorem=20 > for OWD feedback loops aren't effective. How does one design a control = loop around things that are non-parametric? It also begs the question, = what are the feed forward knobs that can actually help? >=20 > Bob >=20 > On Mon, Jul 12, 2021 at 12:07 PM Ben Greear > wrote: >=20 > Measuring one or a few links provides a bit of data, but seems = like if someone is trying to understand > a large and real network, then the OWD between point A and B needs = to just be input into something much > more grand. Assuming real-time OWD data exists between 100 to = 1000 endpoint pairs, has anyone found a way > to visualize this in a useful manner? >=20 > Also, considering something better than ntp may not really scale = to 1000+ endpoints, maybe round-trip > time is only viable way to get this type of data. In that case, = maybe clever logic could use things > like trace-route to get some idea of how long it takes to get = 'onto' the internet proper, and so estimate > the last-mile latency. My assumption is that the last-mile = latency is where most of the pervasive > assymetric network latencies would exist (or just ping 8.8.8.8 = which is 20ms from everywhere due to > $magic). >=20 > Endpoints could also triangulate a bit if needed, using some = anchor points in the network > under test. >=20 > Thanks, > Ben >=20 > On 7/12/21 11:21 AM, Bob McMahon wrote: > > iperf 2 supports OWD and gives full histograms for TCP write to = read, TCP connect times, latency of packets (with UDP), latency of = "frames" with > > simulated video traffic (TCP and UDP), xfer times of bursts = with low duty cycle traffic, and TCP RTT (sampling based.) It also has = support for sampling (per > > interval reports) down to 100 usecs if configured with = --enable-fastsampling, otherwise the fastest sampling is 5 ms. We've = released all this as open source. > > > > OWD only works if the end realtime clocks are synchronized = using a "machine level" protocol such as IEEE 1588 or PTP. Sadly, *most = data centers don't > provide > > sufficient level of clock accuracy and the GPS pulse per second = * to colo and vm customers. > > > > https://iperf2.sourceforge.io/iperf-manpage.html > > > > Bob > > > > On Mon, Jul 12, 2021 at 10:40 AM David P. Reed = = >> wrote: > > > > > > On Monday, July 12, 2021 9:46am, "Livingood, Jason" = > >> said: > > > > > I think latency/delay is becoming seen to be as = important certainly, if not a more direct proxy for end user QoE. This = is all still evolving and I > have > > to say is a super interesting & fun thing to work on. :-) > > > > If I could manage to sell one idea to the management = hierarchy of communications industry CEOs (operators, vendors, ...) it = is this one: > > > > "It's the end-to-end latency, stupid!" > > > > And I mean, by end-to-end, latency to complete a task at a = relevant layer of abstraction. > > > > At the link level, it's packet send to packet receive = completion. > > > > But at the transport level including retransmission = buffers, it's datagram (or message) origination until the = acknowledgement arrives for that > message being > > delivered after whatever number of retransmissions, freeing = the retransmission buffer. > > > > At the WWW level, it's mouse click to display update = corresponding to completion of the request. > > > > What should be noted is that lower level latencies don't = directly predict the magnitude of higher-level latencies. But longer = lower level latencies > almost > > always amplfify higher level latencies. Often non-linearly. > > > > Throughput is very, very weakly related to these latencies, = in contrast. > > > > The amplification process has to do with the presence of = queueing. Queueing is ALWAYS bad for latency, and throughput only helps = if it is in exactly the > > right place (the so-called input queue of the bottleneck = process, which is often a link, but not always). > > > > Can we get that slogan into Harvard Business Review? Can we = get it taught in Managerial Accounting at HBS? (which does address = logistics/supply chain > queueing). > > > > > > > > > > > > > > > > This electronic communication and the information and any files = transmitted with it, or attached to it, are confidential and are = intended solely for the > use of > > the individual or entity to whom it is addressed and may = contain information that is confidential, legally privileged, protected = by privacy laws, or > otherwise > > restricted from disclosure to anyone else. If you are not the = intended recipient or the person responsible for delivering the e-mail = to the intended > recipient, > > you are hereby notified that any use, copying, distributing, = dissemination, forwarding, printing, or copying of this e-mail is = strictly prohibited. If you > > received this e-mail in error, please return the e-mail to the = sender, delete it from your computer, and destroy any printed copy of = it. >=20 >=20 > --=20 > Ben Greear > > Candela Technologies Inc http://www.candelatech.com >=20 >=20 > This electronic communication and the information and any files=20 > transmitted with it, or attached to it, are confidential and are=20 > intended solely for the use of the individual or entity to whom it is=20 > addressed and may contain information that is confidential, legally=20 > privileged, protected by privacy laws, or otherwise restricted from = disclosure to anyone else. If you are not the intended recipient or the = person responsible for delivering the e-mail to the intended recipient, = you are hereby notified that any use, copying, distributing, = dissemination, forwarding, printing, or copying of this e-mail is = strictly prohibited. If you received this e-mail in error, please return = the e-mail to the sender, delete it from your computer, and destroy any = printed copy of it. -- Ben Greear Candela Technologies Inc http://www.candelatech.com _______________________________________________ Bloat mailing list Bloat@lists.bufferbloat.net https://lists.bufferbloat.net/listinfo/bloat