From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yb1-xb30.google.com (mail-yb1-xb30.google.com [IPv6:2607:f8b0:4864:20::b30]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E85A83B2A4; Tue, 11 Oct 2022 13:01:06 -0400 (EDT) Received: by mail-yb1-xb30.google.com with SMTP id 203so17176769ybc.10; Tue, 11 Oct 2022 10:01:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=YmzaSJW3SgA6XQRZRd/dFfZUSS30V9kmLOen1zkW0zg=; b=jHxwSf1F0Nqx/+aJsWvingm7i7XnvbILE/Rjt/xqV2uCRPQgW4WZGlAhfFgSWO9PiD JVbyoBx7pEF2kxDUUAsI/yx7kJFhGJLoiQ7FBpXHe6FjneAawR0nfbVjC/rQdHbKDQc7 891ZtWVfBidV+iyvLBSBImxNvlh6eLlytGZFXxbF8pNn6VYt8p1qxWF6L0vsOTzco4+G 6yDSXw51QKRo3UrDkSrJARqgaY2TiVRLsDheVd4En3lzKUljG+Pt7R/qA0ueCUM3FlEg hXSjpLcajyTQje5E/lRxpkv8Rp+OZH+Qo45wPZT4+mjMgaHP03iVPAsElzErTwIoB6kF EKag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YmzaSJW3SgA6XQRZRd/dFfZUSS30V9kmLOen1zkW0zg=; b=mWz57bie11HjOXZ7VRO+AU65uRn4+WKvGUMWaYKYenApM5vF9X+aAix/LH3oVLetNx om2+bwN3XY9xD8ovQ/gyPneBf0e2JmnHa1DTb4Ae4qhrxB+P0afHLFM7tXziolOZJQHI 4wpN9pxlagAFUxqsqwz9j/0ZtRYzW2ik1dXV8bpCTP9BTuOfMafVe6Nk9bR2HkfjgQVM cSDPosZ+Hml8WkCgGoF82HNwtTr1+o+vy59OG8hzlL/hoy2Y9jts1WyyH5RFTaCzXwRO /aeEOewFQgEWtJrDORYcfOzs0gPYthioCe+NMNtgUMYHngR509+PkakRQjBU0UhC2Ttl hKeA== X-Gm-Message-State: ACrzQf1K9gEe16NK93eTqnUPDaWyhuC571IsJiHPy8kIRsPCOuqk2pZe MgQKvGkIXfLfzBauxlnIRvzkYaGMJttaWQCHRhI= X-Google-Smtp-Source: AMsMyM4ADspz5jUzp/QaR6V1ck1sqAefgbBQ0GEc7AELWMMyfCTW77lBUKGQSfti0Jal5oROCjqS0NAlhjpwbiuGWE8= X-Received: by 2002:a25:7311:0:b0:6c0:7141:586d with SMTP id o17-20020a257311000000b006c07141586dmr14763492ybc.597.1665507666236; Tue, 11 Oct 2022 10:01:06 -0700 (PDT) MIME-Version: 1.0 References: <72674884-9E0D-4645-B5F5-C593CC45A8F0@gmx.de> In-Reply-To: From: Dave Taht Date: Tue, 11 Oct 2022 10:00:52 -0700 Message-ID: To: Bob McMahon Cc: Sebastian Moeller , David Lang , Cake List , Make-Wifi-fast , Rpm , bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Rpm] [Bloat] [Make-wifi-fast] [Cake] The most wonderful video ever about bufferbloat X-BeenThere: rpm@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: revolutions per minute - a new metric for measuring responsiveness List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Oct 2022 17:01:07 -0000 On Tue, Oct 11, 2022 at 9:58 AM Bob McMahon via Rpm wrote: > > > Saturate a link in both directions simultaneously with multiple greedy = flows while measuring load-dependent latency changes for small isochronous = probe flows. > > This functionality is released in iperf 2.1.8 per the bounceback feature = but, unfortunately, OpenWRT doesn't maintain iperf 2 as a package anymore a= nd uses 2.0.13 iperf 2.1.8 was pushed into the openwrt mainline and may appear as of 22.03.1. I'll check. > > CLIENT SPECIFIC OPTIONS > > --bounceback[=3Dn]run a TCP bounceback or rps test with optional number w= rites in a burst per value of n. The default is ten writes every period and= the default period is one second (Note: set size with -l or --len which de= faults to 100 bytes)--bounceback-congest[=3Dup|down|bidir][,n]request a con= current working load or TCP stream(s), defaults to full duplex (or bidir) u= nless the up or down option is provided. The number of TCP streams defaults= to 1 and can be changed via the n value, e.g. --bounceback-congest=3Ddown,= 4 will use four TCP streams from server to the client as the working load. = The IP ToS will be BE (0x0) for working load traffic.--bounceback-hold nreq= uest the server to insert a delay of n milliseconds between its read and wr= ite (default is no delay)--bounceback-period[=3Dn]request the client schedu= le its send(s) every n seconds (default is one second, use zero value for i= mmediate or continuous back to back)--bounceback-no-quickackrequest the ser= ver not set the TCP_QUICKACK socket option (disabling TCP ACK delays) durin= g a bounceback test (see NOTES)--bounceback-txdelay nrequest the client to = delay n seconds between the start of the working load and the bounceback tr= affic (default is no delay) > > On Tue, Oct 11, 2022 at 12:15 AM Sebastian Moeller wrot= e: >> >> Hi Bob, >> >> On 11 October 2022 02:05:40 CEST, Bob McMahon = wrote: >> >It's too big because it's oversized so it's in the size domain. It's >> >basically Little's law's value for the number of items in a queue. >> > >> >*Number of items in the system =3D (the rate items enter and leave the >> >system) x (the average amount of time items spend in the system)* >> > >> > >> >Which gets driven to the standing queue size when the arrival rate >> >exceeds the service rate - so the driving factor isn't the service and >> >arrival rates, but *the queue size *when *any service rate is less than= an >> >arrival rate.* >> >> [SM] You could also argue it is the ratio of arrival to service rates, w= ith the queue size being a measure correlating with how long the system wil= l tolerate ratios larger than one... >> >> >> > >> >In other words, one can find and measure bloat regardless of the >> >enter/leave rates (as long as the leave rate is too slow) and the value= of >> >memory units found will always be the same. >> > >> >Things like prioritizations to jump the line are somewhat of hacks at >> >reducing the service time for a specialized class of packets but nobody >> >really knows which packets should jump. >> >> [SM] Au contraire most everybody 'knows' it is their packets that should= jump ahead of the rest ;) For intermediate hop queues however that endpoin= t perception is not really actionable due to lack of robust and reliable im= portance identifiers on packets. In side a 'domain' dscps might work if tre= ated to strict admission control, but that typically will not help end2end = traffic over the internet. This is BTW why I think FQ is a great concept, a= s it mostly results in the desirable outcome of not picking winners and los= ers (like arbitrarily starving a flow), but I digress. >> >> >Also, nobody can define what >> >working conditions are so that's another problem with this class of tes= ts. >> >> [SM] While real working conditions will be different for each link and p= robably vary over time, it seems achievable to come up with a set of pessim= istic assumptions how to model a challenging work condition against which t= o test potential remedies, assuming that such remedies will also work well = under less challenging conditions, no? >> >> >> > >> >Better maybe just to shrink the queue and eliminate all unneeded queuei= ng >> >delays. >> >> [SM] The 'unneeded' does a lot of work in that sentence ;). I like Van's= ? Description of queues as shock absorbers so queue size will have a lower = acceptable limit assuming users want to achieve 'acceptable' throughput eve= n with existing bursty senders. (Not all applications are suited for pacing= so some level of burstiness seems unavoidable). >> >> >> > Also, measure the performance per "user conditions" which is going >> >to be different for almost every environment (and is correlated to time= and >> >space.) So any engineering solution is fundamentally suboptimal. >> >> [SM] A matter of definition, if the requirement is to cover many user co= nditions the optimality measure simply needs to be changed from per individ= ual condition to over many/all conditions, no? >> >> >Even >> >pacing the source doesn't necessarily do the right thing because that's >> >like waiting in the waitlist while at home vs the restaurant lobby. >> >> [SM] +1. >> >> > Few >> >care about where messages wait (unless the pitch is AQM is the only >> >solution that drives to a self-fulfilling prophecy - that's why the tes= ts >> >have to come up with artificial conditions that can't be simply defined= .) >> >> Hrm, so the RRUL test, while not the end all of bufferbloat/working cond= itions tests, is not that complicated: >> Saturate a link in both directions simultaneously with multiple greedy f= lows while measuring load-dependent latency changes for small isochronous p= robe flows. >> >> Yes, the it would be nice to have additional higher rate probe flows als= o bursty ones to emulate on-linev games, and 'pumped' greedy flows to emula= te DASH 'streaming', and a horde of small greedy flows that mostly end insi= de the initial window and slow start. But at its core existing RRUL already= gives a useful estimate on how a link behaves under saturating loads all t= he while being relatively simple. >> The responsiveness under working condition seems similar in that it trie= s to saturate a link with an increasing number of greedy flows, in a sense = to create a reasonable bad case that ideally rarely happens. >> >> Regards >> Sebastian >> >> >> > >> >Bob >> > >> >On Mon, Oct 10, 2022 at 3:57 PM David Lang wrote: >> > >> >> On Mon, 10 Oct 2022, Bob McMahon via Bloat wrote: >> >> >> >> > I think conflating bufferbloat with latency misses the subtle point= in >> >> that >> >> > bufferbloat is a measurement in memory units more than a measuremen= t in >> >> > time units. The first design flaw is a queue that is too big. This >> >> youtube >> >> > video analogy doesn't help one understand this important point. >> >> >> >> but the queue is only too big because of the time it takes to empty t= he >> >> queue, >> >> which puts us back into the time domain. >> >> >> >> David Lang >> >> >> >> > Another subtle point is that the video assumes AQM as the only solu= tion >> >> and >> >> > ignores others, i.e. pacing at the source(s) and/or faster service >> >> rates. A >> >> > restaurant that let's one call ahead to put their name on the waitl= ist >> >> > doesn't change the wait time. Just because a transport layer slowed= down >> >> > and hasn't congested a downstream queue doesn't mean the e2e latenc= y >> >> > performance will meet the gaming needs as an example. The delay is = still >> >> > there it's just not manifesting itself in a shared queue that may o= r may >> >> > not negatively impact others using that shared queue. >> >> > >> >> > Bob >> >> > >> >> > >> >> > >> >> > On Mon, Oct 10, 2022 at 2:40 AM Sebastian Moeller via Make-wifi-fas= t < >> >> > make-wifi-fast@lists.bufferbloat.net> wrote: >> >> > >> >> >> Hi Erik, >> >> >> >> >> >> >> >> >>> On Oct 10, 2022, at 11:32, Taraldsen Erik >> >> >> wrote: >> >> >>> >> >> >>> On 10/10/2022, 11:09, "Sebastian Moeller" wrote= : >> >> >>> >> >> >>> Nice! >> >> >>> >> >> >>>> On Oct 10, 2022, at 07:52, Taraldsen Erik via Cake < >> >> >> cake@lists.bufferbloat.net> wrote: >> >> >>>> >> >> >>>> It took about 3 hours from the video was release before we got t= he >> >> >> first request to have SQM on the CPE's we manage as a ISP. Final= ly >> >> >> getting some customer response on the issue. >> >> >>> >> >> >>> [SM] Will you be able to bump these requests to higher-ups = and at >> >> >> least change some perception of customer demand for tighter latenc= y >> >> >> performance? >> >> >>> >> >> >>> That would be the hope. >> >> >> >> >> >> [SM} Excellent, hope this plays out as we wish for. >> >> >> >> >> >> >> >> >>> We actually have fq_codel implemented on the two latest generati= ons of >> >> >> DSL routers. Use sync rate as input to set the rate. Works quite= well. >> >> >> >> >> >> [SM] Cool, if I might ask what fraction of the sync are yo= u >> >> >> setting the traffic shaper for and are you doing fine grained over= head >> >> >> accounting (or simply fold that into a grand "de-rating"-factor)? >> >> >> >> >> >> >> >> >>> There is also a bit of traction around speedtest.net's inclusion = of >> >> >> latency under load internally. >> >> >> >> >> >> [SM] Yes, although IIUC they are reporting the interquarti= le >> >> mean >> >> >> for the two loaded latency estimates, which is pretty conservative= and >> >> only >> >> >> really "triggers" for massive consistently elevated latency; so I = expect >> >> >> this to be great for detecting really bad cases, but I fear it is = too >> >> >> conservative and will make a number of problematic links look OK. = But >> >> hey, >> >> >> even that is leaps and bounds better than the old only idle latenc= y >> >> report. >> >> >> >> >> >> >> >> >>> My hope is that some publication in Norway will pick up on that s= core >> >> >> and do a test and get some mainstream publicity with the results. >> >> >> >> >> >> [SM] Inside the EU the challenge is to get national regula= tors >> >> and >> >> >> the BEREC to start bothering about latency-under-load at all, "som= e >> >> >> mainstream publicity" would probably help here as well. >> >> >> >> >> >> Regards >> >> >> Sebastian >> >> >> >> >> >> >> >> >>> >> >> >>> -Erik >> >> >>> >> >> >>> >> >> >>> >> >> >> >> >> >> _______________________________________________ >> >> >> Make-wifi-fast mailing list >> >> >> Make-wifi-fast@lists.bufferbloat.net >> >> >> https://lists.bufferbloat.net/listinfo/make-wifi-fast >> >> > >> >> >_______________________________________________ >> >> Bloat mailing list >> >> Bloat@lists.bufferbloat.net >> >> https://lists.bufferbloat.net/listinfo/bloat >> >> >> > >> >> -- >> Sent from my Android device with K-9 Mail. Please excuse my brevity. > > > This electronic communication and the information and any files transmitt= ed with it, or attached to it, are confidential and are intended solely for= the use of the individual or entity to whom it is addressed and may contai= n information that is confidential, legally privileged, protected by privac= y laws, or otherwise restricted from disclosure to anyone else. If you are = not the intended recipient or the person responsible for delivering the e-m= ail to the intended recipient, you are hereby notified that any use, copyin= g, distributing, dissemination, forwarding, printing, or copying of this e-= mail is strictly prohibited. If you received this e-mail in error, please r= eturn the e-mail to the sender, delete it from your computer, and destroy a= ny printed copy of it._______________________________________________ > Rpm mailing list > Rpm@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/rpm --=20 This song goes out to all the folk that thought Stadia would work: https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-69813666656= 07352320-FXtz Dave T=C3=A4ht CEO, TekLibre, LLC