On Wed, Jun 3, 2015 at 3:27 PM, Dave Taht wrote: > > > On Wed, Jun 3, 2015 at 3:16 PM, Aaron Wood wrote: > >> >> > On the 3800, it never meets the rate, but it's only off by maybe 5%. >>> >>> As Jonathan pointed out already this is in the range of the >>> difference between raw rates and tcp good put, so nothing to write home >>> about ;) >>> >> >> Yeah, I'm not too worried about that 5%, based on that explanation. >> >> >>> >>> > But on my new WRT1900AC, it's wildly off, even over the same >>> performance range (I tested it from 80-220Mbps rates in 20Mbps jumps, and >>> saw from 40-150Mbps. >>> >>> So you started with the WRT1900AC where the wndr3800 dropped >>> off? I wonder maybe the Belkin is also almost linear for the lower range? >> >> >> Yeah, good point on a methodology fail. I'll run another series of tests >> walking up the same series of rate limits and see what I get. >> >> >>> I also note we adjust the quantum based on the rates: >>> from functions .sh: >>> get_mtu() { >>> >> ... snip >> >>> } >>> >>> which we use in the htb invocations via this indirection: >>> LQ="quantum `get_mtu $IFACE $CEIL`” >>> >>> >> That is odd, and that's quite the aggressive curve on quantum, doubling >> every 10-20Mbps. >> >> I did some math, and plotted out the quantum vs. bandwidth based on that >> snippet of code (and assuming a 1500-byte MTU): >> >> >> >> ​And then plotted out the corresponding time in ms that each quantum >> bytes (it is bytes, right?) is on the wire: >> >> >> ​Which I think is a really interesting plot (and here are the points that >> line up with the steps in the script): >> >> kbps = quantum = time >> 20000 = 3000 = 1.2ms >> 30000 = 6000 = 1.6ms >> 40000 = 12000 = 2.4ms >> 50000 = 24000 = 3.84ms >> 60000 = 48000 = 6.4ms >> 80000 = 96000 = 9.6ms >> > > >> So it appears that the goal of these values was to keep increases the >> quantum as rates went up to provide more bytes per operation, but that's >> going to risk adding latency as the time-per-quantum crosses the delay >> target in fq_codel (if I'm understanding this correctly). >> >> So one thing that I can do is play around with this, and see if I can >> keep that quantum time at a linear level (ie, 10ms, which seems _awfully_ >> long), or continue increasing it (which seems like a bad idea). I'd love >> to hear from whoever put this in as to what it's goal was (or was it just >> empirically tuned?) >> > > Empirical and tested only to about 60Mbits. I got back about 15% cpu to do > it this way at the time I did it on the wndr3800. > > and WOW, thx for the analysis! I did not think much about this crossover > point at the time - because we'd maxed on cpu long beforehand. > And most of my testing on x86 has been with this change to the htb quantum entirely disabled and set to 1514. the "production" sqm-scripts and my own hacked up version(s) have differed in this respect for quite some time. (at least 6 months). Great spot on this discrepancy. :egg, otherwise, on face: > I can certainly see this batching interacting with the codel target. > > On the other hand, you gotta not be running out of cpu in the first place. > I am liking where cake is going. > > One of my daydreams is that once we have writable custom ethernet hardware > that we can easily do hardware outbound rate limiting/shaping merely by > programming a register to return a completion interrupt at the set rate > rather than the actual rate. > > > >> > >>> > I have no idea where to start looking for the cause. But for now, I'm >>> just setting my ingress rate MUCH higher than I should, because it's >>> working out to the right value as a result. >>> >>> It would be great to understand why we need to massively >>> under-shape in that situation to get decent shaping and decent latency >>> under load. >>> >> >> Agreed. >> >> -Aaron >> >> _______________________________________________ >> Cerowrt-devel mailing list >> Cerowrt-devel@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cerowrt-devel >> >> > > > -- > Dave Täht > What will it take to vastly improve wifi for everyone? > https://plus.google.com/u/0/explore/makewififast > -- Dave Täht What will it take to vastly improve wifi for everyone? https://plus.google.com/u/0/explore/makewififast