> On the 3800, it never meets the rate, but it's only off by maybe 5%.

As Jonathan pointed out already this is in the range of the difference between raw rates and tcp good put, so nothing to write home about ;)

Yeah, I'm not too worried about that 5%, based on that explanation.

> But on my new WRT1900AC, it's wildly off, even over the same performance range (I tested it from 80-220Mbps rates in 20Mbps jumps, and saw from 40-150Mbps.

So you started with the WRT1900AC where the wndr3800 dropped off? I wonder maybe the Belkin is also almost linear for the lower range?

Yeah, good point on a methodology fail. I'll run another series of tests walking up the same series of rate limits and see what I get.

I also note we adjust the quantum based on the rates:
from functions .sh:
get_mtu() {

... snip

}

which we use in the htb invocations via this indirection:
LQ="quantum `get_mtu $IFACE $CEIL`”

That is odd, and that's quite the aggressive curve on quantum, doubling every 10-20Mbps.

I did some math, and plotted out the quantum vs. bandwidth based on that snippet of code (and assuming a 1500-byte MTU):

And then plotted out the corresponding time in ms that each quantum bytes (it is bytes, right?) is on the wire:

Which I think is a really interesting plot (and here are the points that line up with the steps in the script):

kbps = quantum = time

20000 = 3000 = 1.2ms

30000 = 6000 = 1.6ms

40000 = 12000 = 2.4ms

50000 = 24000 = 3.84ms

60000 = 48000 = 6.4ms

80000 = 96000 = 9.6ms

So it appears that the goal of these values was to keep increases the quantum as rates went up to provide more bytes per operation, but that's going to risk adding latency as the time-per-quantum crosses the delay target in fq_codel (if I'm understanding this correctly).

So one thing that I can do is play around with this, and see if I can keep that quantum time at a linear level (ie, 10ms, which seems _awfully_ long), or continue increasing it (which seems like a bad idea). I'd love to hear from whoever put this in as to what it's goal was (or was it just empirically tuned?)

>
> I have no idea where to start looking for the cause. But for now, I'm just setting my ingress rate MUCH higher than I should, because it's working out to the right value as a result.

It would be great to understand why we need to massively under-shape in that situation to get decent shaping and decent latency under load.

Agreed.

-Aaron