[Cake] cake vs fqcodel with 1 client, 4 servers

[xnor]

>At those speeds, you probably still have a large enough BDP per flow 
>that TCP can respond appropriately.  I'm testing at 512Kbps, where it's 
>easy to get into single-packet BDPs.
I've tried again with 96 connections to the same server, so 196 Kbps 
each, and that reduces the gross download rate from over 17 to below 10 
Mpbs.
The inbound interface still receives 18.7 Mbps and crude latency 
measurements with a ping running at the same time show only a slight 
increase on average.

The only problem I have with this (besides the amount of perfectly good 
received data that needs to be discarded without ECN to keep TCP in 
check... that's terrible protocol design) is that I do get 3x to 4x 
spikes over the steady state latency when the download begins. That's 
with "just" 16 additional connections starting.


>A first-pass dynamic target adjustment is now pushed, and running here. 
>  It doesn't solve the problem completely, probably because it doesn't 
>guarantee 4xMTU, only 1xMTU, but it looks like it might be beginning to 
>help and I'm pretty sure it's the correct approach.  Tuning it from 
>here should be easy.
>
I don't know exactly what you're adjusting and what you're adjusting to, 
but I see two things that would help:

1) Reduce the amount of data that needs to be dropped.
Maybe the "pattern" of how packets are dropped can be improved such that 
on *average* the sender sends at the same rate but overall less data 
needs to be dropped at the receiver.
For example, instead of dropping the just received packet because it 
exceeded some threshold and doing this for each packet it could make 
more sense to drop multiple packets at once, making the sender to slow 
down significantly.
The problem with this of course would be a more pronounced zig-zag 
pattern in the bandwidth (with the implementation either limiting on the 
peaks or average rate).
(I'm not sure if this makes sense given common TCP implementations.)

2) Predict the slope of the bandwidth and slow down the sender 
proactively, especially during slow-start.
If the connection is in an exponential speed up phase then we have to 
drop packets *before* we notice that the current (or even worse: 
average) rate is above the configured rate. The situation gets worse if 
multiple connections are in that phase at the same time, and ideally 
that would have to be accounted for as well.

Those are just some suggestions. I haven't studied cake's implementation 
so maybe it already does something like that.