During the flent rrul test, another machine in my home had started a Steam game update, this opened somewhere between 8-12 connections and saturated my downstream.

The interesting part was the resulting rrul graph, showing perfect latency control during the test, but 100-150ms of bufferbloat at the very start and end of the rrul graph (when rrul is idle, just pinging).

Instead of a Steam download I started an iperf3 client on a remote machine that opened 16 connections to a LAN machine. Same behavior as the Steam download.

At first I thought this was a bug with cake, but no matter how I tuned cake (limiting my 400/40 connection to 50/20 for example), the behavior was similar.

I tried fq_codel+htb, and got similar results to cake, high latency before the rrul test kicked off, great latency while rrul was doing its up and downloads.

So I swapped out my self-compile LEDE router with an Edgerouter ER-X with both stock and LEDE firmware, both with fq_codel and cake (on both stock and LEDE firmware). Same results.

Even without any SQM active, the rrul test still improved the latency of my connection when it's also downloading from 8+ connections at the same time.

I managed to take the flent rrul test out of the equation by starting a hping3 "flood" to the hop next of my cable modem. 

 

This also fixed the latency while many downloads were running by sending about 300KB/s of empty UDP packets to some unresponsive port on my ISPs local gateway.

I'm puzzled by this result, somehow sending 300KB/s of empty packets upstream drastically improves the latency of my connection when it's receiving many downloads.

Equipment used:

Cisco EPC3212 cable modem (8x4 EuroDOCSIS 3.0)

Linksys WRT1900ACS router with LEDE

Ubiquiti Edgerouter ER-X with both EdgeOS (stock) and LEDE