I'm seeing something interesting using fq_codel, protectcli vault and opnsense. The background load going active cause the responses per second to increase and the bb times to go from 15 ms to 10 ms. I added four upstream TCP streams (not shown) during the yellow time slots and the bounceback delay became lesser! rjmcmahon@fedora:~/Code/bloat/iperf2-code$ iperf -c -i 1 -e --bounceback -t 60 --hide-ips ------------------------------------------------------------ Client connecting to (**hidden**), TCP port 5001 with pid 87070 (1/0 flows/load) Bounceback test (req/reply size = 100 Byte/ 100 Byte) (server hold req=0 usecs & tcp_quickack) Bursting request 10 times every 1.00 second(s) TCP congestion control using cubic TOS defaults to 0x0 (dscp=0,ecn=0) and nodelay (Nagle off) TCP window size: 85.3 KByte (default) ------------------------------------------------------------ [ 1] local *.*.*.59%enp4s0 port 56380 connected with *.*.*.123 port 5001 (bb w/quickack req/reply/hold=100/100/0) (icwnd/mss/irtt=14/1448/10624) (ct=10.71 ms) on 2024-12-01 13:49:05.856 (PST) [ ID] Interval Transfer Bandwidth BB cnt=avg/min/max/stdev Rtry Cwnd(pkts)/RTT(var) Tx/Rx bytes RPS(avg) [ 1] 0.00-1.00 sec 1.95 KBytes 16.0 Kbits/sec 10=15.609/11.041/19.933/2.256 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=64 [ 1] 1.00-2.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.806/11.047/16.801/1.918 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=68 [ 1] 2.00-3.00 sec 1.95 KBytes 16.0 Kbits/sec 10=13.946/10.260/16.458/2.416 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=72 [ 1] 3.00-4.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.997/10.714/16.314/1.579 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=67 [ 1] 4.00-5.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.612/11.687/15.938/1.504 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=68 [ 1] 5.00-6.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.919/11.177/15.786/1.339 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=67 [ 1] 6.00-7.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.381/10.485/16.311/2.250 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=70 [ 1] 7.00-8.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.221/10.669/17.170/2.404 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=70 [ 1] 8.00-9.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.417/11.493/16.045/1.400 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=69 [ 1] 9.00-10.00 sec 1.95 KBytes 16.0 Kbits/sec 10=12.815/11.467/13.680/0.694 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=78 *[ 1] 10.00-11.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.200/9.071/11.488/0.903 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=98[ 1] 11.00-12.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.236/8.838/11.345/0.807 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=98[ 1] 12.00-13.00 sec 1.95 KBytes 16.0 Kbits/sec 10=9.560/8.512/11.301/0.782 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=105[ 1] 13.00-14.00 sec 1.95 KBytes 16.0 Kbits/sec 10=9.939/9.245/10.944/0.552 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=101[ 1] 14.00-15.00 sec 1.95 KBytes 16.0 Kbits/sec 10=13.722/12.576/15.569/0.855 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=73[ 1] 15.00-16.00 sec 1.95 KBytes 16.0 Kbits/sec 10=11.856/10.674/12.693/0.624 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=84[ 1] 16.00-17.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.260/8.872/11.085/0.652 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=97[ 1] 17.00-18.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.044/9.152/11.054/0.755 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=100[ 1] 18.00-19.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.199/8.724/11.377/0.928 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=98* [ 1] 19.00-20.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.831/11.103/18.682/1.886 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=67 [ 1] 20.00-21.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.657/11.171/15.684/1.574 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=68 [ 1] 21.00-22.00 sec 1.95 KBytes 16.0 Kbits/sec 10=15.283/10.757/18.079/2.311 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=65 [ 1] 22.00-23.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.960/10.551/16.555/1.991 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=67 [ 1] 23.00-24.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.699/10.847/16.979/1.815 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=68 [ 1] 24.00-25.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.240/10.476/15.544/1.733 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=70 [ 1] 25.00-26.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.230/10.513/15.785/1.972 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=70 [ 1] 26.00-27.00 sec 1.95 KBytes 16.0 Kbits/sec 10=13.530/10.297/15.595/2.433 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=74 Bob On Sun, Dec 1, 2024 at 2:32 PM Frantisek Borsik via Nnagain < nnagain@lists.bufferbloat.net> wrote: > Good catch, Michael. Forwarded it to Sina, CEO Waveform. > > All the best, > > Frank > > Frantisek (Frank) Borsik > > > > https://www.linkedin.com/in/frantisekborsik > > Signal, Telegram, WhatsApp: +421919416714 > > iMessage, mobile: +420775230885 > > Skype: casioa5302ca > > frantisek.borsik@gmail.com > > > On Sun, Dec 1, 2024 at 6:43 PM Michael Richardson via Nnagain < > nnagain@lists.bufferbloat.net> wrote: > >> I hadn't seen https://www.waveform.com/tools/bufferbloat before. >> Very pretty. I tried it just now, at the diner, tethered to my phone, >> hardly >> surprised to get an F. I wish the mobile carriers cared. >> Alas, the suggestions at the bottom include IQrouter, which is no longer >> in >> production. Probably should include OpenWRT One. >> >> -- >> ] Never tell me the odds! | ipv6 mesh >> networks [ >> ] Michael Richardson, Sandelman Software Works | IoT >> architect [ >> ] mcr@sandelman.ca http://www.sandelman.ca/ | ruby on >> rails [ >> >> _______________________________________________ >> Nnagain mailing list >> Nnagain@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/nnagain >> > _______________________________________________ > Nnagain mailing list > Nnagain@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/nnagain >