<div dir="ltr">I'm seeing something interesting using fq_codel, protectcli vault and opnsense. The background load going active cause the responses per second to increase and the bb times to go from 15 ms to 10 ms. I added four upstream TCP streams (not shown) during the yellow time slots and the bounceback delay became lesser!<br><br>rjmcmahon@fedora:~/Code/bloat/iperf2-code$ iperf -c <my-cloud-server> -i 1 -e --bounceback -t 60 --hide-ips<br>------------------------------------------------------------<br>Client connecting to (**hidden**), TCP port 5001 with pid 87070 (1/0 flows/load)<br>Bounceback test (req/reply size = 100 Byte/ 100 Byte) (server hold req=0 usecs & tcp_quickack)<br>Bursting request 10 times every 1.00 second(s)<br>TCP congestion control using cubic<br>TOS defaults to 0x0 (dscp=0,ecn=0) and nodelay (Nagle off)<br>TCP window size: 85.3 KByte (default)<br>------------------------------------------------------------<br>[ 1] local *.*.*.59%enp4s0 port 56380 connected with *.*.*.123 port 5001 (bb w/quickack req/reply/hold=100/100/0) (icwnd/mss/irtt=14/1448/10624) (ct=10.71 ms) on 2024-12-01 13:49:05.856 (PST)<br>[ ID] Interval Transfer Bandwidth BB cnt=avg/min/max/stdev Rtry Cwnd(pkts)/RTT(var) Tx/Rx bytes RPS(avg)<br>[ 1] 0.00-1.00 sec 1.95 KBytes 16.0 Kbits/sec 10=15.609/11.041/19.933/2.256 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=64<br>[ 1] 1.00-2.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.806/11.047/16.801/1.918 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=68<br>[ 1] 2.00-3.00 sec 1.95 KBytes 16.0 Kbits/sec 10=13.946/10.260/16.458/2.416 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=72<br>[ 1] 3.00-4.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.997/10.714/16.314/1.579 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=67<br>[ 1] 4.00-5.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.612/11.687/15.938/1.504 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=68<br>[ 1] 5.00-6.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.919/11.177/15.786/1.339 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=67<br>[ 1] 6.00-7.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.381/10.485/16.311/2.250 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=70<br>[ 1] 7.00-8.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.221/10.669/17.170/2.404 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=70<br>[ 1] 8.00-9.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.417/11.493/16.045/1.400 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=69<br>[ 1] 9.00-10.00 sec 1.95 KBytes 16.0 Kbits/sec 10=12.815/11.467/13.680/0.694 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=78<br><b style="background-color:rgb(255,255,0)">[ 1] 10.00-11.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.200/9.071/11.488/0.903 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=98<br>[ 1] 11.00-12.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.236/8.838/11.345/0.807 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=98<br>[ 1] 12.00-13.00 sec 1.95 KBytes 16.0 Kbits/sec 10=9.560/8.512/11.301/0.782 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=105<br>[ 1] 13.00-14.00 sec 1.95 KBytes 16.0 Kbits/sec 10=9.939/9.245/10.944/0.552 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=101<br>[ 1] 14.00-15.00 sec 1.95 KBytes 16.0 Kbits/sec 10=13.722/12.576/15.569/0.855 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=73<br>[ 1] 15.00-16.00 sec 1.95 KBytes 16.0 Kbits/sec 10=11.856/10.674/12.693/0.624 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=84<br>[ 1] 16.00-17.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.260/8.872/11.085/0.652 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=97<br>[ 1] 17.00-18.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.044/9.152/11.054/0.755 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=100<br>[ 1] 18.00-19.00 sec 1.95 KBytes 16.0 Kbits/sec 10=10.199/8.724/11.377/0.928 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=98</b><br>[ 1] 19.00-20.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.831/11.103/18.682/1.886 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=67<br>[ 1] 20.00-21.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.657/11.171/15.684/1.574 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=68<br>[ 1] 21.00-22.00 sec 1.95 KBytes 16.0 Kbits/sec 10=15.283/10.757/18.079/2.311 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=65<br>[ 1] 22.00-23.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.960/10.551/16.555/1.991 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=67<br>[ 1] 23.00-24.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.699/10.847/16.979/1.815 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=68<br>[ 1] 24.00-25.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.240/10.476/15.544/1.733 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=70<br>[ 1] 25.00-26.00 sec 1.95 KBytes 16.0 Kbits/sec 10=14.230/10.513/15.785/1.972 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=70<br>[ 1] 26.00-27.00 sec 1.95 KBytes 16.0 Kbits/sec 10=13.530/10.297/15.595/2.433 ms 0 0K/(0)/0(0) us Tx=1000 Byte Rx=1000 Byte RPS=74<br><div><br></div><div>Bob</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Dec 1, 2024 at 2:32 PM Frantisek Borsik via Nnagain <<a href="mailto:nnagain@lists.bufferbloat.net">nnagain@lists.bufferbloat.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Good catch, Michael. Forwarded it to Sina, CEO Waveform.</div><div><br></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>All the best,</div><div><br></div><div><p class="MsoNormal" style="color:rgb(34,34,34)">Frank<u></u><u></u></p><p class="MsoNormal" style="color:rgb(34,34,34)"><u></u><u></u></p><p class="MsoNormal" style="color:rgb(34,34,34)">Frantisek (Frank) Borsik<u></u><u></u></p><p class="MsoNormal" style="color:rgb(34,34,34)"><u></u> <u></u></p><p class="MsoNormal" style="color:rgb(34,34,34)"><a href="https://www.linkedin.com/in/frantisekborsik" style="color:rgb(17,85,204)" target="_blank">https://www.linkedin.com/in/frantisekborsik</a><u></u><u></u></p><p class="MsoNormal" style="color:rgb(34,34,34)">Signal, Telegram, WhatsApp: +421919416714 <u></u><u></u></p><p class="MsoNormal" style="color:rgb(34,34,34)">iMessage, mobile: +420775230885<u></u><u></u></p><p class="MsoNormal" style="color:rgb(34,34,34)">Skype: casioa5302ca<u></u><u></u></p><p class="MsoNormal" style="color:rgb(34,34,34)"><a href="mailto:frantisek.borsik@gmail.com" style="color:rgb(17,85,204)" target="_blank">frantisek.borsik@gmail.com</a></p></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Dec 1, 2024 at 6:43 PM Michael Richardson via Nnagain <<a href="mailto:nnagain@lists.bufferbloat.net" target="_blank">nnagain@lists.bufferbloat.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I hadn't seen <a href="https://www.waveform.com/tools/bufferbloat" rel="noreferrer" target="_blank">https://www.waveform.com/tools/bufferbloat</a> before.<br>
Very pretty. I tried it just now, at the diner, tethered to my phone, hardly<br>
surprised to get an F. I wish the mobile carriers cared.<br>
Alas, the suggestions at the bottom include IQrouter, which is no longer in<br>
production. Probably should include OpenWRT One. <br>
<br>
-- <br>
] Never tell me the odds! | ipv6 mesh networks [<br>
] Michael Richardson, Sandelman Software Works | IoT architect [<br>
] <a href="mailto:mcr@sandelman.ca" target="_blank">mcr@sandelman.ca</a> <a href="http://www.sandelman.ca/" rel="noreferrer" target="_blank">http://www.sandelman.ca/</a> | ruby on rails [<br>
<br>
_______________________________________________<br>
Nnagain mailing list<br>
<a href="mailto:Nnagain@lists.bufferbloat.net" target="_blank">Nnagain@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/nnagain" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/nnagain</a><br>
</blockquote></div>
_______________________________________________<br>
Nnagain mailing list<br>
<a href="mailto:Nnagain@lists.bufferbloat.net" target="_blank">Nnagain@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/nnagain" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/nnagain</a><br>
</blockquote></div>