<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 14, 2021 at 2:07 PM Robert Chacon <<a href="mailto:robert.chacon@jackrabbitwireless.com">robert.chacon@jackrabbitwireless.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">> Cool! What kind of performance are you seeing? The README mentions being<br>
> limited by the BPF hash table size, but can you actually shape 2000<br>
> customers on one machine? On what kind of hardware and at what rate(s)?</div><div dir="ltr"><br></div><div>On our production network our peak throughput is 1.5Gbps from 200 clients, and it works very well.</div><div>We use a simple consumer-class AMD 2700X CPU in production because utilization of the shaper VM is ~15% at 1.5Gbps load.<br></div><div>Customers get reliably capped within ±2Mbps of their allocated htb/fq_codel bandwidth, which is very helpful to control network congestion.<br></div><div><br></div><div>Here are some graphs from RRUL performed on our test bench hypervisor: <a href="https://raw.githubusercontent.com/rchac/LibreQoS/main/docs/fq_codel_1000_subs_4G.png" target="_blank">https://raw.githubusercontent.com/rchac/LibreQoS/main/docs/fq_codel_1000_subs_4G.png</a></div><div>In
that example, bandwidth for the "subscriber" client VM was set to
4Gbps. 1000 IPv4 IPs and 1000 IPv6 IPs were in the filter hash table of LibreQoS.<br></div></div></blockquote><div><br></div><div>What I really love about this plot is that it is now very possible for your customers to</div><div>play live music together with reasonable latencies and jitter, under load.</div><div><br></div><div>Existing tools like "jacktrip" should "just work. For a cool talk about the jacktrip</div><div>revolution: <a href="https://www.npr.org/2020/11/21/937043051/musicians-turn-to-new-software-to-play-together-online">https://www.npr.org/2020/11/21/937043051/musicians-turn-to-new-software-to-play-together-online</a></div><div><br></div><div>I'm using cake to keep things under control on my testbed network, using ardour</div><div>as the mixing tool, and achieving about 6ms of inherent latency.</div><div><br></div><div>I am hoping to sink a bit of time into <a href="http://galene.org">galene.org</a> and various web browsers this</div><div>year to finally get closer to what the lola project has been doing for a while on the video front. </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>The test bench server has an AMD 3900X running Ubuntu in Proxmox. 4Gbps utilizes 10% of the VM's 12 cores. Paravirtualized VirtIO network drivers are used and most offloading types are enabled.</div><div>In
our setup, VM networking multiqueue isn't enabled (it kept disrupting traffic flow), so 6Gbps is probably the most it
can achieve like this. Our qdiscs in this VM may be limited to one core because of that.</div><div> I suspect in a non-virtualized setup, or one with multiqueue, it can handle much more throughput.<br></div><div>Either way for now it's surprising to me how well it works and I'm just grateful for it haha.</div><div>Kudos to you and your peers for making fq_codel so efficient!<br></div><div><br></div><div>- Robert<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 14, 2021 at 12:46 PM Toke Høiland-Jørgensen <<a href="mailto:toke@toke.dk" target="_blank">toke@toke.dk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Robert Chacon <<a href="mailto:robert.chacon@jackrabbitwireless.com" target="_blank">robert.chacon@jackrabbitwireless.com</a>> writes:<br>
<br>
> Hello everyone,<br>
><br>
> I am new here, my name is Robert. I operate a small ISP in the US. I wanted<br>
> to post here to thank Dave Täht, as well as the dozens of contributors to<br>
> the fq_codel and cake projects.<br>
<br>
Thank you for reaching out! It's always fun to hear about real-world<br>
deployments of this technology, and it's great to hear that it's working<br>
well for you! :)<br>
<br>
> I created a simple python application that uses htb+fq_codel to shape my<br>
> customers' traffic, and have seen great performance improvements. I am<br>
> maintaining it as an open source project for other ISPs to use at<br>
> <a href="https://github.com/rchac/LibreQoS" rel="noreferrer" target="_blank">https://github.com/rchac/LibreQoS</a><br>
<br>
Cool! What kind of performance are you seeing? The README mentions being<br>
limited by the BPF hash table size, but can you actually shape 2000<br>
customers on one machine? On what kind of hardware and at what rate(s)?<br>
<br>
-Toke<br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr"><div dir="ltr">
<table style="width:450px;font-size:10pt;font-family:Arial,sans-serif" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="font-size:10pt;font-family:Arial,sans-serif;border-right:3px solid rgb(33,150,243);width:180px;padding-right:10px;vertical-align:top" valign="top">
<img alt="photograph" style="border: 0px none; height: auto; width: 120px;" src="https://jackrabbitwireless.com/wp-content/uploads/2018/11/cropped-jackrabbitwirelesslogo-yellow1.png" width="120" border="0"><p style="margin-top:32px;margin-bottom:0px;line-height:1"><b><span style="font-size:12pt;font-family:Arial,sans-serif;color:rgb(33,150,243)">Robert Chacón<br></span></b>
<span style="font-family:Arial,sans-serif;font-size:9pt;color:rgb(0,0,0)">Owner</span>
</p>
</td>
<td valign="top">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="font-size:10pt;color:rgb(68,68,68);font-family:Arial,sans-serif;padding-bottom:5px;padding-left:30px;vertical-align:top;line-height:1.3" valign="top">
<span><span style="color:rgb(0,0,0)"><b>M</b></span><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(0,0,0)"> (915) 730-1472<br></span></span>
<span><span style="color:rgb(0,0,0)"><b>E</b></span><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(0,0,0)"> <a href="mailto:robert.chacon@jackrabbitwireless.com" target="_blank">robert.chacon@jackrabbitwireless.com</a></span></span>
</td>
</tr>
<tr>
<td style="font-size:10pt;color:rgb(0,121,172);font-family:Arial,sans-serif;padding-bottom:5px;padding-left:30px;vertical-align:top;line-height:1.3" valign="top">
<b style="font-family:Arial,sans-serif;font-size:9pt;color:rgb(0,0,0)">JackRabbit Wireless LLC</b>
<span>
<br>
<span style="font-size:10pt;font-family:Arial,sans-serif;color:rgb(0,0,0)">P.O. Box 222111<span><br></span></span>
<span style="font-size:10pt;font-family:Arial,sans-serif;color:rgb(0,0,0)">El Paso, TX 79913</span>
</span>
</td>
</tr>
<tr>
<td style="font-size:10pt;font-family:Arial,sans-serif;padding-bottom:5px;padding-top:5px;padding-left:30px;vertical-align:top;color:rgb(0,121,172)" valign="top">
<a href="http://jackrabbitwireless.com" target="_blank"><b style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(33,150,243)">jackrabbitwireless.com</b></a><br><br>
</td>
</tr>
</tbody>
</table>
</td></tr>
</tbody>
</table>
</div></div>
_______________________________________________<br>
Bloat mailing list<br>
<a href="mailto:Bloat@lists.bufferbloat.net" target="_blank">Bloat@lists.bufferbloat.net</a><br>
<a href="https://lists.bufferbloat.net/listinfo/bloat" rel="noreferrer" target="_blank">https://lists.bufferbloat.net/listinfo/bloat</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">"For a successful technology, reality must take precedence over public relations, for Mother Nature cannot be fooled" - Richard Feynman<br><br><a href="mailto:dave@taht.net" target="_blank">dave@taht.net</a> <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729</div></div></div>