It's been a while since I posted out in the open about what's going on behind-the-scenes with LibreQoS development. So here's a "State of 1.5" summary, as of a few minutes ago.
Unified Configuration
Instead of having configuration split between/etc/lqos.conf
andispConfig.py
, we have all of the configuration in one place -/etc/lqos.conf
. It makes support a lot easier to have a single place to send people, and there's a LOT more validation and sanity checking now.
- New Configuration Format. DONE
- Automatic conversion from 1.4 configuration, including migrations. DONE
Performance
- Merged into the
develop
branch. DONE
- The old RTT system made up to 4 map lookups per packet (!). The new one makes do with 1 lookup, at the expense of only being accurate on the old system by +/- 5%. That's a huge reduction in per-packet workload, so I'm happy with it. Status: Working on my machine, needs cleaning before push
- The new RTT system runs on the input side, so on NICs that do receive-side steering it is now spread between CPUs rather than single CPU. Working on my machine
Bin-Packing
- Enabled eBPF SKB-Metadata and
bpf_xdp_adjust_meta
(which requires 5.5 kernel, but is actually supported by NICs around 5.18+). This allows the XDP side to store the TC and CPU map data in a blob of meta-data accompanying the packet data itself in kernel memory. If support for this is detected (not every NIC does it), it automaticaly passes the data between the XDP and TC flows - which allows to skip an entire LPM lookup on the TC side. I've wanted this for over a year. Works on my machine, improves throughput by 0.5 gbps single stream on my really crappy testbed setup
We're hoping to extend the bin-packing system to be both smarter and to include top-level trees (to avoid "oops, two important things are on one CPU" incidents).
Smart Weight Calculation: partly done. We have a call that builds weights per-customer now. Weights are a combination of:
- (if you have LTS) what did the customer do in this period, last week? This is remarkably predictable, people are really consistent on aggregate.
- What did the customer do in the last 5 minutes? (Doesn't require LTS, reasonably accurate)
The actual binpacking part isn't done yet, but doesn't look excessively tough.
- A fraction of their defined plan.
Per-Flow Analysis
We've had long-running task items to: track RTT per flow, balance the reported host RTT between flows, make it possible to exclude endpoints from reporting (e.g. a UISP server hosted somewhere else), and begin per-ASN and per-target analysis. We've also wanted to have flow information accessible, with a view to future enhancements - and allow a LOT more querying.
- Track TCP flows in real-time. We count bytes/packets, estimate a rate per flow, track RTT in both directions. This is working super-nicely on my test system.
- Track UDP/ICMP in real-time. We're aggregating bytes/packets and estimating a rate per flow.
- Web UI - display RTTs. RTTs are now combined per-host with a much smarter algorithm that can optionally exclude data from a flow that is beneath (threshold bits per second). The actual threshold is still being figured out.
There's a lot more to do here, mostly the analytics and display side. But it is coming along hot and heavy, and looking pretty good.
- Web UI API - you can view the current state of all flows.
Webserver Version
Rocket has been upgraded to the latest and greatest 1.5. A new UI is still coming; it may be a 1.6 item since the scope keeps looking bigger everytime it stares at me.
All the best,
Frank
Frantisek (Frank) Borsik
https://www.linkedin.com/in/frantisekborsik
Signal, Telegram, WhatsApp: +421919416714
iMessage, mobile: +420775230885
Skype: casioa5302ca