[Cerowrt-devel] [Bloat] Latest codel, fq_codel, and pie sim study from cablelabs now available

[Jonathan Morton]

On 2 May, 2013, at 5:20 am, Simon Barber wrote:

> Or one could use more queues in SFQ, so that the chance of 2 streams sharing a queue is small.

CableLabs actually did try that - increasing the number of queues - and found that it made things worse.  This, I think, extends to true "fair queueing" with flows explicitly rather than stochastically identified.  The reason is that with a very large number of flows, the bandwidth (or the packet throughput) is still shared evenly between them, and there is not enough bandwidth in the VoIP flow's share to allow it to work correctly.  With a relatively small number of flow buckets, the responsive flows hashed to the same bucket get out of the way of the unresponsive VoIP flow.

In short, a very large number of flow buckets prioritises BitTorrent over anything latency-sensitive, because BitTorrent uses a very large number of individual flows.

By contrast, putting all the BitTorrent flows into one bucket (or a depressed-priority queue with it's own SFQ buckets), or else elevating the VoIP traffic explicitly to a prioritised queue, would share the bandwidth more favourably to the VoIP flow, allowing it to use as much as it needed.  Either, or indeed both simultaneously, would do the job reasonably well, although an elevated priority queue should be bandwidth limited to a fraction of capacity to avoid the temptation of abuse by bulk flows.  Then there would be no performance objection to using a large number of flow buckets.

I can easily see a four-tier system working for most consumers, just so long as the traffic for each tier can be identified - each tier would have it's own fq_codel queue:

1) Network control traffic, eg. DNS, ICMP, even SYNs and pure ACKs - max 1/16th bandwidth, top priority

2) Latency-sensitive unresponsive flows, eg. VoIP and gaming - max 1/4 bandwidth, high priority

3) Ordinary bulk traffic, eg. web browsing, email, general purpose protocols - no bandwidth limit, normal priority

4) Background traffic, eg. BitTorrent - no bandwidth limit, low priority, voluntarily marked, competes at 1:4 with normal.

Obviously, the classification system implementing that must have some idea of what bandwidth is actually available at any given moment, but it is not necessary to explicitly restrict the top tiers' bandwidth when the link is otherwise idle.  Practical algorithms could be found to approximate the correct behaviour on a saturated link, while simply letting all traffic through on an unsaturated link.  Basic installations could already do this using HTB and assuming a link bandwidth.

Even better, of course, would be some system that allows BitTorrent to yield as though it were a smaller number of flows than it really is.  The "swarm" behaviour is very unusual among network protocols.  LEDBAT via uTP already does a good job on a per-flow basis, but since it's all under control of a single application, the necessary information should be available at that point.  I am reminded of the way Azureus adjusts global bandwidth limits - both incoming and outgoing - to match reality, based on both periodic and continuous measurements.

 - Jonathan Morton