[Cerowrt-devel] cake3 vs sqm+fq_codel at 115/12 mbit (basically comcast´s blast service)

[Jonathan Morton]

Awesome.

Oddly enough, cake3 actually gets slightly less throughput than
htb+fq_codel on the Pentium-MMX. However that's with the simplest possible
htb configuration (since I'm manually typing it in), and no firewall rules
or NAT going on (just a bridge between two Ethernet ports).

A couple of notes on the statistics that are now reported:

The rate for each class is now a threshold rather than a limit. The class
is permitted to use more than that bandwidth (up to the global limit), but
will yield to lower priority classes in that condition. This is consistent
with both user expectations and standard PHB specs, and means that traffic
benefits from high priority markings only if it's appropriately sparse.

On that note, I expect roughly the filtering uses of each class:

0 - background bulk traffic, CS1 marked, ie. BitTorrent. Use as many
parallel connections as you like, without worrying about ordinary traffic.

1 - best effort, the great majority of ordinary traffic - web pages,
software updates, whatever. If in doubt, leave it here (default CS0 lands
here).

2 - elevated priority, bandwidth sensitive traffic, such as streaming video
or a vlan.

3 - low volume, latency sensitive traffic such as VoIP, online games, NTP,
etc. EF traffic lands here.

A minor frustration for me here - firewall rules on ingress are processed
only after the traffic has already passed through ifb. This means I can't
custom mark my inbound traffic.

Three delay statistics are now reported, all of which are based on EWMAs of
packet sojourn times at dequeue. Pk is biased heavily to high delays (so
should usually report on fat flows), Sp to low delays (so should capture
sparse flows), and Av keeps a true average. The concept of a biased EWMA is
borrowed from ReplayGain and the whole "loudness war" problem that it aims
to solve; some broadcast studios (including the BBC) use audio meters which
work this way.

The new set-associative hash function also generates extra statistics. The
same 1024 queues are now divided into 128 sets of 8 "ways", and a tag on
each queue tracks which flow is presently using it. This allows hash
collisions to be resolved in most cases, with limited worst case overhead,
greatly improving flow isolation under severely stressed conditions. (It's
difficult to provoke this on a home network, but offices may well
appreciate this feature.)

The "way miss" counter is incremented whenever an empty queue's tag is
changed to assign it to a new flow, signalling a departure from the fast
path for that packet. Expect to see a small percentage of these with normal
traffic.

The "way indirect hit" counter tracks the situations where a hash collision
would have occurred with a plain hash function, but was resolved by the set
associativity. This is also a departure from the fast path.

The "way collision" counter indicates when even set associative hashing is
insufficient - there are more than 8 distinct flows attempting to occupy
queues in the same set. In such a case, the search for an empty queue is
terminated and the packet is placed in the queue matching the plain hash.
NB: so far this code path is completely untested to my knowledge!

- Jonathan Morton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cerowrt-devel/attachments/20150402/8ae035d9/attachment-0002.html>