[Cake] cake exploration

[Jonathan Morton]

Okay, lots of things to think about all at once.  Cherry-picking:

> 5) Byte Mode-ish handling
> 
> Dropping a single 64 byte packet does little good. You will find in
> the 50 flow tests that a ton of traffic is acks, not being dropped,
> and pie does better in this case than does fq, as it shoots
> wildly at everything, but usually misses the fat packets, where DRR
> will merrily store up an entire
> MTU worth of useless acks when only one is needed.
> 
> So just trying to drop more little packets might be helpful in some cases.
> 
> 6) Ack thinning. I gave what is conventionally called "stretch acks" a
> new name, as stretch acks
> have a deserved reputation as sucking. Well, they dont suck anymore in
> linux, and what I was
> mostly thinking was to drop no more than 2 in a row...
> 
> One thing this would help with is in packing wifi aggregates - which
> have hard limits on the number of packets in a TXOP (42), and a byte
> limit on wireless n of 64k. Sending 41 acks from
> one flow, when you could send the last 2, seems like a big win on
> packing a TXOP.
> 
> (this is something eric proposed, and given the drop rates we now see
> from wifi and the wild and wooly internet I am inclined to agree that
> it is worth fiddling with)
> 
> (I am not huge on it, though)

Okay, dropping individual 64-byte packets does little good.  But transmitting 64-byte packets costs very little.  On the gripping hand, receiving and processing 64-byte packets is probably disproportionately expensive; probably much of our forwarding CPU overhead is per-packet, not per-byte.  (This could be tested by reducing MTU and seeing how much the CPU saturation threshold moves.)

Here’s something important:  ECE-Echo is carried on acks.

As far as wifi is concerned, if you’ve got 42 acks from one flow aggregating into one transmit opportunity, then I’d say that’s a very rare and degenerate case.  Maybe it happens occasionally on real traffic, but it would be a transient condition - not occurring continuously.  So it only matters if there’s competing traffic for the same station, and that’s something we can tackle a different way.

> 7) Macaddr hashing on the nexthop instead of the 5tuple. When used on
> an internal, switched network, it  would be better to try and maximize
> the port usage rather than the 5 tuple in some cases.

We’ve sort-of almost got that already with “dsthost" mode.

> Personally I do not think that the 8 way set associative has is what
> wifi needs for cake, I tend to think we need to "pack" aggregates with
> as many different flows as possible, and randomize how we packet
> them... I think.... maybe….

Wifi needs per-flow fairness on a per-station basis, ie. not hashing the entire 5-tuple at once, but processing different parts of it separately.

That’s potentially a good thing in the more general case, actually; at the moment, with multiple hosts served by a single cake instance, one user can outcompete another by simply using more flows.

> 13)  It might be possible to write a faster codel - and easier to read
> by using a case statement on the 2 core variables in it. The current
> code does not show the 3 way state machine as well as that could, and
> for all I know there is something intelligent we could do with the 4th
> state.

Yes.  Easier to read code may even turn out to be faster (for a variety of reasons), and will certainly be easier for hardware folks to get their heads around.

- Jonathan Morton