[Cake] Control theory and congestion control

[David Lang]

On Mon, 11 May 2015, Jonathan Morton wrote:

>>>>> Congestion control looks like a simple problem too. If there is no 
>>>>> congestion, increase the amount of data in flight; if there is, reduce it. 
>>>>> We even have Explicit Congestion Notification now to tell us that crucial 
>>>>> data point, but we could always infer it from dropped packets before.
>>>>
>>>> I think we critically depend on being able to interpret lost packets as 
>>>> well, as a) not all network nodes use ECN signaling, and b) even those that 
>>>> do can go into “drop-everything” mode if overloaded.
>>>
>>> Yes, but I consider that a degraded mode of operation.  Even if it is, for 
>>> the time being, the dominant mode.
>>>
>>>> 1) Competiton with simple greedy non-ECN flows, if these push the router 
>>>> into the dropping regime how will well behaved ECN flows be able to 
>>>> compete?
>>>
>>> Backwards compatibility for current ECN means dropping non-ECN packets that 
>>> would have been marked.  That works, so we can use it as a model.
>>
>> 	Let me elaborate, what I mean is if we got an ecn reduce slowly signal 
>> on the ecn flow and the router goes into overload, what guarantees that our 
>> flow with the double reduce-slowly ecn signal plus the reduce-hard drop will 
>> end not end up at an disadvantage over greedy non-ecn flows? It probably is 
>> quite simple but I can not see it right now.
>
> There are two possible answers to this:
>
> 1) The most restrictive signal seen during an RTT is the one to react to.  So 
> a “fast down” signal overrides anything else.

sorry for joining in late, but I think you are modeling something that doesn't 
match reality.

are you really going to see two bottlenecks in a given round trip (or even one 
connection)? Since you are ramping up fairly slowly, aren't you far more likely 
to only see one bottleneck (and once you get through that one, you are pretty 
much set through the rest of the link)

That one bottleneck could try to give you a 'fast down' signal, but I think you 
are unlikely to get multiple 'down' signals.

Looking at the network, you have two real scenarios ('server' == fast 
connection, 'client' == slow connection)

Start off with the possible combinations
1. server <-> server
2. server <-> client
3. client <-> server
4. client <-> client

#4 is the same as #1, just with smaller numbers

if both ends have the same available bandwidth (#1 and #4), then the bottleneck 
you are going to have to deal with is the local one

If you have less bandwidth than the other end (#3), the bottleneck that you are 
going to have to deal with is the local one

If you have more bandwidth than the other end (#2), then the bottleneck that you 
are going to have to deal with is the remote one

If you end up with an underprovisioned peer somewhere in the middle, it's either 
lower available bandwidth than either end, or it doesn't matter.

I think the only way you would need a 'fast down' signal is if the route changes 
to go through an underprvisioned peering, or when you have a new flow starting 
to contend with you due to routing changes (if it's a new flow, it should start 
slow and ramp up, so you, and the other affected flows, should all be good with 
a 'slow down' signal)

>>
>> 	But we already have a hard time to convince the operators of the edge 
>> routers (telcos cable cos…) to actually implement something saner than deep 
>> buffers at those devices. If they would at least own up to the head-end 
>> buffers for the downlink we would be in much better shape, and if they would 
>> offer to handle up-link buffer bloat as part of their optional 
>> ISP-router-thingy the issue would be stamped already. But did you look inside 
>> a typical CPE recently, still kernel from the 2.X series, so no 
>> codel/fq_codel and what ever else fixes were found in the several years since 
>> 2.X was the hot new thing…
>
> For CPE at least, there exists a market opportunity for somebody to fill. 
> OpenWRT shows what can be done with existing hardware with some user 
> engagement.  In principle, it’s only a short step from there to a new 
> commercial product that Does the Right Things.

especially if the device is managed by the ISP who knows what speed it should be 
using.

>>>> Is the steady state, potentially outside of the home, link truly likely 
>>>> enough that an non-oscillating congestion controller will effectively work 
>>>> better? In other words would the intermediate node ever signal hold 
>>>> sufficiently often that implementing this stage seems reasonable?

Is there really such a thing as a steady-state link outside of a 'dark fiber' 
point-to-point link used for a single application?  If not, that doesn't sound 
like something really worth optimizing for.

>>> Core routers don’t track flow state, but they are typically provisioned to 
>>> not saturate their links in the first place.
>>
>> 	This I heard quite often; it always makes me wonder whether there is a 
>> better way to design a network to work well at capacity instead of working 
>> around this by simply over-provisining, I thought it is called network 
>> engineering not network-“brute-forcing”…

network engineering at that level requires knowing future usage patterns. That's 
not possible yet, so you have to overbuild to account for the unexpected. This 
is no different than the fact that Bridges are overbuilt to allow for the 
unexpected loads of the future and possible sub-standard materials creeping in 
at some point in the construction process.

Only in Rocket Science where ounces count do they work to eliminate headroom 
(and as the first Space-X landing attempt showed, sometimes they cut too far)

>> 	Both of which pretty much try to keep constant bitrate UDP traffic flows 
>> going I believe, so they only care if the immediate network path and or 
>> alternatives a) has sufficient headroom for the data and b) latency changes 
>> due to path re-routing stay inside the de-jitter/de-lag buffer systems that 
>> are in use; or put differently, these traffic types will not attempt to 
>> saturate a given link by themselves so they are not the most sensitive probes 
>> for network path stability, no?
>
> I fully appreciate that *some* network paths may be unstable, and any 
> congestion control system will need to chase the sweet spot up and down under 
> such conditions.
>
> Most of the time, however, baseline RTT is stable over timescales of the order 
> of minutes, and available bandwidth is dictated by the last-mile link as the 
> bottleneck.  BDP and therefore the ideal cwnd is a simple function of baseline 
> RTT and bandwidth.  Hence there are common scenarios in which a steady-state 
> condition can exist.  That’s enough to justify the “hold” signal.

Unless you prevent other traffic from showing up on the network (phones checking 
e-mail, etc). I don't believe that you are ever going to have stable bandwidth 
available for any noticable timeframe.

David Lang