[Cake] new code point proposed

[Cake] new code point proposed

[Dave Taht]

https://tools.ietf.org/html/draft-you-tsvwg-latency-loss-tradeoff-00



--
Dave Täht
Let's go make home routers and wifi faster! With better software!
https://www.gofundme.com/savewifi

Re: [Cake] new code point proposed

[Jonathan Morton]

> On 5 Apr, 2016, at 21:57, Dave Taht <dave.taht@gmail.com> wrote:
> 
> https://tools.ietf.org/html/draft-you-tsvwg-latency-loss-tradeoff-00

Interesting.  This is obviously written around the DualQ AQM, but it seems feasible to implement within Cake’s framework.  I’m somewhat pleased that this appears to mark a move away from using ECN alone to describe whether DCTCP is in use or not.

Some of the requirements are written in a way that assumes a single queue of fixed maximum size for each of “La" and "Lo", rather than a dynamic, flow-isolated system as Cake is.  It might be reasonable to suggest clarifying the language to take these cases into account.

We can already vary the Codel parameters between tins, which provides an obvious mechanism to make the queue management much more aggressive for “La” traffic, and more relaxed for “Lo” traffic.  I don’t think we need to explicitly limit the queue allocation to “La” traffic as specified.

Another detail which differs from Cake’s view of the world is that neither La nor Lo have priority over each other.  While Cake does not implement strict priority, it does implement soft priority as part of its effort to minimise latency for the upper tins; the most explicit part of this is that bandwidth consumed by higher tins is removed from lower tins’ allocations.  However, this effect could be hidden by making the two DRR weights equal for the lower tin.

Obviously, traffic marked with the existing “low latency intent” DSCPs can be treated as “Lo” traffic when there is no admission control in place, without any requirement for re-marking.  This is consistent with what Cake does anyway.

This seems like a good excuse to overhaul Cake’s Diffserv class allocations.  I could propose a 5-class system:

Tin 0 = LLT “Lo” traffic (inc. existing low-loss & high-throughput classes), 256/256, 100%, increased target & interval.
Tin 1 = Best Effort traffic, 256/256, 100%, standard target & interval.
Tin 2 = LLT “La” traffic (inc. existing low-latency classes), 256/256, 100%, standard target, reduced interval.
Tin 3 = Low Priority traffic, 2048/16, 6.25%, standard target & interval.
Tin 4 = Network Control traffic, 4096/1, 6.25%, increased target & interval.

Note that Tin 4 is almost, but not quite, strictly admission-controlled, discouraging the use of “network control” DSCPs for general traffic.  Tins 0-2 implement LLT as specified.  Tin 3 takes the unusual and counter-intuitive step of placing “low priority” traffic in a high tin, but the effect should be very similar to existing behaviour, due to the soft-priority system and the low bandwidth allocation.

 - Jonathan Morton

Re: [Cake] new code point proposed

[moeller0]

> On Apr 5, 2016, at 22:06 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 5 Apr, 2016, at 21:57, Dave Taht <dave.taht@gmail.com> wrote:
>> 
>> https://tools.ietf.org/html/draft-you-tsvwg-latency-loss-tradeoff-00
> 
> Interesting.  This is obviously written around the DualQ AQM, but it seems feasible to implement within Cake’s framework.  I’m somewhat pleased that this appears to mark a move away from using ECN alone to describe whether DCTCP is in use or not.
> 
> Some of the requirements are written in a way that assumes a single queue of fixed maximum size for each of “La" and "Lo", rather than a dynamic, flow-isolated system as Cake is.  It might be reasonable to suggest clarifying the language to take these cases into account.
> 
> We can already vary the Codel parameters between tins, which provides an obvious mechanism to make the queue management much more aggressive for “La” traffic, and more relaxed for “Lo” traffic.  I don’t think we need to explicitly limit the queue allocation to “La” traffic as specified.
> 
> Another detail which differs from Cake’s view of the world is that neither La nor Lo have priority over each other.  While Cake does not implement strict priority, it does implement soft priority as part of its effort to minimise latency for the upper tins; the most explicit part of this is that bandwidth consumed by higher tins is removed from lower tins’ allocations.  However, this effect could be hidden by making the two DRR weights equal for the lower tin.
> 
> Obviously, traffic marked with the existing “low latency intent” DSCPs can be treated as “Lo” traffic when there is no admission control in place, without any requirement for re-marking.  This is consistent with what Cake does anyway.
> 
> This seems like a good excuse to overhaul Cake’s Diffserv class allocations.  I could propose a 5-class system:
> 
> Tin 0 = LLT “Lo” traffic (inc. existing low-loss & high-throughput classes), 256/256, 100%, increased target & interval.
> Tin 1 = Best Effort traffic, 256/256, 100%, standard target & interval.
> Tin 2 = LLT “La” traffic (inc. existing low-latency classes), 256/256, 100%, standard target, reduced interval.

	This might back fire, as far as I understand interval is the reaction time window for a flow, this needs to be roughly in the ballpark of the RTT, reducing it (significantly) will make the AQM quite trigger happy. This might be in line with the LA proposal, but what if LA traffic has to cross a satellite link?

Best Regards
	Sebastian


> Tin 3 = Low Priority traffic, 2048/16, 6.25%, standard target & interval.
> Tin 4 = Network Control traffic, 4096/1, 6.25%, increased target & interval.
> 
> Note that Tin 4 is almost, but not quite, strictly admission-controlled, discouraging the use of “network control” DSCPs for general traffic.  Tins 0-2 implement LLT as specified.  Tin 3 takes the unusual and counter-intuitive step of placing “low priority” traffic in a high tin, but the effect should be very similar to existing behaviour, due to the soft-priority system and the low bandwidth allocation.
> 
> - Jonathan Morton
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] new code point proposed

[Jonathan Morton]

> On 5 Apr, 2016, at 23:28, moeller0 <moeller0@gmx.de> wrote:
> 
>> Tin 0 = LLT “Lo” traffic (inc. existing low-loss & high-throughput classes), 256/256, 100%, increased target & interval.
>> Tin 1 = Best Effort traffic, 256/256, 100%, standard target & interval.
>> Tin 2 = LLT “La” traffic (inc. existing low-latency classes), 256/256, 100%, standard target, reduced interval.
> 
> 	This might back fire, as far as I understand interval is the reaction time window for a flow, this needs to be roughly in the ballpark of the RTT, reducing it (significantly) will make the AQM quite trigger happy. This might be in line with the LA proposal, but what if LA traffic has to cross a satellite link?

The entire point *is* to make the AQM very trigger-happy for “La" traffic.  By selecting the “La” DSCP (or any other low-latency DSCP, for that matter), the originator of the traffic is requesting that behaviour.  Reduced throughput is an expected side-effect.

Satellite links have nasty effects on latency-sensitive traffic all by themselves.  I don’t think we need to worry too much about that combination.  If the flow uses less than its fair share of bandwidth, the AQM won’t trigger anyway.

In any case, Codel’s behaviour and default parameters are tuned for conventional TCP.  Latency-sensitive traffic generally doesn’t use conventional TCP, so the usual assumptions go out of the window.  I propose retaining the standard “target” parameter on Tin 2 to avoid triggering AQM with a single large packet, but reducing “interval” to make Codel’s behaviour more suitable for UDP and DCTCP traffic.

 - Jonathan Morton

Re: [Cake] new code point proposed

[moeller0]

Hi Jonathan,

> On Apr 5, 2016, at 22:40 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 5 Apr, 2016, at 23:28, moeller0 <moeller0@gmx.de> wrote:
>> 
>>> Tin 0 = LLT “Lo” traffic (inc. existing low-loss & high-throughput classes), 256/256, 100%, increased target & interval.
>>> Tin 1 = Best Effort traffic, 256/256, 100%, standard target & interval.
>>> Tin 2 = LLT “La” traffic (inc. existing low-latency classes), 256/256, 100%, standard target, reduced interval.
>> 
>> 	This might back fire, as far as I understand interval is the reaction time window for a flow, this needs to be roughly in the ballpark of the RTT, reducing it (significantly) will make the AQM quite trigger happy. This might be in line with the LA proposal, but what if LA traffic has to cross a satellite link?
> 
> The entire point *is* to make the AQM very trigger-happy for “La” traffic.  

	Well, yes but not too trigger happy.

> By selecting the “La” DSCP (or any other low-latency DSCP, for that matter), the originator of the traffic is requesting that behaviour.  Reduced throughput is an expected side-effect.
> 
> Satellite links have nasty effects on latency-sensitive traffic all by themselves.  I don’t think we need to worry too much about that combination.  If the flow uses less than its fair share of bandwidth, the AQM won’t trigger anyway.
> 
> In any case, Codel’s behaviour and default parameters are tuned for conventional TCP.  

	Or rather TCP-friendly flows. My point is a well-behaved UDP (SCTP/whatever) flow might respond just as well as a TCP-flow, so TCP versus other protocols does not seem to be the optimal categorization here.

> Latency-sensitive traffic generally doesn’t use conventional TCP, so the usual assumptions go out of the window.  

	I fear this is not correct. Ssh is mostly using TCP and is latency sensitive (to some degree), I agree that it would not be too well served with LA though.

> I propose retaining the standard “target” parameter on Tin 2 to avoid triggering AQM with a single large packet, but reducing “interval” to make Codel’s behaviour more suitable for UDP and DCTCP traffic.

	Okay these are too different beasts: UDP might be too broad a stroke to use here, unresponsive UDP flows maybe. DCTCP will require much tighter target and interval settings anyway and does not seem to be fit for the wider internet (with its assumption of ZERO % ack packet loss). That said, I believe you already made cake fit for short RTT environments (units seems to be micro seconds):
} else if (strcmp(*argv, "datacentre") == 0) {
                        interval = 100;
                        target   =   5;

;)

Regarding UDP the challenge I see is to differentiate between responsive and unresponsive flows. In any case we need to give the endpoint-pair one RTT worth of time to signal packet loss or ECN to the receiver and back to the sender (actually the full true RTT is the shortest time we can expect to see behaviour changes of the sender at the AQM-queue), so reducing interval << RTT will simply prune the flow excessively before the sender can react. If we would know that the sender does not care and will not reduce its rate voluntarily dropping hard early might be a good strategy, but if we see a tcp-friendly udp flow then asking nicely might be better. 
	I fail to see decent heuristics to discriminate between both types (short of keeping a history per flow, and assume each flow to be friendly first and only switch dropping strategy if the flow did not react properly and timely to drops/marks).

Best Regards


> 
> - Jonathan Morton
> 

Re: [Cake] new code point proposed

[Jonathan Morton]

> On 6 Apr, 2016, at 13:30, moeller0 <moeller0@gmx.de> wrote:
> 
>> Latency-sensitive traffic generally doesn’t use conventional TCP, so the usual assumptions go out of the window.  
> 
> I fear this is not correct. Ssh is mostly using TCP and is latency sensitive (to some degree), I agree that it would not be too well served with LA though.

SSH is not so latency-sensitive as to require “La” service; round-trip response times of a quarter second are acceptable, though multiple seconds are not.  With flow isolation, best-effort service is sufficient.

Even in the best-effort tin, Cake provides flow isolation and host isolation, so any remaining latency (assuming we’re not in severe overload) is self-induced by the flow itself.  LLT, as I read it, allows traffic to explicitly indicate that even self-induced latency is unacceptable, using the “La” codepoint.  Hence the more aggressive AQM setting.

However, SSH will work quite happily in an “La” class if ECN is used.  The marking rate may be high in some circumstances, but packets will not be dropped, so application latency remains low.  Some applications may benefit from this behaviour.

> Regarding UDP the challenge I see is to differentiate between responsive and unresponsive flows.

Which we fundamentally can’t do quickly enough to control the queue to “La” standards, so it’s not worth trying for this purpose.

So we should pessimistically assume that “La” marked traffic is unresponsive, and control all of it aggressively.  Responsive traffic which uses an “La” mark will run at lower throughput, which is exactly the expected tradeoff.

> DCTCP will require much tighter target and interval settings anyway and does not seem to be fit for the wider internet (with its assumption of ZERO % ack packet loss).

This is actually a very good point.  DualQ is supposedly designed to allow DCTCP to coexist with conventional traffic, and LLT is essentially a spec for DualQ.  But to maintain minimum latency for DCTCP, both the forward and reverse paths need to be marked “La”.  So if DCTCP really assumes zero reverse-path loss, it will incur a higher round-trip delay due to the reverse path being marked “Lo”.

I do sometimes wonder what those guys are smoking.

 - Jonathan Morton

Re: [Cake] new code point proposed

[moeller0]

Hi Jonathan,

> On Apr 6, 2016, at 19:16 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 6 Apr, 2016, at 13:30, moeller0 <moeller0@gmx.de> wrote:
>> 
>>> Latency-sensitive traffic generally doesn’t use conventional TCP, so the usual assumptions go out of the window.  
>> 
>> I fear this is not correct. Ssh is mostly using TCP and is latency sensitive (to some degree), I agree that it would not be too well served with LA though.
> 
> SSH is not so latency-sensitive as to require “La” service; round-trip response times of a quarter second are acceptable, though multiple seconds are not.  With flow isolation, best-effort service is sufficient.
> 
> Even in the best-effort tin, Cake provides flow isolation and host isolation, so any remaining latency (assuming we’re not in severe overload) is self-induced by the flow itself.  LLT, as I read it, allows traffic to explicitly indicate that even self-induced latency is unacceptable, using the “La” codepoint.  Hence the more aggressive AQM setting.
> 
> However, SSH will work quite happily in an “La” class if ECN is used.  The marking rate may be high in some circumstances, but packets will not be dropped, so application latency remains low.  Some applications may benefit from this behaviour.

	Okay, thanks.

> 
>> Regarding UDP the challenge I see is to differentiate between responsive and unresponsive flows.
> 
> Which we fundamentally can’t do quickly enough to control the queue to “La” standards, so it’s not worth trying for this purpose.
> 
> So we should pessimistically assume that “La” marked traffic is unresponsive, and control all of it aggressively.  Responsive traffic which uses an “La” mark will run at lower throughput, which is exactly the expected tradeoff.

	Color me sceptic here.as all the other DSCPs this will only ever work reliable on egress (and in lab settings). I like the implied trade-offs of LA and LO, but I guess they will get as much exposure/distribution as all the other DSCP schemes (namely, not universal enough to be really useful). I looks like a “cute” new mode to teach cake though to allow actual testing.

> 
>> DCTCP will require much tighter target and interval settings anyway and does not seem to be fit for the wider internet (with its assumption of ZERO % ack packet loss).
> 
> This is actually a very good point.  

	Sorry, that was an accident then ;)

> DualQ is supposedly designed to allow DCTCP to coexist with conventional traffic, and LLT is essentially a spec for DualQ.  But to maintain minimum latency for DCTCP, both the forward and reverse paths need to be marked “La”.  So if DCTCP really assumes zero reverse-path loss, it will incur a higher round-trip delay due to the reverse path being marked “Lo”.

	But the DCTCP traffic is assumed to not leave the building, so to speak?

Best Regards
	Sebastian

> 
> I do sometimes wonder what those guys are smoking.
> 
> - Jonathan Morton
> 

Re: [Cake] new code point proposed

[Dave Taht]

I note that somewhere along the line in the last few days I read a
very good outline of the  ietf proposal to dedicate ECT(1) to a
DCTCP-like use, discussing the benefits and problems and alternate
approaches. Jonathon's ELR was mentioned, but I'll be damned if I can
find it again. It might have been part of thursdays bar bof, don't
remember. I thought it was in iccrg, wasn't.

I have a LOT more hope for repurposing ECT(1) than diffserv markings.

In other news (I am not at ietf) mptcp over bonded radios to an
aircraft is quite neat.

https://buenayreb.conf.meetecho.com/q-meetecho/Index.jsp

Re: [Cake] new code point proposed

[Dave Taht]

this is still not the document I read (which was better), but this is
what was just discussed.

https://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-l4s-id-00
Dave Täht
Let's go make home routers and wifi faster! With better software!
https://www.gofundme.com/savewifi


On Wed, Apr 6, 2016 at 11:11 AM, Dave Taht <dave.taht@gmail.com> wrote:
> I note that somewhere along the line in the last few days I read a
> very good outline of the  ietf proposal to dedicate ECT(1) to a
> DCTCP-like use, discussing the benefits and problems and alternate
> approaches. Jonathon's ELR was mentioned, but I'll be damned if I can
> find it again. It might have been part of thursdays bar bof, don't
> remember. I thought it was in iccrg, wasn't.
>
> I have a LOT more hope for repurposing ECT(1) than diffserv markings.
>
> In other news (I am not at ietf) mptcp over bonded radios to an
> aircraft is quite neat.
>
> https://buenayreb.conf.meetecho.com/q-meetecho/Index.jsp

Re: [Cake] new code point proposed

[Alan Jenkins]

On 06/04/16 21:39, Dave Taht wrote:
> this is still not the document I read (which was better), but this is
> what was just discussed.
>
> https://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-l4s-id-00
> Dave Täht
>
> On Wed, Apr 6, 2016 at 11:11 AM, Dave Taht <dave.taht@gmail.com> wrote:
>> I note that somewhere along the line in the last few days I read a
>> very good outline of the  ietf proposal to dedicate ECT(1) to a
>> DCTCP-like use, discussing the benefits and problems and alternate
>> approaches. Jonathon's ELR was mentioned, but I'll be damned if I can
>> find it again. It might have been part of thursdays bar bof, don't
>> remember. I thought it was in iccrg, wasn't.
>>
>> I have a LOT more hope for repurposing ECT(1) than diffserv markings.

There is an updated version, maybe that's what you wanted :).  The diff 
says there's a new section in particular (2.3 Pre-Requisite Transport 
Layer Behaviour).

https://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-l4s-id-01


I read -00. Am I forgetting something, or does the proposed scaleable 
TCP (DCTCP etc) break deployments of routers with ECN like `codel ecn`?  
`fq_codel` (default ecn) might avoid unfairness but it still doesn't 
sound ideal.  I know they've said ECN routers aren't broadly deployed, 
but it seems like any that do exist would suffer badly.

Alan