From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw1-xc2a.google.com (mail-yw1-xc2a.google.com [IPv6:2607:f8b0:4864:20::c2a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 72DB93B29E for ; Sat, 27 Jul 2019 11:35:41 -0400 (EDT) Received: by mail-yw1-xc2a.google.com with SMTP id q128so21326139ywc.1 for ; Sat, 27 Jul 2019 08:35:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=krose.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=y1TGiQbw/fiy5aL3PXmOZ6htYXVCo/Ju+8/j5MS8GaA=; b=IlgtZMJw8nvR0McFOuS3vr0gJemzAIFVS+izTeX+/0YoEd217vpOtgg9reCn7CDSNZ TxVm+DPIISW8y0XyNb+jMf0QXNWaDCZnJfuV+sWDNHIV/yCNM4CvM8oSJ2VX+6plC/1k gxSCHZljyHG5UM5eFTV7Zf0hicFv6UQZuwrsY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=y1TGiQbw/fiy5aL3PXmOZ6htYXVCo/Ju+8/j5MS8GaA=; b=idQiu0PacJrTmAa+MfgRnZl9ICSjskYX/beULg9vT+aIuo3xUNhDQ49uZWeS9GIQmM m1iRUE8u5N1IJ8oohze8WjTfVjdrOqM/NXeM6uMNog2aMBiNxya2Ro2OkDl0FG6luJPK F64bk2lDRWei+9AXWWPhQU7QEatkYV1a4qICIdd1vRWUdM2JZcWJRztVwtN6nRfrsINg Mkj9pEdrUUu01hYgIifObNpcLHKhvcb97xGVOBwYTXHV2gU5bvMqRi9Hqj1lYLmANZVq otqtW0etAL9vLuSpOAVMNUr/BsXgdxiEtwJjwTlCb9fFFXtFzHtcfxaxtUt5vr2KEne/ IbsQ== X-Gm-Message-State: APjAAAWHUsgo6k0/VNRPRPnUdrjr+dLQEvGA7DPw+BJlhxjGP87WWyQP BpDwFln0xMJ8KNAQMUxJ88yYWPh61j2e1XYSLe0= X-Google-Smtp-Source: APXvYqzcGPxfljbAagn8KAShD/eBewUR/YtphhOIyn/8t+6RCgChDsqhnSeEGa9WIx8GuUw22iLiuQ+771f63uJHD1w= X-Received: by 2002:a81:980d:: with SMTP id p13mr60288734ywg.51.1564241740643; Sat, 27 Jul 2019 08:35:40 -0700 (PDT) MIME-Version: 1.0 References: <350f8dd5-65d4-d2f3-4d65-784c0379f58c@bobbriscoe.net> <40605F1F-A6F5-4402-9944-238F92926EA6@gmx.de> <1563401917.00951412@apps.rackspace.com> <4833615D-4D68-4C95-A35D-BCF9702CEF69@akamai.com> In-Reply-To: <4833615D-4D68-4C95-A35D-BCF9702CEF69@akamai.com> From: Kyle Rose Date: Sat, 27 Jul 2019 11:35:28 -0400 Message-ID: To: "Holland, Jake" Cc: Bob Briscoe , "ecn-sane@lists.bufferbloat.net" , tsvwg IETF list , "David P. Reed" Content-Type: multipart/alternative; boundary="0000000000004b57d9058eab6948" Subject: Re: [Ecn-sane] [tsvwg] per-flow scheduling X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Jul 2019 15:35:41 -0000 --0000000000004b57d9058eab6948 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Right, I understand that under 3168 behavior the sender would react differently to ECE markings than L4S flows would, but I guess I don't understand why a sender willing to misclassify traffic with ECT(1) wouldn't also choose to react non-normatively to ECE markings. On the rest, I think we agree. Kyle On Thu, Jul 25, 2019 at 3:26 PM Holland, Jake wrote: > Hi Kyle, > > > > I almost agree, except that the concern is not about classic flows. > > > > I agree (with caveats) with what Bob and Greg have said before: ordinary > classic flows don=E2=80=99t have an incentive to mis-mark if they=E2=80= =99ll be responding > normally to CE, because a classic flow will back off too aggressively and > starve itself if it=E2=80=99s getting CE marks from the LL queue. > > > > That said, I had a message where I tried to express something similar to > the concerns I think you just raised, with regard to a different category > of flow: > > https://mailarchive.ietf.org/arch/msg/tsvwg/bUu7pLmQo6BhR1mE2suJPPluW3Q > > > > So I agree with the concerns you=E2=80=99ve raised here, and I want to +1= that > aspect of it while also correcting that I don=E2=80=99t think these apply= for > ordinary classic flows, but rather for flows that use application-level > quality metrics to change bit-rates instead responding at the transport > level. > > > > For those flows (which seems to include some of today=E2=80=99s video con= ferencing > traffic), I expect they really would see an advantage by mis-marking > themselves, and will require policing that imposes a policy decision. > Given that, I agree that I don=E2=80=99t see a simple alternative to FQ f= or flows > originating outside the policer=E2=80=99s trust domain when the network i= s fully > utilized. > > > > I hope that makes at least a little sense. > > > > Best regards, > > Jake > > > > *From: *Kyle Rose > *Date: *2019-07-23 at 11:13 > *To: *Bob Briscoe > *Cc: *"ecn-sane@lists.bufferbloat.net" , > tsvwg IETF list , "David P. Reed" > *Subject: *Re: [tsvwg] [Ecn-sane] per-flow scheduling > > > > On Mon, Jul 22, 2019 at 9:44 AM Bob Briscoe wrote: > > Folks, > > As promised, I've pulled together and uploaded the main architectural > arguments about per-flow scheduling that cause concern: > > Per-Flow Scheduling and the End-to-End Argum ent > > > > It runs to 6 pages of reading. But I tried to make the time readers will > have to spend worth it. > > > > Before reading the other responses (poisoning my own thinking), I wanted > to offer my own reaction. In the discussion of figure 1, you seem to impl= y > that there's some obvious choice of bin packing for the flows involved, b= ut > that can't be right. What if the dark green flow has deadlines? Why shoul= d > that be the one that gets only leftover bandwidth? I'll return to this > point in a bit. > > > > The tl;dr summary of the paper seems to be that the L4S approach leaves > the allocation of limited bandwidth up to the endpoints, while FQ > arbitrarily enforces equality in the presence of limited bandwidth; but i= n > reality the bottleneck device needs to make *some* choice when there's a > shortage and flows don't respond. That requires some choice of policy. > > > > In FQ, the chosen policy is to make sure every flow has the ability to ge= t > low latency for itself, but in the absence of some other kind of trusted > signaling allocates an equal proportion of the available bandwidth to eac= h > flow. ISTM this is the best you can do in an adversarial environment, > because anything else can be gamed to get a more than equal share (and > depending on how "flow" is defined, even this can be gamed by opening up > more flows; but this is not a problem unique to FQ). > > > > In L4S, the policy is to assume one queue is well-behaved and one not, an= d > to use the ECT(1) codepoint as a classifier to get into one or the other. > But policy choice doesn't end there: in an uncooperative or adversarial > environment, you can easily get into a situation in which the bottleneck > has to apply policy to several unresponsive flows in the supposedly > well-behaved queue. Note that this doesn't even have to involve bad actor= s > misclassifying on purpose: it could be two uncooperative 200 Mb VR flows > competing for 300 Mb of bandwidth. In this case, L4S falls back to classi= c, > which with DualQ means every flow, not just the uncooperative ones, > suffers. As a user, I don't want my small, responsive flows to suffer whe= n > uncooperative actors decide to exceed the BBW. > > > > Getting back to figure 1, how do you choose the right allocation? With th= e > proposed use of ECT(1) as classifier, you have exactly one bit available = to > decide which queue, and therefore which policy, applies to a flow. Should > all the classic flows get assigned whatever is left after the L4S flows a= re > allocated bandwidth? That hardly seems fair to classic flows. But let's s= ay > this policy is implemented. It then escapes me how this is any different > from the trust problems facing end-to-end DSCP/QoS: why wouldn't everyone > just classify their classic flows as L4S, forcing everything to be treate= d > as classic and getting access to a (greater) share of the overall BBW? Th= en > we're left both with a spent ECT(1) codepoint and a need for FQ or some > other queuing policy to arbitrate between flows, without any bits with > which to implement the high-fidelity congestion signal required to achiev= e > low latency without getting squeezed out. > > > > The bottom line is that I see no way to escape the necessity of something > FQ-like at bottlenecks outside of the sender's trust domain. If FQ can't = be > done in backbone-grade hardware, then the only real answer is pipes in th= e > core big enough to force the bottleneck to live somewhere closer to the > edge, where FQ does scale. > > > > Note that, in a perfect world, FQ wouldn't trigger at all because there > would always be enough bandwidth for everything users wanted to do, but i= n > the real world it seems like the best you can possibly do in the absence = of > trusted information about how to prioritize traffic. IMO, best to think o= f > FQ as a last-ditch measure indicating to the operator that they're gonna > need a bigger pipe than as a steady-state bandwidth allocator. > > > > Kyle > > > --0000000000004b57d9058eab6948 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Right, I understand that under 3168 behavior the send= er would react differently to ECE markings than L4S flows would, but I gues= s I don't understand why a sender willing to misclassify traffic with E= CT(1) wouldn't also choose to react non-normatively to ECE markings. On= the rest, I think we agree.

Kyle

On = Thu, Jul 25, 2019 at 3:26 PM Holland, Jake <jholland@akamai.com> wrote:

Hi Kyle,

=C2=A0

I almost agree, except that the concern is not about= classic flows.

=C2=A0

I agree (with caveats) with what Bob and Greg have s= aid before: ordinary classic flows don=E2=80=99t have an incentive to mis-m= ark if they=E2=80=99ll be responding normally to CE, because a classic flow= will back off too aggressively and starve itself if it=E2=80=99s getting CE marks from the LL queue.

=C2=A0

That said, I had a message where I tried to express = something similar to the concerns I think you just raised, with regard to a= different category of flow:

https://mailarchive.ietf.= org/arch/msg/tsvwg/bUu7pLmQo6BhR1mE2suJPPluW3Q

=C2=A0

So I agree with the concerns you=E2=80=99ve raised h= ere, and I want to +1 that aspect of it while also correcting that I don=E2= =80=99t think these apply for ordinary classic flows, but rather for flows = that use application-level quality metrics to change bit-rates instead responding at the transport level.

=C2=A0

For those flows (which seems to include some of toda= y=E2=80=99s video conferencing traffic), I expect they really would see an = advantage by mis-marking themselves, and will require policing that imposes= a policy decision.=C2=A0 Given that, I agree that I don=E2=80=99t see a simple alternative to FQ for flows originating outsi= de the policer=E2=80=99s trust domain when the network is fully utilized.

=C2=A0

I hope that makes at least a little sense.=

=C2=A0

Best regards,

Jake

=C2=A0

From: Kyle Rose <krose@krose.org>
Date: 2019-07-23 at 11:13
To: Bob Briscoe <ietf@bobbriscoe.net>
Cc: "ecn-sane@lists.bufferbloat.net" <ecn-sane@lists.bufferbloa= t.net>, tsvwg IETF list <tsvwg@ietf.org>, "David P. Reed" <dpreed@deepplum.com&g= t;
Subject: Re: [tsvwg] [Ecn-sane] per-flow scheduling

=C2=A0

On Mon, Jul 22, 2019 at = 9:44 AM Bob Briscoe <ietf@bobbriscoe.net> wrote:

Folks,

As promised, I've pulled together and uploaded the main architectural a= rguments about per-flow scheduling that cause concern:

Per-Flow Scheduling and the End-to-End Argum ent


It runs to 6 pages of reading. But I tried to make the time readers will ha= ve to spend worth it.

=C2=A0

Before reading the other= responses (poisoning my own thinking), I wanted to offer my own reaction. = In the discussion of figure 1, you seem to imply that there's some obvi= ous choice of bin packing for the flows involved, but that can't be right. What if the dark green flow has deadlines? Wh= y should that be the one that gets only leftover bandwidth? I'll return= to this point in a bit.

=C2=A0

The tl;dr summary of the= paper seems to be that the L4S approach leaves the allocation of limited b= andwidth up to the endpoints, while FQ arbitrarily enforces equality in the= presence of limited bandwidth; but in reality the bottleneck device needs to make *some* choice when there's= a shortage and flows don't respond. That requires some choice of polic= y.

=C2=A0

In FQ, the chosen policy= is to make sure every flow has the ability to get low latency for itself, = but in the absence of some other kind of trusted signaling allocates an equ= al proportion of the available bandwidth to each flow. ISTM this is the best you can do in an adversarial environme= nt, because anything else can be gamed to get a more than equal share (and = depending on how "flow" is defined, even this can be gamed by ope= ning up more flows; but this is not a problem unique to FQ).

=C2=A0

In L4S, the policy is to= assume one queue is well-behaved and one not, and to use the ECT(1) codepo= int as a classifier to get into one or the other. But policy choice doesn&#= 39;t end there: in an uncooperative or adversarial environment, you can easily get into a situation in which the bottleneck h= as to apply policy to several unresponsive flows in the supposedly well-beh= aved queue. Note that this doesn't even have to involve bad actors misc= lassifying on purpose: it could be two uncooperative 200 Mb VR flows competing for 300 Mb of bandwidth. In this c= ase, L4S falls back to classic, which with DualQ means every flow, not just= the uncooperative ones, suffers. As a user, I don't want my small, res= ponsive flows to suffer when uncooperative actors decide to exceed the BBW.

=C2=A0

Getting back to figure 1= , how do you choose the right allocation? With the proposed use of ECT(1) a= s classifier, you have exactly one bit available to decide which queue, and= therefore which policy, applies to a flow. Should all the classic flows get assigned whatever is left after the= L4S flows are allocated bandwidth? That hardly seems fair to classic flows= . But let's say this policy is implemented. It then escapes me how this= is any different from the trust problems facing end-to-end DSCP/QoS: why wouldn't everyone just classify their = classic flows as L4S, forcing everything to be treated as classic and getti= ng access to a (greater) share of the overall BBW? Then we're left both= with a spent ECT(1) codepoint and a need for FQ or some other queuing policy to arbitrate between flows, without an= y bits with which to implement the high-fidelity congestion signal required= to achieve low latency without getting squeezed out.

=C2=A0

The bottom line is that = I see no way to escape the necessity of something FQ-like at bottlenecks ou= tside of the sender's trust domain. If FQ can't be done in backbone= -grade hardware, then the only real answer is pipes in the core big enough to force the bottleneck to live somewhere clo= ser to the edge, where FQ does scale.

=C2=A0

Note that, in a perfect = world, FQ wouldn't trigger at all because there would always be enough = bandwidth for everything users wanted to do, but in the real world it seems= like the best you can possibly do in the absence of trusted information about how to prioritize traffic. IMO, best = to think of FQ as a last-ditch measure indicating to the operator that they= 're gonna need a bigger pipe than as a steady-state bandwidth allocator= .

=C2=A0

Kyle

=C2=A0

--0000000000004b57d9058eab6948--