From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 45CA83CB35 for ; Sun, 3 Mar 2019 13:48:07 -0500 (EST) Received: by mail-wr1-x42d.google.com with SMTP id l5so3142677wrw.6 for ; Sun, 03 Mar 2019 10:48:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=JdHvEj6IWyGL6a88seasDoMCKCJ1WDSkxzpzS9wFEd4=; b=C22U98Kwtvk2CA+pjVKs+Om+wb1fWZVQmAXQYgClLxNWRrfzpseL65iL5ef6Xh53n3 Qf9Ag26c+ox6l5R2UZUE0tR7sQqHo3ECJPBe/llET3VItSMT9+5cR94VbqxNivSEyyLe nMLwAbovvRDSJKxikjTDC1sEFSb13OtRBieJfdeWXC03E2TP/sRty+eGeJVfEN9qB+uj /Qr0uPV97NIn2ucJhT3fCc7aWXOHTDIdR+5YAl39kudeHY+ka0djkXVwscbLF7wXBy9l w3otDz4xliJiePAClfL9GjEOSlhlW7Qaxv/kjOXx7IZKvm/AcQX4sw5yNWGO1OmlK+ZN g6sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=JdHvEj6IWyGL6a88seasDoMCKCJ1WDSkxzpzS9wFEd4=; b=uHEKgMMBn9ez61VUNUP0q8K6ARLN4l5u8gna5cu6xeORcA1Ih0KoXuxSKuU7gD3qEV N5rNNNeP9OihjdZjXvRcrh1aOZbkM5CmAk++1LACS2fZDn3OUjC+efhNsKlummYNwsIF G+lDOOxFRpHT2aOtC02B+zZrh1Azsm9Ib9dUev1cpNPK7Um6l94a9wQTiV462KiK0cyn vb5HSG5eEJ07qT32+rRz4c+RsUETexXxBNsy3op6RsWU4rI+4AemL5AXTP/t3cWYz9ZT Gl+EbCrkqeQwFATnF/8IXOKDhlqsenVDSyPnfVUxtIJj4zeX2Fpe6/aa18m/K9WuFbdW rGZQ== X-Gm-Message-State: APjAAAUiMyfTh4cpmrAiTcp77uGJTVyi4ZkpoSPc9+HLETYXj0b85JMy +RCA6FzsGtVfOq3EWSM0mOpMyA== X-Google-Smtp-Source: APXvYqxoza0zikjEYG9prX3qKnPmayjrAaiUOJRfmr26PZZUOo5ci31dQbRU3e4UEVHu9u7bh8vnug== X-Received: by 2002:a05:6000:6:: with SMTP id h6mr9933359wrx.134.1551638886242; Sun, 03 Mar 2019 10:48:06 -0800 (PST) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id y140sm15909592wmd.18.2019.03.03.10.48.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Mar 2019 10:48:05 -0800 (PST) From: Pete Heist Message-Id: <9746402E-41EA-43F9-AAE6-53C038EABFFF@heistp.net> Content-Type: multipart/alternative; boundary="Apple-Mail=_CAA02231-8DA0-4007-B27C-1123853306BC" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Date: Sun, 3 Mar 2019 19:48:04 +0100 In-Reply-To: <511C22FF-64A5-46CA-BFB9-4EA350B76122@gmail.com> Cc: Sebastian Moeller , =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen_via_Cake?= , George Amanakis To: Jonathan Morton References: <72193310-7502-47B8-9554-7F8F9FA23204@heistp.net> <874l8mn9iy.fsf@toke.dk> <417E17B2-17F7-4106-A92D-C5B5AC41D808@gmx.de> <5CC769AC-AD0C-40E8-BFCA-6AD2A8162FFC@gmail.com> <49A8FB42-EC0B-41B7-8CE4-888DF3C2A88D@gmx.de> <542D3BC3-317A-4F3E-A5B6-16FF678DDFAC@gmail.com> <5C8177FD-2605-4CDD-9AA2-F0E5AF9C779B@heistp.net> <1C81557C-2FD5-4C48-BC17-F9E841D08097@gmail.com> <511C22FF-64A5-46CA-BFB9-4EA350B76122@gmail.com> X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] Upstream submission of dual-mode fairness patch X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Mar 2019 18:48:07 -0000 --Apple-Mail=_CAA02231-8DA0-4007-B27C-1123853306BC Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Mar 3, 2019, at 5:40 PM, Jonathan Morton = wrote: >=20 >> On 3 Mar, 2019, at 6:35 pm, Pete Heist wrote: >>=20 >> It almost seems like in ingress mode, dropped packets are being = counted as bytes transferred, when they weren=E2=80=99t. >=20 > Well yes, that's the whole point - at least as far as the shaper is = concerned. Ingress mode is supposed to deal with the case where inbound = packets have already traversed the bottleneck link *before* Cake gets to = choose what to do with them. >=20 > But that shouldn't affect fairness, only total throughput. Fairness = is not handled by the shaper, but by the (very) extended DRR++ = algorithm. It should probably come as no surprise then that if I take pcaps from = the client side and sum cumulative bytes transferred with 1514 * the = number of drops (according to tcp.analysis.lost_segment), I get pretty = close to the same value for each IP. I think that roughly confirms what = we think is happening. = https://www.heistp.net/downloads/ingress_fairness/client/ = IP 1, 16 down: 54968142 bytes transferred 5313 drops (according to tcp.analysis.lost_segment) 54968142 + 1514 * 5313 =3D 63012024 IP 2, 1 down: 63769820 bytes transferred 61 drops (according to tcp.analysis.lost_segment) 63769820 + 1514 * 61 =3D 63862174 Knowing this though, maybe the current behavior is, after all, what we = want. :) I mean, if a host has caused more drops with more simultaneous flows, = then it has filled the pipe with that data, even if it=E2=80=99s been = dropped to signal congestion. Should we penalize other hosts in order to = equalize goodput for all hosts, regardless of how many flows they use? = If a client wants to ensure that they aren=E2=80=99t =E2=80=9Ccharged" = for drops for fairness purposes, they can use ECN. I think it=E2=80=99s subjective and I=E2=80=99m willing to be corrected = or be convinced otherwise, but thanks for bringing me to this point = though=E2=80=A6 --Apple-Mail=_CAA02231-8DA0-4007-B27C-1123853306BC Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Mar 3, 2019, at 5:40 PM, Jonathan Morton <chromatix99@gmail.com> wrote:

On 3 Mar, 2019, at 6:35 = pm, Pete Heist <pete@heistp.net> wrote:

It = almost seems like in ingress mode, dropped packets are being counted as = bytes transferred, when they weren=E2=80=99t.

Well yes, that's the whole point = - at least as far as the shaper is concerned.  Ingress mode is = supposed to deal with the case where inbound packets have already = traversed the bottleneck link *before* Cake gets to choose what to do = with them.

But that shouldn't affect = fairness, only total throughput.  Fairness is not handled by the = shaper, but by the (very) extended DRR++ algorithm.

It should probably come as no surprise = then that if I take pcaps from the client side and sum cumulative bytes = transferred with 1514 * the number of drops (according to = tcp.analysis.lost_segment), I get pretty close to the same value for = each IP. I think that roughly confirms what we think is happening. https://www.heistp.net/downloads/ingress_fairness/client/

IP 1, 16 = down:
  54968142 bytes transferred
 = 5313 drops (according to tcp.analysis.lost_segment)
  = 54968142 + 1514 * 5313 =3D 63012024

IP 2, 1 = down:
  63769820 bytes transferred
 = 61 drops (according to tcp.analysis.lost_segment)
  = 63769820 + 1514 * 61 =3D 63862174

Knowing this though, maybe the current = behavior is, after all, what we want. :)

I mean, if a host has caused more drops = with more simultaneous flows, then it has filled the pipe with that = data, even if it=E2=80=99s been dropped to signal congestion. Should we = penalize other hosts in order to equalize goodput for all hosts, = regardless of how many flows they use? If a client wants to ensure that = they aren=E2=80=99t =E2=80=9Ccharged" for drops for fairness purposes, = they can use ECN.

I think it=E2=80=99s subjective and I=E2=80=99m willing to be = corrected or be convinced otherwise, but thanks for bringing me to this = point though=E2=80=A6

= --Apple-Mail=_CAA02231-8DA0-4007-B27C-1123853306BC--