From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id C4D353B29E for ; Tue, 8 Jan 2019 17:01:21 -0500 (EST) Received: by mail-wm1-x32e.google.com with SMTP id y185so9456364wmd.1 for ; Tue, 08 Jan 2019 14:01:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Jp2kP6AzUjxxHlSOhnfhkBsS1iIUGLIl9VyZDEMKKnI=; b=GymfdZtah/dYKspwelnXTIjlFkYAt+1m02xaYyKsXEJkd2ELJHWcGVl2l//DMjuJ6g 8mmvCdgrdD0VHC1J+ybLheiZjAugssM5HEtNA4FoFK3bFBg8suBetFu72D1JowHa/sH5 AS4mWN93dnQLqBedliyPx9q22H+FODhI9W46s9v00CnFoCh2RHZMokLAM2C0wmMbOjaU gRsfdndhvxstM7p8c6rO6moTDyG+Q4wxnAKL6OTY+Dc2adB2W38gPAaHLWHl7y+0LoMz AjFj9kTqTorrZlJGINAvaNIe71dO9+OESZtGSX+feoXaeeLt0IUSNiAwn+x6Xu06uEIp HXGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Jp2kP6AzUjxxHlSOhnfhkBsS1iIUGLIl9VyZDEMKKnI=; b=adppG+MZjPBVWkMmKV7pxGQq4o5SXohF9syn8OGZcUaY3GG9TJmjip4vEKWfKgfEZi Gg8tW2TFWd3jRj57Dz6TqMzC01iRVNXCDaGuIcsiU3GRbHY4Q/R6nQRxljuG/YItxkSE xPMmxdxie1aK/6KET1S7nOSGisUxmESeNal86znxFnSYq1hlk2fZmaSNCgEN174Ba7Ul rgCvB1/qeyhmjOKdz9liRv+vIdHbIMqq8WuIEBRIS3eVW0XeoGvemWK64yuFLWZNxD9q Bs8TvA+jU+ucIJ0ktxgCakHKoIx3Q/j6YLUuVS/Ho8p8a4xD+ClJmVeHQDKZSP1XcSJH AiJw== X-Gm-Message-State: AJcUukdqMWok+PqgL1HOHehsCwCOFsFu5YVm+ID5NQzUyLqgPlxGVLsr gZM+zgbDEsSYSRYyBbeVqFTkow== X-Google-Smtp-Source: ALg8bN7J4xTS7TH7CjchHsMn4OXLXc9fKoC88OViFHjIhmf+eHgxD4YYb+U8JNCaHn/bhi1xyi7Pjw== X-Received: by 2002:a1c:70b:: with SMTP id 11mr3209730wmh.74.1546984880626; Tue, 08 Jan 2019 14:01:20 -0800 (PST) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id b18sm52656026wrw.83.2019.01.08.14.01.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 Jan 2019 14:01:19 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Pete Heist In-Reply-To: Date: Tue, 8 Jan 2019 23:01:18 +0100 Cc: Cake List Content-Transfer-Encoding: quoted-printable Message-Id: <749130B2-8A84-4F88-AF23-385B438C04DA@heistp.net> References: <87wonjxvss.fsf@toke.dk> <0077CC34-490F-4D76-82D3-BE37B27F2E1C@heistp.net> <49A6DCF8-BE98-47F4-9C66-6B4288390A58@heistp.net> <87tvinxos7.fsf@toke.dk> <87r2drxnal.fsf@toke.dk> <45D43135-318B-48AD-B09B-69BBB034CE12@heistp.net> <87o98vxm57.fsf@toke.dk> <797FCC60-0048-4EF6-80BC-19707E9173FB@heistp.net> <87lg3zxdyr.fsf@toke.dk> <87imz2yiet.fsf@toke.dk> <252DC221-7024-4834-9757-96335372A5A7@heistp.net> <87ftu6yc2y.fsf@toke.dk> <11DD478A-E61D-4D62-92B3-30B9A9A9572E@heistp.net> <874laly07v.fsf@toke.dk> <75345B18-E171-44C8-B059-9F6A6A663CAC@heistp.net> <87sgy4wvqq.fsf@toke.dk> <4FB89A19-314E-4B45-BEF3-83DB04A90C08@heistp.net> <39B9038B-42DF-41B6-9D78-6B71DCB06B2A@heistp.net> To: Dave Taht X-Mailer: Apple Mail (2.3445.9.1) Subject: Re: [Cake] cake infinite loop(?) with hfsc on one-armed router X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2019 22:01:22 -0000 I should have done that: = https://www.heistp.net/downloads/htb_split_gso_patched/ Note that I changed the names in the plots to match the convention of my = first email, but it should be clear which is which and I left all plots = in. The text output is there too as I sometimes like to open several up = in different browser tabs and switch between tabs to compare values. It looks like about 100 usec to me. Throughput also looks consistently = about 0.3 Mbit higher (~1.3%) in the split results. > On Jan 8, 2019, at 9:44 PM, Dave Taht wrote: >=20 > On Tue, Jan 8, 2019 at 12:03 PM Pete Heist wrote: >>=20 >> Here=E2=80=99s the re-test with the patched version and HTB. Looks = like success, nice work! >=20 > I note that I'm big on having the flent.gz files around also. In this > case, by eyeball, split-gso appears to have about 130us less latency, > but a cdf comparison of split vs no-split woud show that more easily. >=20 >> Split GSO on: >>=20 >> = https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_split1.sv= g >> = https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_split2.sv= g >> = https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_split3.sv= g >>=20 >> Split GSO off: >>=20 >> = https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_no_split1= .svg >> = https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_no_split2= .svg >> = https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_no_split3= .svg >>=20 >> Your patch in the latest kernels looks simpler. Bringing the patch = back to prior kernel versions would be appreciated, but I can understand = how 3.16 becomes less and less relevant as time goes on, although, = it=E2=80=99s not at end of life yet. :) >>=20 >> Interesting how download rate control in each of the graphs with GSO = splitting on looks accurate to the point where flent=E2=80=99s = throughput graph scale is at 0.02 Mbit per step, and one can see that = values coming back from netperf are probably quantized to 0.01 Mbit... >>=20 >>> On Jan 7, 2019, at 4:07 PM, Pete Heist wrote: >>>=20 >>> Sorry, that=E2=80=99s without the patch, will give that a try when I = have a chance and post the results, probably tomorrow... >>>=20 >>>> On Jan 7, 2019, at 12:30 PM, Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >>>>=20 >>>> Pete Heist writes: >>>>=20 >>>>>> On Jan 6, 2019, at 9:56 PM, Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >>>>>>=20 >>>>>> Pete Heist writes: >>>>>>=20 >>>>>>> Lastly, is using cake as a leaf to htb risky until a fix is = made? I=E2=80=99ve >>>>>>> been doing that for a while without any apparent issues, though = I=E2=80=99m >>>>>>> hesitating now to try that in a production environment. >>>>>>=20 >>>>>> Hmm, that's a good question. I would expect so; but I would also = expect >>>>>> the issue to show up pretty much straight away, so if you haven't = hit it >>>>>> yet, I may be wrong. I'll do some more digging... Should probably = also >>>>>> try to replicate all this stuff on my own machine :) >>>>>=20 >>>>>=20 >>>>> Ok, after what I=E2=80=99m seeing on my APU1 tests on 3.16.7, = I=E2=80=99m definitely >>>>> not putting split GSO into production. I just turned it on and off >>>>> three times and here=E2=80=99s what I got: >>>>>=20 >>>>> Split GSO on: >>>>>=20 >>>>> = https://www.heistp.net/downloads/htb_split_gso/htb_cake_split_gso.svg >>>>> = https://www.heistp.net/downloads/htb_split_gso/htb_cake_split_gso2.svg >>>>> = https://www.heistp.net/downloads/htb_split_gso/htb_cake_split_gso3.svg >>>>>=20 >>>>> Split GSO off: >>>>>=20 >>>>> = https://www.heistp.net/downloads/htb_split_gso/htb_cake_no_split_gso.svg >>>>> = https://www.heistp.net/downloads/htb_split_gso/htb_cake_no_split_gso2.svg >>>>> = https://www.heistp.net/downloads/htb_split_gso/htb_cake_no_split_gso3.svg >>>>>=20 >>>>> I=E2=80=99ve seen these square waves before with htb and wondered = where they >>>>> came from, and I think we may finally have an answer! What manner = of >>>>> thing causes this I don=E2=80=99t know, but there=E2=80=99s a = chance you may end up >>>>> finding out=E2=80=A6 :) >>>>=20 >>>> Is this without the patch to CAKE that adjusts the qlen? And have = you >>>> tried running with that patch (with HTB)? >>>>=20 >>>> -Toke >>>=20 >>=20 >> _______________________________________________ >> Cake mailing list >> Cake@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cake >=20 >=20 >=20 > --=20 >=20 > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740