From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-x443.google.com (mail-wr1-x443.google.com [IPv6:2a00:1450:4864:20::443]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 0CA7F3B2A4 for ; Fri, 6 Jul 2018 18:12:57 -0400 (EDT) Received: by mail-wr1-x443.google.com with SMTP id h10-v6so5451223wrq.8 for ; Fri, 06 Jul 2018 15:12:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=HfRG8X8eljg87EExB6C0mmF0LHj8PjXhfci5G2pufEQ=; b=LJkhYmL3VX8shNxDEo/kH1w2fpr+2F+w9MOqBU17Rx3xB3NM4EZASgITEYConEpa4k ac3/rxzA47LW5vPonlxuulNGKXk309/ondxGQhmwU9CyN8M8rj0C8gR1t+CVqc6pE+iv 10PiP9pL1feujnQkqcZRmd5TBgczLo9wUp7uN7t9TtaechUNgR2cqjBHrUN8hqc+pZ12 FaPKKmeiERq0nsopA/NxgIxW2MPK6ELGDOkVqLHIwC4vFN8mjKjyl5OVGMYyLm0xpM62 Bm1H0Yq+Riesvj8fT7k+hwfWbccGo/QFa1Cm8TVkY0nOMK9P5HzhU3z5XLaOvumcFt0s kc+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=HfRG8X8eljg87EExB6C0mmF0LHj8PjXhfci5G2pufEQ=; b=WNcDXIT+xVyfcIc6u3gP8w7P0OxaNkMaHVShbWFJWKYNa59Nn1NhhQD1y3IeKXdwLs 2ytA/0Hem7O3miz8+OUHjj3PLre8vOAmBKyfawBAK8jBHFlR8qbmacbGskG1AUn2MRXk BmyY7k3Lzrqur3x7JiMkaMIMdK1v33lIvyiW9N2XKD2p0wmk6vf8qnE6azOdjto51QVD d7hDqhKi7cp9dHYHeM6iTKiHFS5KJ785qcDIjYVN/qArbLDMdKzW4xgKPYEvS0Jlz+6c 0Rb3e4DCiZnnTcKUDklydN30yE0jzgLPSTw2E3+N6qu48h5tFtTk9wAlv/ZLJsIDEIR7 z6Dg== X-Gm-Message-State: AOUpUlGGolLsQMwqi0HpUcI5KJdd4IAqMiBmTr9H/jF55jS/EQLvHlv9 05hfUqcwYn1VVqW5OZPECtQaQw== X-Google-Smtp-Source: AAOMgpe/APpBxMmmRFROHcKRoVlDGB7bnpXV3OBTELR2itIKr0BXO5EKaGgNC3rMWJEQbJDKEQoUkQ== X-Received: by 2002:a5d:4007:: with SMTP id n7-v6mr243977wrp.258.1530915175632; Fri, 06 Jul 2018 15:12:55 -0700 (PDT) Received: from tron.luk.heistp.net (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id s11-v6sm6882814wrm.42.2018.07.06.15.12.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Jul 2018 15:12:54 -0700 (PDT) From: Pete Heist Message-Id: <8F1AFA8F-1799-49AD-AE48-A0BD1B22CFB8@heistp.net> Content-Type: multipart/mixed; boundary="Apple-Mail=_BCED9522-8264-4F3E-A4D8-C58A4AC9D489" Mime-Version: 1.0 (Mac OS X Mail 11.4 \(3445.8.2\)) Date: Sat, 7 Jul 2018 00:12:53 +0200 In-Reply-To: Cc: Cake List To: Dave Taht References: <761C7004-247B-42B4-B56C-2527816826C7@heistp.net> <87y3eoa3wn.fsf@toke.dk> <1CFA01E7-2B3C-42C7-B102-3D0516EDCE43@heistp.net> <82EEB9CD-E441-4639-9592-A2429681EE0E@heistp.net> X-Mailer: Apple Mail (2.3445.8.2) Subject: Re: [Cake] lockup with cake and veth X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2018 22:12:57 -0000 --Apple-Mail=_BCED9522-8264-4F3E-A4D8-C58A4AC9D489 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Ok, it ended up being partly that, and mainly needing to increase kernel = params for max TCP window memory, otherwise there was a hard limit on = what I=E2=80=99d actually end up getting for the TCP window. Rates start getting finicky with these large TCP windows, go too large = and the rates can collapse for some reason, but I have managed to run = about 6Gbit with 100ms RTT through Cake this way without a problem, = either with a single stream or the aggregate of up to four streams. Or = 20Gbit at 20ms RTT. ./qenv.sh on ./run_iperf.sh ./run_flent.sh > On Jul 6, 2018, at 9:40 PM, Dave Taht wrote: >=20 > You ran out of queue to store for that rtt. >=20 > netem limit 100000 or higher for that. > On Fri, Jul 6, 2018 at 12:25 PM Pete Heist wrote: >>=20 >> Only, I wonder if high BDP links can be emulated with netem. Say I = want 10gbit with 100ms rtt. If I use =E2=80=9Cnetem delay 50ms=E2=80=9D = each direction, rtt is correct, but throughput for iperf3 drops from = 30Gbit to 450Mbit. CPU is a few percent, so that=E2=80=99s not the = issue. Fiddling with TCP window (iperf3 -w) just seems to make it worse. = So I=E2=80=99m still figuring out netem, I guess... --Apple-Mail=_BCED9522-8264-4F3E-A4D8-C58A4AC9D489 Content-Disposition: attachment; filename=veth_qm.tgz Content-Type: application/octet-stream; x-unix-mode=0644; name="veth_qm.tgz" Content-Transfer-Encoding: base64 H4sIAB3nP1sAA+1Y227bRhD1q/YrJrQQxEUpiRIlpbJloHD8YCBJE8ttH5pCoMiVRJhc0lxKthHk 3zvD5VUXy0Ecp0U1D6Z3dufM7GXm7GrJ4/n4xm8efEdpofS7Xfoa/W6r/M3kwOh0jH6/1zEN1Bvt fq93AN3vGVQmCxlbEcCBvJeW47ti27hd/f9RWab7Hy3EeOpxETfk/Kl97Nr/fq+f7X/bME3c/26n g/vfeupANsn/fP8PXzQnrmhOLDln7Orsw/jyz4v3Q63d7am90RLlKFH2zEzHcDXs2AP9FgSPG264 NBuxHY5vXeEEt2NpW54rZkNjZZwdRLwR+dwf+9bdsJ652zTqtjJqtD4q90l4Q81s/dKDHLH4T9sa 6orZKDcbJWbMDclASOB33AYZLakpebTkEZMe5yEY7BC8wL5ehDC3wpDj2Fs3ngMugfACyxnLOOKW L+F0CCZMgwh8/jMEAvx7uFlYjk4zXfFjey4keQi6B50W6LpEDzzW8ZjGEpsxl7EeWpHl85hHa76G JtDsaBsEGK2G0W7gHxZeu55XmkC+/1n+X57/+ubdeSO+i5/+jNGZ6VFeb8n/btdM879jtjp9zH/T 7Hf2+f8c8v63q3PGLhcCOJ6L+3iOeQuWhCgI4gZjo/Or3z8wdiFwlfAAuSGPph2IA0C+gHjOlQbo UDbyUer8WsKhA6f6CwPVqQzY6Ozy4sPViLEbLpbIPJgcA5B4SmGRZMVtEF2DwKMuQ8vmEppw47jS lvDKx32DCU9gJxxziyeY8ojVuOPGMPOCieVJsNBXEFIA9twSM54BNAme+5BkkmREf8lUMIgBgcp8 rojKyuyYdhfzKIKfTgfgcA+B5Ybo2e7deH4p83+2AE/tYxf/F/mP/I/ET/zf3vP/s0iF/5+C1Y38 Tv8grW8dtonXX/c7r1tQmDyG1I1e57VZsUG2pmpRzut1jk97daRaA16eNh2+bIoFFrWXOemvE3Zm pKuVgp75DvQ4oW+7RMI/eq83SZb/aQ37Lj525D+mf/H+a/e7xP8dY8//zyKV/D97ezH++OZidDbU NHb28V3WsK1rjoo3bzOF4k7P9ZFpVYoR71n30G35EjNt9Oiho1Uvo8s/ihgwZQv+HGALs02nP8i9 1LrxsxZeqPmCg28Ja4b3W+xzvLwvcYg6STp1Ay50hJHqVjGwIGR97BDbeJMIvGB2n0YCJ/opxaC+ jpd8ZfZN9YhB08gvNdl0UHfiBw4fRO5sHjcnkevMeNPj0/gUTmgUfhwubfyoO8op1S8am95ZBnDi hthLJiCSbgVS9Je6kn/LhkzIV0fwmdXwBWN5QKEMtbqhHYOcu9M401Mkm/QUWkWP9y57HoDOQfsk dBKoEyjUCQJe1cniaPBJ4K7W8hJqOY4agErbkjy1cQWrJXPF61zq0A2rYaRqmpeQK10Ij0x1naBj AYf6Z3LxpeFBfB9yoJoHIccNT0Krf1YgXxpRyRY3bMVWRayirdWqLJCOwijJaaTmhY0KwkNWuc/C 3yJEg+NjVlPbevS4Oa/GHT0cd7ICm/xHyv8PWMqtIXl5SFuHU5hpJMpmouJUS/gIX9Wl8y1Jj+wc 7CsBvG8FmBSHgBb2gWz4xu3femijrz8yFO1PFGpSEDRXLPHe6BSVL6k1A5XpGg27I2pIDbm07HKB 2JRd7AtjgVDV6xAS8wDfr1EUYLQUkE7lJKm5VHLTAl5DNFU/6Z6nKa1WXM+abZOKNquVTamNdmlh pZqubeIKjcp+xSc2y4ZIClqZeDTii7KBXTUgNtHK7KUR0VRis6uxEQ9pm7hQI6qqWir+JOOEFojK UtNiPdq4HkmASUolJK6ezrg5U/gLtHp+W9DgxRA0Df4+phe+WDsxhB/byjwvJKjEA0M/M0CBxGpT t8DPrgY78XHq6/g3foGfIVXxs0vKTnzcjHV8xyvwM6QK/uix+HITPiqxfij80Wb8x66P3LQ+qCzw N65PfhPbiY/5tI4fLQv8DCnBVzeF1RynQxLSj0/4ZMKnlAt4BEtvJ0r56VTlfG6ImaEOd1VDiVHV UHJVNHJds2ZFN7e0hr0vfoJSv+84DY1Ciji9HmIVFsaHfwTpOZY8oAL9r3zz7WUve9nLXvayl73s ZS97+X/KP245gbMAKAAA --Apple-Mail=_BCED9522-8264-4F3E-A4D8-C58A4AC9D489--