From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-x241.google.com (mail-wr0-x241.google.com [IPv6:2a00:1450:400c:c0c::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 374CB3BA8E for ; Tue, 5 Jun 2018 10:24:25 -0400 (EDT) Received: by mail-wr0-x241.google.com with SMTP id 94-v6so2657399wrf.5 for ; Tue, 05 Jun 2018 07:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eventide.io; s=google; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=OaQvqPFa3k9eBS/zJ7uFYm5rxWTSsfdH86vTVFCcAPM=; b=GJoPbWL3mHFTm/3Zxdh2rKbW3n9SW14Hdkb3M0UWeYCewZDpQT98eXAtWchzp6Ypv0 /h8Iv/TRjqDoYC4N3Bld2FcPpj6VP9ZEvH0P1V1QY4LSPSliZxTwFVFAOVRkIr3uFR0E aBTo4++Fw0hloVFVRQFTTS7CJO96qgskLEhMylUsxyQVV2CXh0b5FhQiv4UxjQuCOr++ /4s+0U8fQnjrexB95h/XNnxIUKbwJBJkCnuhkboQ7/TUdZOZ+9r48U6H8HEHkv9cEhW1 /SmtTqOozWO7mxfvjnP5/EjHuxWzDhAbrm6JFJWFtgFrsQu7dtNFTX4JCPGmA1WIvgss xq/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=OaQvqPFa3k9eBS/zJ7uFYm5rxWTSsfdH86vTVFCcAPM=; b=O3B5bsGh6nosVgy23o7ZDSAl4YNXYKwntvZTnyzV/skBUYq2qbLuoMxCyYdnRJ8ZU2 u70tw8A3FzJI85ASSUcgYRiBM/QwIR+zApA0YxNheLfOQGOHz5eipryeMX1xl0MzlEEF tx1J9pKollJT3VCfr9178XZel3AEOaRCAJY4aCFzpEqwxxRqmNRhSZl44jmStaWygFjK Bmuv4yp7wzKCNKPX+oRkXo4fJ+pB28DMJl6ithmnVKKtQaS54S/EjFy0DA//zXgFdOMd Y0w7/hIZYAGORB8wys8wPyJ+dV5xlFpWVOABG85htHuBJ2VXZGcGQ3o9CqYc5Dpbn9OC VG9Q== X-Gm-Message-State: ALKqPwd+dAcbZGNCffSNZNUu1QmFvt3AFmYJvbtWuL9YdaPySptYT9Jf 509mwZl6DXizDayRTlqwqsEwXw== X-Google-Smtp-Source: ADUXVKKm7BFlj9tpCwNNTFUUi7fXo6Mfz25AIysmYvc3PjdcMU5+wKkMadQIKnuJFolHMNTYSd6sEg== X-Received: by 2002:adf:ebc3:: with SMTP id v3-v6mr19461110wrn.33.1528208664268; Tue, 05 Jun 2018 07:24:24 -0700 (PDT) Received: from tron.luk.eventide.io (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id s9-v6sm1582527wmc.34.2018.06.05.07.24.22 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 05 Jun 2018 07:24:22 -0700 (PDT) Content-Type: multipart/mixed; boundary="Apple-Mail=_1937C333-22C4-402B-AE6B-DF963C6B7045" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Pete Heist In-Reply-To: <7F63C1A8-07F2-4523-BF8D-6DA76F6FCF2B@eventide.io> Date: Tue, 5 Jun 2018 16:24:21 +0200 Cc: Georgios Amanakis , Dave Taht , Cake List Message-Id: <038B0692-217D-4202-9485-7B5E821C753B@eventide.io> References: <87muwe9z7j.fsf@toke.dk> <87k1ri9yve.fsf@toke.dk> <877enimhuh.fsf@toke.dk> <9A273CA6-FDF6-485D-B466-8B01D4573BD9@gmail.com> <871sdqmg8l.fsf@toke.dk> <87a7sdcan1.fsf@toke.dk> <877enhc7o0.fsf@toke.dk> <55f626e4c3e0bf56321f9991fd0693d0ef08a66a.camel@gmail.com> <87602zh2p0.fsf@toke.dk> <7F63C1A8-07F2-4523-BF8D-6DA76F6FCF2B@eventide.io> To: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Mailer: Apple Mail (2.3124) Subject: Re: [Cake] Lockup at high speeds X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 14:24:25 -0000 --Apple-Mail=_1937C333-22C4-402B-AE6B-DF963C6B7045 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jun 5, 2018, at 1:46 PM, Pete Heist wrote: >=20 >> On Jun 4, 2018, at 1:26 AM, Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >>=20 >> Georgios Amanakis writes: >>=20 >>> I am trying to reproduce this with veth to no avail. >>> I compiled a net-next kernel with Toke's configuration, and = simplified >>> veth to only a client and a server. >>=20 >> Hmm. Guess it may only be triggered by the way the mlx5 driver calls >> transmit, or something? Guess I'll have to go back and check once I = get >> some spare cycles on that machine... >=20 > As another test, I tried redirecting egress and ingress of the = loopback adapter through a common IFB. I couldn=E2=80=99t reproduce it = either, but I can only hit ~3Gbit total with cake and flent running on = one APU2. >=20 > If anyone wants to try it (faster hardware?), just run the attached = =E2=80=98flentlo.sh' with no arguments as root on a box with cake and = flent installed and netserver running. One run is done with noqueue and = the other with cake datacentre. >=20 > Interestingly, for me the =E2=80=98datacentre' keyword actually both = increases total throughput and reduces rtt in this case (3286Mbit/5ms vs = 2838Mbit/15ms), so I left that in the test. I wonder if the rtt = parameter affects the lockup in any way, but I doubt it. >=20 > Also, this is a case where using netperf UDP_RR reduces total = throughput vs irtt (for me, 2910Mbit vs 3286Mbit), probably due to = competition. Update attached just using lo on egress without IFB (also replacing = ifconfig with ip). Tested on VMWare w/ Debian 9 unstable (kernel = 4.16.0-2) up to 22Gbit and still didn=E2=80=99t see a lockup. This is as = far as I can go with what I have available. George=E2=80=99s veth test = is probably closer to the original config anyway. --Apple-Mail=_1937C333-22C4-402B-AE6B-DF963C6B7045 Content-Disposition: attachment; filename=flentlo.tgz Content-Type: application/octet-stream; name="flentlo.tgz" Content-Transfer-Encoding: base64 H4sIAK+bFlsAA+2V3W7TMBTHd9s8xSGbECCljbumlYY6CRATewaEKjexG2uunTkOE0x7d06Spuvn gIuBQOeXC9vJ3+fLp67Uwnht+2V+8mzEcTxJEmjGcTvGw1E71tNkMgZ2fh4nCRuNJwnEbDQcxScQ P19Ij1Sl5w5DKYQXuVClP6JDmZRP2GlzgfX4j3D6YjBXZjDnZR4E11fvZ9dX7z58nIZKzrUNg9Wq nga3tpxZ8+o13Ac9n8JtpsoUeJZBJr7CWaMEZ62HlN8IyLjnKfaWE2Cs4R4qo9VSeZEFD52tGXpp 7S1tVjg7F8BTP1sq51AW9E4b8yiqXahUBD1VgFbmpnlv+FLUflcxg/9WiFp8PLqcm0wLkMgFKLNw oiwfbZbCd+LOZFUEvToMDEc5kfpuE3i79iSV9sLtuiq4w+RXrjA1b1OrgWuNC2WBxVCdD2HJfZo3 sxgfqe2dyoBdsLoMyhpoKwGidbqOovFzvw7zYSdIsRfj8bNalYRdtHHNcVnCsFkseQHs6efnBWB/ IPuuUZq+O571xpbf61Mpd5o+E/qIyUsY4IeBqTDV4eVLtt1dW/rM3pkD+oNOmg2HuvcJh7gfr7T9 OH/F5UaD7MmxKq5aXQOy/vcA5yo9w99upGEcQ/QJtE25zm3pISrqU5+VuMaTjCKpFpUTER60z6fY Bus3ePUucj+d9BOIsFAMIoths4d+YRa1SyXhM4RnLITpFEJrQvjyFnwuTNBr75JA6B2NlLsiKfdU mMoBVWezh59ROwy7VCMFb/qL72imFBvyWmbsbSUqsb23acW1bNuGVMHfvv0JgiAIgiAIgiAIgiAI giAIgiAIgiAIgvgf+AF8ERbVACgAAA== --Apple-Mail=_1937C333-22C4-402B-AE6B-DF963C6B7045--