From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-x242.google.com (mail-wr0-x242.google.com [IPv6:2a00:1450:400c:c0c::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 226EA3BA8E for ; Tue, 5 Jun 2018 07:46:18 -0400 (EDT) Received: by mail-wr0-x242.google.com with SMTP id y15-v6so2063459wrg.11 for ; Tue, 05 Jun 2018 04:46:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eventide.io; s=google; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=chgCr0wRJHtcICuFe0UrHLaXoh2gM0p/10GkQwkpSGo=; b=QheZm4B9HnNm/CfqDBcey4IkR88ztPOku70+/rA052qTNV4QHOlRE6aRqQH9Zv81xl my1vWDRIMlZKhv3Hz5wwxaY57siBqx9wmXGgUI3bfTUux4y2ZR9ntfTCHG4hjRbH/Srd HHVm4wM/U024ssYejbPCEga7zt5MNnK73z5kvvWaBrrjW4BwMmoRc2NdSvcrvIN5kNh0 Q1lLr4RnMHFm84eNLLqiQF03y6BiXw+9DjuGIvjQcN0OfnNeumTO1p7F5soYt/c+JoKc vhgnGtdRjKl67cZlE0XU/6sGa0Gs2tW/RisvxeisBJboGuRksjzuZLxMqfwWg0zzQBTA OJQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=chgCr0wRJHtcICuFe0UrHLaXoh2gM0p/10GkQwkpSGo=; b=qZ9AKjMCWqMhM6sy2nBbj+SE8qM9/FJYkhBOeiFAXp+a3jupBbMi9QfCQRGncR5fvs 2VHQRuD3K3nXG/5pmApBdXlW2dZxz2iIVqvVTncqhehJHIU9rCjHs/koOUWtHF7Bzb4p zzg15P3rgGHbHEbuiKu3WwJ9iEnB4xoy0YMwixcceRMM7NjBOSaikg1r9k1jpklzTkPR NbLyujdD4xrirGZu623VPAs9y+xfadU36/yWizxvpxGvrOh9Ub1F8fOXizsP2se1rdhK HIUVYjTA66snsV/jsxxZDPymZdvumTt3hiE0Qz0e0VJmPKXQTfN5d/tC/dTsNe7q+19e d+jg== X-Gm-Message-State: ALKqPwfCcbQe5sf7pZyP2GBnHZQTpmTx4idQz9tpC/RONy2q4URNjH6D rLjGx89YklSlZ2Ps22SEQcxcEfaP4Pc= X-Google-Smtp-Source: ADUXVKJrb408iP3KkgIJjSBNFn5EoicHQAAVT4TXPbbDEitwSAvFTKG2w4St135EwZRNU4uSRd/dhw== X-Received: by 2002:adf:8e30:: with SMTP id n45-v6mr18200392wrb.27.1528199177010; Tue, 05 Jun 2018 04:46:17 -0700 (PDT) Received: from tron.luk.eventide.io (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id 44-v6sm107934616wrv.47.2018.06.05.04.46.15 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 05 Jun 2018 04:46:16 -0700 (PDT) Content-Type: multipart/mixed; boundary="Apple-Mail=_BDFCDAB2-2DE4-4E0D-8310-4B1296821A14" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Pete Heist In-Reply-To: <87602zh2p0.fsf@toke.dk> Date: Tue, 5 Jun 2018 13:46:15 +0200 Cc: Georgios Amanakis , Dave Taht , Cake List Message-Id: <7F63C1A8-07F2-4523-BF8D-6DA76F6FCF2B@eventide.io> References: <87muwe9z7j.fsf@toke.dk> <87k1ri9yve.fsf@toke.dk> <877enimhuh.fsf@toke.dk> <9A273CA6-FDF6-485D-B466-8B01D4573BD9@gmail.com> <871sdqmg8l.fsf@toke.dk> <87a7sdcan1.fsf@toke.dk> <877enhc7o0.fsf@toke.dk> <55f626e4c3e0bf56321f9991fd0693d0ef08a66a.camel@gmail.com> <87602zh2p0.fsf@toke.dk> To: =?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Mailer: Apple Mail (2.3124) Subject: Re: [Cake] Lockup at high speeds X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 11:46:18 -0000 --Apple-Mail=_BDFCDAB2-2DE4-4E0D-8310-4B1296821A14 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Jun 4, 2018, at 1:26 AM, Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >=20 > Georgios Amanakis writes: >=20 >> I am trying to reproduce this with veth to no avail. >> I compiled a net-next kernel with Toke's configuration, and = simplified >> veth to only a client and a server. >=20 > Hmm. Guess it may only be triggered by the way the mlx5 driver calls > transmit, or something? Guess I'll have to go back and check once I = get > some spare cycles on that machine... As another test, I tried redirecting egress and ingress of the loopback = adapter through a common IFB. I couldn=E2=80=99t reproduce it either, = but I can only hit ~3Gbit total with cake and flent running on one APU2. If anyone wants to try it (faster hardware?), just run the attached = =E2=80=98flentlo.sh' with no arguments as root on a box with cake and = flent installed and netserver running. One run is done with noqueue and = the other with cake datacentre. Interestingly, for me the =E2=80=98datacentre' keyword actually both = increases total throughput and reduces rtt in this case (3286Mbit/5ms vs = 2838Mbit/15ms), so I left that in the test. I wonder if the rtt = parameter affects the lockup in any way, but I doubt it. Also, this is a case where using netperf UDP_RR reduces total throughput = vs irtt (for me, 2910Mbit vs 3286Mbit), probably due to competition. Pete --Apple-Mail=_BDFCDAB2-2DE4-4E0D-8310-4B1296821A14 Content-Disposition: attachment; filename=flentlo.sh Content-Type: application/octet-stream; name="flentlo.sh" Content-Transfer-Encoding: 7bit #!/bin/bash IFB_IFACE="ifblo" IFACE="lo" qos_on() { modprobe act_mirred # add ifb device ip link add name $IFB_IFACE type ifb tc qdisc add dev $IFACE handle ffff: ingress ifconfig $IFB_IFACE up # redirect ingress to ifb tc filter add dev $IFACE parent ffff: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ${IFB_IFACE} # redirect egress to ifb tc qdisc add dev $IFACE root handle 1: prio bands 2 priomap 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 tc filter add dev $IFACE parent 1: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ${IFB_IFACE} # add cake to ifb tc qdisc add dev ${IFB_IFACE} root cake datacentre nonat unlimited } qos_off() { tc qdisc del dev ${IFB_IFACE} root > /dev/null 2>&1 ifconfig ${IFB_IFACE} down > /dev/null 2>&1 tc qdisc del dev ${IFACE} handle ffff: ingress > /dev/null 2>&1 ip link delete dev ${IFB_IFACE} > /dev/null 2>&1 tc qdisc del dev $IFACE root > /dev/null 2>&1 } run() { flent rrul_be -l 60 -H localhost -p all_scaled --figure-width=10 --figure-height=7.5 -t $1 -o ${1}.png } if [ "$1" == "on" ]; then qos_on elif [ "$1" == "off" ]; then qos_off elif [ "$1" == "run" ]; then qos_off qos_on run "$2" flent -i *.gz else qos_off run noqueue qos_on run ifb qos_off flent -i *.gz fi --Apple-Mail=_BDFCDAB2-2DE4-4E0D-8310-4B1296821A14--