Re: [Cake] Lockup at high speeds

[Pete Heist <pete@eventide.io>]

[-- Attachment #1: Type: text/plain, Size: 1366 bytes --]


> On Jun 4, 2018, at 1:26 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Georgios Amanakis <gamanakis@gmail.com> writes:
> 
>> I am trying to reproduce this with veth to no avail.
>> I compiled a net-next kernel with Toke's configuration, and simplified
>> veth to only a client and a server.
> 
> Hmm. Guess it may only be triggered by the way the mlx5 driver calls
> transmit, or something? Guess I'll have to go back and check once I get
> some spare cycles on that machine...

As another test, I tried redirecting egress and ingress of the loopback adapter through a common IFB. I couldn’t reproduce it either, but I can only hit ~3Gbit total with cake and flent running on one APU2.

If anyone wants to try it (faster hardware?), just run the attached ‘flentlo.sh' with no arguments as root on a box with cake and flent installed and netserver running. One run is done with noqueue and the other with cake datacentre.

Interestingly, for me the ‘datacentre' keyword actually both increases total throughput and reduces rtt in this case (3286Mbit/5ms vs 2838Mbit/15ms), so I left that in the test. I wonder if the rtt parameter affects the lockup in any way, but I doubt it.

Also, this is a case where using netperf UDP_RR reduces total throughput vs irtt (for me, 2910Mbit vs 3286Mbit), probably due to competition.

Pete

[-- Attachment #2: flentlo.sh --]
[-- Type: application/octet-stream, Size: 1316 bytes --]

#!/bin/bash

IFB_IFACE="ifblo"
IFACE="lo"

qos_on() {
	modprobe act_mirred

	# add ifb device
	ip link add name $IFB_IFACE type ifb
	tc qdisc add dev $IFACE handle ffff: ingress
	ifconfig $IFB_IFACE up
	
	# redirect ingress to ifb
	tc filter add dev $IFACE parent ffff: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ${IFB_IFACE}
	
	# redirect egress to ifb
	tc qdisc add dev $IFACE root handle 1: prio bands 2 priomap 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
	tc filter add dev $IFACE parent 1: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ${IFB_IFACE}

	# add cake to ifb
	tc qdisc add dev ${IFB_IFACE} root cake datacentre nonat unlimited
}

qos_off() {
	tc qdisc del dev ${IFB_IFACE} root > /dev/null 2>&1
	ifconfig ${IFB_IFACE} down > /dev/null 2>&1
	tc qdisc del dev ${IFACE} handle ffff: ingress > /dev/null 2>&1
	ip link delete dev ${IFB_IFACE} > /dev/null 2>&1
	tc qdisc del dev $IFACE root > /dev/null 2>&1
}

run() {
	flent rrul_be -l 60 -H localhost -p all_scaled --figure-width=10 --figure-height=7.5 -t $1 -o ${1}.png
}

if [ "$1" == "on" ]; then
	qos_on
elif [ "$1" == "off" ]; then
	qos_off
elif [ "$1" == "run" ]; then
	qos_off
	qos_on
	run "$2"
	flent -i *.gz
else
	qos_off
	run noqueue
	qos_on
	run ifb
	qos_off
	flent -i *.gz
fi