From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x230.google.com (mail-qt0-x230.google.com [IPv6:2607:f8b0:400d:c0d::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B8DFA3B2A4 for ; Mon, 27 Nov 2017 12:38:46 -0500 (EST) Received: by mail-qt0-x230.google.com with SMTP id q13so28692401qtb.5 for ; Mon, 27 Nov 2017 09:38:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=bDoyqQscgBTYAlKr2TP9j06+dEZV7NRCt8OSQWujXZg=; b=R6LkXRp3G9CHBybnNy7vThtGOnfKpuIBNo+7N/wAe2LZmcw48hPiLY4YxY9dl+ra+c /s8rnkjyeK13XAlJ1ZpG/rBbyedS4EsOD50w+xwIMli+OFg2Ss6SGtv5aWrOJfhRK+rj 7RlAKvDTJiMgwquoBlTdN8aBBeeUcQ9OOnpIWMRY4q49OLX1ZtgqEiMwr7LCMfUYOzPc UkhV2UYxBqh7DNGb61TORAP8d1Nfp7KQ65G5GK/C1J7UkxPi0pGE3Ozui6nOXS9o/cpX yxZe54t9ghVzLyDNggNbaqd7pjmqatbv5c2gOiMzuuimQPPHoFqbiOw7vupUJrhzZT2A Jx8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=bDoyqQscgBTYAlKr2TP9j06+dEZV7NRCt8OSQWujXZg=; b=H/1nFdjpZO+pwwtAjn2RiDb9Bcg75BkYtw+DVucfBOYju/LUXDvTVW+RqjDFVqdpgJ OAMKWogpzIGR1ptVk5M4cosJrLChXbx2Je7IEevvyf3ytIENxStDSnaCIn/sGEbQc1NE qaO8aug70g22mU0JqlBTb40r/Ybw5wOreNrLJTWrVh4L2kLAGXrs/WDQ1k+NrrwMo5zE RlsmO+0b3bKh5MfumuCfCTyRLh8601CEJk6cUNC/ayY6dX2WfcqNTteqvMX8+BrEtjRZ lDnYu6fMCnBx0e5hv7HJqNtpUR/sVw8QxASmbGpWrzpEudtGs7tgV/2ohcoBJ5clpqWu MRvA== X-Gm-Message-State: AJaThX79vClEaXrNXQlz17jg9ZIzLS+sSoxnkq7jOP3BA4SA5vvYyCjq Uq8/u1AQS4qr7cBHHH5nw9XUFZMMHqES7PiP8eU= X-Google-Smtp-Source: AGs4zMb9iXDVQ1Us81Z6LVaEnZjgtxBDbkzdEe9YzpJJXlojXQRkGysc6DgBgoHGxZWQF3lSqXUz+y3GhN/ql0t56sk= X-Received: by 10.200.36.203 with SMTP id t11mr32689208qtt.277.1511804326240; Mon, 27 Nov 2017 09:38:46 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.193.93 with HTTP; Mon, 27 Nov 2017 09:38:45 -0800 (PST) In-Reply-To: <2A5F940F-F713-4578-8123-5CAD98A9C4C3@gmx.de> References: <85E1A7B2-8AA7-418A-BE43-209A1EC8881A@gmail.com> <87d1447z9w.fsf@toke.dk> <27F95EB1-490B-404C-8F77-98646B6159E7@gmail.com> <1C937A63-CEC1-4173-8812-EA2A85972B73@gmail.com> <20D304DC-494E-4A00-9B39-1E9F4B0F0CB6@gmail.com> <67B1612D-895D-4E3A-8CBD-21580B470696@gmail.com> <2A5F940F-F713-4578-8123-5CAD98A9C4C3@gmx.de> From: Dave Taht Date: Mon, 27 Nov 2017 09:38:45 -0800 Message-ID: To: Sebastian Moeller Cc: Georgios Amanakis , Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] cake flenter results round 1 X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Nov 2017 17:38:46 -0000 On Mon, Nov 27, 2017 at 9:34 AM, Sebastian Moeller wrote: > But 444.35 + 443.65 =3D 888, no? My bad. I miss-read the test setup. Pre-coffee here, though, that caused an adrenalin spike. Yea! per host fairness 1v12! and correct bandwidth on this cpu. :whew: > >> On Nov 27, 2017, at 18:33, Dave Taht wrote: >> >> georgios >> >> the result you got was "fair", but you shoul have seen something >> closer to 900mbit than 400. >> >> On Mon, Nov 27, 2017 at 8:17 AM, Georgios Amanakis = wrote: >>> Dear Pete, >>> >>> I am trying to replicate the unfair behaviour you are seeing with >>> dual-{src,dst}host, albeit on different hardware and I am getting a fai= r >>> distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients).= All >>> running Archlinux, latest cake and patched iproute2-4.14.1, connected w= ith >>> Gbit ethernet, TSO/GSO/GRO enabled. >>> >>> Qdisc setup: >>> ---------------- >>> Router: >>> qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3 >>> dual-dsthost rtt 100.0ms raw >>> >>> Client A(kernel default): >>> qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantu= m >>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn >>> >>> Client B (kernel default): >>> qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quan= tum >>> 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn >>> ---------------- >>> >>> >>> Cli: >>> ---------------- >>> Router: >>> netserver & >>> >>> Client A: >>> flent tcp_1down -H router >>> >>> Client B: >>> flent tcp_12down -H router >>> ---------------- >>> >>> >>> Results: >>> ---------------- >>> Router: >>> qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost= rtt >>> 100.0ms raw >>> Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 reque= ues >>> 0) >>> backlog 0b 0p requeues 0 >>> memory used: 1224872b of 15140Kb >>> capacity estimate: 900Mbit >>> Bulk Best Effort Voice >>> thresh 56250Kbit 900Mbit 225Mbit >>> target 5.0ms 5.0ms 5.0ms >>> interval 100.0ms 100.0ms 100.0ms >>> pk_delay 14us 751us 7us >>> av_delay 2us 642us 1us >>> sp_delay 1us 1us 1us >>> pkts 109948 4601651 14315 >>> bytes 160183242 6964893773 1618242 >>> way_inds 0 21009 0 >>> way_miss 160 188 5 >>> way_cols 0 0 0 >>> drops 0 10 0 >>> marks 0 0 0 >>> ack_drop 0 0 0 >>> sp_flows 0 1 1 >>> bk_flows 1 0 0 >>> un_flows 0 0 0 >>> max_len 7570 68130 1022 >>> >>> >>> Client A: >>> avg median # data pts >>> Ping (ms) ICMP : 0.11 0.08 ms 350 >>> TCP download : 443.65 430.38 Mbits/s 301 >>> >>> >>> Client B: >>> avg median # data pts >>> Ping (ms) ICMP : 0.09 0.06 ms 350 >>> TCP download avg : 37.03 35.87 Mbits/s 301 >>> TCP download sum : 444.35 430.40 Mbits/s 301 >>> TCP download::1 : 37.00 35.87 Mbits/s 301 >>> TCP download::10 : 37.01 35.87 Mbits/s 301 >>> TCP download::11 : 37.02 35.87 Mbits/s 301 >>> TCP download::12 : 37.00 35.87 Mbits/s 301 >>> TCP download::2 : 37.03 35.87 Mbits/s 301 >>> TCP download::3 : 36.99 35.87 Mbits/s 301 >>> TCP download::4 : 37.03 35.87 Mbits/s 301 >>> TCP download::5 : 37.07 35.87 Mbits/s 301 >>> TCP download::6 : 37.00 35.87 Mbits/s 301 >>> TCP download::7 : 37.12 35.87 Mbits/s 301 >>> TCP download::8 : 37.05 35.87 Mbits/s 301 >>> TCP download::9 : 37.03 35.87 Mbits/s 301 >>> ---------------- >>> >>> Does this suggest that it is indeed a problem of an underpowered CPU in= your >>> case? >>> >>> George >>> >>> >>> On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist wrot= e: >>>> >>>> >>>> On Nov 27, 2017, at 3:48 PM, Jonathan Morton >>>> wrote: >>>> >>>> It's not at all obvious how we'd detect that. Packets are staying in = the >>>> queue for less time than the codel target, which is exactly what you'd= get >>>> if you weren't saturated at all. >>>> >>>> That makes complete sense when you put it that way. Cake has no way of >>>> knowing why the input rate is lower than expected, even if it=E2=80=99= s part of the >>>> cause. >>>> >>>> I don=E2=80=99t think flent can know this either. It can=E2=80=99t eas= ily know the cause >>>> for its total output to be lower than expected. >>>> >>>> All I know is, this is a common problem in deployments, particularly o= n >>>> low-end hardware like ER-Xs, that can be tricky for users to figure ou= t. >>>> >>>> I don=E2=80=99t even think monitoring CPU in general would work. The C= PU could be >>>> high because it=E2=80=99s doing other calculations, but there=E2=80=99= s still enough for >>>> cake at a low rate, and there=E2=80=99s no need to warn in that case. = I=E2=80=99d be >>>> interested in any ideas on how to know this is happening in the system= as a >>>> whole. So far, there are just various clues that one needs to piece to= gether >>>> (no or few drops or marks, less total throughput that expected, high c= pu >>>> without other external usage, etc). Then it needs to be proven with a = test. >>>> >>>> Anyway thanks, your clue was what I needed! I need to remember to revi= ew >>>> the qdisc stats when something unexpected happens. >>>> >>>> _______________________________________________ >>>> Cake mailing list >>>> Cake@lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/cake >>>> >>> >>> >>> _______________________________________________ >>> Cake mailing list >>> Cake@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/cake >>> >> >> >> >> -- >> >> Dave T=C3=A4ht >> CEO, TekLibre, LLC >> http://www.teklibre.com >> Tel: 1-669-226-2619 >> _______________________________________________ >> Cake mailing list >> Cake@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cake > --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619