From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-x234.google.com (mail-qk0-x234.google.com [IPv6:2607:f8b0:400d:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id D01C83B2A4 for ; Mon, 27 Nov 2017 12:33:13 -0500 (EST) Received: by mail-qk0-x234.google.com with SMTP id o6so33424146qkh.3 for ; Mon, 27 Nov 2017 09:33:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=93F6a9tSuaGCYmqeS6kvbqRTUE2VkrxYwL3HWiNXi7I=; b=rgLI/v9Iz7PANPp4ZidD968wIYnYxihFNn42IZuBnuGbe2OhPJp+L65nvysfhrN6iv cS/OTPpwZHxGLVGzTYIT1EV7DC33yJTqLl4CTl4N9VMq7aWxQFBr7L8oCptN+6SpGY7e QONPmz4MGS1bUwnpKG/ee2cl7ywgWrD8XPHKoRJHqf97izgQ9OXf7M9TCLzRcthk+1IR bsxMfWyiH6AnH+L2PCfS819A45ckvATUHEg3PXy6Y1aAhkvUVfn1Yug/y9uq1mAB5poy +mLDjszgdJ/3tY22ySjQA5jc20EVFgHmRiOQLOANhx36avkjrdPa2k6Y4q+po9igG9d6 P5qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=93F6a9tSuaGCYmqeS6kvbqRTUE2VkrxYwL3HWiNXi7I=; b=IPQn6b8/Wti9LDKV4LpuB+n24EQA9U1CpiliuMxmkY8GWy/Tx1Cv874ieVT1tcxzMr wK55OlPgsJupltJSjc/r0/Zv7LwfTSO4xI64T0AtHCftCF9JA2J8A151ul8fNF1tklIZ F/kx3qKSTbiG4QdeMrK0G+1Ythuiaviz1Q/4gnxNbGNfTtbVd3LgSRc8CDDoGE6OmKwo h1TLgC/lGWCMbMK3ElRzx/BvfEEJ26v+Yi30+J9n7PfzuOd0SeYPm8zTbt4Gtjc+6NnI RZ32vhIP6TR4UQe+zpsdzmreyx7F7DRM2z5LqEB+jo6gnuOojLuDASjPeLzAqjilU3dY 7mXw== X-Gm-Message-State: AJaThX4AwTa/HbdLm1v7H079QB9f3p2Y71KFXwRO5w1FAivhB0DRpVY1 EuoyoGKqYl7uQJpwYPHBDuDmQSnrCBJgl/K5R8c= X-Google-Smtp-Source: AGs4zMZfFGhmPyRjGro6uePFhnro3KYGdyiIfRVl3JnpDTu4/G8dbhLiLS+iv8AtjwmPmGrDja+wWR2zvFOZEz0lqr4= X-Received: by 10.55.188.6 with SMTP id m6mr22125312qkf.75.1511803993320; Mon, 27 Nov 2017 09:33:13 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.193.93 with HTTP; Mon, 27 Nov 2017 09:33:12 -0800 (PST) In-Reply-To: References: <85E1A7B2-8AA7-418A-BE43-209A1EC8881A@gmail.com> <87d1447z9w.fsf@toke.dk> <27F95EB1-490B-404C-8F77-98646B6159E7@gmail.com> <1C937A63-CEC1-4173-8812-EA2A85972B73@gmail.com> <20D304DC-494E-4A00-9B39-1E9F4B0F0CB6@gmail.com> <67B1612D-895D-4E3A-8CBD-21580B470696@gmail.com> From: Dave Taht Date: Mon, 27 Nov 2017 09:33:12 -0800 Message-ID: To: Georgios Amanakis Cc: Pete Heist , Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] cake flenter results round 1 X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Nov 2017 17:33:13 -0000 georgios the result you got was "fair", but you shoul have seen something closer to 900mbit than 400. On Mon, Nov 27, 2017 at 8:17 AM, Georgios Amanakis wr= ote: > Dear Pete, > > I am trying to replicate the unfair behaviour you are seeing with > dual-{src,dst}host, albeit on different hardware and I am getting a fair > distribution. Hardware are Xeon E3-1220Lv2 (router), i3-3110M(Clients). A= ll > running Archlinux, latest cake and patched iproute2-4.14.1, connected wit= h > Gbit ethernet, TSO/GSO/GRO enabled. > > Qdisc setup: > ---------------- > Router: > qdisc cake 8003: dev ens4 root refcnt 2 bandwidth 900Mbit diffserv3 > dual-dsthost rtt 100.0ms raw > > Client A(kernel default): > qdisc fq_codel 0: dev eno2 root refcnt 2 limit 10240p flows 1024 quantum > 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn > > Client B (kernel default): > qdisc fq_codel 0: dev enp1s0 root refcnt 2 limit 10240p flows 1024 quantu= m > 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn > ---------------- > > > Cli: > ---------------- > Router: > netserver & > > Client A: > flent tcp_1down -H router > > Client B: > flent tcp_12down -H router > ---------------- > > > Results: > ---------------- > Router: > qdisc cake 8003: root refcnt 2 bandwidth 900Mbit diffserv3 dual-dsthost r= tt > 100.0ms raw > Sent 7126680117 bytes 4725904 pkt (dropped 10, overlimits 4439745 requeu= es > 0) > backlog 0b 0p requeues 0 > memory used: 1224872b of 15140Kb > capacity estimate: 900Mbit > Bulk Best Effort Voice > thresh 56250Kbit 900Mbit 225Mbit > target 5.0ms 5.0ms 5.0ms > interval 100.0ms 100.0ms 100.0ms > pk_delay 14us 751us 7us > av_delay 2us 642us 1us > sp_delay 1us 1us 1us > pkts 109948 4601651 14315 > bytes 160183242 6964893773 1618242 > way_inds 0 21009 0 > way_miss 160 188 5 > way_cols 0 0 0 > drops 0 10 0 > marks 0 0 0 > ack_drop 0 0 0 > sp_flows 0 1 1 > bk_flows 1 0 0 > un_flows 0 0 0 > max_len 7570 68130 1022 > > > Client A: > avg median # data pts > Ping (ms) ICMP : 0.11 0.08 ms 350 > TCP download : 443.65 430.38 Mbits/s 301 > > > Client B: > avg median # data pts > Ping (ms) ICMP : 0.09 0.06 ms 350 > TCP download avg : 37.03 35.87 Mbits/s 301 > TCP download sum : 444.35 430.40 Mbits/s 301 > TCP download::1 : 37.00 35.87 Mbits/s 301 > TCP download::10 : 37.01 35.87 Mbits/s 301 > TCP download::11 : 37.02 35.87 Mbits/s 301 > TCP download::12 : 37.00 35.87 Mbits/s 301 > TCP download::2 : 37.03 35.87 Mbits/s 301 > TCP download::3 : 36.99 35.87 Mbits/s 301 > TCP download::4 : 37.03 35.87 Mbits/s 301 > TCP download::5 : 37.07 35.87 Mbits/s 301 > TCP download::6 : 37.00 35.87 Mbits/s 301 > TCP download::7 : 37.12 35.87 Mbits/s 301 > TCP download::8 : 37.05 35.87 Mbits/s 301 > TCP download::9 : 37.03 35.87 Mbits/s 301 > ---------------- > > Does this suggest that it is indeed a problem of an underpowered CPU in y= our > case? > > George > > > On Mon, Nov 27, 2017 at 10:53 AM, Pete Heist wrote: >> >> >> On Nov 27, 2017, at 3:48 PM, Jonathan Morton >> wrote: >> >> It's not at all obvious how we'd detect that. Packets are staying in th= e >> queue for less time than the codel target, which is exactly what you'd g= et >> if you weren't saturated at all. >> >> That makes complete sense when you put it that way. Cake has no way of >> knowing why the input rate is lower than expected, even if it=E2=80=99s = part of the >> cause. >> >> I don=E2=80=99t think flent can know this either. It can=E2=80=99t easil= y know the cause >> for its total output to be lower than expected. >> >> All I know is, this is a common problem in deployments, particularly on >> low-end hardware like ER-Xs, that can be tricky for users to figure out. >> >> I don=E2=80=99t even think monitoring CPU in general would work. The CPU= could be >> high because it=E2=80=99s doing other calculations, but there=E2=80=99s = still enough for >> cake at a low rate, and there=E2=80=99s no need to warn in that case. I= =E2=80=99d be >> interested in any ideas on how to know this is happening in the system a= s a >> whole. So far, there are just various clues that one needs to piece toge= ther >> (no or few drops or marks, less total throughput that expected, high cpu >> without other external usage, etc). Then it needs to be proven with a te= st. >> >> Anyway thanks, your clue was what I needed! I need to remember to review >> the qdisc stats when something unexpected happens. >> >> _______________________________________________ >> Cake mailing list >> Cake@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cake >> > > > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake > --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619