From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 3183E3CB3C for ; Mon, 12 Nov 2018 12:00:19 -0500 (EST) Received: by mail-qk1-x731.google.com with SMTP id 189so14479073qkj.8 for ; Mon, 12 Nov 2018 09:00:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=WI+MfNq6lsNrlMFOwj/dflqIEdLv9eH0BTk0JPkbNKU=; b=u6mnstLfm0uvXyyT3tqFlW4iln6WqD8k5qktHxY2e99ilkC+QdWUY2Rux7txL7ToeM V1U8sRkU3LX+2Z3mpvs3FOG5sCWiUaBIvIKqUhIeo5+XPUwGMW9p8Mjmz5jom+qFCf6D NXbZwgdcWfT+eNB4f6SOXgqc8IIx+n4oTL74iCa+C78GPuayg3lH8SAHQDtWsUH8qYxN PYZfQjwqXYY1k2wk0VY2TbwwPvTkXt2bwFdyX2UScgTGagQAJPH26IOV7CsYDxssQv7k SiMlLl93UbPwc8D4dq24nNW8wLvJEOLe670eaTbv/gYSzLnNY7sPXe6j9c3EKt1ZaQT6 q0xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=WI+MfNq6lsNrlMFOwj/dflqIEdLv9eH0BTk0JPkbNKU=; b=YNlMBvIDE4NYi9k5TjRGUvzkHuaGRIgmvgS6LuWXB3coy9q3U/heQqOtDXstDiY2Gl 1byHqLS1F94LLr4/MLtCoZw3aoesaUj41ucwPXbqZB2cLt/dXCVdak0R4yERtL7QH5du /4umYLzPOKVYYQBKV4ngw1WNpguLSf4r1TwvQDXzMSLU/VAmjhtwZnrS1+xuuuALBokp L1G8ljCV/eplm+BJtZQCLeqmonyMbXUFTSP6Cy4GSWviDqOpq5GjpKZrcEawxWdtgzNZ HzxMERH1l2KWDxuHKFjjJEIH/014Mw9geGby2jpujFylfo9ORzRHejiBQNeoXp+8T5K1 yDnQ== X-Gm-Message-State: AGRZ1gLw6CdZXkHJA/jFfSgd56pi1rn4dNiK49N2667HUxNvaPwaleKn 9gNYEoniXZNNuzywYnrtdtz+4m0eb2AZ8fqoa/s= X-Google-Smtp-Source: AJdET5f/AgeouEu94sjyOPZpQX5WHM8F0to/+p3bhYfDZ2zbolpgyJMlqSyI4IlvstyDAiaTHbETtI2gvd2gShjjKMg= X-Received: by 2002:a37:9ce:: with SMTP id 197mr1594227qkj.164.1542042017597; Mon, 12 Nov 2018 09:00:17 -0800 (PST) MIME-Version: 1.0 References: <87va529v0l.fsf@taht.net> In-Reply-To: From: Dave Taht Date: Mon, 12 Nov 2018 09:00:05 -0800 Message-ID: To: Mikael Abrahamsson Cc: =?UTF-8?Q?Dave_T=C3=A4ht?= , bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] excellent result with OpenWrt 18.06.1 + CAKE on FTTH X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Nov 2018 17:00:19 -0000 On Mon, Nov 12, 2018 at 8:21 AM Mikael Abrahamsson wrote= : > > On Mon, 12 Nov 2018, Dave Taht wrote: > > > Don't use this connection much, do you? :) > > Last 4 week average is 300 kilobit/s up and 3000 kilobit/s down. So no. > Mostly streaming Netflix and similar things. > > >> marks 512 > > > > and that you have at least one device with ecn enabled. Would this be > > OSX or IOS perhaps? > > I typically turn it on on all devices I remember to turn it on. There are > plenty of iOS devices in the household, but also ECN enabled OSX machines= . > > > I don't suppose you have someone else "across town" you could run some > > benchmarks against? > > Surely. I can run anything you need, I have 1GE ubuntu machine > ~3ms away. What tests do you want me to run? I have ubuntu laptop here I > can run wired tests with. It already has flent installed, so just tell me > what you want me to do and test. If you want me to change qdisc settings > I'm going to need good instructions, I am not proficient in changing thos= e > settings. cake on or off would be a good start, then fq_codel with simple.qos. irtt stats are more accurate at this high resolution, irtt is now packaged and started automagically for ubuntu, and picked up automatically by flent if available. do check to see if it's on both boxes and that the relevant port is not firewalled. #/bin/sh F=3D"flent -x -s .02" # run flent in high resolution - even 20ms samples isn't good enough anymore! S=3Dyour_server T=3D"a meaningful title" $F -H $S -t "$T" rrul $F -H $S -t "$T" rrul_be $F -H $S -t "$T-100" -l 20 --socket-stats --te=3Dupload_streams=3D100 tcp_n= up $F -H $S -t "$T-500" -l 20 --socket-stats --te=3Dupload_streams=3D500 tcp_n= up $F -H $S -t "$T" --te=3Ddownload_streams=3D100 tcp_ndown Of these the most interesting one in cake's case is actually the tcn_nup test, vs fq_codel, vs baseline I note that -s .02 and socket stats will EAT GB of ram to process and take minutes to finalize. If you feel really ambitious, ecn/no behavior on your tcps on tcp_nup is rather interesting at this load. > > > Similarly, a cpu number under load. I note here, that splitting GSO > > has a big cost, (primarily in routing table lookup) and you can at > > these speeds, probably disable it. > > sirq% peaks out around 35-40% when doing download at 250 megabit/s. Aroun= d > 10% when doing upload at 100 megabit/s. Armada 385 is nice. Yep. I've universaly switched to that on "the low end" when I can't get an APU2. If only the wifi was better. > > I was also expecting 64k here. I imagine you are using modern linuxes > > that don't overuse TSO anymore, and osx and windows never got into it t= o > > the extreme that linux did. > > root@wrt1200-hemma:~# uname -a > Linux wrt1200-hemma 4.14.63 #0 SMP Wed Aug 15 20:42:39 2018 armv7l GNU/Li= nux > > > > -- > Mikael Abrahamsson email: swmike@swm.pp.se --=20 Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740