From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.taht.net (mail.taht.net [IPv6:2a01:7e00::f03c:91ff:feae:7028]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 907423B29E for ; Tue, 28 Nov 2017 17:34:14 -0500 (EST) Received: from nemesis.taht.net (unknown [IPv6:2603:3024:1536:86f0:2e0:4cff:fec1:1206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.taht.net (Postfix) with ESMTPSA id 5E6BF21341; Tue, 28 Nov 2017 22:34:13 +0000 (UTC) From: Dave Taht To: Pete Heist Cc: Cake List References: <745FEC66-95A7-40E1-A8FA-57714D3AB6AC@gmail.com> <87zi76xlw5.fsf@nemesis.taht.net> <6F2894AD-87EA-4EFC-918E-625E49EDA977@gmail.com> Date: Tue, 28 Nov 2017 14:34:11 -0800 In-Reply-To: <6F2894AD-87EA-4EFC-918E-625E49EDA977@gmail.com> (Pete Heist's message of "Tue, 28 Nov 2017 21:35:06 +0100") Message-ID: <87o9nmxcbg.fsf@nemesis.taht.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] cake flenter results round 2 X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Nov 2017 22:34:14 -0000 Pete Heist writes: >> On Nov 28, 2017, at 8:07 PM, Dave Taht wrote: >>=20 >> Pete Heist writes: >>=20 >>> *** Round 3 Plans: >>>=20 >>> * Use netem to test a spread of simulated rtts and bandwidths. >>=20 >> Since you are leveraging a few too few boxes, attached are my current >> scripts for fiddling a bit with network namespaces. I added individual >> ssh, irtt, etc, servers so that things like flent's ssh stuff should >> just work for polling stats. > > You mean a few too few virtual boxes, not necessarily physical ones, righ= t? :) > In other words, there are different topologies that can be used for testi= ng. In > your scripts you=E2=80=99re simulating "Internet access", where client is= the family or > organization for example and server is stuff on the Internet: > > client - middlebox - delay - server > > In my earlier point-to-point WiFi testing I was simulating an ISP=E2=80= =99s backhaul: > > client - client_router - station ----- ap - server_router - server > > In my current testing I=E2=80=99m simulating, well, something far less us= eful if I think about it- two boxes blasting traffic to one another over a = cable and trying to improve queueing delays between them: > > client - client_router --- server_router - server Yes, in this case, the local TCP stack tends to interact better with fq_codel than cake does, due in part to the NET_XMIT_CN support there. I confess that my fav takeaway of your results thus far has been - it doesn't crash. It still doesn't crash, here, either. We could upstream it this week. :) > > I hadn=E2=80=99t realized how heavy-weight traffic generation for anythin= g beyond 4/4 flows would be at Gbit rates, or how confusing and trivial som= e of these results would be. > > So beyond just my vague idea to =E2=80=9Csimulate a spread of rtts and ba= ndwidths=E2=80=9D, I see I need a topology change to produce something more= useful. I think there are still two options: Yes, I wouldn't trust the result with netem on what you got very far. > 1) Point-to-point WiFi again, where I=E2=80=99d be using two NSM5s and te= sting over > short range at rates of up to 100 Mbps. I would probably try to get FreeN= et=E2=80=99s > APU version 1 boxes into action again so I=E2=80=99d have physical device= s for each of > the six roles above. I wish I didn=E2=80=99t have to use those RTL8111Es,= but that=E2=80=99s how > it is. > > 2) Your =E2=80=9CInternet access=E2=80=9D setup. Either I can get the vet= h stuff into action on > a single physical APU2 (powerful enough?), or I can try to set up four ph= ysical > boxes with the same topology (or if I were tricky, try to spread the four= roles > across two boxes). Getting the veth stuff setup for a basic test should be plug and go with what I gave you. And you do have 4 cores, so try a few tests at 900Mbit to see what happens. I think george has the most powerful box we have readily available, and perhaps it would be easier for him to give netns a go. I cannot trust the results we get from the cloud (too many unknown VMs sharing the hardware)... so I keep thinking, for christmas, I will finally get around to replacing snapon (which is doing lede build and server duty in sweden). I am presently getting in a good 40+ minute nap in between kernel builds, which that would also solve. (I like my naps tho). All my computers are pretty low power - only the core I5 nucs and laptops have fans which only kick in on builds - and I like a quiet work environment, so trying to build the fastest box possible without howling fans would be ideal. snapon (6 cores) cost about 2.5k when we got it (5? years ago). It looks like for about the same price we can get to at least 8 cores. Things start going pear-shaped on (for example) the AMD threadripper (16 cores) - or Xeons, at 1k for the cpu at min (but a 30 second kernel build) We could try to get a dedicated rack mount somewhere, but I don't know where, or how much. I rather enjoyed what esr pulled off with "the great beast". He had different requirements - in my case I'd like a sharable box for builds and simulations. Were these actual simulations (e.g. ns3) the virtual clock would be ok to run in the cloud, but since we're on bare metal, we'd need bare metal. Worse, to do this truly right, actually starting to fiddle with 10Gig+ hardware in a realistic topology could also be of use. Arguably those of you in Northern Europe need space heaters more than I do... PS:=20 I note the veth file I sent had two errors in it - vdaemons was meant to be vssh.sh and the netem delay component should have had a limit 100000 to it. I'm still using these simplistic shell scripts 'cause I havent found anything better to construct topologies with (want ipv6), as yet, and I should probably get around to a public repo for 'em. > > #1 would be useful for FreeNet. Would it also be useful for Cake testing = in general, or would you prefer more #2 results at this stage (i.e. simulat= ing dsl, cable, satellite, etc)? I'm really happy with this stuff right now. I haven't taken apart the tarball from the last attempt yet. 16x1, 10x1 results with ack filtering against DSL speeds and typical cable speeds seem plausible.