From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 7E70B3CB37 for ; Fri, 26 Jul 2019 11:32:27 -0400 (EDT) Received: by mail-io1-xd2f.google.com with SMTP id j5so101478937ioj.8 for ; Fri, 26 Jul 2019 08:32:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=K4YV+1PdqVdscv/Cyt1/i5ET43CY/utL+cMfeSI1TX4=; b=HMKuSDAHF9uZr/n3ZSz8Gl4ew2baCW+D06IiqkJIN0ZH53a6imdgon7uVdk0Dehr4/ Vda3WnAkfCxWl/j7STwUYmCmaHL96joobMJx6PRSAfdZNpy0EYGznxpRFS1MVg6tjA3A Gwyfmr2mhU0LBsrZ7ptMreAeAQUKOMww/lGzbJKShxGrAceL6IPDEsuDJDU3ceN2Y+Au kKtkmpkf5D65Ynrj7y97cPxOU1x2R7QTbgiLdFaBIsTk2Spe7W3gwT9gcWzVZMtduALi k7amMsSTdQVQFzvxLYWM1lJe6ThWVmB9LmJXyXF2jIJHgU+d5Z2DByjjiTSL7FwHAPl0 amcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=K4YV+1PdqVdscv/Cyt1/i5ET43CY/utL+cMfeSI1TX4=; b=AuQLCUFaeZ5/apk1yjWK0w4gx1kKq0yB/sDqwp2zhsl1KnLH7E7iMBb9rSVX8veR3p Ox0WpA6L4DHUYjwTNDVaXGTg3WFFqhA6meMTj4UtBUn/NnW4ezmvAOKkFS+78ptR6ueJ wAGLGOMmaQZnE8CsTAVlEEWnzIGw8X2u7wUmnUnLz6+nlkUcbqDSsGt51XS+xlknegtZ wHztpQFjer46+gsHBzkD8kx3POR1TIzWYRF2KPWlZ52S4fad2SxZo730o6M9wYyz/5jZ nn6rD9ip9nm+nf7C7e1+OC2072o62kkh6xl7o8mw4T22EFijPRH/yykHFIt2XrDX4WjC RtKQ== X-Gm-Message-State: APjAAAV7wzAhKdHkkO1MFD8ArJL37dD/Ei6YpHmjdw4c+LQKeQ9QcAYD zT5IA1UdHGib8mW0LiresSi2UqGwkvSlJ+gYl5s= X-Google-Smtp-Source: APXvYqykE9ai0NxuObLE9YrVM1Fuiie7Bu/iga2uA400yJxXlikb+MMt3vPXt3TyUniQBQqZUK1MQyBBrtcxQhIBs0I= X-Received: by 2002:a02:7303:: with SMTP id y3mr745616jab.97.1564155146825; Fri, 26 Jul 2019 08:32:26 -0700 (PDT) MIME-Version: 1.0 References: <364514D5-07F2-4388-A2CD-35ED1AE38405@akamai.com> <1238A446-6E05-4A55-8B3B-878C8F39FC75@gmail.com> <17B33B39-D25A-432C-9037-3A4835CCC0E1@gmail.com> <52F85CFC-B7CF-4C7A-88B8-AE0879B3CCFE@gmail.com> <87ef2myqzv.fsf@taht.net> <803D9CA8-220E-4F98-9B8E-6CE2916C3100@gmail.com> <0079BC6B-4792-48ED-90D3-D9A69407F316@gmx.de> <22af0671-fdd0-0953-fc96-55b34beb0be9@bobbriscoe.net> <3EB0D59D-69A7-4730-BCDF-10E5C61EF987@heistp.net> In-Reply-To: From: Dave Taht Date: Fri, 26 Jul 2019 08:32:15 -0700 Message-ID: To: Pete Heist Cc: "De Schepper, Koen (Nokia - BE/Antwerp)" , "ecn-sane@lists.bufferbloat.net" , "tsvwg@ietf.org" , Neal Cardwell Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Ecn-sane] The state of l4s, bbrv2, sce? X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jul 2019 15:32:27 -0000 I did miss a couple details On Fri, Jul 26, 2019 at 8:05 AM Dave Taht wrote: > > Changing the title.... > > I hope to be able to add some features and boxes to the worldwide > flent fleet to gather up some more data. Simple stuff includes trying > to verify more fully worldwide what happens when you twiddle the ecn > bits, mildly longer term look at what happens when conflicting > interpretations > of these bits are in play somewhere on the path, bit longer than that > getting an openwrt build up as a middlebox and vm, and then finally, > finally > see what happens on a couple kinds of wifi. > > There's now a flent server in mumbai, in particular, which I hope will > shed some insight as to the state of networks in india, long term, on > a variety > of fronts. But none of it's ready lacking a good release to freeze on. > > 1) BBRv2 is now available for public hacking. I had a good readthrough > last night. > > The published tree applies cleanly (with a small patch) to net-next. > I've had a chance to read through the code (lots of good changes to > bbr!). > > Although neal was careful to say in iccrg the optional ecn mode uses > "dctcp/l4s-style signalling", he did not identify how that was > actually applied > at the middleboxes, and the supplied test scripts > (gtests/net/tcp/bbr/nsperf) don't do that. All we know is that it's > set to kick in at 20 packets. Is it fq_codel's ce_threshold? red? pie? > dualpi? Does it revert to drop on overload? > > Is it running on bare metal? 260us is at the bare bottom of what linux > can schedule reliably, vms are much worse. > > Couple notes: > > BBRv2 doesn't use ect(1) as an identifier. > > The chromium release has no support for ecn at all. > > Adding back in the stuff I'd first done to rfc3168 bbrv1 looks > straightforward, making it do sce, less so. I note that at lower rates a cap of cwnd 2 instead of 4 seems seems feasibl= e. > 2) To clarify something from the l4s team, are the results you've been > presenting for years all from the 3.19 kernel? bsd? microsoft? ns2? > ns3? what? > > The code on github is not worth testing against currently? It does > have some needed features like a setsockopt for using up ect(1). Were these tests with gro/tso enabled? > should I use the issue tracker for that? I have some comments on > dualpi in addition to my outstanding question about pie's default of > drop at 10% mark > rate vs dualpi's 0. Notably it's set to 1000 packets now (fq_codel > defaults to 10,000 and we switched to memory limits both in it and > cake given a modern > packet's dynamic range of 64b to 64k). I've observed 10gige can be in > the 2-3k packets range... has dualpi been tested above 1gige yet? > > 3) The current patches for sce need to get rebased for net-next. The > sch_cake mods are easy but as the dctcp code did morph a bit since sce > work forked it as did the other tcps. I took a stab at forward porting > it to net-next, but I figure that development is hot and heavy and > some patches will land after ietf. I do not mind taking a stab again > at cleaning it up (helps me to understand what's going on), as how the > algos currently (as of, like, yesterday) work is clear to me... what > I'd like to do at least is also add 'em to the out of tree > fq_codel_fast implementation. Another issue on the tcp front in this patchset was disabling iw10 as a burst. I do strongly agree with that, pacing it, and or reverting to iw4, then pacing (as it's not been taken up by netbsd or osx either) would make this stuff gentler at lower rates. Is the ramp function as needed with iw4 in play? > > Did I miss anything about the current state of things? > > My basic testbed is a string of containers on a couple 12 core boxes > on bare metal, and more advanced is the openwrt stuff part of my wifi > lab. That's > presently almost all 4.14 based on arm, mips, and x86, running both on > real hardware and in emulation. > > On Fri, Jul 26, 2019 at 6:10 AM Pete Heist wrote: > > > > > > > On Jul 25, 2019, at 12:14 PM, De Schepper, Koen (Nokia - BE/Antwerp) = wrote: > > > > > > We have the testbed running our reference kernel version 3.19 with th= e drop patch. Let me know if you want to see the difference in behavior bet= ween the =E2=80=9Cgood=E2=80=9D DCTCP and the =E2=80=9Cdeteriorated=E2=80= =9D DCTCP in the latest kernels too. There were several issues introduced w= hich made DCTCP both more aggressive, and currently less aggressive. It cal= ls for better regression tests (for Prague at least) to make sure it=E2=80= =99s behavior is not changed too drastically by new updates. If enough peop= le are interested, we can organize a session in one of the available rooms. > > > > > > Pete, Jonathan, > > > > > > Also for testing further your tests, let me know when you are availab= le. > > > > Regarding testing, we now have a five node setup in our test environmen= t running a mixture of tcp-prague and dualq kernels to cover the scenarios = Jon outlined earlier. With what little time we=E2=80=99ve had for it this w= eek, we=E2=80=99ve only done some basic tests, and seem to be seeing behavi= or similar to what we saw at the hackathon, but we can discuss specific res= ults following IETF 105. > > > > Our intention is to coordinate a public effort to create reproducible t= est scenarios for L4S using flent. Details to follow post-conference. We do= feel it=E2=80=99s important that all of our Linux testing be on modern 5.1= + kernels, as the 3.19 series was end of life as of May 2015 (https://lwn.n= et/Articles/643934/), so we'll try to keep up to date with any patches you = might have for the newer kernels. > > > > Overall, I think we=E2=80=99ve improved the cooperation between the tea= ms this week (from zero to a little bit :), which should hopefully help mov= e both projects along... > > _______________________________________________ > > Ecn-sane mailing list > > Ecn-sane@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/ecn-sane > > > > -- > > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 --=20 Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740