From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 7487E3CB36; Sun, 24 Mar 2019 18:31:12 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1553466670; bh=U35HUADE6VQCojJlmwXoMCJ7TDJFXmquNMYBjsNEdZs=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=JYU7jZWllcqvS0QbKXIiMmu+kjodaIJzpH1x0C1Heo+/BJdaSIdoNtzDW2vRz7GRi wyyJNrua6fGnO6QxEFwSj1aIXWlZsbSWipkMmHlg54cvuVNauF2WwNswGtzyQ9f6k4 Y2YFGSr8oIIzf2EeGvYKr44fj6Zfjgz2VYT4/0O4= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from hms-beagle2.lan ([77.179.191.128]) by mail.gmx.com (mrgmx001 [212.227.17.190]) with ESMTPSA (Nemesis) id 0MfW5D-1hK6h33BLn-00P6QI; Sun, 24 Mar 2019 23:31:10 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) From: Sebastian Moeller In-Reply-To: Date: Sun, 24 Mar 2019 23:31:09 +0100 Cc: Luca Muscariello , Cake List , ecn-sane@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: References: <3F3F78F9-8F97-42F2-9005-F8449489BECF@gmail.com> <42B7B2FB-94FC-4CA6-B75E-CADB61FE3173@gmail.com> <6047E899-078F-4E08-A29C-B87D268A4E5E@heistp.net> <69EA569B-4AD2-4C4B-937B-9ABDD563B120@gmail.com> To: =?utf-8?Q?Dave_T=C3=A4ht?= X-Mailer: Apple Mail (2.3445.9.1) X-Provags-ID: V03:K1:h2bG/l5cSBiv7uUg/ys/LgbxdU1ugf7pEuhFdV9qh/NTrUMlQ8j JLvlsRlYFCiudChuHMhB3J0+cwFlrwK0tRK5wfIGoy35rnGspI3XWri5q27RJ2eD9fayLcX 48jOfE9E3t4VmKnLAzvaQGpRHHmRqW4LcNJV9eHzHzoUr/Wj4Vop715QC0HwJe/KyplnZfC 2/rEKfl42z61FXZCJlCyQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:6aluRy1/Prk=:Y52YMBuDTnKamti5oiZ22r s+0moncbsBkpKpFfqAPnIfWUnkxBWyZZpNjv9BblXQY6FePlD3eADl4w06eSvNL4vzfjgw2lT cyeshnSHLyKW1lEx03iMCBEpyWfpYIqHpoS9KLu/m5eDHS9d62n3dYG/Im038OK8qimQ6KYiX YpoO/4nvf0xyvV9fubq9mbo57Cs3hzII9L1XOAWy2PIJhdikfcMyA/aR/sFHzEN7KRoT35Mut Suj3ZVf6NVqhu7KD0LqOkMjq2SEA3eYHU7VvP3wfqtqZoL4S63jFUHNvnLbxLZa434dVlZ+/n kNvnjEkd/IHYADBqtBebm2egek7vhi/okPLs9t+cY5oSg2zWPEgR/OjcB72XzH9dLz+zu04Xm /TmzHwTRX5f+K/KlTYh3voHvuxOTF3SkiQy2G2s/gQiL5mg7CFIfcy0xWsLM9wYXy6Wa0QGgt hN4Bf3BdUioyUozZNukU9TOhC0QPGC9m4rpMhDBx/m+eos5Se7lc1AdYPsf79r5kG5pnBXlHu u+XbPg2IzZ0DGdS3UCHqXcMFyLqER0ecHJk5Xtq820jiOjNsGhMtMQzIfS+JzSdvtnrrDAy4+ jqzy708WY019hVBM/krUp89e+hzTbduXYXc6Ne0YyT6EUPieIu9fI6Tz7EjcuMsbDKBoYFgdL pQvxBe8u0KEa3uI9wRdwqRybX2NSSdFVRr+hhGSxABKIQerUwHAoyMLyGPArPvhH8ea4/PLSl fc8LvnVUMQDK3mFbpBpvQvdd2jZkjsL8YuM5zgjH0LhZD33juxQQ0GVTRz4hZVrEwTYAYtNyF /2LAxJm74PCQqAOSG0Uv5Gn02giHBX6JUGDxvjeVRGvnWR4qYbAqgbnpl1h5Z01a8Nfs00qFn cNkMaHsB/1G/eNE6aqWFN7RLL1HQfqJ0SlFuX1DRdaNBnH//l6eg9kqhQNCyztiGLxC13IdXZ WCWDsAk70xA== Subject: Re: [Ecn-sane] [Cake] The two SCE tests I have in mind X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Mar 2019 22:31:12 -0000 Hi All, while I am late to the party, I could run a client side test tonight. = Nothing special, just a residential O2/Telefonica (via L2-bitstream = access) VDSL2 50/10 link in Germany. Would a capture of that be of any = help? If yes I am happy to test, otherwise I will wait for the dust to = settle. BTW, later this week it would be great to create sharable client and = server VMs, just as the LLLLS folks should start distributing, but that = certainly is for after the meeting. P.S.: how likely is that meeting to be on time (I need to figure out how = to schedule work around this)? Best Regards, or better Ahoy to the crew of the buffer boat Sebastian > On Mar 24, 2019, at 11:30, Dave Taht wrote: >=20 > On Sun, Mar 24, 2019 at 10:30 AM Luca Muscariello > wrote: >>=20 >> We have done something like that a year ago in our team in Cisco >> for a project that you Dave are aware of but that is out of scope for = these lists. >>=20 >> It is important to have vantage points in residential and enterprise = networks. >=20 > It is hard to get dedicated testing points in those networks. However > providing tools so more volunteers can participate in a larger test > has always proven worthwhile. >=20 >> Cloud to Cloud never goes to transit and the big Clouds have global = footprints. >> So you can traverse the planet inside the Cloud WAN (100% bit = consistency) and then traverse a safe peering >> point where the adjacency has been well set (or not). >=20 > cloud to cloud should be interesting. >=20 > Anyway, the first sce enabled version of cake for public testing is > now up at cake-sce-de.teklibre.net. (dns still propigating) It is rate > limited to 40Mbits and thus with 10 flows going, self congests and > thus marks with both SCE and CE. Inititial tests across our cloudy > backbone have been good, no reordering, no drops, while SCE marking > like crazy. The needed code to setup your own server is in my github > sch_cake and tc-adv repos. >=20 > We hope to roll that out over linode, google cloud, and aws over the > coming days. >=20 > A typical test right now, looks like this: >=20 > # server side install if you want to set up your own test server > # after an: >=20 > apt-get install flent netperf irtt build-essential flex bison > git clone https://github.com/dtaht/sch_cake > git clone https://github.com/dtaht/tc-adv > git clone https://github.com/dtaht/fq_codel_fast > git clone https://github.com/dtaht/fq_codel_fast > git clone https://github.com/tohojo/sqm-scripts >=20 > for i in sch_cake tc-adv fq_codel_fast sqm-scripts > do > cd $i; make; make install; cd .. > done >=20 > tc qdisc replace dev eth0 root cake sce bandwidth 40mbit >=20 > # Client side test >=20 > tcpdump -i eth0 -s 128 -w mylocationinfo.cap > flent -H cake-sce-de.teklibre.net -l 20 -t my_location_info > --te=3Ddownload_streams tcp_ndown > killall tcpdump >=20 > Go looking at the cap for sack, drops, reorders, and the sce bit being = excerted. >=20 > (flent's --socket-stats option got broken in a recent tc release which > we need to fix for that and qdisc stats) >=20 > if you are running your own server, you can now also see the > sce marking rate via: >=20 > tc -s qdisc show dev eth0 >=20 > qdisc cake 8002: root refcnt 2 bandwidth 40Mbit diffserv3 > triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw > overhead 0 sce >=20 > Sent 37542361 bytes 25522 pkt (dropped 0, overlimits 30923 requeues 0) > backlog 0b 0p requeues 0 > memory used: 124568b of 4Mb > capacity estimate: 40Mbit > min/max network layer size: 42 / 1514 > min/max overhead-adjusted size: 42 / 1514 > average network hdr offset: 14 >=20 >=20 > Bulk Best Effort Voice >=20 > thresh 2500Kbit 40Mbit 10Mbit > target 7.3ms 5.0ms 5.0ms > interval 102.3ms 100.0ms 100.0ms > pk_delay 0us 528us 347us > av_delay 0us 201us 10us > sp_delay 0us 4us 5us > backlog 0b 0b 0b > pkts 0 25209 313 > bytes 0 37500795 41566 > way_inds 0 0 0 > way_miss 0 106 5 > way_cols 0 0 0 > drops 0 0 0 > marks 0 24 0 > ack_drop 0 0 0 > sp_flows 0 1 0 > bk_flows 0 0 1 > un_flows 0 0 0 > max_len 0 3028 1752 > quantum 300 1220 305 > sce 0 23105 0 >=20 >=20 > setting up the sqm-scripts to use fq_codel_fast is a matter of setting > the right parameters in a /etc/sqm/eth0.iface.conf file for bandwidth > (40mbit) and qos model (simplest.qos) and >=20 > IQDISC_OPTS=3D"ce_threshold 2us" # these are artificially low - 1-2ms > should be closer to "right" on cloudy devices for a real, rather than > test, deployment > EQDISC_OPTS=3D"ce_threshold 2us" >=20 >=20 >> IPv4/IPv6 of course we'll see a big difference in favour of IPv6. >> What Michael has shown is his paper makes a lot of sense to me. >> Depending on the use case hit ratio can be very high or very low. >=20 > I am very behind on reading papers this week. >=20 >>=20 >>=20 >>=20 >>=20 >> On Sun, Mar 24, 2019 at 9:55 AM Dave Taht = wrote: >>>=20 >>> 1) The research into whether bit flipping to the extent that SCE = will >>> do has not been done yet. The study of ECT(0) vs ECT(1) behavior >>> transiting to CE was a little lightweight. >>>=20 >>> To test this we going to fire up a ton of nanodes in various data >>> centers, with low SCE thresholds, and low bandwidths, to flip lots = of >>> bits, and test between the data centers and from as many vantage >>> points around the net as we can get - do packet captures as well as >>> flent tests >>>=20 >>> as a control, set up identical boxes, with SCE disabled, in the same >>> data centers. >>>=20 >>> Setup flent, irtt, iperf3. >>>=20 >>> 2) Diffserv bit preservation test >>>=20 >>> The research going by on the tsvwg mailing seems a bit dated. It is >>> very straightforward to use irtt to test to see what udp codepoints >>> survive e2e, and to also leverage this testbed setup. Similarly, >>> netperf can easily be used to mark tcp. We do not have a good packet >>> cap tool to verify that the bits are being set right, however I = think >>> irtt can be modified to check for correctness here and produce a >>> report. >>>=20 >>> On Sun, Mar 24, 2019 at 1:45 AM Jonathan Morton = wrote: >>>>=20 >>>>> On 24 Mar, 2019, at 8:37 am, Pete Heist wrote: >>>>>=20 >>>>> I should theoretically arrive at the boat today some time after = 3pm, having picked up a mini HDMI to HDMI adapter, which we can use with = the cable that=E2=80=99s there=E2=80=A6 >>>>=20 >>>> Awesome. I'm also setting up a Linux VM on my Mac, which should = help things along. >>>>=20 >>>> We're bringing up some actual hardware with the SCE-enabled Cake on = it now. Dave wants to investigate various theoretical phenomena the = Internet might exhibit with a mixture of ECN codepoints; I just want to = be sure it actually works as intended, before I move on to fiddling with = TCP. >>>>=20 >>>> - Jonathan Morton >>>>=20 >>>> _______________________________________________ >>>> Cake mailing list >>>> Cake@lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/cake >>>=20 >>>=20 >>>=20 >>> -- >>>=20 >>> Dave T=C3=A4ht >>> CTO, TekLibre, LLC >>> http://www.teklibre.com >>> Tel: 1-831-205-9740 >>> _______________________________________________ >>> Cake mailing list >>> Cake@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/cake >=20 >=20 >=20 > --=20 >=20 > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 > _______________________________________________ > Cake mailing list > Cake@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/cake