From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 9415A3CB36; Sun, 24 Mar 2019 08:33:45 -0400 (EDT) Received: by mail-qt1-x844.google.com with SMTP id k2so7424758qtm.1; Sun, 24 Mar 2019 05:33:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=YYQ6Tw1BzyNZui9CTw2bDsX1YWijvz7vcg5h9cQ7Zo0=; b=RG9wxBfaS6vXpLujUhkWudLh5Vn2Jr4MSpMKcrycjCFPKuuRyEbmt6qF5d5Edz3br+ gGU1qUj0l8SJBPjebcNYVS1z7ZcZu5zykAzbE5CIDms63a/9rMu4bDq8VhO6O9iFtBBU w+7yr1GCjDXdGEjc5a1V9VPXnsZH03KNw0M7IDypGTr5KOeNqIa5Brj1gJLCq7O7ZQzU XdMzDW4toMXep0s2fuiucgzzuxo7R9TvynIbcNozqUUugDe4goPyqzwjNHPiUsT9NAyV HWveMw8gtKfehEUPESo1hCM0gjSQ4BXmpNhf3REgbVE6QcuLHb3/eDahNbAuWRDWhio2 E8sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=YYQ6Tw1BzyNZui9CTw2bDsX1YWijvz7vcg5h9cQ7Zo0=; b=XiU6mRsQnYUEHaD6hdQqEYp7hL5ahJWX150klu9CaeUZMI0R757Bv/1D0uV4FsA9hZ D52kpJH6vkFUD65IysvyiMxU0VBAfH/LmwLg4GriyoLsWRJ3bK+ZwRQ2wt7hkRHWxDfX VfuurVP209vzYg1U9gJnogW62FHbUakSCMCf6FLUl7MlOe0LC+b3wPOPtFUcM8GsSk+k Qd+v/RSp5voNNKU/pmkLBhpuygsMRrdvxK//7mzi5TXD421i8VYtZKttCnx1PpBrUFCk m0FNOyWO2zddp5Fng90GFp13dgreL+VCfkV79AuJEg0Vzyh0chqzC247x/ZlHUjtmgzF emRg== X-Gm-Message-State: APjAAAVhPqbtlNKe8CrPQvTetOOqjURbaT9AIKQ4s9jLgYbIPBW8ci6V TAj2CpagrZLR2zQC5YQ/Lc7sCt1fw9CQzOBdPDw= X-Google-Smtp-Source: APXvYqylMjzdqoL1DpyBMb0XKHSWAxqYOIlAJL5hPUMdiD7hGhXYdVX4iZwGg7pfi0QoCxDuzpKQR/AKz/uhQ+1fkzU= X-Received: by 2002:ac8:30e8:: with SMTP id w37mr16524320qta.136.1553430825063; Sun, 24 Mar 2019 05:33:45 -0700 (PDT) MIME-Version: 1.0 References: <3F3F78F9-8F97-42F2-9005-F8449489BECF@gmail.com> <42B7B2FB-94FC-4CA6-B75E-CADB61FE3173@gmail.com> <6047E899-078F-4E08-A29C-B87D268A4E5E@heistp.net> <69EA569B-4AD2-4C4B-937B-9ABDD563B120@gmail.com> In-Reply-To: From: Dave Taht Date: Sun, 24 Mar 2019 05:33:32 -0700 Message-ID: To: Luca Muscariello Cc: Jonathan Morton , ecn-sane@lists.bufferbloat.net, Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] The two SCE tests I have in mind X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Mar 2019 12:33:45 -0000 I made two goofs in this post. --te=3Ddownload_streams=3D10 and the server and client need to be negiotiating ecn: https://www.bufferbloat.net/projects/cerowrt/wiki/Enable_ECN/ So far as I know the OSX section is out of date, osx always negotiates ecn now. I do not have data on ios's current configuration of ecn enablement. On Sun, Mar 24, 2019 at 3:30 AM Dave Taht wrote: > > On Sun, Mar 24, 2019 at 10:30 AM Luca Muscariello > wrote: > > > > We have done something like that a year ago in our team in Cisco > > for a project that you Dave are aware of but that is out of scope for t= hese lists. > > > > It is important to have vantage points in residential and enterprise ne= tworks. > > It is hard to get dedicated testing points in those networks. However > providing tools so more volunteers can participate in a larger test > has always proven worthwhile. > > > Cloud to Cloud never goes to transit and the big Clouds have global foo= tprints. > > So you can traverse the planet inside the Cloud WAN (100% bit consisten= cy) and then traverse a safe peering > > point where the adjacency has been well set (or not). > > cloud to cloud should be interesting. > > Anyway, the first sce enabled version of cake for public testing is > now up at cake-sce-de.teklibre.net. (dns still propigating) It is rate > limited to 40Mbits and thus with 10 flows going, self congests and > thus marks with both SCE and CE. Inititial tests across our cloudy > backbone have been good, no reordering, no drops, while SCE marking > like crazy. The needed code to setup your own server is in my github > sch_cake and tc-adv repos. > > We hope to roll that out over linode, google cloud, and aws over the > coming days. > > A typical test right now, looks like this: > > # server side install if you want to set up your own test server > # after an: > > apt-get install flent netperf irtt build-essential flex bison > git clone https://github.com/dtaht/sch_cake > git clone https://github.com/dtaht/tc-adv > git clone https://github.com/dtaht/fq_codel_fast > git clone https://github.com/dtaht/fq_codel_fast > git clone https://github.com/tohojo/sqm-scripts > > for i in sch_cake tc-adv fq_codel_fast sqm-scripts > do > cd $i; make; make install; cd .. > done > > tc qdisc replace dev eth0 root cake sce bandwidth 40mbit > > # Client side test > > tcpdump -i eth0 -s 128 -w mylocationinfo.cap > flent -H cake-sce-de.teklibre.net -l 20 -t my_location_info > --te=3Ddownload_streams tcp_ndown > killall tcpdump > > Go looking at the cap for sack, drops, reorders, and the sce bit being ex= certed. > > (flent's --socket-stats option got broken in a recent tc release which > we need to fix for that and qdisc stats) > > if you are running your own server, you can now also see the > sce marking rate via: > > tc -s qdisc show dev eth0 > > qdisc cake 8002: root refcnt 2 bandwidth 40Mbit diffserv3 > triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw > overhead 0 sce > > Sent 37542361 bytes 25522 pkt (dropped 0, overlimits 30923 requeues 0) > backlog 0b 0p requeues 0 > memory used: 124568b of 4Mb > capacity estimate: 40Mbit > min/max network layer size: 42 / 1514 > min/max overhead-adjusted size: 42 / 1514 > average network hdr offset: 14 > > > Bulk Best Effort Voice > > thresh 2500Kbit 40Mbit 10Mbit > target 7.3ms 5.0ms 5.0ms > interval 102.3ms 100.0ms 100.0ms > pk_delay 0us 528us 347us > av_delay 0us 201us 10us > sp_delay 0us 4us 5us > backlog 0b 0b 0b > pkts 0 25209 313 > bytes 0 37500795 41566 > way_inds 0 0 0 > way_miss 0 106 5 > way_cols 0 0 0 > drops 0 0 0 > marks 0 24 0 > ack_drop 0 0 0 > sp_flows 0 1 0 > bk_flows 0 0 1 > un_flows 0 0 0 > max_len 0 3028 1752 > quantum 300 1220 305 > sce 0 23105 0 > > > setting up the sqm-scripts to use fq_codel_fast is a matter of setting > the right parameters in a /etc/sqm/eth0.iface.conf file for bandwidth > (40mbit) and qos model (simplest.qos) and > > IQDISC_OPTS=3D"ce_threshold 2us" # these are artificially low - 1-2ms > should be closer to "right" on cloudy devices for a real, rather than > test, deployment > EQDISC_OPTS=3D"ce_threshold 2us" > > > > IPv4/IPv6 of course we'll see a big difference in favour of IPv6. > > What Michael has shown is his paper makes a lot of sense to me. > > Depending on the use case hit ratio can be very high or very low. > > I am very behind on reading papers this week. > > > > > > > > > > > On Sun, Mar 24, 2019 at 9:55 AM Dave Taht wrote: > >> > >> 1) The research into whether bit flipping to the extent that SCE will > >> do has not been done yet. The study of ECT(0) vs ECT(1) behavior > >> transiting to CE was a little lightweight. > >> > >> To test this we going to fire up a ton of nanodes in various data > >> centers, with low SCE thresholds, and low bandwidths, to flip lots of > >> bits, and test between the data centers and from as many vantage > >> points around the net as we can get - do packet captures as well as > >> flent tests > >> > >> as a control, set up identical boxes, with SCE disabled, in the same > >> data centers. > >> > >> Setup flent, irtt, iperf3. > >> > >> 2) Diffserv bit preservation test > >> > >> The research going by on the tsvwg mailing seems a bit dated. It is > >> very straightforward to use irtt to test to see what udp codepoints > >> survive e2e, and to also leverage this testbed setup. Similarly, > >> netperf can easily be used to mark tcp. We do not have a good packet > >> cap tool to verify that the bits are being set right, however I think > >> irtt can be modified to check for correctness here and produce a > >> report. > >> > >> On Sun, Mar 24, 2019 at 1:45 AM Jonathan Morton wrote: > >> > > >> > > On 24 Mar, 2019, at 8:37 am, Pete Heist wrote: > >> > > > >> > > I should theoretically arrive at the boat today some time after 3p= m, having picked up a mini HDMI to HDMI adapter, which we can use with the = cable that=E2=80=99s there=E2=80=A6 > >> > > >> > Awesome. I'm also setting up a Linux VM on my Mac, which should hel= p things along. > >> > > >> > We're bringing up some actual hardware with the SCE-enabled Cake on = it now. Dave wants to investigate various theoretical phenomena the Intern= et might exhibit with a mixture of ECN codepoints; I just want to be sure i= t actually works as intended, before I move on to fiddling with TCP. > >> > > >> > - Jonathan Morton > >> > > >> > _______________________________________________ > >> > Cake mailing list > >> > Cake@lists.bufferbloat.net > >> > https://lists.bufferbloat.net/listinfo/cake > >> > >> > >> > >> -- > >> > >> Dave T=C3=A4ht > >> CTO, TekLibre, LLC > >> http://www.teklibre.com > >> Tel: 1-831-205-9740 > >> _______________________________________________ > >> Cake mailing list > >> Cake@lists.bufferbloat.net > >> https://lists.bufferbloat.net/listinfo/cake > > > > -- > > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 --=20 Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740