From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AF0AA3CB37; Sun, 24 Mar 2019 06:30:27 -0400 (EDT) Received: by mail-qk1-x72d.google.com with SMTP id c1so3712364qkk.4; Sun, 24 Mar 2019 03:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=u+8HkzSKBh3FgTWsTJqxIN2CaygGnKoB3al2fAqsysQ=; b=llKSOL2ZsdXJ7kp+6d2B3h4Zx3qj5UF7g2vdV1v8YDin/jIqj6qgPIelhZhJQrH+es 8BhjEnlP9uZ6UMq/rHocpADyZMS0pviZfT1w6xdDiPg9B5pVJoCxyZ8CGamph9218RHx l2OcgU0l2XVRDlA/v7az3ghgjPApG0ZdGQciXM7Om9+ZnpIiBWAeVxscKVJ1FFogN9tN 3DeiWeV8y2xmM8roq8uauTJE86F/yeEqFrU10w9ErBYwoRTslI5wuwG3sGOQ4nBKkcV/ whWsTdVyLjlWP/P88uwhgAPIbW2DhW/bdA+5yj5TjEkAByipReUPlZSTOl91HSEAmsxe qKpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=u+8HkzSKBh3FgTWsTJqxIN2CaygGnKoB3al2fAqsysQ=; b=GKtsvNVGoAPIgXMQy+F+er3elrjLYrQ46K3ev+46VNfnjEoinLNO9nBWulDjeJI1nI IFUKMONSPfR7SC9ab9ytPtpKOeCfEHShimiRy45lOV3zBCVKFbOTe0CtBIaBi0JV/8DX 2osKBctmrxFLsioWnRBdUA9KzN8uKeTUwfmDsipf7XS4crZoZAv7LEBc382RHHwG2s9n YIBOUGuSQDYVpK8Gr+6jOQK+QAkweMtR0MJMNQtvwEYWdehd15wlhsszVkyybJN6ZpYS 0DeYcMB0oDYNMLlB2Z/gzhaQ8jVy5wBic12zj/optUGCWrTYN5HXCue2yV1KU+vR/q8E vjpA== X-Gm-Message-State: APjAAAWPUs4UwV9KkJ9wIGO8GmJYMITW39OeeNKassbqVJv0qQedrRLI MMvP3iNtjCQfnasulPaXTPluvCxu1hqSumnoFkM= X-Google-Smtp-Source: APXvYqxnpJ99L/GAxB8oaouiQanSuGdbOWbHOix9F7gPkiJ7wXin++f0X7rYeEIIlKl++oypzOjlSR8GKRbAsqHq1aA= X-Received: by 2002:a05:620a:1424:: with SMTP id k4mr14602797qkj.17.1553423427092; Sun, 24 Mar 2019 03:30:27 -0700 (PDT) MIME-Version: 1.0 References: <3F3F78F9-8F97-42F2-9005-F8449489BECF@gmail.com> <42B7B2FB-94FC-4CA6-B75E-CADB61FE3173@gmail.com> <6047E899-078F-4E08-A29C-B87D268A4E5E@heistp.net> <69EA569B-4AD2-4C4B-937B-9ABDD563B120@gmail.com> In-Reply-To: From: Dave Taht Date: Sun, 24 Mar 2019 11:30:15 +0100 Message-ID: To: Luca Muscariello Cc: Jonathan Morton , ecn-sane@lists.bufferbloat.net, Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] The two SCE tests I have in mind X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Mar 2019 10:30:27 -0000 On Sun, Mar 24, 2019 at 10:30 AM Luca Muscariello wrote: > > We have done something like that a year ago in our team in Cisco > for a project that you Dave are aware of but that is out of scope for the= se lists. > > It is important to have vantage points in residential and enterprise netw= orks. It is hard to get dedicated testing points in those networks. However providing tools so more volunteers can participate in a larger test has always proven worthwhile. > Cloud to Cloud never goes to transit and the big Clouds have global footp= rints. > So you can traverse the planet inside the Cloud WAN (100% bit consistency= ) and then traverse a safe peering > point where the adjacency has been well set (or not). cloud to cloud should be interesting. Anyway, the first sce enabled version of cake for public testing is now up at cake-sce-de.teklibre.net. (dns still propigating) It is rate limited to 40Mbits and thus with 10 flows going, self congests and thus marks with both SCE and CE. Inititial tests across our cloudy backbone have been good, no reordering, no drops, while SCE marking like crazy. The needed code to setup your own server is in my github sch_cake and tc-adv repos. We hope to roll that out over linode, google cloud, and aws over the coming days. A typical test right now, looks like this: # server side install if you want to set up your own test server # after an: apt-get install flent netperf irtt build-essential flex bison git clone https://github.com/dtaht/sch_cake git clone https://github.com/dtaht/tc-adv git clone https://github.com/dtaht/fq_codel_fast git clone https://github.com/dtaht/fq_codel_fast git clone https://github.com/tohojo/sqm-scripts for i in sch_cake tc-adv fq_codel_fast sqm-scripts do cd $i; make; make install; cd .. done tc qdisc replace dev eth0 root cake sce bandwidth 40mbit # Client side test tcpdump -i eth0 -s 128 -w mylocationinfo.cap flent -H cake-sce-de.teklibre.net -l 20 -t my_location_info --te=3Ddownload_streams tcp_ndown killall tcpdump Go looking at the cap for sack, drops, reorders, and the sce bit being exce= rted. (flent's --socket-stats option got broken in a recent tc release which we need to fix for that and qdisc stats) if you are running your own server, you can now also see the sce marking rate via: tc -s qdisc show dev eth0 qdisc cake 8002: root refcnt 2 bandwidth 40Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw overhead 0 sce Sent 37542361 bytes 25522 pkt (dropped 0, overlimits 30923 requeues 0) backlog 0b 0p requeues 0 memory used: 124568b of 4Mb capacity estimate: 40Mbit min/max network layer size: 42 / 1514 min/max overhead-adjusted size: 42 / 1514 average network hdr offset: 14 Bulk Best Effort Voice thresh 2500Kbit 40Mbit 10Mbit target 7.3ms 5.0ms 5.0ms interval 102.3ms 100.0ms 100.0ms pk_delay 0us 528us 347us av_delay 0us 201us 10us sp_delay 0us 4us 5us backlog 0b 0b 0b pkts 0 25209 313 bytes 0 37500795 41566 way_inds 0 0 0 way_miss 0 106 5 way_cols 0 0 0 drops 0 0 0 marks 0 24 0 ack_drop 0 0 0 sp_flows 0 1 0 bk_flows 0 0 1 un_flows 0 0 0 max_len 0 3028 1752 quantum 300 1220 305 sce 0 23105 0 setting up the sqm-scripts to use fq_codel_fast is a matter of setting the right parameters in a /etc/sqm/eth0.iface.conf file for bandwidth (40mbit) and qos model (simplest.qos) and IQDISC_OPTS=3D"ce_threshold 2us" # these are artificially low - 1-2ms should be closer to "right" on cloudy devices for a real, rather than test, deployment EQDISC_OPTS=3D"ce_threshold 2us" > IPv4/IPv6 of course we'll see a big difference in favour of IPv6. > What Michael has shown is his paper makes a lot of sense to me. > Depending on the use case hit ratio can be very high or very low. I am very behind on reading papers this week. > > > > > On Sun, Mar 24, 2019 at 9:55 AM Dave Taht wrote: >> >> 1) The research into whether bit flipping to the extent that SCE will >> do has not been done yet. The study of ECT(0) vs ECT(1) behavior >> transiting to CE was a little lightweight. >> >> To test this we going to fire up a ton of nanodes in various data >> centers, with low SCE thresholds, and low bandwidths, to flip lots of >> bits, and test between the data centers and from as many vantage >> points around the net as we can get - do packet captures as well as >> flent tests >> >> as a control, set up identical boxes, with SCE disabled, in the same >> data centers. >> >> Setup flent, irtt, iperf3. >> >> 2) Diffserv bit preservation test >> >> The research going by on the tsvwg mailing seems a bit dated. It is >> very straightforward to use irtt to test to see what udp codepoints >> survive e2e, and to also leverage this testbed setup. Similarly, >> netperf can easily be used to mark tcp. We do not have a good packet >> cap tool to verify that the bits are being set right, however I think >> irtt can be modified to check for correctness here and produce a >> report. >> >> On Sun, Mar 24, 2019 at 1:45 AM Jonathan Morton = wrote: >> > >> > > On 24 Mar, 2019, at 8:37 am, Pete Heist wrote: >> > > >> > > I should theoretically arrive at the boat today some time after 3pm,= having picked up a mini HDMI to HDMI adapter, which we can use with the ca= ble that=E2=80=99s there=E2=80=A6 >> > >> > Awesome. I'm also setting up a Linux VM on my Mac, which should help = things along. >> > >> > We're bringing up some actual hardware with the SCE-enabled Cake on it= now. Dave wants to investigate various theoretical phenomena the Internet= might exhibit with a mixture of ECN codepoints; I just want to be sure it = actually works as intended, before I move on to fiddling with TCP. >> > >> > - Jonathan Morton >> > >> > _______________________________________________ >> > Cake mailing list >> > Cake@lists.bufferbloat.net >> > https://lists.bufferbloat.net/listinfo/cake >> >> >> >> -- >> >> Dave T=C3=A4ht >> CTO, TekLibre, LLC >> http://www.teklibre.com >> Tel: 1-831-205-9740 >> _______________________________________________ >> Cake mailing list >> Cake@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cake --=20 Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740