Re: [Cake] The two SCE tests I have in mind

[Dave Taht <dave.taht@gmail.com>]

On Sun, Mar 24, 2019 at 10:30 AM Luca Muscariello
<luca.muscariello@gmail.com> wrote:
>
> We have done something like that a year ago in our team in Cisco
> for a project that you Dave are aware of but that is out of scope for these lists.
>
> It is important to have vantage points in residential and enterprise networks.

It is hard to get dedicated testing points in those networks. However
providing tools so more volunteers can participate in a larger test
has always proven worthwhile.

> Cloud to Cloud never goes to transit and the big Clouds have global footprints.
> So you can traverse the planet inside the Cloud WAN (100% bit consistency) and then traverse a safe peering
> point where the adjacency  has been well set (or not).

cloud to cloud should be interesting.

Anyway, the first sce enabled version of cake for public testing is
now up at cake-sce-de.teklibre.net. (dns still propigating) It is rate
limited to 40Mbits and thus with 10 flows going, self congests and
thus marks with both SCE and CE. Inititial tests across our cloudy
backbone have been good, no reordering, no drops, while SCE marking
like crazy. The needed code to setup your own server is in my github
sch_cake and tc-adv repos.

We hope to roll that out over linode, google cloud, and aws over the
coming days.

A typical test right now, looks like this:

# server side install if you want to set up your own test server
# after an:

apt-get install flent netperf irtt build-essential flex bison
git clone https://github.com/dtaht/sch_cake
git clone https://github.com/dtaht/tc-adv
git clone https://github.com/dtaht/fq_codel_fast
git clone https://github.com/dtaht/fq_codel_fast
git clone https://github.com/tohojo/sqm-scripts

for i in sch_cake tc-adv fq_codel_fast sqm-scripts
do
cd $i; make; make install; cd ..
done

tc qdisc replace dev eth0 root cake sce bandwidth 40mbit

# Client side test

tcpdump -i eth0 -s 128 -w mylocationinfo.cap
flent -H  cake-sce-de.teklibre.net -l 20 -t my_location_info
--te=download_streams tcp_ndown
killall tcpdump

Go looking at the cap for sack, drops, reorders, and the sce bit being excerted.

(flent's --socket-stats option got broken in a recent tc release which
we need to fix for that and qdisc stats)

if you are running your own server, you can now also see the
sce marking rate via:

tc -s qdisc show dev eth0

qdisc cake 8002: root refcnt 2 bandwidth 40Mbit diffserv3
triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw
overhead 0 sce

 Sent 37542361 bytes 25522 pkt (dropped 0, overlimits 30923 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 124568b of 4Mb
 capacity estimate: 40Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14


                   Bulk  Best Effort        Voice

  thresh       2500Kbit       40Mbit       10Mbit
  target          7.3ms        5.0ms        5.0ms
  interval      102.3ms      100.0ms      100.0ms
  pk_delay          0us        528us        347us
  av_delay          0us        201us         10us
  sp_delay          0us          4us          5us
  backlog            0b           0b           0b
  pkts                0        25209          313
  bytes               0     37500795        41566
  way_inds            0            0            0
  way_miss            0          106            5
  way_cols            0            0            0
  drops               0            0            0
  marks               0           24            0
  ack_drop            0            0            0
  sp_flows            0            1            0
  bk_flows            0            0            1
  un_flows            0            0            0
  max_len             0         3028         1752
  quantum           300         1220          305
  sce                 0        23105            0


setting up the sqm-scripts to use fq_codel_fast is a matter of setting
the right parameters in a /etc/sqm/eth0.iface.conf file for bandwidth
(40mbit) and qos model (simplest.qos) and

IQDISC_OPTS="ce_threshold 2us" # these are artificially low - 1-2ms
should be closer to "right" on cloudy devices for a real, rather than
test, deployment
EQDISC_OPTS="ce_threshold 2us"


> IPv4/IPv6 of course we'll see a big difference in favour of IPv6.
> What Michael has shown is his paper makes a lot of sense to me.
> Depending on the use case hit ratio can be very high or very low.

I am very behind on reading papers this week.

>
>
>
>
> On Sun, Mar 24, 2019 at 9:55 AM Dave Taht <dave.taht@gmail.com> wrote:
>>
>> 1) The research into whether bit flipping to the extent that SCE will
>> do has not been done yet. The study of ECT(0) vs ECT(1) behavior
>> transiting to CE was a little lightweight.
>>
>> To test this we going to fire up a ton of nanodes in various data
>> centers, with low SCE thresholds, and low bandwidths, to flip lots of
>> bits, and test between the data centers and from as many vantage
>> points around the net as we can get - do packet captures as well as
>> flent tests
>>
>> as a control, set up identical boxes, with SCE disabled, in the same
>> data centers.
>>
>> Setup flent, irtt, iperf3.
>>
>> 2) Diffserv bit preservation test
>>
>> The research going by on the tsvwg mailing seems a bit dated. It is
>> very straightforward to use irtt to test to see what udp codepoints
>> survive e2e, and to also leverage this testbed setup. Similarly,
>> netperf can easily be used to mark tcp. We do not have a good packet
>> cap tool to verify that the bits are being set right, however I think
>> irtt can be modified to check for correctness here and produce a
>> report.
>>
>> On Sun, Mar 24, 2019 at 1:45 AM Jonathan Morton <chromatix99@gmail.com> wrote:
>> >
>> > > On 24 Mar, 2019, at 8:37 am, Pete Heist <pete@heistp.net> wrote:
>> > >
>> > > I should theoretically arrive at the boat today some time after 3pm, having picked up a mini HDMI to HDMI adapter, which we can use with the cable that’s there…
>> >
>> > Awesome.  I'm also setting up a Linux VM on my Mac, which should help things along.
>> >
>> > We're bringing up some actual hardware with the SCE-enabled Cake on it now.  Dave wants to investigate various theoretical phenomena the Internet might exhibit with a mixture of ECN codepoints; I just want to be sure it actually works as intended, before I move on to fiddling with TCP.
>> >
>> >  - Jonathan Morton
>> >
>> > _______________________________________________
>> > Cake mailing list
>> > Cake@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/cake
>>
>>
>>
>> --
>>
>> Dave Täht
>> CTO, TekLibre, LLC
>> http://www.teklibre.com
>> Tel: 1-831-205-9740
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake



-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740