[Cake] The two SCE tests I have in mind
Dave Taht
dave.taht at gmail.com
Sun Mar 24 06:30:15 EDT 2019
On Sun, Mar 24, 2019 at 10:30 AM Luca Muscariello
<luca.muscariello at gmail.com> wrote:
>
> We have done something like that a year ago in our team in Cisco
> for a project that you Dave are aware of but that is out of scope for these lists.
>
> It is important to have vantage points in residential and enterprise networks.
It is hard to get dedicated testing points in those networks. However
providing tools so more volunteers can participate in a larger test
has always proven worthwhile.
> Cloud to Cloud never goes to transit and the big Clouds have global footprints.
> So you can traverse the planet inside the Cloud WAN (100% bit consistency) and then traverse a safe peering
> point where the adjacency has been well set (or not).
cloud to cloud should be interesting.
Anyway, the first sce enabled version of cake for public testing is
now up at cake-sce-de.teklibre.net. (dns still propigating) It is rate
limited to 40Mbits and thus with 10 flows going, self congests and
thus marks with both SCE and CE. Inititial tests across our cloudy
backbone have been good, no reordering, no drops, while SCE marking
like crazy. The needed code to setup your own server is in my github
sch_cake and tc-adv repos.
We hope to roll that out over linode, google cloud, and aws over the
coming days.
A typical test right now, looks like this:
# server side install if you want to set up your own test server
# after an:
apt-get install flent netperf irtt build-essential flex bison
git clone https://github.com/dtaht/sch_cake
git clone https://github.com/dtaht/tc-adv
git clone https://github.com/dtaht/fq_codel_fast
git clone https://github.com/dtaht/fq_codel_fast
git clone https://github.com/tohojo/sqm-scripts
for i in sch_cake tc-adv fq_codel_fast sqm-scripts
do
cd $i; make; make install; cd ..
done
tc qdisc replace dev eth0 root cake sce bandwidth 40mbit
# Client side test
tcpdump -i eth0 -s 128 -w mylocationinfo.cap
flent -H cake-sce-de.teklibre.net -l 20 -t my_location_info
--te=download_streams tcp_ndown
killall tcpdump
Go looking at the cap for sack, drops, reorders, and the sce bit being excerted.
(flent's --socket-stats option got broken in a recent tc release which
we need to fix for that and qdisc stats)
if you are running your own server, you can now also see the
sce marking rate via:
tc -s qdisc show dev eth0
qdisc cake 8002: root refcnt 2 bandwidth 40Mbit diffserv3
triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw
overhead 0 sce
Sent 37542361 bytes 25522 pkt (dropped 0, overlimits 30923 requeues 0)
backlog 0b 0p requeues 0
memory used: 124568b of 4Mb
capacity estimate: 40Mbit
min/max network layer size: 42 / 1514
min/max overhead-adjusted size: 42 / 1514
average network hdr offset: 14
Bulk Best Effort Voice
thresh 2500Kbit 40Mbit 10Mbit
target 7.3ms 5.0ms 5.0ms
interval 102.3ms 100.0ms 100.0ms
pk_delay 0us 528us 347us
av_delay 0us 201us 10us
sp_delay 0us 4us 5us
backlog 0b 0b 0b
pkts 0 25209 313
bytes 0 37500795 41566
way_inds 0 0 0
way_miss 0 106 5
way_cols 0 0 0
drops 0 0 0
marks 0 24 0
ack_drop 0 0 0
sp_flows 0 1 0
bk_flows 0 0 1
un_flows 0 0 0
max_len 0 3028 1752
quantum 300 1220 305
sce 0 23105 0
setting up the sqm-scripts to use fq_codel_fast is a matter of setting
the right parameters in a /etc/sqm/eth0.iface.conf file for bandwidth
(40mbit) and qos model (simplest.qos) and
IQDISC_OPTS="ce_threshold 2us" # these are artificially low - 1-2ms
should be closer to "right" on cloudy devices for a real, rather than
test, deployment
EQDISC_OPTS="ce_threshold 2us"
> IPv4/IPv6 of course we'll see a big difference in favour of IPv6.
> What Michael has shown is his paper makes a lot of sense to me.
> Depending on the use case hit ratio can be very high or very low.
I am very behind on reading papers this week.
>
>
>
>
> On Sun, Mar 24, 2019 at 9:55 AM Dave Taht <dave.taht at gmail.com> wrote:
>>
>> 1) The research into whether bit flipping to the extent that SCE will
>> do has not been done yet. The study of ECT(0) vs ECT(1) behavior
>> transiting to CE was a little lightweight.
>>
>> To test this we going to fire up a ton of nanodes in various data
>> centers, with low SCE thresholds, and low bandwidths, to flip lots of
>> bits, and test between the data centers and from as many vantage
>> points around the net as we can get - do packet captures as well as
>> flent tests
>>
>> as a control, set up identical boxes, with SCE disabled, in the same
>> data centers.
>>
>> Setup flent, irtt, iperf3.
>>
>> 2) Diffserv bit preservation test
>>
>> The research going by on the tsvwg mailing seems a bit dated. It is
>> very straightforward to use irtt to test to see what udp codepoints
>> survive e2e, and to also leverage this testbed setup. Similarly,
>> netperf can easily be used to mark tcp. We do not have a good packet
>> cap tool to verify that the bits are being set right, however I think
>> irtt can be modified to check for correctness here and produce a
>> report.
>>
>> On Sun, Mar 24, 2019 at 1:45 AM Jonathan Morton <chromatix99 at gmail.com> wrote:
>> >
>> > > On 24 Mar, 2019, at 8:37 am, Pete Heist <pete at heistp.net> wrote:
>> > >
>> > > I should theoretically arrive at the boat today some time after 3pm, having picked up a mini HDMI to HDMI adapter, which we can use with the cable that’s there…
>> >
>> > Awesome. I'm also setting up a Linux VM on my Mac, which should help things along.
>> >
>> > We're bringing up some actual hardware with the SCE-enabled Cake on it now. Dave wants to investigate various theoretical phenomena the Internet might exhibit with a mixture of ECN codepoints; I just want to be sure it actually works as intended, before I move on to fiddling with TCP.
>> >
>> > - Jonathan Morton
>> >
>> > _______________________________________________
>> > Cake mailing list
>> > Cake at lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/cake
>>
>>
>>
>> --
>>
>> Dave Täht
>> CTO, TekLibre, LLC
>> http://www.teklibre.com
>> Tel: 1-831-205-9740
>> _______________________________________________
>> Cake mailing list
>> Cake at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
--
Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740
More information about the Cake
mailing list