[Cake] cake flenter results round 2

[Cake] cake flenter results round 2

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 4363 bytes --]

http://www.drhleny.cz/bufferbloat/cake/round2/ <http://www.drhleny.cz/bufferbloat/cake/round2/>

Round 2 Tarball: http://www.drhleny.cz/bufferbloat/cake/round2.tgz <http://www.drhleny.cz/bufferbloat/cake/round2.tgz>

*** Notes/Analysis ***

* bql tests similar to last time:

http://www.drhleny.cz/bufferbloat/cake/round2/bql_csrt_rrulbe_eg_fq_codel_nolimit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/bql_csrt_rrulbe_eg_fq_codel_nolimit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round2/bql_csrt_rrulbe_eg_cakeeth_nolimit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/bql_csrt_rrulbe_eg_cakeeth_nolimit/index.html>

* 8/8 flows tests and n/n flows tests in general still a bit mysterious with too much for me to unpack:

http://www.drhleny.cz/bufferbloat/cake/round2/nflows_vs_rate_8_8_eg_fq_codel_200mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/nflows_vs_rate_8_8_eg_fq_codel_200mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round2/nflows_vs_rate_8_8_eg_cakeeth_200mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/nflows_vs_rate_8_8_eg_cakeeth_200mbit/index.html>

* first voip tests with irtt show it’s working, but these will be more meaningful at lower rates:

http://www.drhleny.cz/bufferbloat/cake/round2/voip_voiprrul_eg_fq_codel_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/voip_voiprrul_eg_fq_codel_900mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round2/voip_voiprrul_eg_cakeeth_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/voip_voiprrul_eg_cakeeth_900mbit/index.html>

* still don’t think I managed to get udp flood to work, must be doing something wrong:

http://www.drhleny.cz/bufferbloat/cake/round2/udpflood_eg_fq_codel_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/udpflood_eg_fq_codel_900mbit/index.html>
http://www.drhleny.cz/bufferbloat/cake/round2/udpflood_eg_cakeeth_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/udpflood_eg_cakeeth_900mbit/index.html>

* new cake host isolation tests across a range of RTTs show fairness improvements up to rtt 20ms, then nothing significant after that:

http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_100us_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_100us_eg_cakeeth_src_cakeeth_dst_500mbit/index.html> (2.22)
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_1ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_1ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html> (1.7)
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_2ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_2ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html> (1.6)
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_3ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_3ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html> (1.42)
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_5ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_5ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html> (1.31)
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_8ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_8ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html> (1.16)
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_10ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_10ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html> (1.12)
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_20ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_20ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html> (1.02)
http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_40ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/hostiso_cake_40ms_eg_cakeeth_src_cakeeth_dst_500mbit/index.html> (1.017)

*** Round 3 Plans:

* Use netem to test a spread of simulated rtts and bandwidths.


[-- Attachment #2: Type: text/html, Size: 6013 bytes --]

Re: [Cake] cake flenter results round 2

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 699 bytes --]


> On Nov 28, 2017, at 3:11 PM, Pete Heist <peteheist@gmail.com> wrote:
> 
> * still don’t think I managed to get udp flood to work, must be doing something wrong:
> 
> http://www.drhleny.cz/bufferbloat/cake/round2/udpflood_eg_fq_codel_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/udpflood_eg_fq_codel_900mbit/index.html>
> http://www.drhleny.cz/bufferbloat/cake/round2/udpflood_eg_cakeeth_900mbit/index.html <http://www.drhleny.cz/bufferbloat/cake/round2/udpflood_eg_cakeeth_900mbit/index.html>
Never mind, udp flood tests are working and copied up now. They'll probably be more useful at lower bandwidths.

Nice to see OWD, IPDV and loss in the voip tests… :)


[-- Attachment #2: Type: text/html, Size: 1484 bytes --]

Re: [Cake] cake flenter results round 2

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

Pete Heist <peteheist@gmail.com> writes:

>
> *** Round 3 Plans:
>
> * Use netem to test a spread of simulated rtts and bandwidths.

Since you are leveraging a few too few boxes, attached are my current
scripts for fiddling a bit with network namespaces. I added individual
ssh, irtt, etc, servers so that things like flent's ssh stuff should
just work for polling stats.

To get it setup, run

# ./vsetup.sh # basically go read *.sh
# ./vcake.sh # go read it
# ip netns exec server flent -H 10.10.2.2 yourtest

What's got me stopped is the network namespaces don't seem to let me
have an /etc/hosts file for dns lookup, as I'd wanted to have names
like mbox-l, mbox-r and the above IP, be "client".


[-- Attachment #2: veth.tgz --]
[-- Type: application/x-gtar-compressed, Size: 2432 bytes --]

Re: [Cake] cake flenter results round 2

[Pete Heist]

> On Nov 28, 2017, at 8:07 PM, Dave Taht <dave@taht.net> wrote:
> 
> Pete Heist <peteheist@gmail.com> writes:
> 
>> *** Round 3 Plans:
>> 
>> * Use netem to test a spread of simulated rtts and bandwidths.
> 
> Since you are leveraging a few too few boxes, attached are my current
> scripts for fiddling a bit with network namespaces. I added individual
> ssh, irtt, etc, servers so that things like flent's ssh stuff should
> just work for polling stats.

You mean a few too few virtual boxes, not necessarily physical ones, right? :) In other words, there are different topologies that can be used for testing. In your scripts you’re simulating "Internet access", where client is the family or organization for example and server is stuff on the Internet:

client - middlebox - delay - server

In my earlier point-to-point WiFi testing I was simulating an ISP’s backhaul:

client - client_router - station ----- ap - server_router - server

In my current testing I’m simulating, well, something far less useful if I think about it- two boxes blasting traffic to one another over a cable and trying to improve queueing delays between them:

client - client_router --- server_router - server

I hadn’t realized how heavy-weight traffic generation for anything beyond 4/4 flows would be at Gbit rates, or how confusing and trivial some of these results would be.

So beyond just my vague idea to “simulate a spread of rtts and bandwidths”, I see I need a topology change to produce something more useful. I think there are still two options:

1) Point-to-point WiFi again, where I’d be using two NSM5s and testing over short range at rates of up to 100 Mbps. I would probably try to get FreeNet’s APU version 1 boxes into action again so I’d have physical devices for each of the six roles above. I wish I didn’t have to use those RTL8111Es, but that’s how it is.

2) Your “Internet access” setup. Either I can get the veth stuff into action on a single physical APU2 (powerful enough?), or I can try to set up four physical boxes with the same topology (or if I were tricky, try to spread the four roles across two boxes).

#1 would be useful for FreeNet. Would it also be useful for Cake testing in general, or would you prefer more #2 results at this stage (i.e. simulating dsl, cable, satellite, etc)?

Re: [Cake] cake flenter results round 2

[Dave Taht]

Pete Heist <peteheist@gmail.com> writes:

>> On Nov 28, 2017, at 8:07 PM, Dave Taht <dave@taht.net> wrote:
>> 
>> Pete Heist <peteheist@gmail.com> writes:
>> 
>>> *** Round 3 Plans:
>>> 
>>> * Use netem to test a spread of simulated rtts and bandwidths.
>> 
>> Since you are leveraging a few too few boxes, attached are my current
>> scripts for fiddling a bit with network namespaces. I added individual
>> ssh, irtt, etc, servers so that things like flent's ssh stuff should
>> just work for polling stats.
>
> You mean a few too few virtual boxes, not necessarily physical ones, right? :)
> In other words, there are different topologies that can be used for testing. In
> your scripts you’re simulating "Internet access", where client is the family or
> organization for example and server is stuff on the Internet:
>
> client - middlebox - delay - server
>
> In my earlier point-to-point WiFi testing I was simulating an ISP’s backhaul:
>
> client - client_router - station ----- ap - server_router - server
>
> In my current testing I’m simulating, well, something far less useful if I think about it- two boxes blasting traffic to one another over a cable and trying to improve queueing delays between them:
>
> client - client_router --- server_router - server

Yes, in this case, the local TCP stack tends to interact better with
fq_codel than cake does, due in part to the NET_XMIT_CN support there.

I confess that my fav takeaway of your results thus far has been - it
doesn't crash. It still doesn't crash, here, either. We could upstream
it this week. :)

>
> I hadn’t realized how heavy-weight traffic generation for anything beyond 4/4 flows would be at Gbit rates, or how confusing and trivial some of these results would be.
>
> So beyond just my vague idea to “simulate a spread of rtts and bandwidths”, I see I need a topology change to produce something more useful. I think there are still two options:

Yes, I wouldn't trust the result with netem on what you got very far.

> 1) Point-to-point WiFi again, where I’d be using two NSM5s and testing over
> short range at rates of up to 100 Mbps. I would probably try to get FreeNet’s
> APU version 1 boxes into action again so I’d have physical devices for each of
> the six roles above. I wish I didn’t have to use those RTL8111Es, but that’s how
> it is.
>
> 2) Your “Internet access” setup. Either I can get the veth stuff into action on
> a single physical APU2 (powerful enough?), or I can try to set up four physical
> boxes with the same topology (or if I were tricky, try to spread the four roles
> across two boxes).

Getting the veth stuff setup for a basic test should be plug and go with
what I gave you. And you do have 4 cores, so try a few tests at 900Mbit
to see what happens.

I think george has the most powerful box we have readily available, and
perhaps it would be easier for him to give netns a go. I cannot trust
the results we get from the cloud (too many unknown VMs sharing the
hardware)... so I keep thinking, for christmas, I will finally get
around to replacing snapon (which is doing lede build and server duty in
sweden).

I am presently getting in a good 40+ minute nap in between kernel
builds, which that would also solve. (I like my naps tho). All my
computers are pretty low power - only the core I5 nucs and laptops have
fans which only kick in on builds - and I like a quiet work environment,
so trying to build the fastest box possible without howling fans would
be ideal.

snapon (6 cores) cost about 2.5k when we got it (5? years ago). It looks
like for about the same price we can get to at least 8 cores. Things
start going pear-shaped on (for example) the AMD threadripper (16 cores)
- or Xeons, at 1k for the cpu at min (but a 30 second kernel build)

We could try to get a dedicated rack mount somewhere, but I don't know
where, or how much.

I rather enjoyed what esr pulled off with "the great beast". He had
different requirements - in my case I'd like a sharable box for builds
and simulations. Were these actual simulations (e.g. ns3) the virtual
clock would be ok to run in the cloud, but since we're on bare metal,
we'd need bare metal.

Worse, to do this truly right, actually starting to fiddle with 10Gig+
hardware in a realistic topology could also be of use. Arguably those of
you in Northern Europe need space heaters more than I do...

PS: 
I note the veth file I sent had two errors in it - vdaemons was meant to
be vssh.sh and the netem delay component should have had a limit 100000
to it. I'm still using these simplistic shell scripts 'cause I havent
found anything better to construct topologies with (want ipv6), as yet,
and I should probably get around to a public repo for 'em.

>
> #1 would be useful for FreeNet. Would it also be useful for Cake testing in general, or would you prefer more #2 results at this stage (i.e. simulating dsl, cable, satellite, etc)?

I'm really happy with this stuff right now. I haven't taken apart the
tarball from the last attempt yet. 16x1, 10x1 results with ack filtering
against DSL speeds and typical cable speeds seem plausible.

Re: [Cake] cake flenter results round 2

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 88 bytes --]


A diffserv 200Mbit result would be good.

We are utterly out of cpu at 900mbits here.


[-- Attachment #2: not_winning_on_cpu.png --]
[-- Type: image/png, Size: 220032 bytes --]

Re: [Cake] cake flenter results round 2

[Pete Heist]

> On Nov 28, 2017, at 11:52 PM, Dave Taht <dave@taht.net> wrote:
> 
> A diffserv 200Mbit result would be good.
> 
> We are utterly out of cpu at 900mbits here.
> 
> <not_winning_on_cpu.png>

Wow, I see flent’s combination plots are handy though.

Stuff to sort in irtt also. Merely setting the source IP of an outgoing packet doubles allocated heap when it calls into x/net code. Super.

Ok, here’s a quick veth try but I gotta get to sleep so I’m not investigating much ’til later. I wonder if 4.9.0-4 is enough:

root@apu2a:/home/sysadmin/src/veth# cat settings.example 
export HOSTS="server client delay mbox"

# I’ve been a bad boy and did sudo make install in iproute2-cake-next
export TC=/sbin/tc
root@apu2a:/home/sysadmin/src/veth# ./vsetup.sh
Cannot remove namespace file "/var/run/netns/server": No such file or directory
Cannot remove namespace file "/var/run/netns/client": No such file or directory
Cannot remove namespace file "/var/run/netns/delay": No such file or directory
Cannot remove namespace file "/var/run/netns/mbox": No such file or directory
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
RTNETLINK answers: File exists
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
root@apu2a:/home/sysadmin/src/veth# ./vcake.sh 
root@apu2a:/home/sysadmin/src/veth# ip netns exec server flent -H 10.10.2.2 rrul_be
Started Flent 1.1.1-git-5daa2b3 using Python 2.7.13.
Starting rrul_be test. Expected run time: 70 seconds.
WARNING: Program exited non-zero.
Runner class: NetperfDemoRunner
Command: /usr/bin/netperf -P 0 -v 0 -D -0.20 -4 -Y CS0,CS0 -H 10.10.2.2 -p 12865 -t UDP_RR -l 70 -F /dev/urandom    --   -H 10.10.2.2 -k THROUGHPUT,LOCAL_CONG_CONTROL,REMOTE_CONG_CONTROL,TRANSPORT_MSS,LOCAL_TRANSPORT_RETRANS,REMOTE_TRANSPORT_RETRANS,LOCAL_SOCKET_TOS,REMOTE_SOCKET_TOS,DIRECTION,ELAPSED_TIME,PROTOCOL,LOCAL_SEND_SIZE,LOCAL_RECV_SIZE,REMOTE_SEND_SIZE,REMOTE_RECV_SIZE  
Return code: 255
Stdout: establish control: are you sure there is a netserver listening on 10.10.2.2 at port 12865?
establish_control could not establish the control connection from 0.0.0.0 port 0 address family AF_INET to 10.10.2.2 port 12865 address family AF_INET

...

[Cake] Fwd: Re: cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 3058 bytes --]

I got Dave's netns scripts working on my server. Will try to run rrul tests
on a 200/10mbit tonight.

George

On Nov 28, 2017 7:16 PM, "Pete Heist" <peteheist@gmail.com> wrote:

>
> > On Nov 28, 2017, at 11:52 PM, Dave Taht <dave@taht.net> wrote:
> >
> > A diffserv 200Mbit result would be good.
> >
> > We are utterly out of cpu at 900mbits here.
> >
> > <not_winning_on_cpu.png>
>
> Wow, I see flent’s combination plots are handy though.
>
> Stuff to sort in irtt also. Merely setting the source IP of an outgoing
> packet doubles allocated heap when it calls into x/net code. Super.
>
> Ok, here’s a quick veth try but I gotta get to sleep so I’m not
> investigating much ’til later. I wonder if 4.9.0-4 is enough:
>
> root@apu2a:/home/sysadmin/src/veth# cat settings.example
> export HOSTS="server client delay mbox"
>
> # I’ve been a bad boy and did sudo make install in iproute2-cake-next
> export TC=/sbin/tc
> root@apu2a:/home/sysadmin/src/veth# ./vsetup.sh
> Cannot remove namespace file "/var/run/netns/server": No such file or
> directory
> Cannot remove namespace file "/var/run/netns/client": No such file or
> directory
> Cannot remove namespace file "/var/run/netns/delay": No such file or
> directory
> Cannot remove namespace file "/var/run/netns/mbox": No such file or
> directory
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> RTNETLINK answers: File exists
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> root@apu2a:/home/sysadmin/src/veth# ./vcake.sh
> root@apu2a:/home/sysadmin/src/veth# ip netns exec server flent -H
> 10.10.2.2 rrul_be
> Started Flent 1.1.1-git-5daa2b3 using Python 2.7.13.
> Starting rrul_be test. Expected run time: 70 seconds.
> WARNING: Program exited non-zero.
> Runner class: NetperfDemoRunner
> Command: /usr/bin/netperf -P 0 -v 0 -D -0.20 -4 -Y CS0,CS0 -H 10.10.2.2 -p
> 12865 -t UDP_RR -l 70 -F /dev/urandom    --   -H 10.10.2.2 -k
> THROUGHPUT,LOCAL_CONG_CONTROL,REMOTE_CONG_CONTROL,TRANSPORT_
> MSS,LOCAL_TRANSPORT_RETRANS,REMOTE_TRANSPORT_RETRANS,LOCAL_
> SOCKET_TOS,REMOTE_SOCKET_TOS,DIRECTION,ELAPSED_TIME,
> PROTOCOL,LOCAL_SEND_SIZE,LOCAL_RECV_SIZE,REMOTE_SEND_SIZE,REMOTE_RECV_SIZE
> Return code: 255
> Stdout: establish control: are you sure there is a netserver listening on
> 10.10.2.2 at port 12865?
> establish_control could not establish the control connection from 0.0.0.0
> port 0 address family AF_INET to 10.10.2.2 port 12865 address family AF_INET
>
> ...
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>

[-- Attachment #2: Type: text/html, Size: 4456 bytes --]

Re: [Cake] cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 4257 bytes --]

I just ran some rrul tests, but I am getting a strange behaviour.
Setup:
server        --        delay        --        mbox        --        client
netserver         50ms/50ms         45/900mbit

cli:
https://drive.google.com/open?id=10CwMwLItL6bVSyMTp8-kVXWuFBAG6ft2
1) ./vsetup.sh (includes vcake.sh)
2) ip netns exec server netserver
3) ip netns exec client flent ....

rrul1 flent:
https://drive.google.com/open?id=1XB1k68fv4PCsRhSbmbCMvqOLk7Dv08bh
rrul1 tc qdisc show:
https://drive.google.com/open?id=1mKEYq7iDauOL5A9lCat_jAUgE7Eq6et8

rrul2 flent:
https://drive.google.com/open?id=19OhJdOtMLcsNM66Jy-Edn7rD8TC9WelJ
rrul2 tc qdisc show:
https://drive.google.com/open?id=1HtgjwvmTbr7XQKRp4XOqVvkz8FzCxdTk

rrul1 looks good, while rrul2 has a very strange trace. Look at the
upload/ping. The setup was exactly the same between these two. If I lower
the mbox bandwidth to 10/200mbit I don't get this behaviour anymore, the
traces look like rrul1.

Am I doing something wrong?

George

On Tue, Nov 28, 2017 at 7:20 PM, Georgios Amanakis <gamanakis@gmail.com>
wrote:

>
>
> I got Dave's netns scripts working on my server. Will try to run rrul
> tests on a 200/10mbit tonight.
>
> George
>
> On Nov 28, 2017 7:16 PM, "Pete Heist" <peteheist@gmail.com> wrote:
>
>>
>> > On Nov 28, 2017, at 11:52 PM, Dave Taht <dave@taht.net> wrote:
>> >
>> > A diffserv 200Mbit result would be good.
>> >
>> > We are utterly out of cpu at 900mbits here.
>> >
>> > <not_winning_on_cpu.png>
>>
>> Wow, I see flent’s combination plots are handy though.
>>
>> Stuff to sort in irtt also. Merely setting the source IP of an outgoing
>> packet doubles allocated heap when it calls into x/net code. Super.
>>
>> Ok, here’s a quick veth try but I gotta get to sleep so I’m not
>> investigating much ’til later. I wonder if 4.9.0-4 is enough:
>>
>> root@apu2a:/home/sysadmin/src/veth# cat settings.example
>> export HOSTS="server client delay mbox"
>>
>> # I’ve been a bad boy and did sudo make install in iproute2-cake-next
>> export TC=/sbin/tc
>> root@apu2a:/home/sysadmin/src/veth# ./vsetup.sh
>> Cannot remove namespace file "/var/run/netns/server": No such file or
>> directory
>> Cannot remove namespace file "/var/run/netns/client": No such file or
>> directory
>> Cannot remove namespace file "/var/run/netns/delay": No such file or
>> directory
>> Cannot remove namespace file "/var/run/netns/mbox": No such file or
>> directory
>> net.ipv4.ip_forward = 1
>> net.ipv6.conf.all.forwarding = 1
>> net.ipv4.ip_forward = 1
>> net.ipv6.conf.all.forwarding = 1
>> net.ipv4.ip_forward = 1
>> net.ipv6.conf.all.forwarding = 1
>> net.ipv4.ip_forward = 1
>> net.ipv6.conf.all.forwarding = 1
>> RTNETLINK answers: File exists
>> net.ipv4.ip_forward = 1
>> net.ipv6.conf.all.forwarding = 1
>> net.ipv4.ip_forward = 1
>> net.ipv6.conf.all.forwarding = 1
>> net.ipv4.ip_forward = 1
>> net.ipv6.conf.all.forwarding = 1
>> net.ipv4.ip_forward = 1
>> net.ipv6.conf.all.forwarding = 1
>> root@apu2a:/home/sysadmin/src/veth# ./vcake.sh
>> root@apu2a:/home/sysadmin/src/veth# ip netns exec server flent -H
>> 10.10.2.2 rrul_be
>> Started Flent 1.1.1-git-5daa2b3 using Python 2.7.13.
>> Starting rrul_be test. Expected run time: 70 seconds.
>> WARNING: Program exited non-zero.
>> Runner class: NetperfDemoRunner
>> Command: /usr/bin/netperf -P 0 -v 0 -D -0.20 -4 -Y CS0,CS0 -H 10.10.2.2
>> -p 12865 -t UDP_RR -l 70 -F /dev/urandom    --   -H 10.10.2.2 -k
>> THROUGHPUT,LOCAL_CONG_CONTROL,REMOTE_CONG_CONTROL,TRANSPORT_
>> MSS,LOCAL_TRANSPORT_RETRANS,REMOTE_TRANSPORT_RETRANS,LOCAL_S
>> OCKET_TOS,REMOTE_SOCKET_TOS,DIRECTION,ELAPSED_TIME,PROTOCOL,
>> LOCAL_SEND_SIZE,LOCAL_RECV_SIZE,REMOTE_SEND_SIZE,REMOTE_RECV_SIZE
>> Return code: 255
>> Stdout: establish control: are you sure there is a netserver listening on
>> 10.10.2.2 at port 12865?
>> establish_control could not establish the control connection from 0.0.0.0
>> port 0 address family AF_INET to 10.10.2.2 port 12865 address family AF_INET
>>
>> ...
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
>>
>
>

[-- Attachment #2: Type: text/html, Size: 6677 bytes --]

Re: [Cake] cake flenter results round 2

[Dave Taht]

the astounding figure is dropping ~70% of all packets, and tcp still
works at all.

682411 pkt (dropped 1511579 ...)

is your 2nd result repeatable? What happens with just "ack-filter" not
aggressive?

I keep thinking that a way to coax more performance out of 40GigE+
might be to drop more acks (as correctly as possible) before they even
hit the server.


On Tue, Nov 28, 2017 at 7:29 PM, Georgios Amanakis <gamanakis@gmail.com> wrote:
> I just ran some rrul tests, but I am getting a strange behaviour.
> Setup:
> server        --        delay        --        mbox        --        client
> netserver         50ms/50ms         45/900mbit
>
> cli:
> https://drive.google.com/open?id=10CwMwLItL6bVSyMTp8-kVXWuFBAG6ft2
> 1) ./vsetup.sh (includes vcake.sh)
> 2) ip netns exec server netserver
> 3) ip netns exec client flent ....
>
> rrul1 flent:
> https://drive.google.com/open?id=1XB1k68fv4PCsRhSbmbCMvqOLk7Dv08bh
> rrul1 tc qdisc show:
> https://drive.google.com/open?id=1mKEYq7iDauOL5A9lCat_jAUgE7Eq6et8
>
> rrul2 flent:
> https://drive.google.com/open?id=19OhJdOtMLcsNM66Jy-Edn7rD8TC9WelJ
> rrul2 tc qdisc show:
> https://drive.google.com/open?id=1HtgjwvmTbr7XQKRp4XOqVvkz8FzCxdTk
>
> rrul1 looks good, while rrul2 has a very strange trace. Look at the
> upload/ping. The setup was exactly the same between these two. If I lower
> the mbox bandwidth to 10/200mbit I don't get this behaviour anymore, the
> traces look like rrul1.
>
> Am I doing something wrong?
>
> George
>
> On Tue, Nov 28, 2017 at 7:20 PM, Georgios Amanakis <gamanakis@gmail.com>
> wrote:
>>
>>
>>
>> I got Dave's netns scripts working on my server. Will try to run rrul
>> tests on a 200/10mbit tonight.
>>
>> George
>>
>> On Nov 28, 2017 7:16 PM, "Pete Heist" <peteheist@gmail.com> wrote:
>>>
>>>
>>> > On Nov 28, 2017, at 11:52 PM, Dave Taht <dave@taht.net> wrote:
>>> >
>>> > A diffserv 200Mbit result would be good.
>>> >
>>> > We are utterly out of cpu at 900mbits here.
>>> >
>>> > <not_winning_on_cpu.png>
>>>
>>> Wow, I see flent’s combination plots are handy though.
>>>
>>> Stuff to sort in irtt also. Merely setting the source IP of an outgoing
>>> packet doubles allocated heap when it calls into x/net code. Super.
>>>
>>> Ok, here’s a quick veth try but I gotta get to sleep so I’m not
>>> investigating much ’til later. I wonder if 4.9.0-4 is enough:
>>>
>>> root@apu2a:/home/sysadmin/src/veth# cat settings.example
>>> export HOSTS="server client delay mbox"
>>>
>>> # I’ve been a bad boy and did sudo make install in iproute2-cake-next
>>> export TC=/sbin/tc
>>> root@apu2a:/home/sysadmin/src/veth# ./vsetup.sh
>>> Cannot remove namespace file "/var/run/netns/server": No such file or
>>> directory
>>> Cannot remove namespace file "/var/run/netns/client": No such file or
>>> directory
>>> Cannot remove namespace file "/var/run/netns/delay": No such file or
>>> directory
>>> Cannot remove namespace file "/var/run/netns/mbox": No such file or
>>> directory
>>> net.ipv4.ip_forward = 1
>>> net.ipv6.conf.all.forwarding = 1
>>> net.ipv4.ip_forward = 1
>>> net.ipv6.conf.all.forwarding = 1
>>> net.ipv4.ip_forward = 1
>>> net.ipv6.conf.all.forwarding = 1
>>> net.ipv4.ip_forward = 1
>>> net.ipv6.conf.all.forwarding = 1
>>> RTNETLINK answers: File exists
>>> net.ipv4.ip_forward = 1
>>> net.ipv6.conf.all.forwarding = 1
>>> net.ipv4.ip_forward = 1
>>> net.ipv6.conf.all.forwarding = 1
>>> net.ipv4.ip_forward = 1
>>> net.ipv6.conf.all.forwarding = 1
>>> net.ipv4.ip_forward = 1
>>> net.ipv6.conf.all.forwarding = 1
>>> root@apu2a:/home/sysadmin/src/veth# ./vcake.sh
>>> root@apu2a:/home/sysadmin/src/veth# ip netns exec server flent -H
>>> 10.10.2.2 rrul_be
>>> Started Flent 1.1.1-git-5daa2b3 using Python 2.7.13.
>>> Starting rrul_be test. Expected run time: 70 seconds.
>>> WARNING: Program exited non-zero.
>>> Runner class: NetperfDemoRunner
>>> Command: /usr/bin/netperf -P 0 -v 0 -D -0.20 -4 -Y CS0,CS0 -H 10.10.2.2
>>> -p 12865 -t UDP_RR -l 70 -F /dev/urandom    --   -H 10.10.2.2 -k
>>> THROUGHPUT,LOCAL_CONG_CONTROL,REMOTE_CONG_CONTROL,TRANSPORT_MSS,LOCAL_TRANSPORT_RETRANS,REMOTE_TRANSPORT_RETRANS,LOCAL_SOCKET_TOS,REMOTE_SOCKET_TOS,DIRECTION,ELAPSED_TIME,PROTOCOL,LOCAL_SEND_SIZE,LOCAL_RECV_SIZE,REMOTE_SEND_SIZE,REMOTE_RECV_SIZE
>>> Return code: 255
>>> Stdout: establish control: are you sure there is a netserver listening on
>>> 10.10.2.2 at port 12865?
>>> establish_control could not establish the control connection from 0.0.0.0
>>> port 0 address family AF_INET to 10.10.2.2 port 12865 address family AF_INET
>>>
>>> ...
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>>
>>
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

With ack-filter-aggressive I get results similar to rrul1 in ~10% of the
runs, 90% are similar to rrul2.
With ack-filter I haven't managed yet to get a rrul1 result, all of them
are similar to rrul2.
Will experiment more today.

George

On Tue, Nov 28, 2017 at 11:01 PM, Dave Taht <dave.taht@gmail.com> wrote:

> the astounding figure is dropping ~70% of all packets, and tcp still
> works at all.
>
> 682411 pkt (dropped 1511579 ...)
>
> is your 2nd result repeatable? What happens with just "ack-filter" not
> aggressive?
>
> I keep thinking that a way to coax more performance out of 40GigE+
> might be to drop more acks (as correctly as possible) before they even
> hit the server.
>
>

[-- Attachment #2: Type: text/html, Size: 1122 bytes --]

Re: [Cake] cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 2378 bytes --]

I am troubled by the number of data points flent reports for some pings and
uploads in this setup. A typical ack-filter result, similar to rrul2 I
posted before, looks like this:
Summary of rrul test run 'rrul_cakeeth_ds3_900mbit_45mbit_ack' (at
2017-11-29 14:37:5
5.719180):

                             avg       median          # data pts
 Ping (ms) ICMP   :       100.29       100.00 ms              337
 Ping (ms) UDP BE :       297.62       100.20 ms              156
 Ping (ms) UDP BK :       409.84       100.20 ms              109
 Ping (ms) UDP EF :       414.94       100.20 ms              121
 Ping (ms) avg    :       374.13       100.05 ms              343
 TCP download BE  :       206.39       217.48 Mbits/s         301
 TCP download BK  :       163.05       163.81 Mbits/s         301
 TCP download CS5 :       206.40       217.50 Mbits/s         301
 TCP download EF  :       203.18       215.23 Mbits/s         301
 TCP download avg :       194.75       202.59 Mbits/s         301
 TCP download sum :       779.02       810.36 Mbits/s         301
 TCP totals       :       781.55       810.66 Mbits/s         301
 TCP upload BE    :         1.86        24.68 Mbits/s          23
 TCP upload BK    :         0.15         1.72 Mbits/s          24
 TCP upload CS5   :         0.25         2.26 Mbits/s          30
 TCP upload EF    :         0.27         2.15 Mbits/s          29
 TCP upload avg   :         0.63         7.59 Mbits/s          36
 TCP upload sum   :         2.53        30.38 Mbits/s          36

Shouldn't all "# data pts" be ~300?

George

On Wed, Nov 29, 2017 at 9:34 AM, Georgios Amanakis <gamanakis@gmail.com>
wrote:

> With ack-filter-aggressive I get results similar to rrul1 in ~10% of the
> runs, 90% are similar to rrul2.
> With ack-filter I haven't managed yet to get a rrul1 result, all of them
> are similar to rrul2.
> Will experiment more today.
>
> George
>
> On Tue, Nov 28, 2017 at 11:01 PM, Dave Taht <dave.taht@gmail.com> wrote:
>
>> the astounding figure is dropping ~70% of all packets, and tcp still
>> works at all.
>>
>> 682411 pkt (dropped 1511579 ...)
>>
>> is your 2nd result repeatable? What happens with just "ack-filter" not
>> aggressive?
>>
>> I keep thinking that a way to coax more performance out of 40GigE+
>> might be to drop more acks (as correctly as possible) before they even
>> hit the server.
>>
>>

[-- Attachment #2: Type: text/html, Size: 3725 bytes --]

Re: [Cake] cake flenter results round 2

[Toke Høiland-Jørgensen]

Georgios Amanakis <gamanakis@gmail.com> writes:

> I am troubled by the number of data points flent reports for some pings and
> uploads in this setup. A typical ack-filter result, similar to rrul2 I
> posted before, looks like this:
> Summary of rrul test run 'rrul_cakeeth_ds3_900mbit_45mbit_ack' (at
> 2017-11-29 14:37:5
> 5.719180):
>
>                              avg       median          # data pts
>  Ping (ms) ICMP   :       100.29       100.00 ms              337
>  Ping (ms) UDP BE :       297.62       100.20 ms              156
>  Ping (ms) UDP BK :       409.84       100.20 ms              109
>  Ping (ms) UDP EF :       414.94       100.20 ms              121
>  Ping (ms) avg    :       374.13       100.05 ms              343
>  TCP download BE  :       206.39       217.48 Mbits/s         301
>  TCP download BK  :       163.05       163.81 Mbits/s         301
>  TCP download CS5 :       206.40       217.50 Mbits/s         301
>  TCP download EF  :       203.18       215.23 Mbits/s         301
>  TCP download avg :       194.75       202.59 Mbits/s         301
>  TCP download sum :       779.02       810.36 Mbits/s         301
>  TCP totals       :       781.55       810.66 Mbits/s         301
>  TCP upload BE    :         1.86        24.68 Mbits/s          23
>  TCP upload BK    :         0.15         1.72 Mbits/s          24
>  TCP upload CS5   :         0.25         2.26 Mbits/s          30
>  TCP upload EF    :         0.27         2.15 Mbits/s          29
>  TCP upload avg   :         0.63         7.59 Mbits/s          36
>  TCP upload sum   :         2.53        30.38 Mbits/s          36
>
> Shouldn't all "# data pts" be ~300?

Ideally, yes. However, netperf works by estimating after how many bytes
it will need to emit the next data point. If this estimate is wrong
(which most often happens at very low bandwidths as you are seeing
here), the number of data points can be a lot lower. The average values
should still be correct, though; they are calculated at the end of the
test as bytes transmitted / test duration.

Something similar happens with the UDP results, with the added
complication that these can stop entirely when there is packet loss.
Flent has recently learned how to support the irtt tool
(https://github.com/peteheist/irtt/) for the UDP latency measurements,
which emits isochronous latency probes in the same way as the regular
ping tool. If you install irtt and use the latest git version of Flent,
it should be picked up automatically and used for the UDP ping
measurements.

-Toke

Re: [Cake] cake flenter results round 2

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1164 bytes --]


> On Nov 29, 2017, at 3:50 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Georgios Amanakis <gamanakis@gmail.com <mailto:gamanakis@gmail.com>> writes:
> 
>> I am troubled by the number of data points flent reports for some pings and
>> uploads in this setup. A typical ack-filter result, similar to rrul2 I
>> posted before, looks like this:
> Flent has recently learned how to support the irtt tool
> (https://github.com/peteheist/irtt/ <https://github.com/peteheist/irtt/>) for the UDP latency measurements,
> which emits isochronous latency probes in the same way as the regular
> ping tool. If you install irtt and use the latest git version of Flent,
> it should be picked up automatically and used for the UDP ping
> measurements.

In case you install irtt, let me know if you have any problems. It’s currently only in source form, but should be easy to install after you have Go set up. https://github.com/peteheist/irtt#installation. There are also two systemd service files in the tree in case you want to start the server at boot.

(That was also informative for me about how netperf decides when to emit a data point…)


[-- Attachment #2: Type: text/html, Size: 8492 bytes --]

Re: [Cake] cake flenter results round 2

[Toke Høiland-Jørgensen]

> (That was also informative for me about how netperf decides when to
> emit a data point…)

In that case I can add that the stated reason for this way of doing
things is performance (i.e., emitting data points should not interfere
with transfer performance). This is mostly an issue on systems where
getting time is expensive; which is not the case on modern Linux
systems. But I'm not entirely sure that the optimisation only has
historical reasons; it may be that some systems supported by Netperf
still has this issue...

-Toke

Re: [Cake] cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 2217 bytes --]

I did some more testing. Same setup as before, I varied the amount of delay:

server        --        delay        --        mbox        --        client
netserver             Xms/Xms             45/900mbit

Cake config:
qdisc cake 801b: dev mbox.l root refcnt 2 bandwidth 45Mbit diffserv3
triple-isolate ack-filter-aggressive rtt 100.0ms noatm overhead 38
via-ethernet mpu 84
qdisc cake 801c: dev mbox.r root refcnt 2 bandwidth 900Mbit diffserv3
triple-isolate rtt 100.0ms noatm overhead 38 via-ethernet mpu 84

Results:
delay 10ms (rtt) flent:
https://drive.google.com/open?id=1hq_MRtocoDglTqxvAHoZvo932ThLBQaC
delay 10ms (rtt) stat:
https://drive.google.com/open?id=1kTnpreQzpRn-7iO6i85eXVf8GjJYg19e

delay 20ms (rtt) flent:
https://drive.google.com/open?id=1Ollbqg7BzM4RiPuSH-tiIuaE8vnKu5tg
delay 20ms (rtt) stats:
https://drive.google.com/open?id=1nwS80SJmnVtubIXyYgBCIQdom_QfSSKB

delay 40ms (rtt) flent:
https://drive.google.com/open?id=1nWUo82_L8_GobR1xbKms-jGhkNwT5msx
delay 40ms (rtt) stats:
https://drive.google.com/open?id=1oYfERh57fKHomVHb4z0dHQtFtP2U2aWs

delay 80ms (rtt) flent:
https://drive.google.com/open?id=17j2T12Xmbi10i-0drHOgdc1x1NL8zAto
delay 80ms (rtt) stats:
https://drive.google.com/open?id=1e8cf5z4xDXYMbY8Q1rMvJd8J8F5OOcth

delay 100ms (rtt) flent:
https://drive.google.com/open?id=1vg-A92eFc7AMSOuBgj-sRnANBMJda9og
delay 100ms (rtt) stats:
https://drive.google.com/open?id=1_WojJPa8h9JmNvmWjW9Gos8ShtvM-zt0

I will repeat these with ack-filter instead of ack-filter-aggressive.

George


On Wed, Nov 29, 2017 at 10:44 AM, Toke Høiland-Jørgensen <toke@toke.dk>
wrote:

> > (That was also informative for me about how netperf decides when to
> > emit a data point…)
>
> In that case I can add that the stated reason for this way of doing
> things is performance (i.e., emitting data points should not interfere
> with transfer performance). This is mostly an issue on systems where
> getting time is expensive; which is not the case on modern Linux
> systems. But I'm not entirely sure that the optimisation only has
> historical reasons; it may be that some systems supported by Netperf
> still has this issue...
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 3666 bytes --]

Re: [Cake] cake flenter results round 2

[Dave Taht]

I just want to verify that you increased the netem limit by a lot in the scripts?

tc qdisc add dev whatever root netem delay 10ms limit 100000


Georgios Amanakis <gamanakis@gmail.com> writes:

> I did some more testing. Same setup as before, I varied the amount of delay:
>
> server -- delay -- mbox -- client
> netserver Xms/Xms 45/900mbit
>
> Cake config:
> qdisc cake 801b: dev mbox.l root refcnt 2 bandwidth 45Mbit diffserv3
> triple-isolate ack-filter-aggressive rtt 100.0ms noatm overhead 38 via-ethernet
> mpu 84
> qdisc cake 801c: dev mbox.r root refcnt 2 bandwidth 900Mbit diffserv3
> triple-isolate rtt 100.0ms noatm overhead 38 via-ethernet mpu 84
>
> Results:
> delay 10ms (rtt) flent:
> https://drive.google.com/open?id=1hq_MRtocoDglTqxvAHoZvo932ThLBQaC
> delay 10ms (rtt) stat:
> https://drive.google.com/open?id=1kTnpreQzpRn-7iO6i85eXVf8GjJYg19e
>
> delay 20ms (rtt) flent:
> https://drive.google.com/open?id=1Ollbqg7BzM4RiPuSH-tiIuaE8vnKu5tg
> delay 20ms (rtt) stats:
> https://drive.google.com/open?id=1nwS80SJmnVtubIXyYgBCIQdom_QfSSKB
>
> delay 40ms (rtt) flent:
> https://drive.google.com/open?id=1nWUo82_L8_GobR1xbKms-jGhkNwT5msx
> delay 40ms (rtt) stats:
> https://drive.google.com/open?id=1oYfERh57fKHomVHb4z0dHQtFtP2U2aWs
>
> delay 80ms (rtt) flent:
> https://drive.google.com/open?id=17j2T12Xmbi10i-0drHOgdc1x1NL8zAto
> delay 80ms (rtt) stats:
> https://drive.google.com/open?id=1e8cf5z4xDXYMbY8Q1rMvJd8J8F5OOcth
>
> delay 100ms (rtt) flent:
> https://drive.google.com/open?id=1vg-A92eFc7AMSOuBgj-sRnANBMJda9og
> delay 100ms (rtt) stats:
> https://drive.google.com/open?id=1_WojJPa8h9JmNvmWjW9Gos8ShtvM-zt0
>
> I will repeat these with ack-filter instead of ack-filter-aggressive.
>
> George
>
> On Wed, Nov 29, 2017 at 10:44 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>
>     > (That was also informative for me about how netperf decides when to
>     > emit a data point…)
>     
>     In that case I can add that the stated reason for this way of doing
>     things is performance (i.e., emitting data points should not interfere
>     with transfer performance). This is mostly an issue on systems where
>     getting time is expensive; which is not the case on modern Linux
>     systems. But I'm not entirely sure that the optimisation only has
>     historical reasons; it may be that some systems supported by Netperf
>     still has this issue...
>     
>     -Toke
>     
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 393 bytes --]

I didn't, it remained at the default value: 1000.

I only modified the delay parameter in:
"ip netns exec delay tc qdisc replace dev delay.r root netem delay 50ms"


On Wed, Nov 29, 2017 at 12:44 PM, Dave Taht <dave@taht.net> wrote:

>
> I just want to verify that you increased the netem limit by a lot in the
> scripts?
>
> tc qdisc add dev whatever root netem delay 10ms limit 100000
>
>
>

[-- Attachment #2: Type: text/html, Size: 828 bytes --]

[Cake] Fwd: cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 905 bytes --]

---------- Forwarded message ----------
From: Georgios Amanakis <gamanakis@gmail.com>
Date: Wed, Nov 29, 2017 at 12:50 PM
Subject: Re: [Cake] cake flenter results round 2
To: Dave Taht <dave@taht.net>


To avoid a misunderstanding, the delay parameter in both:
"ip netns exec delay tc qdisc replace dev delay.r root netem delay 50ms"
"ip netns exec delay tc qdisc replace dev delay.l root netem delay 50ms"

On Wed, Nov 29, 2017 at 12:49 PM, Georgios Amanakis <gamanakis@gmail.com>
wrote:

> I didn't, it remained at the default value: 1000.
>
> I only modified the delay parameter in:
> "ip netns exec delay tc qdisc replace dev delay.r root netem delay 50ms"
>
>
> On Wed, Nov 29, 2017 at 12:44 PM, Dave Taht <dave@taht.net> wrote:
>
>>
>> I just want to verify that you increased the netem limit by a lot in the
>> scripts?
>>
>> tc qdisc add dev whatever root netem delay 10ms limit 100000
>>
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 2031 bytes --]

Re: [Cake] Fwd: cake flenter results round 2

[Dave Taht]

Georgios Amanakis <gamanakis@gmail.com> writes:

> ---------- Forwarded message ----------
> From: Georgios Amanakis <gamanakis@gmail.com>
> Date: Wed, Nov 29, 2017 at 12:50 PM
> Subject: Re: [Cake] cake flenter results round 2
> To: Dave Taht <dave@taht.net>
>
> To avoid a misunderstanding, the delay parameter in both:
> "ip netns exec delay tc qdisc replace dev delay.r root netem delay 50ms"
> "ip netns exec delay tc qdisc replace dev delay.l root netem delay 50ms"

I would strongly suspect you are seeing drops in these qdiscs also
without the increased limit, at the increased delay, at 900mbit (go
check with ip netns exec delay tc -s qdisc show)

My original scripts were targeted at 16Mbit/1mbit and thus I didn't
change the limit.

>
> On Wed, Nov 29, 2017 at 12:49 PM, Georgios Amanakis <gamanakis@gmail.com> wrote:
>
>     I didn't, it remained at the default value: 1000.
>     
>     I only modified the delay parameter in:
>     "ip netns exec delay tc qdisc replace dev delay.r root netem delay 50ms"
>     
>     
>     
>     
>     
>     
>     On Wed, Nov 29, 2017 at 12:44 PM, Dave Taht <dave@taht.net> wrote:
>     
>     
>         I just want to verify that you increased the netem limit by a lot in the
>         scripts?
>         
>         tc qdisc add dev whatever root netem delay 10ms limit 100000
>         
>         
>         
>         
>         
>
>     
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] Fwd: cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 1313 bytes --]

I completely neglected this. It turns out sometimes netem drops, sometimes
it doesn't.
This would also explain the different behaviour I am getting (rrul1)
sometimes.
I guess it is because netem doesn't drop in this case.

I am attaching a new file with both cases (I could replicate them again).
Delay's netem and mbox's cake stats are in the txt files.

Is a limit of 4000 a sane value?
Calculated as (0.05s * 900mbit/s / 8bit/byte / 1500bytes/packet)??

George

On Wed, Nov 29, 2017 at 1:00 PM, Dave Taht <dave@taht.net> wrote:

> Georgios Amanakis <gamanakis@gmail.com> writes:
>
> > ---------- Forwarded message ----------
> > From: Georgios Amanakis <gamanakis@gmail.com>
> > Date: Wed, Nov 29, 2017 at 12:50 PM
> > Subject: Re: [Cake] cake flenter results round 2
> > To: Dave Taht <dave@taht.net>
> >
> > To avoid a misunderstanding, the delay parameter in both:
> > "ip netns exec delay tc qdisc replace dev delay.r root netem delay 50ms"
> > "ip netns exec delay tc qdisc replace dev delay.l root netem delay 50ms"
>
> I would strongly suspect you are seeing drops in these qdiscs also
> without the increased limit, at the increased delay, at 900mbit (go
> check with ip netns exec delay tc -s qdisc show)
>
> My original scripts were targeted at 16Mbit/1mbit and thus I didn't
> change the limit.
>
>

[-- Attachment #1.2: Type: text/html, Size: 2107 bytes --]

[-- Attachment #2: ackaggr_netem.tgz --]
[-- Type: application/x-gzip, Size: 195804 bytes --]

Re: [Cake] Fwd: cake flenter results round 2

[Dave Taht]

there is no performance impact of using really high values for netem
limit. Stick with 100000. :)

(well, there is a cache impact, but that's the cost of correct simulation)

On Wed, Nov 29, 2017 at 10:19 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
> I completely neglected this. It turns out sometimes netem drops, sometimes
> it doesn't.
> This would also explain the different behaviour I am getting (rrul1)
> sometimes.
> I guess it is because netem doesn't drop in this case.
>
> I am attaching a new file with both cases (I could replicate them again).
> Delay's netem and mbox's cake stats are in the txt files.
>
> Is a limit of 4000 a sane value?
> Calculated as (0.05s * 900mbit/s / 8bit/byte / 1500bytes/packet)??
>
> George
>
> On Wed, Nov 29, 2017 at 1:00 PM, Dave Taht <dave@taht.net> wrote:
>>
>> Georgios Amanakis <gamanakis@gmail.com> writes:
>>
>> > ---------- Forwarded message ----------
>> > From: Georgios Amanakis <gamanakis@gmail.com>
>> > Date: Wed, Nov 29, 2017 at 12:50 PM
>> > Subject: Re: [Cake] cake flenter results round 2
>> > To: Dave Taht <dave@taht.net>
>> >
>> > To avoid a misunderstanding, the delay parameter in both:
>> > "ip netns exec delay tc qdisc replace dev delay.r root netem delay 50ms"
>> > "ip netns exec delay tc qdisc replace dev delay.l root netem delay 50ms"
>>
>> I would strongly suspect you are seeing drops in these qdiscs also
>> without the increased limit, at the increased delay, at 900mbit (go
>> check with ip netns exec delay tc -s qdisc show)
>>
>> My original scripts were targeted at 16Mbit/1mbit and thus I didn't
>> change the limit.
>>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] Fwd: cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 2340 bytes --]

With netem's limit set at 100000, I repeated the tests for ack-aggressive,
ack, noack.

Setup:
server        --        delay        --        mbox        --        client
netserver             50ms/50ms             45/900mbit

George

On Wed, Nov 29, 2017 at 1:51 PM, Dave Taht <dave.taht@gmail.com> wrote:

> there is no performance impact of using really high values for netem
> limit. Stick with 100000. :)
>
> (well, there is a cache impact, but that's the cost of correct simulation)
>
> On Wed, Nov 29, 2017 at 10:19 AM, Georgios Amanakis <gamanakis@gmail.com>
> wrote:
> > I completely neglected this. It turns out sometimes netem drops,
> sometimes
> > it doesn't.
> > This would also explain the different behaviour I am getting (rrul1)
> > sometimes.
> > I guess it is because netem doesn't drop in this case.
> >
> > I am attaching a new file with both cases (I could replicate them again).
> > Delay's netem and mbox's cake stats are in the txt files.
> >
> > Is a limit of 4000 a sane value?
> > Calculated as (0.05s * 900mbit/s / 8bit/byte / 1500bytes/packet)??
> >
> > George
> >
> > On Wed, Nov 29, 2017 at 1:00 PM, Dave Taht <dave@taht.net> wrote:
> >>
> >> Georgios Amanakis <gamanakis@gmail.com> writes:
> >>
> >> > ---------- Forwarded message ----------
> >> > From: Georgios Amanakis <gamanakis@gmail.com>
> >> > Date: Wed, Nov 29, 2017 at 12:50 PM
> >> > Subject: Re: [Cake] cake flenter results round 2
> >> > To: Dave Taht <dave@taht.net>
> >> >
> >> > To avoid a misunderstanding, the delay parameter in both:
> >> > "ip netns exec delay tc qdisc replace dev delay.r root netem delay
> 50ms"
> >> > "ip netns exec delay tc qdisc replace dev delay.l root netem delay
> 50ms"
> >>
> >> I would strongly suspect you are seeing drops in these qdiscs also
> >> without the increased limit, at the increased delay, at 900mbit (go
> >> check with ip netns exec delay tc -s qdisc show)
> >>
> >> My original scripts were targeted at 16Mbit/1mbit and thus I didn't
> >> change the limit.
> >>
> >
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
> >
>
>
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619
>

[-- Attachment #1.2: Type: text/html, Size: 3757 bytes --]

[-- Attachment #2: ack_noack.tgz --]
[-- Type: application/x-gzip, Size: 346696 bytes --]

Re: [Cake] Fwd: cake flenter results round 2

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 138 bytes --]

So that looks pretty sane for both variants.

Something with a lot more flows, like rrul_50_down, or rrul_be_nflows
would be interesting.

[-- Attachment #2: total_bandwidth_900_45mbit.png --]
[-- Type: image/png, Size: 108178 bytes --]

[-- Attachment #3: upload_totals.png --]
[-- Type: image/png, Size: 46855 bytes --]

[Cake] Fwd: Fwd: cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 688 bytes --]

---------- Forwarded message ----------
From: Georgios Amanakis <gamanakis@gmail.com>
Date: Wed, Nov 29, 2017 at 3:37 PM
Subject: Re: [Cake] Fwd: cake flenter results round 2
To: Dave Taht <dave.taht@gmail.com>


rrul_50_down for ack-aggressive, ack, noack.
netserver and irtt on server.
flent on client.

Same setup as above:
server        --        delay        --        mbox        --        client
netserver             50ms/50ms             45/900mbit

George


On Wed, Nov 29, 2017 at 3:06 PM, Dave Taht <dave.taht@gmail.com> wrote:

> So that looks pretty sane for both variants.
>
> Something with a lot more flows, like rrul_50_down, or rrul_be_nflows
> would be interesting.
>

[-- Attachment #1.2: Type: text/html, Size: 1604 bytes --]

[-- Attachment #2: rrul50down_ack_noack.tgz --]
[-- Type: application/x-gzip, Size: 915703 bytes --]

Re: [Cake] Fwd: cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 258 bytes --]

Testing rrul_be_nflows with 64 down and 16 up, ack-aggressive, ack, noack.
This time with plots.

Same setup as above:
server        --        delay        --        mbox        --        client
netserver             50ms/50ms             45/900mbit

George

[-- Attachment #1.2: Type: text/html, Size: 810 bytes --]

[-- Attachment #2: total_bandwidth.png --]
[-- Type: image/png, Size: 79354 bytes --]

[-- Attachment #3: upload_totals.png --]
[-- Type: image/png, Size: 53530 bytes --]

[-- Attachment #4: download_totals.png --]
[-- Type: image/png, Size: 53019 bytes --]

[-- Attachment #5: latency_totals.png --]
[-- Type: image/png, Size: 57233 bytes --]

[-- Attachment #6: rrulbe_d64_u16_ack_noack.tgz --]
[-- Type: application/x-gzip, Size: 1395044 bytes --]

Re: [Cake] Fwd: cake flenter results round 2

[Dave Taht]

Georgios Amanakis <gamanakis@gmail.com> writes:

> Testing rrul_be_nflows with 64 down and 16 up, ack-aggressive, ack, noack.
> This time with plots.

Hmm. So with the more flows we don't get as much queue to filter out.

>
> Same setup as above:
> server -- delay -- mbox -- client
> netserver 50ms/50ms 45/900mbit
>
> George
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake flenter results round 2

[Dave Taht]

Also, it is easier (for me at least) to download a tarball of all your
results for a given run, rather than each one individually.

I am thinking we can rename ack-filter-aggressive to
ack-filter-too-damn-aggressive.

Dave Taht <dave@taht.net> writes:

> I just want to verify that you increased the netem limit by a lot in the scripts?
>
> tc qdisc add dev whatever root netem delay 10ms limit 100000
>
>
> Georgios Amanakis <gamanakis@gmail.com> writes:
>
>> I did some more testing. Same setup as before, I varied the amount of delay:
>>
>> server -- delay -- mbox -- client
>> netserver Xms/Xms 45/900mbit
>>
>> Cake config:
>> qdisc cake 801b: dev mbox.l root refcnt 2 bandwidth 45Mbit diffserv3
>> triple-isolate ack-filter-aggressive rtt 100.0ms noatm overhead 38 via-ethernet
>> mpu 84
>> qdisc cake 801c: dev mbox.r root refcnt 2 bandwidth 900Mbit diffserv3
>> triple-isolate rtt 100.0ms noatm overhead 38 via-ethernet mpu 84
>>
>> Results:
>> delay 10ms (rtt) flent:
>> https://drive.google.com/open?id=1hq_MRtocoDglTqxvAHoZvo932ThLBQaC
>> delay 10ms (rtt) stat:
>> https://drive.google.com/open?id=1kTnpreQzpRn-7iO6i85eXVf8GjJYg19e
>>
>> delay 20ms (rtt) flent:
>> https://drive.google.com/open?id=1Ollbqg7BzM4RiPuSH-tiIuaE8vnKu5tg
>> delay 20ms (rtt) stats:
>> https://drive.google.com/open?id=1nwS80SJmnVtubIXyYgBCIQdom_QfSSKB
>>
>> delay 40ms (rtt) flent:
>> https://drive.google.com/open?id=1nWUo82_L8_GobR1xbKms-jGhkNwT5msx
>> delay 40ms (rtt) stats:
>> https://drive.google.com/open?id=1oYfERh57fKHomVHb4z0dHQtFtP2U2aWs
>>
>> delay 80ms (rtt) flent:
>> https://drive.google.com/open?id=17j2T12Xmbi10i-0drHOgdc1x1NL8zAto
>> delay 80ms (rtt) stats:
>> https://drive.google.com/open?id=1e8cf5z4xDXYMbY8Q1rMvJd8J8F5OOcth
>>
>> delay 100ms (rtt) flent:
>> https://drive.google.com/open?id=1vg-A92eFc7AMSOuBgj-sRnANBMJda9og
>> delay 100ms (rtt) stats:
>> https://drive.google.com/open?id=1_WojJPa8h9JmNvmWjW9Gos8ShtvM-zt0
>>
>> I will repeat these with ack-filter instead of ack-filter-aggressive.
>>
>> George
>>
>> On Wed, Nov 29, 2017 at 10:44 AM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>>
>>     > (That was also informative for me about how netperf decides when to
>>     > emit a data point…)
>>     
>>     In that case I can add that the stated reason for this way of doing
>>     things is performance (i.e., emitting data points should not interfere
>>     with transfer performance). This is mostly an issue on systems where
>>     getting time is expensive; which is not the case on modern Linux
>>     systems. But I'm not entirely sure that the optimisation only has
>>     historical reasons; it may be that some systems supported by Netperf
>>     still has this issue...
>>     
>>     -Toke
>>     
>>
>>
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

[Cake] Fwd: cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 517 bytes --]

---------- Forwarded message ----------
From: Georgios Amanakis <gamanakis@gmail.com>
Date: Wed, Nov 29, 2017 at 12:59 PM
Subject: Re: [Cake] cake flenter results round 2
To: Dave Taht <dave@taht.net>


Of course :)

On Wed, Nov 29, 2017 at 12:54 PM, Dave Taht <dave@taht.net> wrote:

>
> Also, it is easier (for me at least) to download a tarball of all your
> results for a given run, rather than each one individually.
>
> I am thinking we can rename ack-filter-aggressive to
> ack-filter-too-damn-aggressive.
>
>

[-- Attachment #1.2: Type: text/html, Size: 1076 bytes --]

[-- Attachment #2: ackaggr.tgz --]
[-- Type: application/x-gzip, Size: 565125 bytes --]

Re: [Cake] cake flenter results round 2

[Pete Heist]

> On Nov 29, 2017, at 4:44 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
>> (That was also informative for me about how netperf decides when to
>> emit a data point…)
> 
> In that case I can add that the stated reason for this way of doing
> things is performance (i.e., emitting data points should not interfere
> with transfer performance). This is mostly an issue on systems where
> getting time is expensive; which is not the case on modern Linux
> systems. But I'm not entirely sure that the optimisation only has
> historical reasons; it may be that some systems supported by Netperf
> still has this issue...

Nice, I do respect the level of thought put into netperf…

Re: [Cake] cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 992 bytes --]

@Pete @Toke just saw your responses. Thank you very much for the
explanation. I will give irtt and flent-git a try, and maybe build an aur
package for archlinux.

George

On Wed, Nov 29, 2017 at 11:08 AM, Pete Heist <peteheist@gmail.com> wrote:

>
> > On Nov 29, 2017, at 4:44 PM, Toke Høiland-Jørgensen <toke@toke.dk>
> wrote:
> >
> >> (That was also informative for me about how netperf decides when to
> >> emit a data point…)
> >
> > In that case I can add that the stated reason for this way of doing
> > things is performance (i.e., emitting data points should not interfere
> > with transfer performance). This is mostly an issue on systems where
> > getting time is expensive; which is not the case on modern Linux
> > systems. But I'm not entirely sure that the optimisation only has
> > historical reasons; it may be that some systems supported by Netperf
> > still has this issue...
>
> Nice, I do respect the level of thought put into netperf…
>
>

[-- Attachment #2: Type: text/html, Size: 1436 bytes --]

Re: [Cake] cake flenter results round 2

[Toke Høiland-Jørgensen]

Georgios Amanakis <gamanakis@gmail.com> writes:

> @Pete @Toke just saw your responses. Thank you very much for the
> explanation. I will give irtt and flent-git a try, and maybe build an
> aur package for archlinux.

There's already an AUR package for flent-git; haven't gotten around to
making one for irtt, but am fine with you beating me to it ;)

-Toke

Re: [Cake] cake flenter results round 2

[Toke Høiland-Jørgensen]

Pete Heist <peteheist@gmail.com> writes:

>> On Nov 29, 2017, at 4:44 PM, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> 
>>> (That was also informative for me about how netperf decides when to
>>> emit a data point…)
>> 
>> In that case I can add that the stated reason for this way of doing
>> things is performance (i.e., emitting data points should not interfere
>> with transfer performance). This is mostly an issue on systems where
>> getting time is expensive; which is not the case on modern Linux
>> systems. But I'm not entirely sure that the optimisation only has
>> historical reasons; it may be that some systems supported by Netperf
>> still has this issue...
>
> Nice, I do respect the level of thought put into netperf…

Yup, netperf is the result of decades of optimisation, and widely
trusted, especially in the kernel community. Which is the main reason
why I chose it when first designing Flent (which was originally called
netperf-wrapper for this reason :)).

The flip side of this is that netperf does have its oddities and nooks
and crannies, and can feel arcane at times. And the code is somewhat
intimidating. So a clean slate tool like irtt definitely has its place
as well ;)

-Toke

Re: [Cake] cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 202 bytes --]

I just installed flent-git (thanks for the aur) and irtt, was more
straightforward than I expected :)
In my setup netserver runs on server, flent runs on client. Where should I
run irtt server?

George

[-- Attachment #2: Type: text/html, Size: 301 bytes --]

Re: [Cake] cake flenter results round 2

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 416 bytes --]

On Wed, Nov 29, 2017 at 5:24 PM Georgios Amanakis <gamanakis@gmail.com>
wrote:

> I just installed flent-git (thanks for the aur) and irtt, was more
> straightforward than I expected :)
> In my setup netserver runs on server, flent runs on client. Where should I
> run irtt server?
>

irtt server should be run on server as well. “irtt server” without any
options should be good enough for testing...

[-- Attachment #2: Type: text/html, Size: 680 bytes --]

Re: [Cake] cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

Results with irtt/flent-git. Same setup as before:

server      --      delay      --      mbox      --      client
netserver          50ms/50ms        45/900mbit

Cake config:
qdisc cake 801b: dev mbox.l root refcnt 2 bandwidth 45Mbit diffserv3
triple-isolate ack-filter-aggressive rtt 100.0ms noatm overhead 38
via-ethernet mpu 84
qdisc cake 801c: dev mbox.r root refcnt 2 bandwidth 900Mbit diffserv3
triple-isolate rtt 100.0ms noatm overhead 38 via-ethernet mpu 84

Results:
delay 100ms (rtt) flent:
https://drive.google.com/open?id=11N1QbRLJh8Akxh0iT1OKfzt7FCYutN9c
delay 100ms (rtt) stat:
https://drive.google.com/open?id=1QMaySbK5qW9IQcBZPgRSO4kNWTsQPDh8

[-- Attachment #2: Type: text/html, Size: 932 bytes --]

Re: [Cake] cake flenter results round 2

[Toke Høiland-Jørgensen]

Georgios Amanakis <gamanakis@gmail.com> writes:

> Results with irtt/flent-git. Same setup as before:

Looks like that is still using netperf for the UDP measurements. Flent
does some sanity checks on irtt before using it, which may be failing.
Running flent with -v should give some hints...

-Toke

[Cake] Fwd: cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 677 bytes --]

My fault, I had forgotten to include ~/go/bin in PATH.

New files:
delay 100ms (rtt) flent: https://drive.google.com/open?
id=13HAAvDtkBCPNu5FC61Cge_1vcd5LXuWY
delay 100ms (rtt) stat: https://drive.google.com/open?id=1THM-
F0MiuCntMz7lx3m7vA1FvHYZSNHC



On Wed, Nov 29, 2017 at 11:40 AM, Toke Høiland-Jørgensen <toke@toke.dk>
wrote:

> Georgios Amanakis <gamanakis@gmail.com> writes:
>
> > Results with irtt/flent-git. Same setup as before:
>
> Looks like that is still using netperf for the UDP measurements. Flent
> does some sanity checks on irtt before using it, which may be failing.
> Running flent with -v should give some hints...
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1458 bytes --]

Re: [Cake] cake flenter results round 2

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 3105 bytes --]

@Pete I think you need to start netserver on the client first (in your
case, you are running flent on the server): "ip netns exec client netserver"

On Tue, Nov 28, 2017 at 7:16 PM, Pete Heist <peteheist@gmail.com> wrote:

>
> > On Nov 28, 2017, at 11:52 PM, Dave Taht <dave@taht.net> wrote:
> >
> > A diffserv 200Mbit result would be good.
> >
> > We are utterly out of cpu at 900mbits here.
> >
> > <not_winning_on_cpu.png>
>
> Wow, I see flent’s combination plots are handy though.
>
> Stuff to sort in irtt also. Merely setting the source IP of an outgoing
> packet doubles allocated heap when it calls into x/net code. Super.
>
> Ok, here’s a quick veth try but I gotta get to sleep so I’m not
> investigating much ’til later. I wonder if 4.9.0-4 is enough:
>
> root@apu2a:/home/sysadmin/src/veth# cat settings.example
> export HOSTS="server client delay mbox"
>
> # I’ve been a bad boy and did sudo make install in iproute2-cake-next
> export TC=/sbin/tc
> root@apu2a:/home/sysadmin/src/veth# ./vsetup.sh
> Cannot remove namespace file "/var/run/netns/server": No such file or
> directory
> Cannot remove namespace file "/var/run/netns/client": No such file or
> directory
> Cannot remove namespace file "/var/run/netns/delay": No such file or
> directory
> Cannot remove namespace file "/var/run/netns/mbox": No such file or
> directory
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> RTNETLINK answers: File exists
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> net.ipv4.ip_forward = 1
> net.ipv6.conf.all.forwarding = 1
> root@apu2a:/home/sysadmin/src/veth# ./vcake.sh
> root@apu2a:/home/sysadmin/src/veth# ip netns exec server flent -H
> 10.10.2.2 rrul_be
> Started Flent 1.1.1-git-5daa2b3 using Python 2.7.13.
> Starting rrul_be test. Expected run time: 70 seconds.
> WARNING: Program exited non-zero.
> Runner class: NetperfDemoRunner
> Command: /usr/bin/netperf -P 0 -v 0 -D -0.20 -4 -Y CS0,CS0 -H 10.10.2.2 -p
> 12865 -t UDP_RR -l 70 -F /dev/urandom    --   -H 10.10.2.2 -k
> THROUGHPUT,LOCAL_CONG_CONTROL,REMOTE_CONG_CONTROL,TRANSPORT_
> MSS,LOCAL_TRANSPORT_RETRANS,REMOTE_TRANSPORT_RETRANS,
> LOCAL_SOCKET_TOS,REMOTE_SOCKET_TOS,DIRECTION,ELAPSED_
> TIME,PROTOCOL,LOCAL_SEND_SIZE,LOCAL_RECV_SIZE,REMOTE_SEND_
> SIZE,REMOTE_RECV_SIZE
> Return code: 255
> Stdout: establish control: are you sure there is a netserver listening on
> 10.10.2.2 at port 12865?
> establish_control could not establish the control connection from 0.0.0.0
> port 0 address family AF_INET to 10.10.2.2 port 12865 address family AF_INET
>
> ...
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>

[-- Attachment #2: Type: text/html, Size: 3876 bytes --]

Re: [Cake] cake flenter results round 2

[Pete Heist]

> On Nov 29, 2017, at 4:42 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
> 
> @Pete I think you need to start netserver on the client first (in your case, you are running flent on the server): "ip netns exec client netserver"

Thanks. I think I’m going to have more success testing at much lower rates than you on my hardware. At 100/1000 the tcp totals end up at around 84/202.

It will be interesting to see if I can get a whole flenter run to work, but for sanity I’ll start at something like the setup in qdisc_vdsl.sh…

Re: [Cake] cake flenter results round 2

[Pete Heist]

> On Nov 29, 2017, at 9:19 AM, Pete Heist <peteheist@gmail.com> wrote:
> 
>> On Nov 29, 2017, at 4:42 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
>> 
>> @Pete I think you need to start netserver on the client first (in your case, you are running flent on the server): "ip netns exec client netserver"
> 
> Thanks. I think I’m going to have more success testing at much lower rates than you on my hardware. At 100/1000 the tcp totals end up at around 84/202.
> 
> It will be interesting to see if I can get a whole flenter run to work, but for sanity I’ll start at something like the setup in qdisc_vdsl.sh…

Flenter is running now. I had to change the setup to add another box because flenter was written with p2p connections in mind with routers (middle boxes) at each end of the link and symmetric bandwidths, at least in the roles I’m using. So my topology is now (matching flenter roles):

cli - crt - delay - srt - srv

I’m still figuring out how this will work. I only use netem on the delay box, with rate 20mbit delay 10ms on both delay.l and delay.r:

root@apu2a:/home/sysadmin/src/veth# ip netns exec delay tc qdisc
qdisc noqueue 0: dev lo root refcnt 2 
qdisc netem 1: dev delay.r root refcnt 2 limit 1000 delay 10.0ms rate 20Mbit
qdisc netem 1: dev delay.l root refcnt 2 limit 1000 delay 10.0ms rate 20Mbit

Naturally, it will become the bottleneck link. Only the tests with rate limiting below that will be useful. But while rtt is as I expect...

root@apu2a:/home/sysadmin/src/veth# ip netns exec cli ping 10.10.0.1
PING 10.10.0.1 (10.10.0.1) 56(84) bytes of data.
64 bytes from 10.10.0.1: icmp_seq=1 ttl=61 time=20.3 ms
64 bytes from 10.10.0.1: icmp_seq=2 ttl=61 time=20.2 ms

bandwidth is not, so it looks like I’d have to use tbf or htb if I want hard limits:

root@apu2a:/home/sysadmin/src/veth# ip netns exec cli iperf3 -c 10.10.0.1
Connecting to host 10.10.0.1, port 5201
[  4] local 10.10.3.100 port 51222 connected to 10.10.0.1 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.59 MBytes  13.4 Mbits/sec    1   45.2 KBytes       
[  4]   1.00-2.00   sec  3.04 MBytes  25.5 Mbits/sec    0   80.6 KBytes       
[  4]   2.00-3.00   sec  4.47 MBytes  37.5 Mbits/sec    0    115 KBytes       
[  4]   3.00-4.00   sec  5.65 MBytes  47.4 Mbits/sec    0    147 KBytes       
[  4]   4.00-5.00   sec  6.84 MBytes  57.3 Mbits/sec    0    178 KBytes       
[  4]   5.00-6.00   sec  7.95 MBytes  66.7 Mbits/sec    0    209 KBytes       
[  4]   6.00-7.00   sec  9.01 MBytes  75.6 Mbits/sec    0    240 KBytes       
[  4]   7.00-8.00   sec  10.3 MBytes  86.0 Mbits/sec    0    296 KBytes       
[  4]   8.00-9.00   sec  13.4 MBytes   113 Mbits/sec    0    417 KBytes       
[  4]   9.00-10.00  sec  17.5 MBytes   146 Mbits/sec    0    568 KBytes       

Anyway I’ll post the run later and we’ll see…