[Cake] cake flenter results round 3

[Cake] cake flenter results round 3

[Pete Heist]

[-- Attachment #1: Type: text/plain, Size: 1836 bytes --]

http://www.drhleny.cz/bufferbloat/cake/round3/ <http://www.drhleny.cz/bufferbloat/cake/round3/>

Round 3 Tarball: http://www.drhleny.cz/bufferbloat/cake/round3.tgz <http://www.drhleny.cz/bufferbloat/cake/round3.tgz>

*** Notes/Analysis ***

* And another couple hundred tests showing Cake doesn’t crash, at 20/20mbit either!

* I’m now using netns (thanks Dave/Georgios) with a five box setup: client, client router, delay, server router and server. Delay is set up to simulate a 20mbit/20mbit symmetric connection with “bandwidth 20mbit delay 10ms” on each of its two interfaces. That apparently doesn’t hard limit the bandwidth, but no matter, I do get a 20ms RTT as expected, and any tests with rate limiting use a default limit of 18mbit, which seems to control the queue well. The unlimited tests can probably be ignored.

* Now that my CPU can actually run this test, Cake shows good results vs fq_codel on 32/32 nflows:

http://www.drhleny.cz/bufferbloat/cake/round3/nflows_32_32_eg_fq_codel_18.0mbit/index.html
http://www.drhleny.cz/bufferbloat/cake/round3/nflows_32_32_eg_cake_18.0mbit/index.html

* OWD, IPDV (jitter) and lost packets looking good for a VoIP conversation with dual-whatever. The VoIP conversation is simulated along with a normal RRUL test (4/4 flows), a torrent, and tcp_12down simultaneously (family of four?)

http://www.drhleny.cz/bufferbloat/cake/round3/flowiso_mix_eg_cake_dsrc_cake_ddst_18.0mbit/voip-voip_delay_loss.svg

*** Round 4 Plans

* All I can do easily with this setup without more effort is vary rtt, bandwidth, loss or whatever else netem can do. Or I can add other tests with spreads of parameters. Sorry, no asymmetric rates or useful ack filtering tests. Anyone care for high rtt tests, low bandwidth tests like 2/2mbit, or anything else in particular?


[-- Attachment #2: Type: text/html, Size: 2868 bytes --]

Re: [Cake] cake flenter results round 3

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 292 bytes --]

High RTT with relatively high bandwidth is traditionally considered
challenging, and is representative of Australasian and satellite
connections.  Definitely do some of those.

Low bandwidth should also be interesting.  DSL at 4/1 Mbit is typical entry
level package here.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 353 bytes --]

Re: [Cake] cake flenter results round 3

[Dave Taht]

On Wed, Nov 29, 2017 at 3:18 PM, Pete Heist <peteheist@gmail.com> wrote:
> http://www.drhleny.cz/bufferbloat/cake/round3/
>
> Round 3 Tarball: http://www.drhleny.cz/bufferbloat/cake/round3.tgz
>
> *** Notes/Analysis ***
>
> * And another couple hundred tests showing Cake doesn’t crash, at 20/20mbit
> either!
>
> * I’m now using netns (thanks Dave/Georgios) with a five box setup: client,
> client router, delay, server router and server. Delay is set up to simulate
> a 20mbit/20mbit symmetric connection with “bandwidth 20mbit delay 10ms” on
> each of its two interfaces. That apparently doesn’t hard limit the
> bandwidth, but no matter, I do get a 20ms RTT as expected, and any tests
> with rate limiting use a default limit of 18mbit, which seems to control the
> queue well. The unlimited tests can probably be ignored.
>
> * Now that my CPU can actually run this test, Cake shows good results vs
> fq_codel on 32/32 nflows:
>
> http://www.drhleny.cz/bufferbloat/cake/round3/nflows_32_32_eg_fq_codel_18.0mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round3/nflows_32_32_eg_cake_18.0mbit/index.html
>
> * OWD, IPDV (jitter) and lost packets looking good for a VoIP conversation
> with dual-whatever. The VoIP conversation is simulated along with a normal
> RRUL test (4/4 flows), a torrent, and tcp_12down simultaneously (family of
> four?)
>
> http://www.drhleny.cz/bufferbloat/cake/round3/flowiso_mix_eg_cake_dsrc_cake_ddst_18.0mbit/voip-voip_delay_loss.svg
>
> *** Round 4 Plans
>
> * All I can do easily with this setup without more effort is vary rtt,
> bandwidth, loss or whatever else netem can do. Or I can add other tests with
> spreads of parameters. Sorry, no asymmetric rates or useful ack filtering
> tests. Anyone care for high rtt tests, low bandwidth tests like 2/2mbit, or
> anything else in particular?

well, he says, shyly, there's a new option to netem, called slotting.
delete the rate, "slot 4ms 4ms bytes 10k 16"

>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 884 bytes --]

I gave high RTT with high bandwidth a try:
server   --   delay  --  mbox  --  client
netserver   300/300ms  45/900mbit  flent

I had to run flent with "-s 0.61" in order to avoid errors with fping dying
prematurely.
Comparing noack, ack, and ack-aggressive, ack seems to give the highest
download rates.
Cake was setup with "rtt 600ms".

George

On Wed, Nov 29, 2017 at 6:25 PM, Jonathan Morton <chromatix99@gmail.com>
wrote:

> High RTT with relatively high bandwidth is traditionally considered
> challenging, and is representative of Australasian and satellite
> connections.  Definitely do some of those.
>
> Low bandwidth should also be interesting.  DSL at 4/1 Mbit is typical
> entry level package here.
>
> - Jonathan Morton
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>
>

[-- Attachment #1.2: Type: text/html, Size: 1620 bytes --]

[-- Attachment #2: rrul_delay600ms_ack_noack.tgz --]
[-- Type: application/x-gzip, Size: 243475 bytes --]

[-- Attachment #3: totals.png --]
[-- Type: image/png, Size: 126979 bytes --]

Re: [Cake] cake flenter results round 3

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 118 bytes --]

I'd like to see variations using Cake's "internet", "oceanic" and
"satellite" presets at this RTT.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 195 bytes --]

Re: [Cake] cake flenter results round 3

[Dave Taht]

Georgios Amanakis <gamanakis@gmail.com> writes:

> I gave high RTT with high bandwidth a try:
> server -- delay -- mbox -- client
> netserver 300/300ms 45/900mbit flent

Neat.

An actual satellite link would be interesting to test against, as
whatever we are trying here beats against whatever they have already in
place.



>
> I had to run flent with "-s 0.61" in order to avoid errors with fping dying
> prematurely.
> Comparing noack, ack, and ack-aggressive, ack seems to give the highest download
> rates.
> Cake was setup with "rtt 600ms".
>
> George
>
> On Wed, Nov 29, 2017 at 6:25 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>     High RTT with relatively high bandwidth is traditionally considered
>     challenging, and is representative of Australasian and satellite
>     connections. Definitely do some of those.
>
>     Low bandwidth should also be interesting. DSL at 4/1 Mbit is typical entry
>     level package here.
>
>     - Jonathan Morton
>     
>
>     _______________________________________________
>     Cake mailing list
>     Cake@lists.bufferbloat.net
>     https://lists.bufferbloat.net/listinfo/cake
>     
>     
>
>
>
>
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 462 bytes --]

Same setup as above:
server   --   delay  --  mbox  --  client
netserver   300/300ms  45/900mbit  flent

Variation between internet, oceanic, and satellite. No ack filtering.
Upload in "internet" (rtt 100ms) is somewhat lower than the others.

George


On Thu, Nov 30, 2017 at 12:22 PM, Jonathan Morton <chromatix99@gmail.com>
wrote:

> I'd like to see variations using Cake's "internet", "oceanic" and
> "satellite" presets at this RTT.
>
> - Jonathan Morton
>

[-- Attachment #1.2: Type: text/html, Size: 1038 bytes --]

[-- Attachment #2: totals.png --]
[-- Type: image/png, Size: 121164 bytes --]

[-- Attachment #3: rrul_delay600ms_cakertt.tgz --]
[-- Type: application/x-gzip, Size: 236751 bytes --]

Re: [Cake] cake flenter results round 3

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 625 bytes --]

Looks okay - but now with ack filtering?

- Jonathan Morton

On 30 Nov 2017 20:07, "Georgios Amanakis" <gamanakis@gmail.com> wrote:

> Same setup as above:
> server   --   delay  --  mbox  --  client
> netserver   300/300ms  45/900mbit  flent
>
> Variation between internet, oceanic, and satellite. No ack filtering.
> Upload in "internet" (rtt 100ms) is somewhat lower than the others.
>
> George
>
>
> On Thu, Nov 30, 2017 at 12:22 PM, Jonathan Morton <chromatix99@gmail.com>
> wrote:
>
>> I'd like to see variations using Cake's "internet", "oceanic" and
>> "satellite" presets at this RTT.
>>
>> - Jonathan Morton
>>
>
>

[-- Attachment #2: Type: text/html, Size: 1495 bytes --]

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 414 bytes --]

I would like to point out what Pete said:
in my setup the mbox shapes both uplink and downlink with cake, which is
mostly untrue in the real world since the router has no control of the
downlink.

How can we simulate this better?


On Thu, Nov 30, 2017 at 1:13 PM, Jonathan Morton <chromatix99@gmail.com>
wrote:

> Looks okay - but now with ack filtering?
>
> - Jonathan Morton
>
>
I am getting to that :)

George

[-- Attachment #2: Type: text/html, Size: 979 bytes --]

Re: [Cake] cake flenter results round 3

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 120 bytes --]

If you use netem to simulate the physical link, you could test Cake's
ingress mode downstream of it.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 163 bytes --]

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 334 bytes --]

Could you provide an example how to rate limit with netem?
What does "slot 4ms 4ms bytes 10k 16" Dave provided do?

George

On Thu, Nov 30, 2017 at 1:27 PM, Jonathan Morton <chromatix99@gmail.com>
wrote:

> If you use netem to simulate the physical link, you could test Cake's
> ingress mode downstream of it.
>
> - Jonathan Morton
>

[-- Attachment #2: Type: text/html, Size: 769 bytes --]

Cannot Install cake qdisc to CentOS7

[Yutaka]

Hi, stuff.This is only report , and not any claim.

[ippan@localhost ~]$ cd sch_cake
[ippan@localhost sch_cake]$ ls
Makefile  cobalt.h  codel5_compat.h   pkt_sched.h
cobalt.c  codel5.h  gen_cake_const.c  sch_cake.c
[ippan@localhost sch_cake]$ sudo make ; make install
[sudo] ippan のパスワード:
make -C /lib/modules/3.10.0-693.5.2.el7.x86_64/build 
SUBDIRS=/home/ippan/sch_cake modules 
LDFLAGS_MODULE="--build-id=0xdfb2f6ccfbea7c413acec97d8440e344b94ca8e7" 
CFLAGS_MODULE="-DCAKE_VERSION=\\\"dfb2f6ccfbea7c413acec97d8440e344b94ca8e7\\\""
make: *** /lib/modules/3.10.0-693.5.2.el7.x86_64/build: 
そのようなファイルやディレクトリはありません.  中止.
make: *** [default] エラー 2
install -v -m 644 sch_cake.ko 
/lib/modules/3.10.0-693.5.2.el7.x86_64/kernel/net/sched/
install: `sch_cake.ko' を stat できません: 
そのようなファイルやディレクトリはありません
make: *** [install] エラー 1
[ippan@localhost sch_cake]$ make
make -C /lib/modules/3.10.0-693.5.2.el7.x86_64/build 
SUBDIRS=/home/ippan/sch_cake modules 
LDFLAGS_MODULE="--build-id=0xdfb2f6ccfbea7c413acec97d8440e344b94ca8e7" 
CFLAGS_MODULE="-DCAKE_VERSION=\\\"dfb2f6ccfbea7c413acec97d8440e344b94ca8e7\\\""
make: *** /lib/modules/3.10.0-693.5.2.el7.x86_64/build: 
そのようなファイルやディレクトリはありません.  中止.
make: *** [default] エラー 2
[ippan@localhost sch_cake]$ make
make -C /lib/modules/3.10.0-693.5.2.el7.x86_64/build 
SUBDIRS=/home/ippan/sch_cake modules 
LDFLAGS_MODULE="--build-id=0xdfb2f6ccfbea7c413acec97d8440e344b94ca8e7" 
CFLAGS_MODULE="-DCAKE_VERSION=\\\"dfb2f6ccfbea7c413acec97d8440e344b94ca8e7\\\""
make[1]: ディレクトリ `/usr/src/kernels/3.10.0-693.5.2.el7.x86_64' 
に入ります
   CC [M]  /home/ippan/sch_cake/sch_cake.o
/home/ippan/sch_cake/sch_cake.c:60:27: 致命的エラー: net/flow_keys.h: 
そのようなファイルやディレクトリはありません
  #include <net/flow_keys.h>
                            ^
コンパイルを停止しました。
make[2]: *** [/home/ippan/sch_cake/sch_cake.o] エラー 1
make[1]: *** [_module_/home/ippan/sch_cake] エラー 2
make[1]: ディレクトリ `/usr/src/kernels/3.10.0-693.5.2.el7.x86_64' 
から出ます
make: *** [default] エラー 2

Yutaka.

Re: [Cake] Cannot Install cake qdisc to CentOS7

[Yutaka]

On 2017年12月01日 04:33, Georgios Amanakis wrote:
> You are using a very old kernel: 3.10.0-693.5.2.el7.x86_64
> Is there any change to upgrade to a newer one?
> 
> George
> 
> On Thu, Nov 30, 2017 at 2:27 PM, Yutaka via Cake 
> <cake@lists.bufferbloat.net <mailto:cake@lists.bufferbloat.net>> wrote:
> 
>     _______________________________________________
>     Cake mailing list
>     Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>     https://lists.bufferbloat.net/listinfo/cake
>     <https://lists.bufferbloat.net/listinfo/cake>
> 
> 
>     ---------- Forwarded message ----------
>     From: Yutaka <intruder_tkyf@yahoo.fr <mailto:intruder_tkyf@yahoo.fr>>
>     To: cake@lists.bufferbloat.net <mailto:cake@lists.bufferbloat.net>
>     Cc:
>     Bcc:
>     Date: Fri, 1 Dec 2017 04:26:59 +0900
>     Subject: Cannot Install cake qdisc to CentOS7
>     Hi, stuff.This is only report , and not any claim.
> 
>     [ippan@localhost ~]$ cd sch_cake
>     [ippan@localhost sch_cake]$ ls
>     Makefile  cobalt.h  codel5_compat.h   pkt_sched.h
>     cobalt.c  codel5.h  gen_cake_const.c  sch_cake.c
>     [ippan@localhost sch_cake]$ sudo make ; make install
>     [sudo] ippan のパスワード:
>     make -C /lib/modules/3.10.0-693.5.2.el7.x86_64/build
>     SUBDIRS=/home/ippan/sch_cake modules
>     LDFLAGS_MODULE="--build-id=0xdfb2f6ccfbea7c413acec97d8440e344b94ca8e7"
>     CFLAGS_MODULE="-DCAKE_VERSION=\\\"dfb2f6ccfbea7c413acec97d8440e344b94ca8e7\\\""
>     make: *** /lib/modules/3.10.0-693.5.2.el7.x86_64/build: そのような
>     ファイルやディレクトリはありません.  中止.
>     make: *** [default] エラー 2
>     install -v -m 644 sch_cake.ko
>     /lib/modules/3.10.0-693.5.2.el7.x86_64/kernel/net/sched/
>     install: `sch_cake.ko' を stat できません: そのようなファイルやディ
>     レクトリはありません
>     make: *** [install] エラー 1
>     [ippan@localhost sch_cake]$ make
>     make -C /lib/modules/3.10.0-693.5.2.el7.x86_64/build
>     SUBDIRS=/home/ippan/sch_cake modules
>     LDFLAGS_MODULE="--build-id=0xdfb2f6ccfbea7c413acec97d8440e344b94ca8e7"
>     CFLAGS_MODULE="-DCAKE_VERSION=\\\"dfb2f6ccfbea7c413acec97d8440e344b94ca8e7\\\""
>     make: *** /lib/modules/3.10.0-693.5.2.el7.x86_64/build: そのような
>     ファイルやディレクトリはありません.  中止.
>     make: *** [default] エラー 2
>     [ippan@localhost sch_cake]$ make
>     make -C /lib/modules/3.10.0-693.5.2.el7.x86_64/build
>     SUBDIRS=/home/ippan/sch_cake modules
>     LDFLAGS_MODULE="--build-id=0xdfb2f6ccfbea7c413acec97d8440e344b94ca8e7"
>     CFLAGS_MODULE="-DCAKE_VERSION=\\\"dfb2f6ccfbea7c413acec97d8440e344b94ca8e7\\\""
>     make[1]: ディレクトリ `/usr/src/kernels/3.10.0-693.5.2.el7.x86_64'
>     に入ります
>        CC [M]  /home/ippan/sch_cake/sch_cake.o
>     /home/ippan/sch_cake/sch_cake.c:60:27: 致命的エラー:
>     net/flow_keys.h: そのようなファイルやディレクトリはありません
>       #include <net/flow_keys.h>
>                                 ^
>     コンパイルを停止しました。
>     make[2]: *** [/home/ippan/sch_cake/sch_cake.o] エラー 1
>     make[1]: *** [_module_/home/ippan/sch_cake] エラー 2
>     make[1]: ディレクトリ `/usr/src/kernels/3.10.0-693.5.2.el7.x86_64'
>     から出ます
>     make: *** [default] エラー 2
> 
>     Yutaka.
> 
> 
> 

Re: [Cake] Cannot Install cake qdisc to CentOS7

[Yutaka]

Thank you George, I will try to think to change.
Yutaka.

On 2017年12月01日 04:33, Georgios Amanakis wrote:
> You are using a very old kernel: 3.10.0-693.5.2.el7.x86_64
> Is there any change to upgrade to a newer one?
> 
> George
> 
> On Thu, Nov 30, 2017 at 2:27 PM, Yutaka via Cake 
> <cake@lists.bufferbloat.net <mailto:cake@lists.bufferbloat.net>> wrote:
> 
>     _______________________________________________
>     Cake mailing list
>     Cake@lists.bufferbloat.net <mailto:Cake@lists.bufferbloat.net>
>     https://lists.bufferbloat.net/listinfo/cake
>     <https://lists.bufferbloat.net/listinfo/cake>
> 
> 
>     ---------- Forwarded message ----------
>     From: Yutaka <intruder_tkyf@yahoo.fr <mailto:intruder_tkyf@yahoo.fr>>
>     To: cake@lists.bufferbloat.net <mailto:cake@lists.bufferbloat.net>
>     Cc:
>     Bcc:
>     Date: Fri, 1 Dec 2017 04:26:59 +0900
>     Subject: Cannot Install cake qdisc to CentOS7
>     Hi, stuff.This is only report , and not any claim.
> 
>     [ippan@localhost ~]$ cd sch_cake
>     [ippan@localhost sch_cake]$ ls
>     Makefile  cobalt.h  codel5_compat.h   pkt_sched.h
>     cobalt.c  codel5.h  gen_cake_const.c  sch_cake.c
>     [ippan@localhost sch_cake]$ sudo make ; make install
>     [sudo] ippan のパスワード:
>     make -C /lib/modules/3.10.0-693.5.2.el7.x86_64/build
>     SUBDIRS=/home/ippan/sch_cake modules
>     LDFLAGS_MODULE="--build-id=0xdfb2f6ccfbea7c413acec97d8440e344b94ca8e7"
>     CFLAGS_MODULE="-DCAKE_VERSION=\\\"dfb2f6ccfbea7c413acec97d8440e344b94ca8e7\\\""
>     make: *** /lib/modules/3.10.0-693.5.2.el7.x86_64/build: そのような
>     ファイルやディレクトリはありません.  中止.
>     make: *** [default] エラー 2
>     install -v -m 644 sch_cake.ko
>     /lib/modules/3.10.0-693.5.2.el7.x86_64/kernel/net/sched/
>     install: `sch_cake.ko' を stat できません: そのようなファイルやディ
>     レクトリはありません
>     make: *** [install] エラー 1
>     [ippan@localhost sch_cake]$ make
>     make -C /lib/modules/3.10.0-693.5.2.el7.x86_64/build
>     SUBDIRS=/home/ippan/sch_cake modules
>     LDFLAGS_MODULE="--build-id=0xdfb2f6ccfbea7c413acec97d8440e344b94ca8e7"
>     CFLAGS_MODULE="-DCAKE_VERSION=\\\"dfb2f6ccfbea7c413acec97d8440e344b94ca8e7\\\""
>     make: *** /lib/modules/3.10.0-693.5.2.el7.x86_64/build: そのような
>     ファイルやディレクトリはありません.  中止.
>     make: *** [default] エラー 2
>     [ippan@localhost sch_cake]$ make
>     make -C /lib/modules/3.10.0-693.5.2.el7.x86_64/build
>     SUBDIRS=/home/ippan/sch_cake modules
>     LDFLAGS_MODULE="--build-id=0xdfb2f6ccfbea7c413acec97d8440e344b94ca8e7"
>     CFLAGS_MODULE="-DCAKE_VERSION=\\\"dfb2f6ccfbea7c413acec97d8440e344b94ca8e7\\\""
>     make[1]: ディレクトリ `/usr/src/kernels/3.10.0-693.5.2.el7.x86_64'
>     に入ります
>        CC [M]  /home/ippan/sch_cake/sch_cake.o
>     /home/ippan/sch_cake/sch_cake.c:60:27: 致命的エラー:
>     net/flow_keys.h: そのようなファイルやディレクトリはありません
>       #include <net/flow_keys.h>
>                                 ^
>     コンパイルを停止しました。
>     make[2]: *** [/home/ippan/sch_cake/sch_cake.o] エラー 1
>     make[1]: *** [_module_/home/ippan/sch_cake] エラー 2
>     make[1]: ディレクトリ `/usr/src/kernels/3.10.0-693.5.2.el7.x86_64'
>     から出ます
>     make: *** [default] エラー 2
> 
>     Yutaka.
> 
> 
> 

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 918 bytes --]

Again, same setup as previously:
server   --   delay  --  mbox  --  client
netserver   300/300ms  45/900mbit  flent

This time variations of both cake rtt and ack filtering.

George

On Thu, Nov 30, 2017 at 1:13 PM, Jonathan Morton <chromatix99@gmail.com>
wrote:

> Looks okay - but now with ack filtering?
>
> - Jonathan Morton
>
> On 30 Nov 2017 20:07, "Georgios Amanakis" <gamanakis@gmail.com> wrote:
>
>> Same setup as above:
>> server   --   delay  --  mbox  --  client
>> netserver   300/300ms  45/900mbit  flent
>>
>> Variation between internet, oceanic, and satellite. No ack filtering.
>> Upload in "internet" (rtt 100ms) is somewhat lower than the others.
>>
>> George
>>
>>
>> On Thu, Nov 30, 2017 at 12:22 PM, Jonathan Morton <chromatix99@gmail.com>
>> wrote:
>>
>>> I'd like to see variations using Cake's "internet", "oceanic" and
>>> "satellite" presets at this RTT.
>>>
>>> - Jonathan Morton
>>>
>>
>>

[-- Attachment #1.2: Type: text/html, Size: 2358 bytes --]

[-- Attachment #2: totals_ack.png --]
[-- Type: image/png, Size: 124519 bytes --]

[-- Attachment #3: totals_noack.png --]
[-- Type: image/png, Size: 127079 bytes --]

[-- Attachment #4: totals_ackaggr.png --]
[-- Type: image/png, Size: 131391 bytes --]

[-- Attachment #5: rrul_delay600ms_cakertt_ack.tgz --]
[-- Type: application/x-gzip, Size: 724864 bytes --]

Re: [Cake] cake flenter results round 3

[Toke Høiland-Jørgensen]

Georgios Amanakis <gamanakis@gmail.com> writes:

> I had to run flent with "-s 0.61" in order to avoid errors with fping dying
> prematurely.

When did fping die, and did you still get results? There's a watchdog
timer to keep it from running too long, but maybe that is a bit too
aggressive?

-Toke

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 1128 bytes --]

I reran the last test two more times, with slightly different results.

George

On Thu, Nov 30, 2017 at 3:17 PM, Georgios Amanakis <gamanakis@gmail.com>
wrote:

> Again, same setup as previously:
> server   --   delay  --  mbox  --  client
> netserver   300/300ms  45/900mbit  flent
>
> This time variations of both cake rtt and ack filtering.
>
> George
>
>
> On Thu, Nov 30, 2017 at 1:13 PM, Jonathan Morton <chromatix99@gmail.com>
> wrote:
>
>> Looks okay - but now with ack filtering?
>>
>> - Jonathan Morton
>>
>> On 30 Nov 2017 20:07, "Georgios Amanakis" <gamanakis@gmail.com> wrote:
>>
>>> Same setup as above:
>>> server   --   delay  --  mbox  --  client
>>> netserver   300/300ms  45/900mbit  flent
>>>
>>> Variation between internet, oceanic, and satellite. No ack filtering.
>>> Upload in "internet" (rtt 100ms) is somewhat lower than the others.
>>>
>>> George
>>>
>>>
>>> On Thu, Nov 30, 2017 at 12:22 PM, Jonathan Morton <chromatix99@gmail.com
>>> > wrote:
>>>
>>>> I'd like to see variations using Cake's "internet", "oceanic" and
>>>> "satellite" presets at this RTT.
>>>>
>>>> - Jonathan Morton
>>>>
>>>
>>>
>

[-- Attachment #1.2: Type: text/html, Size: 2973 bytes --]

[-- Attachment #2: totals_ack_2.png --]
[-- Type: image/png, Size: 117596 bytes --]

[-- Attachment #3: totals_noack_2.png --]
[-- Type: image/png, Size: 116121 bytes --]

[-- Attachment #4: totals_ackaggr_2.png --]
[-- Type: image/png, Size: 120255 bytes --]

[-- Attachment #5: totals_ack_3.png --]
[-- Type: image/png, Size: 114414 bytes --]

[-- Attachment #6: totals_noack_3.png --]
[-- Type: image/png, Size: 119766 bytes --]

[-- Attachment #7: totals_ackaggr_3.png --]
[-- Type: image/png, Size: 125070 bytes --]

[-- Attachment #8: rrul_delay600ms_cakertt_ack_2.tgz --]
[-- Type: application/x-gzip, Size: 696155 bytes --]

[-- Attachment #9: rrul_delay600ms_cakertt_ack_3.tgz --]
[-- Type: application/x-gzip, Size: 699547 bytes --]

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 1218 bytes --]

I still got results but did not trust them.
The error occurred at the end of the run.

===========8<===========
NetperfDemoRunner TCP upload CS5 finished
NetperfDemoRunner TCP upload EF finished
TimerRunner Watchdog [Ping (ms) ICMP] finished
NetperfDemoRunner TCP upload BK finished
WARNING: Program exited non-zero.
Runner class: PingRunner
Command: /usr/bin/fping  -D -p 200 -c 350   10.10.0.1
Return code: 1
Stdout:
Stderr:
10.10.0.1 : xmt/rcv/%loss = 350/0/100%

WARNING: Command produced no valid data.
Runner class: PingRunner
Command: /usr/bin/fping  -D -p 200 -c 350   10.10.0.1
Return code: 1
Stdout:
Stderr:
10.10.0.1 : xmt/rcv/%loss = 350/0/100%

PingRunner Ping (ms) ICMP finished
CpuStatsRunner cpu_stats_localhost finished
===========8<===========

George

On Thu, Nov 30, 2017 at 4:03 PM, Toke Høiland-Jørgensen <toke@toke.dk>
wrote:

> Georgios Amanakis <gamanakis@gmail.com> writes:
>
> > I had to run flent with "-s 0.61" in order to avoid errors with fping
> dying
> > prematurely.
>
> When did fping die, and did you still get results? There's a watchdog
> timer to keep it from running too long, but maybe that is a bit too
> aggressive?
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1775 bytes --]

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 1664 bytes --]

To correct myself, fping did not die prematurely, it was getting no
responses from the remote host because its p parameter was too short at
default for an RTT of 600ms. After looking at the source, I realized that
it is affected but the s parameter of flent, and changed that instead.

On Nov 30, 2017 4:11 PM, "Georgios Amanakis" <gamanakis@gmail.com> wrote:

> I still got results but did not trust them.
> The error occurred at the end of the run.
>
> ===========8<===========
> NetperfDemoRunner TCP upload CS5 finished
> NetperfDemoRunner TCP upload EF finished
> TimerRunner Watchdog [Ping (ms) ICMP] finished
> NetperfDemoRunner TCP upload BK finished
> WARNING: Program exited non-zero.
> Runner class: PingRunner
> Command: /usr/bin/fping  -D -p 200 -c 350   10.10.0.1
> Return code: 1
> Stdout:
> Stderr:
> 10.10.0.1 : xmt/rcv/%loss = 350/0/100%
>
> WARNING: Command produced no valid data.
> Runner class: PingRunner
> Command: /usr/bin/fping  -D -p 200 -c 350   10.10.0.1
> Return code: 1
> Stdout:
> Stderr:
> 10.10.0.1 : xmt/rcv/%loss = 350/0/100%
>
> PingRunner Ping (ms) ICMP finished
> CpuStatsRunner cpu_stats_localhost finished
> ===========8<===========
>
> George
>
> On Thu, Nov 30, 2017 at 4:03 PM, Toke Høiland-Jørgensen <toke@toke.dk>
> wrote:
>
>> Georgios Amanakis <gamanakis@gmail.com> writes:
>>
>> > I had to run flent with "-s 0.61" in order to avoid errors with fping
>> dying
>> > prematurely.
>>
>> When did fping die, and did you still get results? There's a watchdog
>> timer to keep it from running too long, but maybe that is a bit too
>> aggressive?
>>
>> -Toke
>>
>
>

[-- Attachment #2: Type: text/html, Size: 2444 bytes --]

Re: [Cake] cake flenter results round 3

[Toke Høiland-Jørgensen]

Georgios Amanakis <gamanakis@gmail.com> writes:

> I still got results but did not trust them.
> The error occurred at the end of the run.
>
> ===========8<===========
> NetperfDemoRunner TCP upload CS5 finished
> NetperfDemoRunner TCP upload EF finished
> TimerRunner Watchdog [Ping (ms) ICMP] finished

Yeah, the watchdog timer is firing...

> NetperfDemoRunner TCP upload BK finished
> WARNING: Program exited non-zero.
> Runner class: PingRunner
> Command: /usr/bin/fping  -D -p 200 -c 350   10.10.0.1
> Return code: 1
> Stdout:
> Stderr:
> 10.10.0.1 : xmt/rcv/%loss = 350/0/100%

However, I'm not sure the 100% packet loss is because of that (but
rather the reverse). Are you sure the other host is responding to pings?

-Toke

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

Yes it is. Tested with both ping and "fping -p610".

On Nov 30, 2017 4:32 PM, "Toke Høiland-Jørgensen" <toke@toke.dk> wrote:

> Georgios Amanakis <gamanakis@gmail.com> writes:
>
> > I still got results but did not trust them.
> > The error occurred at the end of the run.
> >
> > ===========8<===========
> > NetperfDemoRunner TCP upload CS5 finished
> > NetperfDemoRunner TCP upload EF finished
> > TimerRunner Watchdog [Ping (ms) ICMP] finished
>
> Yeah, the watchdog timer is firing...
>
> > NetperfDemoRunner TCP upload BK finished
> > WARNING: Program exited non-zero.
> > Runner class: PingRunner
> > Command: /usr/bin/fping  -D -p 200 -c 350   10.10.0.1
> > Return code: 1
> > Stdout:
> > Stderr:
> > 10.10.0.1 : xmt/rcv/%loss = 350/0/100%
>
> However, I'm not sure the 100% packet loss is because of that (but
> rather the reverse). Are you sure the other host is responding to pings?
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1371 bytes --]

Re: [Cake] cake flenter results round 3

[Toke Høiland-Jørgensen]

Georgios Amanakis <gamanakis@gmail.com> writes:

> To correct myself, fping did not die prematurely, it was getting no
> responses from the remote host because its p parameter was too short at
> default for an RTT of 600ms. After looking at the source, I realized that
> it is affected but the s parameter of flent, and changed that instead.

Ah, gotcha. That makes sense. So fping will ignore replies that arrive
after it sent the next packet? That's kinda odd... Does regular ping
behave the same way?

-Toke

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

I don't really know. When I ran "ping 10.10.0.1" it just worked. fping
-p250 didn't.

On Nov 30, 2017 4:38 PM, "Toke Høiland-Jørgensen" <toke@toke.dk> wrote:

> Georgios Amanakis <gamanakis@gmail.com> writes:
>
> > To correct myself, fping did not die prematurely, it was getting no
> > responses from the remote host because its p parameter was too short at
> > default for an RTT of 600ms. After looking at the source, I realized that
> > it is affected but the s parameter of flent, and changed that instead.
>
> Ah, gotcha. That makes sense. So fping will ignore replies that arrive
> after it sent the next packet? That's kinda odd... Does regular ping
> behave the same way?
>
> -Toke
>

[-- Attachment #2: Type: text/html, Size: 1087 bytes --]

Cannot install tc-adv(for cake) on Fedora 27

[Yutaka]

Hi , George and stuff.This is only report for advance.
Yutaka.

[ippan@localhost tc-adv]$ make
make[1]: ディレクトリ '/home/ippan/tc-adv/lib' に入ります
make[1]: 'all' に対して行うべき事はありません.
make[1]: ディレクトリ '/home/ippan/tc-adv/lib' から出ます
make[1]: ディレクトリ '/home/ippan/tc-adv/tc' に入ります
gcc -Wall -Wstrict-prototypes  -Wmissing-prototypes 
-Wmissing-declarations -Wold-style-definition -Wformat=2 -O2 
-I../include -DRESOLVE_HOSTNAMES -DLIBDIR=\"/usr/lib\" 
-DCONFDIR=\"/etc/iproute2\" -D_GNU_SOURCE  -DHAVE_SETNS -DHAVE_ELF 
-DCONFIG_GACT -DCONFIG_GACT_PROB -DYY_NO_INPUT   -c -o tc_core.o tc_core.c
tc_core.c: 関数 ‘tc_calc_size_table’ 内:
tc_core.c:189:29: エラー: ‘UINT16_MAX’ undeclared (first use in this 
function); did you mean ‘__INT16_MAX__’?
    if ((sz >> s->size_log) > UINT16_MAX) {
                              ^~~~~~~~~~
                              __INT16_MAX__
tc_core.c:189:29: 備考: 
未宣言の識別子は出現した各関数内で一回のみ報告されます
make[1]: *** [<ビルトイン>: tc_core.o] エラー 1
make[1]: ディレクトリ '/home/ippan/tc-adv/tc' から出ます
make: *** [Makefile:48: all] エラー 2

[ippan@localhost tc-adv]$ uname -a
Linux localhost.localdomain 4.13.15-300.fc27.x86_64 #1 SMP Tue Nov 21 
21:10:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[ippan@localhost tc-adv]$

Re: [Cake] Cannot install tc-adv(for cake) on Fedora 27

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 43 bytes --]

What's the gcc version?

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 86 bytes --]

Re: [Cake] Cannot install tc-adv(for cake) on Fedora 27

[Bret Towe]

[-- Attachment #1: Type: text/plain, Size: 376 bytes --]

it needs an include added
#include <std-int>
working from memory and away from my computer to verify

On Nov 30, 2017 5:40 PM, "Jonathan Morton" <chromatix99@gmail.com> wrote:

> What's the gcc version?
>
> - Jonathan Morton
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>
>

[-- Attachment #2: Type: text/html, Size: 911 bytes --]

Re: [Cake] Cannot install tc-adv(for cake) on Fedora 27

[Yutaka]

Hi , Jonathan.

gcc version is 7.2.1

Yutaka

名前         : gcc
バージョン   : 7.2.1
リリース     : 2.fc27
アーキテクチ : x86_64
サイズ       : 51 M
ソース       : gcc-7.2.1-2.fc27.src.rpm
リポジトリ   : @System
 From repo    : fedora
概要         : Various compilers (C, C++, Objective-C, Java, ...)
URL          : http://gcc.gnu.org
ライセンス   : GPLv3+ and GPLv3+ with exceptions and GPLv2+ with exceptions 
and
              : LGPLv2+ and BSD
説明         : The gcc package contains the GNU Compiler Collection 
version 7.
              : You'll need this package in order to compile C code.


On 2017年12月01日 10:40, Jonathan Morton wrote:
> What's the gcc version?
> 
> - Jonathan Morton
> 

Re: [Cake] Cannot install tc-adv(for cake) on Fedora 27

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 115 bytes --]

It would need stdint.h and limits.h.  It's possible one of those has been
left out of the code.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 162 bytes --]

Re: [Cake] Cannot install tc-adv(for cake) on Fedora 27

[Bret Towe]

[-- Attachment #1: Type: text/plain, Size: 230 bytes --]

here is the fix i had used

On Thu, Nov 30, 2017 at 7:32 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
> It would need stdint.h and limits.h.  It's possible one of those has been
> left out of the code.
>
> - Jonathan Morton

[-- Attachment #2: tc-adv-compile-fix.patch --]
[-- Type: text/x-patch, Size: 253 bytes --]

diff --git a/tc/tc_core.c b/tc/tc_core.c
index 46eaefb..eb925cc 100644
--- a/tc/tc_core.c
+++ b/tc/tc_core.c
@@ -12,6 +12,7 @@
 
 #include <stdio.h>
 #include <stdlib.h>
+#include <stdint.h>
 #include <unistd.h>
 #include <syslog.h>
 #include <fcntl.h>

Re: [Cake] cake flenter results round 3

[Toke Høiland-Jørgensen]

Georgios Amanakis <gamanakis@gmail.com> writes:

> I don't really know. When I ran "ping 10.10.0.1" it just worked. fping
> -p250 didn't.

What about ping -i0.2?

-Toke

Re: [Cake] cake flenter results round 3

[Dave Taht]

All this testing has inspired me to go pick up a cheap ($500!!!) 12 core
Xeon box I found on craigslist, this afternoon. I'm tired of napping
through kernel builds, and needed a space heater.

If anyone else could use a build box and sim station pls send me a ssh key.

I'd like to get some mips/arm/whatever qemu instances up also.

Assuming it proves reliable, I'll converge it with my 10TB nas for
packet captures.

Georgios Amanakis <gamanakis@gmail.com> writes:

> Could you provide an example how to rate limit with netem?

tc qdisc add dev eth0 root netem rate 20mbit

That however creates a tail-drop per-packet queue. You can attach a
sub-qdisc that more closely emulates the real link. Or use HTB. Or
leverage SQM.

Thing is, we've been living in a fantasyland (cake in both directions),
and I'm thinking at least in the downstream direction, attaching a CMTS
or DSLAM emulating tail drop queue would be useful (and depressingly, I
am thinking that ack filtering will lead to even worse bloat on an
un-caked head end).

Your typical Cisco CMTS seemed to be 1000 1.5k packets max.

Arris CMTSes (more common) - I can't tell. My best guess is that they have a
1000-2000 packet limit with GRO enabled, per subscriber, no matter the rate.

Another way to emulate a range of CMTSes is to put a bfifo limit 256k, 512k,
1024k.

DSLAMs have seemed to be in the 64k range.

> What does "slot 4ms 4ms bytes 10k 16" Dave provided do?

This is an attempt to emulate bursty mac (like wifi)'s behaviors.

It will accumulate 4ms worth of packets, then deliver them all in a
bunch. The "interesting" thing to me is the probable effect of the
microburst on cake's deficit scheduler (and new ack filter)

By additionally specifying bytes and packets you can get a max rate out
of it, otherwise it is bound by the speed of qdisc_watchdog_schedule_ns.

I goofed on the line above, it is actually:

slot 4ms 4ms bytes Xk packets Y

slot 4ms # minimum delay for a slot
     4ms # max delay for a slot (it is REALLY hard to reason about
     things with these set for realism[1], so 4ms 4ms is sane for rrul
     testing. A one way test might be 1ms 4ms, with the ack-mostly side
     being 1ms 1ms.)
     bytes 10k # deliver X kbytes per slot. In this case,
           10k * 1000/4ms per slot = ~25Mbit/sec
     packets Y - 802.11n does have a limit (up to 42) 



[1] worse, using 1ms-4ms would create a uniform distribution where the
real distribution is very different and long tailed. (that's work
pending)

>
> George
>
> On Thu, Nov 30, 2017 at 1:27 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>     If you use netem to simulate the physical link, you could test Cake's
>     ingress mode downstream of it.
>
>     - Jonathan Morton
>     
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake flenter results round 3

[Dave Taht]

Georgios Amanakis <gamanakis@gmail.com> writes:

> I would like to point out what Pete said:
> in my setup the mbox shapes both uplink and downlink with cake, which is mostly
> untrue in the real world since the router has no control of the downlink.
>
> How can we simulate this better? 

I just answered this (for cmts, dslam, and wifi) in another part of one
of these these threads.

My brain is beginning to explode with the permutations...

There's one other basic topology change worth having in "veth version 2"
- setting up 4 servers at different RTTs. And what the heck, 4 clients.

I'm running a bunch of errands today, (inc getting a new box). My
original intent was to try to upstream cake tonight, but what the heck,
let's beat it up some more. :)

In particular, I was expecting the sfq results to degrade a LOT at
higher RTTs, but haven't looked.

>
> On Thu, Nov 30, 2017 at 1:13 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>     Looks okay - but now with ack filtering?
>
>     - Jonathan Morton
>     
>
>     
>
> I am getting to that :) 
>
> George
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake flenter results round 3

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 2202 bytes --]

happy-joy-joy-making tcp rtt stats cake vs fq_codel and sfq



On Wed, Nov 29, 2017 at 3:18 PM, Pete Heist <peteheist@gmail.com> wrote:
> http://www.drhleny.cz/bufferbloat/cake/round3/
>
> Round 3 Tarball: http://www.drhleny.cz/bufferbloat/cake/round3.tgz
>
> *** Notes/Analysis ***
>
> * And another couple hundred tests showing Cake doesn’t crash, at 20/20mbit
> either!
>
> * I’m now using netns (thanks Dave/Georgios) with a five box setup: client,
> client router, delay, server router and server. Delay is set up to simulate
> a 20mbit/20mbit symmetric connection with “bandwidth 20mbit delay 10ms” on
> each of its two interfaces. That apparently doesn’t hard limit the
> bandwidth, but no matter, I do get a 20ms RTT as expected, and any tests
> with rate limiting use a default limit of 18mbit, which seems to control the
> queue well. The unlimited tests can probably be ignored.
>
> * Now that my CPU can actually run this test, Cake shows good results vs
> fq_codel on 32/32 nflows:
>
> http://www.drhleny.cz/bufferbloat/cake/round3/nflows_32_32_eg_fq_codel_18.0mbit/index.html
> http://www.drhleny.cz/bufferbloat/cake/round3/nflows_32_32_eg_cake_18.0mbit/index.html
>
> * OWD, IPDV (jitter) and lost packets looking good for a VoIP conversation
> with dual-whatever. The VoIP conversation is simulated along with a normal
> RRUL test (4/4 flows), a torrent, and tcp_12down simultaneously (family of
> four?)
>
> http://www.drhleny.cz/bufferbloat/cake/round3/flowiso_mix_eg_cake_dsrc_cake_ddst_18.0mbit/voip-voip_delay_loss.svg
>
> *** Round 4 Plans
>
> * All I can do easily with this setup without more effort is vary rtt,
> bandwidth, loss or whatever else netem can do. Or I can add other tests with
> spreads of parameters. Sorry, no asymmetric rates or useful ack filtering
> tests. Anyone care for high rtt tests, low bandwidth tests like 2/2mbit, or
> anything else in particular?
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

[-- Attachment #2: fq_codel_v_cake_32_32_flows_20mbit.png --]
[-- Type: image/png, Size: 570559 bytes --]

[-- Attachment #3: sfq_v_cake_32_32_flows_20mbit.png --]
[-- Type: image/png, Size: 2694200 bytes --]

Re: [Cake] cake flenter results round 3

[Dave Taht]

That's really hard to argue with. An fq_codel same conditions (and
pfifo and pie) might be interesting here.

On Thu, Nov 30, 2017 at 9:13 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
> I gave high RTT with high bandwidth a try:
> server   --   delay  --  mbox  --  client
> netserver   300/300ms  45/900mbit  flent
>
> I had to run flent with "-s 0.61" in order to avoid errors with fping dying
> prematurely.
> Comparing noack, ack, and ack-aggressive, ack seems to give the highest
> download rates.
> Cake was setup with "rtt 600ms".
>
> George
>
> On Wed, Nov 29, 2017 at 6:25 PM, Jonathan Morton <chromatix99@gmail.com>
> wrote:
>>
>> High RTT with relatively high bandwidth is traditionally considered
>> challenging, and is representative of Australasian and satellite
>> connections.  Definitely do some of those.
>>
>> Low bandwidth should also be interesting.  DSL at 4/1 Mbit is typical
>> entry level package here.
>>
>> - Jonathan Morton
>>
>>
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
>>
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 2292 bytes --]

cake vs sfq and fq_codel looks great! You already have fq_codel in the
graphs, do you mean codel?

In terms of simulating CMTS I am trying to simulate a cable connection.
Is the following reasonable?
===========8<===========
servers --- delay --- isp --- mbox --- clients

delay:
tc qdisc replace dev delay.r root netem delay ZZms rate XXXmbit limit 100000
tc qdisc replace dev delay.l root netem delay ZZms rate XXXmbit limit 100000

isp:
tc qdisc replace dev isp.l root bfifo limit 1024k
tc qdisc replace dev isp.r root bfifo limit 1024k

mbox:
tc qdisc replace dev mbox.l root cake bandwidth YYYmbit
tc qdisc replace dev mbox.r root cake bandwidth YYYmbit
===========8<===========

George

On Fri, Dec 1, 2017 at 2:23 PM, Dave Taht <dave.taht@gmail.com> wrote:

> That's really hard to argue with. An fq_codel same conditions (and
> pfifo and pie) might be interesting here.
>
> On Thu, Nov 30, 2017 at 9:13 AM, Georgios Amanakis <gamanakis@gmail.com>
> wrote:
> > I gave high RTT with high bandwidth a try:
> > server   --   delay  --  mbox  --  client
> > netserver   300/300ms  45/900mbit  flent
> >
> > I had to run flent with "-s 0.61" in order to avoid errors with fping
> dying
> > prematurely.
> > Comparing noack, ack, and ack-aggressive, ack seems to give the highest
> > download rates.
> > Cake was setup with "rtt 600ms".
> >
> > George
> >
> > On Wed, Nov 29, 2017 at 6:25 PM, Jonathan Morton <chromatix99@gmail.com>
> > wrote:
> >>
> >> High RTT with relatively high bandwidth is traditionally considered
> >> challenging, and is representative of Australasian and satellite
> >> connections.  Definitely do some of those.
> >>
> >> Low bandwidth should also be interesting.  DSL at 4/1 Mbit is typical
> >> entry level package here.
> >>
> >> - Jonathan Morton
> >>
> >>
> >> _______________________________________________
> >> Cake mailing list
> >> Cake@lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/cake
> >>
> >
> >
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
> >
>
>
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619
>

[-- Attachment #2: Type: text/html, Size: 3915 bytes --]

Re: [Cake] cake flenter results round 3

[Dave Taht]

On Fri, Dec 1, 2017 at 11:29 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
> cake vs sfq and fq_codel looks great! You already have fq_codel in the
> graphs, do you mean codel?
>
> In terms of simulating CMTS I am trying to simulate a cable connection.
> Is the following reasonable?
> ===========8<===========
> servers --- delay --- isp --- mbox --- clients
>
> delay:
> tc qdisc replace dev delay.r root netem delay ZZms rate XXXmbit limit 100000
> tc qdisc replace dev delay.l root netem delay ZZms rate XXXmbit limit 100000

There is not much need to specify the rate here, just the delay. You
could like say 10Gbit, if you wanted.

> isp:
> tc qdisc replace dev isp.l root bfifo limit 1024k
> tc qdisc replace dev isp.r root bfifo limit 1024k

No, because this is not the bottleneck queue. isp.r

    tc qdisc del dev isp.r root # a little uncertain of what happens with handle
    tc qdisc replace dev isp.r root handle 1: netem rate YYYmbit \
             limit 100000
    tc qdisc add dev isp.r parent 1:1 handle 10:1 bfifo limit 1024k

I think.

> mbox:
> tc qdisc replace dev mbox.l root cake bandwidth YYYmbit
> tc qdisc replace dev mbox.r root cake bandwidth YYYmbit

    tc qdisc del dev mbox.r root # to leave it unshaped
    # or put cake back in to shape it after the isp shaper above.



> ===========8<===========
>
> George
>
> On Fri, Dec 1, 2017 at 2:23 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>
>> That's really hard to argue with. An fq_codel same conditions (and
>> pfifo and pie) might be interesting here.
>>
>> On Thu, Nov 30, 2017 at 9:13 AM, Georgios Amanakis <gamanakis@gmail.com>
>> wrote:
>> > I gave high RTT with high bandwidth a try:
>> > server   --   delay  --  mbox  --  client
>> > netserver   300/300ms  45/900mbit  flent
>> >
>> > I had to run flent with "-s 0.61" in order to avoid errors with fping
>> > dying
>> > prematurely.
>> > Comparing noack, ack, and ack-aggressive, ack seems to give the highest
>> > download rates.
>> > Cake was setup with "rtt 600ms".
>> >
>> > George
>> >
>> > On Wed, Nov 29, 2017 at 6:25 PM, Jonathan Morton <chromatix99@gmail.com>
>> > wrote:
>> >>
>> >> High RTT with relatively high bandwidth is traditionally considered
>> >> challenging, and is representative of Australasian and satellite
>> >> connections.  Definitely do some of those.
>> >>
>> >> Low bandwidth should also be interesting.  DSL at 4/1 Mbit is typical
>> >> entry level package here.
>> >>
>> >> - Jonathan Morton
>> >>
>> >>
>> >> _______________________________________________
>> >> Cake mailing list
>> >> Cake@lists.bufferbloat.net
>> >> https://lists.bufferbloat.net/listinfo/cake
>> >>
>> >
>> >
>> > _______________________________________________
>> > Cake mailing list
>> > Cake@lists.bufferbloat.net
>> > https://lists.bufferbloat.net/listinfo/cake
>> >
>>
>>
>>
>> --
>>
>> Dave Täht
>> CEO, TekLibre, LLC
>> http://www.teklibre.com
>> Tel: 1-669-226-2619
>
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 3

[Dave Taht]

On Fri, Dec 1, 2017 at 11:45 AM, Dave Taht <dave.taht@gmail.com> wrote:
> On Fri, Dec 1, 2017 at 11:29 AM, Georgios Amanakis <gamanakis@gmail.com> wrote:
>> cake vs sfq and fq_codel looks great! You already have fq_codel in the
>> graphs, do you mean codel?

Heh. I meant codel, yes.

>> In terms of simulating CMTS I am trying to simulate a cable connection.
>> Is the following reasonable?
>> ===========8<===========
>> servers --- delay --- isp --- mbox --- clients
>>
>> delay:
>> tc qdisc replace dev delay.r root netem delay ZZms rate XXXmbit limit 100000
>> tc qdisc replace dev delay.l root netem delay ZZms rate XXXmbit limit 100000
>
> There is not much need to specify the rate here, just the delay. You
> could like say 10Gbit, if you wanted.
>
>> isp:
>> tc qdisc replace dev isp.l root bfifo limit 1024k
>> tc qdisc replace dev isp.r root bfifo limit 1024k
>
> No, because this is not the bottleneck queue. isp.r
>
>     tc qdisc del dev isp.r root # a little uncertain of what happens with handle
>     tc qdisc replace dev isp.r root handle 1: netem rate YYYmbit \
>              limit 100000
>     tc qdisc add dev isp.r parent 1:1 handle 10:1 bfifo limit 1024k
>
> I think.
>
>> mbox:
>> tc qdisc replace dev mbox.l root cake bandwidth YYYmbit
>> tc qdisc replace dev mbox.r root cake bandwidth YYYmbit
>
>     tc qdisc del dev mbox.r root # to leave it unshaped
>     # or put cake back in to shape it after the isp shaper above.
>
>
>
>> ===========8<===========
>>
>> George
>>
>> On Fri, Dec 1, 2017 at 2:23 PM, Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>> That's really hard to argue with. An fq_codel same conditions (and
>>> pfifo and pie) might be interesting here.
>>>
>>> On Thu, Nov 30, 2017 at 9:13 AM, Georgios Amanakis <gamanakis@gmail.com>
>>> wrote:
>>> > I gave high RTT with high bandwidth a try:
>>> > server   --   delay  --  mbox  --  client
>>> > netserver   300/300ms  45/900mbit  flent
>>> >
>>> > I had to run flent with "-s 0.61" in order to avoid errors with fping
>>> > dying
>>> > prematurely.
>>> > Comparing noack, ack, and ack-aggressive, ack seems to give the highest
>>> > download rates.
>>> > Cake was setup with "rtt 600ms".
>>> >
>>> > George
>>> >
>>> > On Wed, Nov 29, 2017 at 6:25 PM, Jonathan Morton <chromatix99@gmail.com>
>>> > wrote:
>>> >>
>>> >> High RTT with relatively high bandwidth is traditionally considered
>>> >> challenging, and is representative of Australasian and satellite
>>> >> connections.  Definitely do some of those.
>>> >>
>>> >> Low bandwidth should also be interesting.  DSL at 4/1 Mbit is typical
>>> >> entry level package here.
>>> >>
>>> >> - Jonathan Morton
>>> >>
>>> >>
>>> >> _______________________________________________
>>> >> Cake mailing list
>>> >> Cake@lists.bufferbloat.net
>>> >> https://lists.bufferbloat.net/listinfo/cake
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > Cake mailing list
>>> > Cake@lists.bufferbloat.net
>>> > https://lists.bufferbloat.net/listinfo/cake
>>> >
>>>
>>>
>>>
>>> --
>>>
>>> Dave Täht
>>> CEO, TekLibre, LLC
>>> http://www.teklibre.com
>>> Tel: 1-669-226-2619
>>
>>
>
>
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 929 bytes --]

Thank you Dave!
I am playing around with this setup:
server -- delay -- isp -- mbox -- client

delay:
tc qdisc replace dev ${host}.l root netem delay 50ms limit 100000
tc qdisc replace dev ${host}.r root netem delay 50ms limit 100000

isp:
tc qdisc replace dev ${host}.l root handle 1:0 netem rate 900mbit limit
100000
tc qdisc replace dev ${host}.r root handle 1:0 netem rate 900mbit limit
100000
tc qdisc replace dev ${host}.r parent 1:1 handle 10:1 bfifo limit 1024k


I played with mbox's qdiscs (3 variations).
mbox:
tc qdisc replace dev ${host}.l root cake bandwidth 890mbit
tc qdisc replace dev ${host}.r root cake bandwidth 890mbit


mbox:
tc qdisc replace dev ${host}.l root codel
tc qdisc replace dev ${host}.r root codel


mbox:
tc qdisc replace dev ${host}.l root fq maxrate 890mbit
tc qdisc replace dev ${host}.r root fq maxrate 890mbit


rrul_be_nflows, 32 upstream, 32 downstream. In every case cake wins.

George

[-- Attachment #1.2: Type: text/html, Size: 1214 bytes --]

[-- Attachment #2: box_totals_cake_codel.png --]
[-- Type: image/png, Size: 66959 bytes --]

[-- Attachment #3: box_totals_cake_fq.png --]
[-- Type: image/png, Size: 60263 bytes --]

[-- Attachment #4: rrul_be_nflows-2017-12-01T160646.489200.nflows_vs_rate_32_32_cake_890mbit.flent.gz --]
[-- Type: application/x-gzip, Size: 976251 bytes --]

[-- Attachment #5: rrul_be_nflows-2017-12-01T160849.976928.nflows_vs_rate_32_32_codel_900mbit.flent.gz --]
[-- Type: application/x-gzip, Size: 960411 bytes --]

[-- Attachment #6: rrul_be_nflows-2017-12-01T161606.795041.nflows_vs_rate_32_32_fq_890mbit.flent.gz --]
[-- Type: application/x-gzip, Size: 850898 bytes --]

[-- Attachment #7: tcp_rtt_cake_codel.png --]
[-- Type: image/png, Size: 182506 bytes --]

[-- Attachment #8: tcp_rtt_cake_fq.png --]
[-- Type: image/png, Size: 165338 bytes --]

Re: [Cake] cake flenter results round 3

[Dave Taht]

Georgios Amanakis <gamanakis@gmail.com> writes:

> Thank you Dave!
> I am playing around with this setup:
> server -- delay -- isp -- mbox -- client
>
> delay:
> tc qdisc replace dev ${host}.l root netem delay 50ms limit 100000
> tc qdisc replace dev ${host}.r root netem delay 50ms limit 100000
>
> isp:
> tc qdisc replace dev ${host}.l root handle 1:0 netem rate 900mbit limit 100000
> tc qdisc replace dev ${host}.r root handle 1:0 netem rate 900mbit limit 100000
> tc qdisc replace dev ${host}.r parent 1:1 handle 10:1 bfifo limit 1024k
>
> I played with mbox's qdiscs (3 variations).
> mbox:
> tc qdisc replace dev ${host}.l root cake bandwidth 890mbit
> tc qdisc replace dev ${host}.r root cake bandwidth 890mbit
>
> mbox:

a reference run with

tc qdisc del dev mbox.l root
tc qdisc del dev mbox.r root

# this may already be our default.

tc qdisc add dev mbox.l root pfifo-fast limit 1000
tc qdisc add dev mbox.r root pfifo-fast limit 1000

> tc qdisc replace dev ${host}.r root codel

Here a definition of a win has to also include a shaper, like
cake's integral one.

tc qdisc replace dev ${host}.r root handle 1:0 netem \
   rate 890mbit limit 100000
tc qdisc replace dev ${host}.r parent 1:1 handle 10:1 \
   codel

(same for .l)

I note that I still trust htb's shaper further tha netem's...

(substitute pie, fq, sfq, etc for codel above)

(did I jump for joy too soon on the fq_codel vs sfq front?)

>
> mbox:
> tc qdisc replace dev ${host}.l root fq maxrate 890mbit
> tc qdisc replace dev ${host}.r root fq maxrate 890mbit

maxrate only puts a cap on each flows peak sending rate.

> rrul_be_nflows, 32 upstream, 32 downstream. In every case cake wins.
>
> George
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

[Cake] Fwd: cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 1680 bytes --]

---------- Forwarded message ----------
From: Georgios Amanakis <gamanakis@gmail.com>
Date: Fri, Dec 1, 2017 at 8:25 PM
Subject: Re: [Cake] cake flenter results round 3
To: Dave Taht <dave.taht@gmail.com>


The only thing in the CMTS setup 900mbit that appears to be close to cake
is rate limiting with netem and applying codel
 as sub-qdisc.

host=mbox
tc qdisc replace dev ${host}.l root handle 1:0 netem rate 890mbit
tc qdisc replace dev ${host}.r root handle 1:0 netem rate 890mbit
tc qdisc replace dev ${host}.l parent 1:1 handle 10:1 codel
tc qdisc replace dev ${host}.r parent 1:1 handle 10:1 codel


On Fri, Dec 1, 2017 at 4:41 PM, Georgios Amanakis <gamanakis@gmail.com>
wrote:

> Thank you Dave!
> I am playing around with this setup:
> server -- delay -- isp -- mbox -- client
>
> delay:
> tc qdisc replace dev ${host}.l root netem delay 50ms limit 100000
> tc qdisc replace dev ${host}.r root netem delay 50ms limit 100000
>
> isp:
> tc qdisc replace dev ${host}.l root handle 1:0 netem rate 900mbit limit
> 100000
> tc qdisc replace dev ${host}.r root handle 1:0 netem rate 900mbit limit
> 100000
> tc qdisc replace dev ${host}.r parent 1:1 handle 10:1 bfifo limit 1024k
>
>
> I played with mbox's qdiscs (3 variations).
> mbox:
> tc qdisc replace dev ${host}.l root cake bandwidth 890mbit
> tc qdisc replace dev ${host}.r root cake bandwidth 890mbit
>
>
> mbox:
> tc qdisc replace dev ${host}.l root codel
> tc qdisc replace dev ${host}.r root codel
>
>
> mbox:
> tc qdisc replace dev ${host}.l root fq maxrate 890mbit
> tc qdisc replace dev ${host}.r root fq maxrate 890mbit
>
>
> rrul_be_nflows, 32 upstream, 32 downstream. In every case cake wins.
>
> George
>
>

[-- Attachment #1.2: Type: text/html, Size: 2518 bytes --]

[-- Attachment #2: tcp_rtt_cake_netem_codel.png --]
[-- Type: image/png, Size: 172662 bytes --]

[-- Attachment #3: box_totals_cake_netem_codel.png --]
[-- Type: image/png, Size: 58567 bytes --]

[-- Attachment #4: rrul_be_nflows-2017-12-01T201958.740500.nflows_vs_rate_32_32_netem_codel_900mbit.flent.gz --]
[-- Type: application/x-gzip, Size: 968265 bytes --]

[-- Attachment #5: rrul_be_nflows-2017-12-01T160646.489200.nflows_vs_rate_32_32_cake_890mbit.flent.gz --]
[-- Type: application/x-gzip, Size: 976251 bytes --]

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 314 bytes --]

>Here a definition of a win has to also include a shaper, like
>cake's integral one.

I realized this an hour ago, but forgot to include "limit 100000" in netem
config. Running that now.

> tc qdisc add dev mbox.l root pfifo-fast limit 1000

I will also do this reference run.
Thank you for your guidance!

George

[-- Attachment #2: Type: text/html, Size: 523 bytes --]

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 68 bytes --]

>(substitute pie, fq, sfq, etc for codel above)
Will also do these.

[-- Attachment #2: Type: text/html, Size: 126 bytes --]

Re: [Cake] cake flenter results round 3

[Dave Taht]

On Fri, Dec 1, 2017 at 5:31 PM, Georgios Amanakis <gamanakis@gmail.com> wrote:
>>Here a definition of a win has to also include a shaper, like
>>cake's integral one.
>
> I realized this an hour ago, but forgot to include "limit 100000" in netem
> config. Running that now.
>
>> tc qdisc add dev mbox.l root pfifo-fast limit 1000

Our rule of thumb for token bucket shapers has been 5-15% so a run
against 900 of 855 or lower might show
codel/pie/fq_codel working better. cake is deficit based, and it's
looking good at 900v890. A way to test
cake's aqm more directly against codel or pie is to use cake in
"flowblind" mode.

> I will also do this reference run.
> Thank you for your guidance!

thank you for tackling some load! These are some of the fastest speeds
and widest workloads we've ever tried these qdiscs at.

I picked up two 12 core xeons today. I went to get my usual cheap
samsung EVO SSD, got one, saw an optane on sale, got one of those for
the main box. I'm a sucker for latency reductions.

They are way too loud, tho.

> George
>



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 3

[George Amanakis]

[-- Attachment #1: Type: text/plain, Size: 211 bytes --]

Another run using the CMTS topology.
ISP limited at 200/10mbit.
Reference: pfifo limit 1000.
Cake or HTB/{codel,pie,fq,sfq} limited at 180/9mbit.
RTT 100ms.
rrul_be_nflows, 32 download, 8 upload streams.

George

[-- Attachment #2: box_totals_cmts_200_10_latency.png --]
[-- Type: image/png, Size: 70542 bytes --]

[-- Attachment #3: box_totals_cmts_200_10_uploads.png --]
[-- Type: image/png, Size: 74605 bytes --]

[-- Attachment #4: box_totals_cmts_200_10.png --]
[-- Type: image/png, Size: 88622 bytes --]

[-- Attachment #5: rrulbe_cmts_32_8_cake_200mbit_10mbit.tgz --]
[-- Type: application/x-compressed-tar, Size: 2718419 bytes --]

Re: [Cake] cake flenter results round 3

[Dave Taht]

On Sat, Dec 2, 2017 at 9:47 AM, George Amanakis <gamanakis@gmail.com> wrote:
> Another run using the CMTS topology.
> ISP limited at 200/10mbit.
> Reference: pfifo limit 1000.
> Cake or HTB/{codel,pie,fq,sfq} limited at 180/9mbit.

That's a little discouraging. That's sch_fq, not fq_codel? (please add fq_codel)

> RTT 100ms.
> rrul_be_nflows, 32 download, 8 upload streams.
>
> George



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 3

[George Amanakis]

[-- Attachment #1: Type: text/plain, Size: 194 bytes --]

On Sat, 2017-12-02 at 09:59 -0800, Dave Taht wrote:
> 
> That's a little discouraging. That's sch_fq, not fq_codel? (please
> add fq_codel)
> 

fq is sch_fq. I added fq_codel in the attachments.

[-- Attachment #2: rrulbe_cmts_32_8_cake_200mbit_10mbit.tgz --]
[-- Type: application/x-compressed-tar, Size: 3131437 bytes --]

[-- Attachment #3: box_totals_cmts_200_10_latency.png --]
[-- Type: image/png, Size: 77061 bytes --]

[-- Attachment #4: box_totals_cmts_200_10_uploads.png --]
[-- Type: image/png, Size: 79442 bytes --]

[-- Attachment #5: box_totals_cmts_200_10.png --]
[-- Type: image/png, Size: 95134 bytes --]

Re: [Cake] cake flenter results round 3

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 1464 bytes --]

On Sat, Dec 2, 2017 at 9:59 AM, Dave Taht <dave.taht@gmail.com> wrote:
> On Sat, Dec 2, 2017 at 9:47 AM, George Amanakis <gamanakis@gmail.com> wrote:
>> Another run using the CMTS topology.
>> ISP limited at 200/10mbit.
>> Reference: pfifo limit 1000.
>> Cake or HTB/{codel,pie,fq,sfq} limited at 180/9mbit.
>
> That's a little discouraging. That's sch_fq, not fq_codel? (please add fq_codel)

Ah, OK, I  get it. (please do add fq_codel. cake (all modes) with ack
filtering. A 20ms result would be good too.)

The ginormous default buffer size for sch_fq leads to seconds of tcp
RTT, but the latency number for the measurement flows is low because
it's fq. sfq amazingly is holding it's own (which it didn't used to),
with it's mere 128 buffers at this RTT, it girates around 200+ms rtt
for tcp.. I imagine that tcp pacing is helping it a lot here, even at
these long rtts...

And the surprising winner at the 98th percentile is... cake flowblind.
There are major differences in the aqm algorithm between codel and
cake...

(all reasonable news.. along with my perpetual joy at "it doesn't crash")

There are a few other puzzling things in this data-set.


>
>> RTT 100ms.
>> rrul_be_nflows, 32 download, 8 upload streams.
>>
>> George
>
>
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

[-- Attachment #2: flowbind_win_rtt.svg --]
[-- Type: image/svg+xml, Size: 502499 bytes --]

Re: [Cake] cake flenter results round 3

[Dave Taht]

On Sat, Dec 2, 2017 at 10:30 AM, George Amanakis <gamanakis@gmail.com> wrote:
> On Sat, 2017-12-02 at 09:59 -0800, Dave Taht wrote:
>>
>> That's a little discouraging. That's sch_fq, not fq_codel? (please
>> add fq_codel)

>>
>
> fq is sch_fq. I added fq_codel in the attachments.

Curiouser and curiouser. both cake normal and flowblind are getting
their more consistent TCP RTTs at the cost of bandwidth compared to
codel/fq_codel/pie/etc.

If there is a way to elide pfifo from your (presumably automatically
generated) summary plots, they'd be easier to read.
( I live and die by the tarballs anyway, but...)

I didn't manage to get either Xeon to boot last night, I end up with a
blank screen of death...


-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

On Dec 2, 2017 1:34 PM, "Dave Taht" <dave.taht@gmail.com> wrote:


Ah, OK, I  get it. (please do add fq_codel. cake (all modes) with ack
filtering. A 20ms result would be good too.)


By cake in all modes I assume you mean modes of flow isolation while
setting it at diffserv3(default)?

[-- Attachment #2: Type: text/html, Size: 788 bytes --]

Re: [Cake] cake flenter results round 3

[Kristjan Onu]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

Hi Jonathan,

> Low bandwidth should also be interesting.  DSL at 4/1 Mbit is typical 
> entry level package here.

I have a 5/1 Mbit cable connection. I setup Cake like this:
$TC qdisc add dev enp4s0 root cake bandwidth 850Kbit ack-filter

I ran Flent like this:
flent rrul --output=rrul.png --plot=all
	--host=netperf.bufferbloat.net --length=60

This produced the graphs attached. Let me know if you have questions or 
want other tests.

Regards,

Kristjan

[-- Attachment #2: rrul_cake.png --]
[-- Type: image/png, Size: 175302 bytes --]

Re: [Cake] cake flenter results round 3

[Kristjan Onu]

[-- Attachment #1: Type: text/plain, Size: 1229 bytes --]

Hello again,

Results in my last email were accidentally using fq_codel for the 
ingress qdisc. Now I set it up with cake, commit 402f05c. The commands I 
used are listed below.

Regards,

Kristjan

export TC=~/local/src/iproute2-cake-next/tc/tc

$TC qdisc add dev enp4s0 root cake bandwidth 850Kbit docsis nat dual-srchost ack-filter

ip link add name ifb0 type ifb

$TC qdisc del dev enp4s0 ingress

$TC qdisc add dev enp4s0 handle ffff: ingress

$TC qdisc del dev ifb0 root

$TC qdisc add dev ifb0 root cake bandwidth 4250Kbit docsis nat dual-dsthost ingress

ip link set ifb0 up

$TC filter add dev enp4s0 parent ffff: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb0


Kristjan Onu wrote:
> Hi Jonathan,
>
>> Low bandwidth should also be interesting. DSL at 4/1 Mbit is typical 
>> entry level package here.
>
> I have a 5/1 Mbit cable connection. I setup Cake like this:
> $TC qdisc add dev enp4s0 root cake bandwidth 850Kbit ack-filter
>
> I ran Flent like this:
> flent rrul --output=rrul.png --plot=all
>     --host=netperf.bufferbloat.net --length=60
>
> This produced the graphs attached. Let me know if you have questions 
> or want other tests.
>
> Regards,
>
> Kristjan


[-- Attachment #2: rrul_cake.png --]
[-- Type: image/png, Size: 144463 bytes --]

[-- Attachment #3: rrul_fq_codel.png --]
[-- Type: image/png, Size: 137012 bytes --]

[-- Attachment #4: rrul-2018-01-07T145826.782602.flent.gz --]
[-- Type: application/gzip, Size: 58396 bytes --]

[-- Attachment #5: rrul-2018-01-07T150555.359811.cake.flent.gz --]
[-- Type: application/gzip, Size: 76792 bytes --]

[Cake] Fwd: cake flenter results round 3

[Georgios Amanakis]

[-- Attachment #1.1: Type: text/plain, Size: 942 bytes --]

I cannot attach the flent.gz files as they exceed the message size limit of
the list. Hopefully Dave got them.

---------- Forwarded message ----------
From: Georgios Amanakis <gamanakis@gmail.com>
Date: Fri, Dec 1, 2017 at 10:20 PM
Subject: Re: [Cake] cake flenter results round 3
To: Dave Taht <dave.taht@gmail.com>
Cc: Cake List <cake@lists.bufferbloat.net>


Just finished testing. CMTS Setup:
server   --   delay   --   isp   --   mbox   --   client
netserver     100ms      900mbit     810mbit      flent

I am attaching the setup, tried to implement all of Dave's suggestions.
I used the files:
vsetup.sh # topology
vcake.sh{.htb} # qdiscs
sshd.sh # start netserver, irtt etc.
stats.sh # generate the txt files with qdisc stats

Tested pfifo, cake, cake_flowblind, netem_codel, htb_codel, htb_pie,
htb_fq, htb_sfq.

In terms of latency cake_flowblind seems to be as good as
htb_{codel,pie,fq}.
htb_sfq outperforms it slightly.

George

[-- Attachment #1.2: Type: text/html, Size: 1556 bytes --]

[-- Attachment #2: box_totals.png --]
[-- Type: image/png, Size: 109302 bytes --]

[-- Attachment #3: setup.tgz --]
[-- Type: application/x-gzip, Size: 6777 bytes --]