[Cake] Trouble getting decent performance with CAKE

[Cake] Trouble getting decent performance with CAKE

[jmarks2]

[-- Attachment #1.1: Type: text/plain, Size: 1730 bytes --]

Dear CAKE experts,

I write for your help in getting CAKE to perform on my router as I
understand it should.

 

I replaced my aging Asus home router with a Ubiquiti Edgerouter 4
specifically so I could take advantage of the port of CAKE to it.

 

I am in the US, with VDSL2 service provided by AT&T at 100/20. I now have
all unfiltered traffic delivered by the AT&T modem to the eth0 port on the
router.

 

My desktop runs Windows. So, to take advantage of flent to quantify CAKE
performance, I stood up a Linux Mint VM and ran flent rrul tests before CAKE
installation, and after.

 

The first attached datafile, labeled qdisc pfifo fast, is the performance
before CAKE. The other file, called 20/95 ack filtered bridged ptm, is my
latest attempt with configuring CAKE flags. 

My major issue is with Ingress. Egress seems fine.

For ingress, I see very little difference between performances with and
without CAKE. And the overall throughput is far less than 100 mbit.

 

The queuing disciplines on the router are:

 

jeremymarks@ubnt:~$ sudo tc qdisc show

 

qdisc cake 1: dev eth0 root refcnt 2 bandwidth 20Mbit besteffort
dual-srchost nat nowash ack-filter split-gso rtt 100.0ms ptm overhead 22
no-sce

qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------

qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0
1 1 1 1 1 1 1 1

qdisc cake 2: dev ifb0 root refcnt 2 bandwidth 95Mbit besteffort
dual-dsthost nat nowash ingress ack-filter split-gso rtt 100.0ms ptm
overhead 22 no-sce

 

I must be making a major mistake in configuring CAKE.

 

I would be grateful for any help or advice in getting this amazing program
to work as it should. 

 

Many thanks for any help.

Jeremy

 

 


[-- Attachment #1.2: Type: text/html, Size: 5552 bytes --]

[-- Attachment #2: rrul-2021-01-24T110301.456648.qdisc_pfifo_fast.flent.gz --]
[-- Type: application/x-gzip, Size: 121758 bytes --]

[-- Attachment #3: rrul-2021-01-24T162210.973709.20_95_ack_filtered_bridged-ptm.flent.gz --]
[-- Type: application/x-gzip, Size: 119863 bytes --]

Re: [Cake] Trouble getting decent performance with CAKE

[xnor]

Haven't looked at your measurements, but what's the issue? Is it 
throughput or latency? Because I have a 750 MHz single core mips router 
here that handles 100 Mbps... barely.
Latency on ingress (and traffic shaping in general on ingress) sucks 
compared to egress.

Why is there an imq0 device? Haven't those been obsolete for several 
years? What kernel are you running?

I assume you have a filter with tc-action mirred egress set up, but just 
to be sure could you post the output of:
$ tc filter show dev eth0 parent ffff:

Also, there's little point in having the ack-filter on on the ingress cake.

Re: [Cake] Trouble getting decent performance with CAKE

[Dave Taht]

As much as I like to get the fustest with the mostest with the rrul
test, it is often easier to test up and down separately.

do :

tc qdisc delete ifb0 root
tc qdisc delete eth0 root

I am not sure what imq is being used for....

Can you share the output of tc -s

instead?

a simple upload test would then be:

(unless you made cake the default)

flent -t pfifo_fast -H wherever --socket-stats -x --te=upload_streams=4 tcp_nup

tc qdisc add dev eth0 root cake all_your_options

flent -t cake -H wherever --socket-stats -x --te=upload_streams=4 tcp_nup

the download setting up tc mirred I assume you are doing with sqm? Do that.

flent -t -H wherever --socket-stats -x --te=download_streams=4 tcp_ndown

then send tc -s

It's entirely feasible your so-called 100mbit service is running at
far far less than 100mbit in the first place.

Another thing to check is if your modem is connecting to the device at
100Mbit or a gbit. SOME - not enough - dsl modems actually connect at
100Mbit and/or joy of joys excert hw flow control.
95Mbit is gonna be slightly too high in former case for cake to
control things, try 92mbit, or less.

On Tue, Jan 26, 2021 at 4:22 PM <jmarks2@gmail.com> wrote:
>
> Dear CAKE experts,
>
> I write for your help in getting CAKE to perform on my router as I understand it should.
>
>
>
> I replaced my aging Asus home router with a Ubiquiti Edgerouter 4 specifically so I could take advantage of the port of CAKE to it.
>
>
>
> I am in the US, with VDSL2 service provided by AT&T at 100/20. I now have all unfiltered traffic delivered by the AT&T modem to the eth0 port on the router.
>
>
>
> My desktop runs Windows. So, to take advantage of flent to quantify CAKE performance, I stood up a Linux Mint VM and ran flent rrul tests before CAKE installation, and after.
>
>
>
> The first attached datafile, labeled qdisc pfifo fast, is the performance before CAKE. The other file, called 20/95 ack filtered bridged ptm, is my latest attempt with configuring CAKE flags.
>
> My major issue is with Ingress. Egress seems fine.
>
> For ingress, I see very little difference between performances with and without CAKE. And the overall throughput is far less than 100 mbit.
>
>
>
> The queuing disciplines on the router are:
>
>
>
> jeremymarks@ubnt:~$ sudo tc qdisc show
>
>
>
> qdisc cake 1: dev eth0 root refcnt 2 bandwidth 20Mbit besteffort dual-srchost nat nowash ack-filter split-gso rtt 100.0ms ptm overhead 22 no-sce
>
> qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
>
> qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>
> qdisc cake 2: dev ifb0 root refcnt 2 bandwidth 95Mbit besteffort dual-dsthost nat nowash ingress ack-filter split-gso rtt 100.0ms ptm overhead 22 no-sce
>
>
>
> I must be making a major mistake in configuring CAKE.
>
>
>
> I would be grateful for any help or advice in getting this amazing program to work as it should.
>
>
>
> Many thanks for any help.
>
> Jeremy
>
>
>
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Cake] Trouble getting decent performance with CAKE

[jmarks2]

Thanks so much for getting back to me.
First, I am not so expert in these things, so please excuse my questions about providing the info you have requested and implementing your suggestions.

1. I don’t know what imq0 is used for, either. 
     I don't have a script to run CAKE at system start because I haven't got it working well enough. So, at reboot, the out-of-the box qdiscs are:

	qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
	qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
	qdisc pfifo_fast 0: dev ifb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1

     You can see that the imq0 device is present at startup. A quick search suggests that Ubiquiti uses this for native QOS during egress. If it's not clear what it's doing and if it's interfering, I will happily remove it, if I knew how. The worst that happens is that the system crashes and I have to reboot.

2. At startup (so no CAKE), sudo tc qdisc -s show:

	qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
	 Sent 2055477419 bytes 2270158 pkt (dropped 0, overlimits 0 requeues 0)
	 backlog 0b 0p requeues 0

	qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 	Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 	backlog 0b 0p requeues 0

	qdisc pfifo_fast 0: dev ifb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
	 Sent 2087261885 bytes 2270207 pkt (dropped 0, overlimits 0 requeues 1)
 	backlog 0b 0p requeues 1

3. I set up CAKE with the following commands:
	sudo tc qdisc add dev eth0 root handle 1:0 cake bandwidth...[switches]
	
	sudo set interfaces input ifb0
	sudo set interfaces ethernet eth0 redirect ifb0
	sudo tc qdisc add dev ifb0 root handle 2:0 cake bandwidth...[switches]

     Do these commands make CAKE the default, or can I choose a qdisc to use when running flent? If so, how do I do this?

4. Regarding your suggestion: "the download setting up tc mirred I assume you are doing with sqm? Do that."

     This is outside my knowledge, and I have not done anything like that.  
     I have read an alternative (older?) way to set up CAKE for ingress: 
	sudo ip link add name ifb4eth0 type ifb
	sudo tc qdisc del dev eth0 ingress
	sudo tc qdisc add dev eth0 handle ffff: ingress
	sudo tc qdisc del dev ifb4eth0 root
	sudo tc qdisc add dev ifb4eth0 root cake bandwidth...
	sudo ifconfig ifb4eth1 up
	sudo tc filter add dev eth1 parent ffff: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb4eth1

    I suspect the first 6 lines of this code achieve the same thing as the first two lines of code that I used (see point 3, above). Am I right?
    Is the filter that is specified in the last line what you are suggesting I add?

This is about as much as I can address with what you have suggested, without a super long email.

Please feel free to correct my mistakes in understanding.
If you have suggestions based on what I have written, I would so grateful. I am really looking forward to hearing what you think.
This learning curve is a little steep.

Many thanks.


-----Original Message-----
From: Dave Taht <dave.taht@gmail.com> 
Sent: Tuesday, January 26, 2021 11:19 PM
To: jmarks2@gmail.com
Cc: Cake List <cake@lists.bufferbloat.net>
Subject: Re: [Cake] Trouble getting decent performance with CAKE

As much as I like to get the fustest with the mostest with the rrul test, it is often easier to test up and down separately.

do :

tc qdisc delete ifb0 root
tc qdisc delete eth0 root

I am not sure what imq is being used for....

Can you share the output of tc -s

instead?

a simple upload test would then be:

(unless you made cake the default)

flent -t pfifo_fast -H wherever --socket-stats -x --te=upload_streams=4 tcp_nup

tc qdisc add dev eth0 root cake all_your_options

flent -t cake -H wherever --socket-stats -x --te=upload_streams=4 tcp_nup

the download setting up tc mirred I assume you are doing with sqm? Do that.

flent -t -H wherever --socket-stats -x --te=download_streams=4 tcp_ndown

then send tc -s

It's entirely feasible your so-called 100mbit service is running at far far less than 100mbit in the first place.

Another thing to check is if your modem is connecting to the device at 100Mbit or a gbit. SOME - not enough - dsl modems actually connect at 100Mbit and/or joy of joys excert hw flow control.
95Mbit is gonna be slightly too high in former case for cake to control things, try 92mbit, or less.

On Tue, Jan 26, 2021 at 4:22 PM <jmarks2@gmail.com> wrote:
>
> Dear CAKE experts,
>
> I write for your help in getting CAKE to perform on my router as I understand it should.
>
>
>
> I replaced my aging Asus home router with a Ubiquiti Edgerouter 4 specifically so I could take advantage of the port of CAKE to it.
>
>
>
> I am in the US, with VDSL2 service provided by AT&T at 100/20. I now have all unfiltered traffic delivered by the AT&T modem to the eth0 port on the router.
>
>
>
> My desktop runs Windows. So, to take advantage of flent to quantify CAKE performance, I stood up a Linux Mint VM and ran flent rrul tests before CAKE installation, and after.
>
>
>
> The first attached datafile, labeled qdisc pfifo fast, is the performance before CAKE. The other file, called 20/95 ack filtered bridged ptm, is my latest attempt with configuring CAKE flags.
>
> My major issue is with Ingress. Egress seems fine.
>
> For ingress, I see very little difference between performances with and without CAKE. And the overall throughput is far less than 100 mbit.
>
>
>
> The queuing disciplines on the router are:
>
>
>
> jeremymarks@ubnt:~$ sudo tc qdisc show
>
>
>
> qdisc cake 1: dev eth0 root refcnt 2 bandwidth 20Mbit besteffort 
> dual-srchost nat nowash ack-filter split-gso rtt 100.0ms ptm overhead 
> 22 no-sce
>
> qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
>
> qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1 
> 2 0 0 1 1 1 1 1 1 1 1
>
> qdisc cake 2: dev ifb0 root refcnt 2 bandwidth 95Mbit besteffort 
> dual-dsthost nat nowash ingress ack-filter split-gso rtt 100.0ms ptm 
> overhead 22 no-sce
>
>
>
> I must be making a major mistake in configuring CAKE.
>
>
>
> I would be grateful for any help or advice in getting this amazing program to work as it should.
>
>
>
> Many thanks for any help.
>
> Jeremy
>
>
>
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



--
"For a successful technology, reality must take precedence over public relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Cake] Trouble getting decent performance with CAKE

[Dave Taht]

You need the tc mirred to get stuff into ifb0. I don't know what the
redirect command you are using does. tc -s qdisc will show stats for
the cake instance if that was all you need on this platform.


On Wed, Jan 27, 2021 at 9:49 AM <jmarks2@gmail.com> wrote:
>
> Thanks so much for getting back to me.
> First, I am not so expert in these things, so please excuse my questions about providing the info you have requested and implementing your suggestions.
>
> 1. I don’t know what imq0 is used for, either.
>      I don't have a script to run CAKE at system start because I haven't got it working well enough. So, at reboot, the out-of-the box qdiscs are:
>
>         qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
>         qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>         qdisc pfifo_fast 0: dev ifb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>
>      You can see that the imq0 device is present at startup. A quick search suggests that Ubiquiti uses this for native QOS during egress. If it's not clear what it's doing and if it's interfering, I will happily remove it, if I knew how. The worst that happens is that the system crashes and I have to reboot.
>
> 2. At startup (so no CAKE), sudo tc qdisc -s show:
>
>         qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
>          Sent 2055477419 bytes 2270158 pkt (dropped 0, overlimits 0 requeues 0)
>          backlog 0b 0p requeues 0
>
>         qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>         Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>         backlog 0b 0p requeues 0
>
>         qdisc pfifo_fast 0: dev ifb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>          Sent 2087261885 bytes 2270207 pkt (dropped 0, overlimits 0 requeues 1)
>         backlog 0b 0p requeues 1
>
> 3. I set up CAKE with the following commands:
>         sudo tc qdisc add dev eth0 root handle 1:0 cake bandwidth...[switches]
>
>         sudo set interfaces input ifb0
>         sudo set interfaces ethernet eth0 redirect ifb0
>         sudo tc qdisc add dev ifb0 root handle 2:0 cake bandwidth...[switches]
>
>      Do these commands make CAKE the default, or can I choose a qdisc to use when running flent? If so, how do I do this?
>
> 4. Regarding your suggestion: "the download setting up tc mirred I assume you are doing with sqm? Do that."
>
>      This is outside my knowledge, and I have not done anything like that.
>      I have read an alternative (older?) way to set up CAKE for ingress:
>         sudo ip link add name ifb4eth0 type ifb
>         sudo tc qdisc del dev eth0 ingress
>         sudo tc qdisc add dev eth0 handle ffff: ingress
>         sudo tc qdisc del dev ifb4eth0 root
>         sudo tc qdisc add dev ifb4eth0 root cake bandwidth...
>         sudo ifconfig ifb4eth1 up
>         sudo tc filter add dev eth1 parent ffff: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb4eth1
>
>     I suspect the first 6 lines of this code achieve the same thing as the first two lines of code that I used (see point 3, above). Am I right?
>     Is the filter that is specified in the last line what you are suggesting I add?
>
> This is about as much as I can address with what you have suggested, without a super long email.
>
> Please feel free to correct my mistakes in understanding.
> If you have suggestions based on what I have written, I would so grateful. I am really looking forward to hearing what you think.
> This learning curve is a little steep.
>
> Many thanks.
>
>
> -----Original Message-----
> From: Dave Taht <dave.taht@gmail.com>
> Sent: Tuesday, January 26, 2021 11:19 PM
> To: jmarks2@gmail.com
> Cc: Cake List <cake@lists.bufferbloat.net>
> Subject: Re: [Cake] Trouble getting decent performance with CAKE
>
> As much as I like to get the fustest with the mostest with the rrul test, it is often easier to test up and down separately.
>
> do :
>
> tc qdisc delete ifb0 root
> tc qdisc delete eth0 root
>
> I am not sure what imq is being used for....
>
> Can you share the output of tc -s
>
> instead?
>
> a simple upload test would then be:
>
> (unless you made cake the default)
>
> flent -t pfifo_fast -H wherever --socket-stats -x --te=upload_streams=4 tcp_nup
>
> tc qdisc add dev eth0 root cake all_your_options
>
> flent -t cake -H wherever --socket-stats -x --te=upload_streams=4 tcp_nup
>
> the download setting up tc mirred I assume you are doing with sqm? Do that.
>
> flent -t -H wherever --socket-stats -x --te=download_streams=4 tcp_ndown
>
> then send tc -s
>
> It's entirely feasible your so-called 100mbit service is running at far far less than 100mbit in the first place.
>
> Another thing to check is if your modem is connecting to the device at 100Mbit or a gbit. SOME - not enough - dsl modems actually connect at 100Mbit and/or joy of joys excert hw flow control.
> 95Mbit is gonna be slightly too high in former case for cake to control things, try 92mbit, or less.
>
> On Tue, Jan 26, 2021 at 4:22 PM <jmarks2@gmail.com> wrote:
> >
> > Dear CAKE experts,
> >
> > I write for your help in getting CAKE to perform on my router as I understand it should.
> >
> >
> >
> > I replaced my aging Asus home router with a Ubiquiti Edgerouter 4 specifically so I could take advantage of the port of CAKE to it.
> >
> >
> >
> > I am in the US, with VDSL2 service provided by AT&T at 100/20. I now have all unfiltered traffic delivered by the AT&T modem to the eth0 port on the router.
> >
> >
> >
> > My desktop runs Windows. So, to take advantage of flent to quantify CAKE performance, I stood up a Linux Mint VM and ran flent rrul tests before CAKE installation, and after.
> >
> >
> >
> > The first attached datafile, labeled qdisc pfifo fast, is the performance before CAKE. The other file, called 20/95 ack filtered bridged ptm, is my latest attempt with configuring CAKE flags.
> >
> > My major issue is with Ingress. Egress seems fine.
> >
> > For ingress, I see very little difference between performances with and without CAKE. And the overall throughput is far less than 100 mbit.
> >
> >
> >
> > The queuing disciplines on the router are:
> >
> >
> >
> > jeremymarks@ubnt:~$ sudo tc qdisc show
> >
> >
> >
> > qdisc cake 1: dev eth0 root refcnt 2 bandwidth 20Mbit besteffort
> > dual-srchost nat nowash ack-filter split-gso rtt 100.0ms ptm overhead
> > 22 no-sce
> >
> > qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
> >
> > qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1
> > 2 0 0 1 1 1 1 1 1 1 1
> >
> > qdisc cake 2: dev ifb0 root refcnt 2 bandwidth 95Mbit besteffort
> > dual-dsthost nat nowash ingress ack-filter split-gso rtt 100.0ms ptm
> > overhead 22 no-sce
> >
> >
> >
> > I must be making a major mistake in configuring CAKE.
> >
> >
> >
> > I would be grateful for any help or advice in getting this amazing program to work as it should.
> >
> >
> >
> > Many thanks for any help.
> >
> > Jeremy
> >
> >
> >
> >
> >
> > _______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
>
>
>
> --
> "For a successful technology, reality must take precedence over public relations, for Mother Nature cannot be fooled" - Richard Feynman
>
> dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729
>


-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Cake] Trouble getting decent performance with CAKE

[Jeremy Marks]

I think the redirect command (in that alternate code block that I didn't use) is from older code, so it's no longer needed. 

I will reimplement CAKE, adding the mirred filter. 
Do you think I can safely remove the imq0 interface? The stats (thank you!) suggest to me that it's not being used. Removing may not be necessary anyway. 

After the bandwidth is not being consumed by others behind this router, I will run dsl reports and flent with and without CAKE, and report back.
I really appreciate the help. 




On Jan 27, 2021, at 11:51 AM, Dave Taht <dave.taht@gmail.com> wrote:

You need the tc mirred to get stuff into ifb0. I don't know what the
redirect command you are using does. tc -s qdisc will show stats for
the cake instance if that was all you need on this platform.


On Wed, Jan 27, 2021 at 9:49 AM <jmarks2@gmail.com> wrote:
> 
> Thanks so much for getting back to me.
> First, I am not so expert in these things, so please excuse my questions about providing the info you have requested and implementing your suggestions.
> 
> 1. I don’t know what imq0 is used for, either.
>     I don't have a script to run CAKE at system start because I haven't got it working well enough. So, at reboot, the out-of-the box qdiscs are:
> 
>        qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
>        qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>        qdisc pfifo_fast 0: dev ifb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
> 
>     You can see that the imq0 device is present at startup. A quick search suggests that Ubiquiti uses this for native QOS during egress. If it's not clear what it's doing and if it's interfering, I will happily remove it, if I knew how. The worst that happens is that the system crashes and I have to reboot.
> 
> 2. At startup (so no CAKE), sudo tc qdisc -s show:
> 
>        qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
>         Sent 2055477419 bytes 2270158 pkt (dropped 0, overlimits 0 requeues 0)
>         backlog 0b 0p requeues 0
> 
>        qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>        Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>        backlog 0b 0p requeues 0
> 
>        qdisc pfifo_fast 0: dev ifb0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
>         Sent 2087261885 bytes 2270207 pkt (dropped 0, overlimits 0 requeues 1)
>        backlog 0b 0p requeues 1
> 
> 3. I set up CAKE with the following commands:
>        sudo tc qdisc add dev eth0 root handle 1:0 cake bandwidth...[switches]
> 
>        sudo set interfaces input ifb0
>        sudo set interfaces ethernet eth0 redirect ifb0
>        sudo tc qdisc add dev ifb0 root handle 2:0 cake bandwidth...[switches]
> 
>     Do these commands make CAKE the default, or can I choose a qdisc to use when running flent? If so, how do I do this?
> 
> 4. Regarding your suggestion: "the download setting up tc mirred I assume you are doing with sqm? Do that."
> 
>     This is outside my knowledge, and I have not done anything like that.
>     I have read an alternative (older?) way to set up CAKE for ingress:
>        sudo ip link add name ifb4eth0 type ifb
>        sudo tc qdisc del dev eth0 ingress
>        sudo tc qdisc add dev eth0 handle ffff: ingress
>        sudo tc qdisc del dev ifb4eth0 root
>        sudo tc qdisc add dev ifb4eth0 root cake bandwidth...
>        sudo ifconfig ifb4eth1 up
>        sudo tc filter add dev eth1 parent ffff: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb4eth1
> 
>    I suspect the first 6 lines of this code achieve the same thing as the first two lines of code that I used (see point 3, above). Am I right?
>    Is the filter that is specified in the last line what you are suggesting I add?
> 
> This is about as much as I can address with what you have suggested, without a super long email.
> 
> Please feel free to correct my mistakes in understanding.
> If you have suggestions based on what I have written, I would so grateful. I am really looking forward to hearing what you think.
> This learning curve is a little steep.
> 
> Many thanks.
> 
> 
> -----Original Message-----
> From: Dave Taht <dave.taht@gmail.com>
> Sent: Tuesday, January 26, 2021 11:19 PM
> To: jmarks2@gmail.com
> Cc: Cake List <cake@lists.bufferbloat.net>
> Subject: Re: [Cake] Trouble getting decent performance with CAKE
> 
> As much as I like to get the fustest with the mostest with the rrul test, it is often easier to test up and down separately.
> 
> do :
> 
> tc qdisc delete ifb0 root
> tc qdisc delete eth0 root
> 
> I am not sure what imq is being used for....
> 
> Can you share the output of tc -s
> 
> instead?
> 
> a simple upload test would then be:
> 
> (unless you made cake the default)
> 
> flent -t pfifo_fast -H wherever --socket-stats -x --te=upload_streams=4 tcp_nup
> 
> tc qdisc add dev eth0 root cake all_your_options
> 
> flent -t cake -H wherever --socket-stats -x --te=upload_streams=4 tcp_nup
> 
> the download setting up tc mirred I assume you are doing with sqm? Do that.
> 
> flent -t -H wherever --socket-stats -x --te=download_streams=4 tcp_ndown
> 
> then send tc -s
> 
> It's entirely feasible your so-called 100mbit service is running at far far less than 100mbit in the first place.
> 
> Another thing to check is if your modem is connecting to the device at 100Mbit or a gbit. SOME - not enough - dsl modems actually connect at 100Mbit and/or joy of joys excert hw flow control.
> 95Mbit is gonna be slightly too high in former case for cake to control things, try 92mbit, or less.
> 
> On Tue, Jan 26, 2021 at 4:22 PM <jmarks2@gmail.com> wrote:
>> 
>> Dear CAKE experts,
>> 
>> I write for your help in getting CAKE to perform on my router as I understand it should.
>> 
>> 
>> 
>> I replaced my aging Asus home router with a Ubiquiti Edgerouter 4 specifically so I could take advantage of the port of CAKE to it.
>> 
>> 
>> 
>> I am in the US, with VDSL2 service provided by AT&T at 100/20. I now have all unfiltered traffic delivered by the AT&T modem to the eth0 port on the router.
>> 
>> 
>> 
>> My desktop runs Windows. So, to take advantage of flent to quantify CAKE performance, I stood up a Linux Mint VM and ran flent rrul tests before CAKE installation, and after.
>> 
>> 
>> 
>> The first attached datafile, labeled qdisc pfifo fast, is the performance before CAKE. The other file, called 20/95 ack filtered bridged ptm, is my latest attempt with configuring CAKE flags.
>> 
>> My major issue is with Ingress. Egress seems fine.
>> 
>> For ingress, I see very little difference between performances with and without CAKE. And the overall throughput is far less than 100 mbit.
>> 
>> 
>> 
>> The queuing disciplines on the router are:
>> 
>> 
>> 
>> jeremymarks@ubnt:~$ sudo tc qdisc show
>> 
>> 
>> 
>> qdisc cake 1: dev eth0 root refcnt 2 bandwidth 20Mbit besteffort
>> dual-srchost nat nowash ack-filter split-gso rtt 100.0ms ptm overhead
>> 22 no-sce
>> 
>> qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
>> 
>> qdisc pfifo_fast 0: dev imq0 root refcnt 2 bands 3 priomap  1 2 2 2 1
>> 2 0 0 1 1 1 1 1 1 1 1
>> 
>> qdisc cake 2: dev ifb0 root refcnt 2 bandwidth 95Mbit besteffort
>> dual-dsthost nat nowash ingress ack-filter split-gso rtt 100.0ms ptm
>> overhead 22 no-sce
>> 
>> 
>> 
>> I must be making a major mistake in configuring CAKE.
>> 
>> 
>> 
>> I would be grateful for any help or advice in getting this amazing program to work as it should.
>> 
>> 
>> 
>> Many thanks for any help.
>> 
>> Jeremy
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
> 
> 
> 
> --
> "For a successful technology, reality must take precedence over public relations, for Mother Nature cannot be fooled" - Richard Feynman
> 
> dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729
> 


-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Cake] Trouble getting decent performance with CAKE

[xnor]

 > I think the redirect command (in that alternate code block that I 
didn't use) is from older code, so it's no longer needed.
 >> 3. I set up CAKE with the following commands:
 >>        sudo tc qdisc add dev eth0 root handle 1:0 cake 
bandwidth...[switches]
 >>
 >>        sudo set interfaces input ifb0
 >>        sudo set interfaces ethernet eth0 redirect ifb0
 >>        sudo tc qdisc add dev ifb0 root handle 2:0 cake 
bandwidth...[switches]

What is that supposed to do? What is "set"? Which package does "set" 
come from?
It seems very weird to have a command that is a shell built-in...


 >> 4. Regarding your suggestion: "the download setting up tc mirred I 
assume you are doing with sqm? Do that."
 >>
 >>     This is outside my knowledge, and I have not done anything like 
that.
 >>     I have read an alternative (older?) way to set up CAKE for ingress:
 >>        sudo ip link add name ifb4eth0 type ifb
 >>        sudo tc qdisc del dev eth0 ingress
 >>        sudo tc qdisc add dev eth0 handle ffff: ingress
 >>        sudo tc qdisc del dev ifb4eth0 root
 >>        sudo tc qdisc add dev ifb4eth0 root cake bandwidth...
 >>        sudo ifconfig ifb4eth1 up
 >>        sudo tc filter add dev eth1 parent ffff: protocol all prio 10 
u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb4eth1

No, this is the *proper* way, but there are mistakes in your example. It 
adds the tc filter to eth1 instead of eth0. You probably also don't want 
the flowid.


 > Do you think I can safely remove the imq0 interface? The stats (thank 
you!) suggest to me that it's not being used. Removing may not be 
necessary anyway.

As I said before, imq has been obsolete for a couple of years. Iirc it 
hasn't been shipped with the kernel for years.
imq was used for QoS back in the day. It extended iptables to do the 
redirection.
ifb is the "modern" replacement and works at a lower level, hence the 
need for the tc-mirred

It looks to me like the linux distribution on your router is doing some 
nonstandard things and possibly runs custom QoS scripts that could 
conflict with your setup. If that is the case you should ask the 
manufacturer for support.

Re: [Cake] Trouble getting decent performance with CAKE

[xnor]

 > Thank you for following up.
 >
 > I think the best way to answer your questions is to point to the site 
where the code for CAKE for the Ubiquiti Edgerouter, and the EdgeOS 
system, is available, along with instructions on how to install and 
implement as a sticky at the top.
 >
 > You will also see that there are 29 pages of discussion with the 
developer, dating from 4 years ago, the latest from this week. In 
general, this port and the associated directions on installation scripts 
have been well accepted.
 >
 > 
https://community.ui.com/questions/Cake-and-FQ-PIE-compiled-for-the-EdgeRouter-devices/fc1ff27c-f321-4344-8737-fcc755cae8a2?page=1
 >
 > I would be very interested to learn your thoughts.
 > Many thanks.

I see that ubiquiti has their own configuration mode/utility [1] and 
"set interfaces .." is one of the accepted configuration commands.
What it does under the hood is a matter of reading the manual or even 
looking at the code, but I guess it results in running pretty much the 
same tc commands that were posted before.

If the imq device stays unused then you can simply ignore it, or 
blacklist the module to not even load it on boot. (Please don't ask me 
how to do this on EdgeOS.)

That aside, I still don't understand what your issue is.


[1] 
https://help.ui.com/hc/en-us/articles/204960094-EdgeRouter-Configuration-and-Operational-Mode


PS: Please send your responses also to the mailing list, thanks.