[Cake] 900mbit tbf + fq_codel vs cake on the apu2

[Cake] 900mbit tbf + fq_codel vs cake on the apu2

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 379 bytes --]

can't achieve full throughput.

root@apu2:/etc/sqm# tc qdisc replace dev enp3s0 handle 10: root tbf
burst 64k rate 900000000 latency 60ms
root@apu2:/etc/sqm# tc qdisc add dev enp3s0 parent 10:0 fq_codel

verses

tc qdisc add dev enp3s0 root cake flows besteffort bandwidth 900mbit


-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

[-- Attachment #2: tbf.png --]
[-- Type: image/png, Size: 49329 bytes --]

[-- Attachment #3: tcp_nup-2018-08-04T141337.403229.tbf_64k_70ms_fq_codel_900mbit.flent.gz --]
[-- Type: application/gzip, Size: 316028 bytes --]

[-- Attachment #4: tcp_nup-2018-08-04T141759.900953.cake_besteffort_flows_900mbit.flent.gz --]
[-- Type: application/gzip, Size: 314456 bytes --]

Re: [Cake] 900mbit tbf + fq_codel vs cake on the apu2

[Pete Heist]

The APU hardware begs for smp support...

> On Aug 4, 2018, at 11:31 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> can't achieve full throughput.
> 
> root@apu2:/etc/sqm# tc qdisc replace dev enp3s0 handle 10: root tbf
> burst 64k rate 900000000 latency 60ms
> root@apu2:/etc/sqm# tc qdisc add dev enp3s0 parent 10:0 fq_codel
> 
> verses
> 
> tc qdisc add dev enp3s0 root cake flows besteffort bandwidth 900mbit
> 
> 
> -- 
> 
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619
> <tbf.png><tcp_nup-2018-08-04T141337.403229.tbf_64k_70ms_fq_codel_900mbit.flent.gz><tcp_nup-2018-08-04T141759.900953.cake_besteffort_flows_900mbit.flent.gz>_______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] 900mbit tbf + fq_codel vs cake on the apu2

[Dave Taht]

It begs for faster code.
On Wed, Aug 22, 2018 at 1:51 AM Pete Heist <pete@heistp.net> wrote:
>
> The APU hardware begs for smp support...
>
> > On Aug 4, 2018, at 11:31 PM, Dave Taht <dave.taht@gmail.com> wrote:
> >
> > can't achieve full throughput.
> >
> > root@apu2:/etc/sqm# tc qdisc replace dev enp3s0 handle 10: root tbf
> > burst 64k rate 900000000 latency 60ms
> > root@apu2:/etc/sqm# tc qdisc add dev enp3s0 parent 10:0 fq_codel
> >
> > verses
> >
> > tc qdisc add dev enp3s0 root cake flows besteffort bandwidth 900mbit
> >
> >
> > --
> >
> > Dave Täht
> > CEO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-669-226-2619
> > <tbf.png><tcp_nup-2018-08-04T141337.403229.tbf_64k_70ms_fq_codel_900mbit.flent.gz><tcp_nup-2018-08-04T141759.900953.cake_besteffort_flows_900mbit.flent.gz>_______________________________________________
> > Cake mailing list
> > Cake@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cake
>


-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] 900mbit tbf + fq_codel vs cake on the apu2

[Dave Taht]

adding a basic shaper to fq_codel itself is kind of trivial. You need
to check if you are shaping

https://github.com/dtaht/sch_tart/blob/master/sch_tart.c#L329

Calculate the next start time:
https://github.com/dtaht/sch_tart/blob/master/sch_tart.c#L392

and calc the rate in setup and init the watchdog.

this was 40% faster than tbf + fq_codel in the good ole days and I
sometimes think about just adding it into fq_codel itself, where, in a
system already using it elsewhere, keeps the icache hotter as well.
I'm going to discard the work in sch_tart and try fiddling with
fq_codel again perhaps, but, also over the years, we've found that
multiple variables (flows, target, interval) were effectively
constants in 99% of cases, and I did want to restore my original work
on codel having saner responses to overload that I'd done in earlier
versions of cake... dunno. I *personally* need an inbound shaper that
cracks 120mbit on mips hardware.

...
I hadn't thought about additionally parallelizing the workload (which
has problems like adding way more queues than I'd like) much.
...

Now, done cleverly, shaping is also parralizable, with an atomic add
across this core variable, or a periodic rcu'd merge step of the
separate bandwidth clocks running.

The "clever" part is that :I've never figured out how two+ instances
of a qdisc (being directed by sch_mq) could share a tiny bit of data.
There *are* some per-cpu stats rcu'd in qdiscs and filters at this
point. and perhaps the per interface (as opposed to per qdisc) stats
are rcu'd enough to pull from periodically to compensate.

OFFTOPIC: seeing this go by gives me more hope on the rx path than
I've had in a while

https://lwn.net/SubscriberLink/763056/f9a20ec24b8d29dd/

Re: [Cake] 900mbit tbf + fq_codel vs cake on the apu2

[Pete Heist]

> On Aug 22, 2018, at 5:10 PM, Dave Taht <dave.taht@gmail.com> wrote:
> 
> adding a basic shaper to fq_codel itself is kind of trivial. You need
> to check if you are shaping
> 
> this was 40% faster than tbf + fq_codel in the good ole days and I

Aha, ok, I didn't there was that much to be gained in serial performance.

> versions of cake... dunno. I *personally* need an inbound shaper that
> cracks 120mbit on mips hardware.

The FreeNet guys upgraded our sector to AC last week, so I switched to a PowerBeam AC (wrecked my legs straddling the peak of a 45 degree roof for half a day). Anyway, I would also like if my ER-X (32-bit mips) had some more headroom to run something codel-like. It can do it, but it starts to struggle. Mine is currently an htb+cake setup.

> ...
> I hadn't thought about additionally parallelizing the workload (which
> has problems like adding way more queues than I'd like) much.

I thought there was already some discussion about that wrt lockless qdisc support, which doesn’t mean adding more queues, or multiple qdisc instances, does it? Perhaps my understanding of what that does and use of the term “smp” was loose...

Re: [Cake] 900mbit tbf + fq_codel vs cake on the apu2

[Dave Taht]

On Wed, Aug 22, 2018 at 9:08 AM Pete Heist <pete@heistp.net> wrote:
>
>
> > On Aug 22, 2018, at 5:10 PM, Dave Taht <dave.taht@gmail.com> wrote:
> >
> > adding a basic shaper to fq_codel itself is kind of trivial. You need
> > to check if you are shaping
> >
> > this was 40% faster than tbf + fq_codel in the good ole days and I
>
> Aha, ok, I didn't there was that much to be gained in serial performance.
>
> > versions of cake... dunno. I *personally* need an inbound shaper that
> > cracks 120mbit on mips hardware.
>
> The FreeNet guys upgraded our sector to AC last week, so I switched to a PowerBeam AC (wrecked my legs straddling the peak of a 45 degree roof for half a day). Anyway, I would also like if my ER-X (32-bit mips) had some more headroom to run something codel-like. It can do it, but it starts to struggle. Mine is currently an htb+cake setup.

well, how much cpu can you claw back with tbf + fq_codel?

note that tbf IS sensitive to the burst size. 64k might be too much.
Or too little.

> > ...
> > I hadn't thought about additionally parallelizing the workload (which
> > has problems like adding way more queues than I'd like) much.
>
> I thought there was already some discussion about that wrt lockless qdisc support, which doesn’t mean adding more queues, or multiple qdisc instances, does it? Perhaps my understanding of what that does and use of the term “smp” was loose...



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619