* [Cake] testers wanted for the "cake" qdisc on mainstream platforms
@ 2017-11-23 17:33 Dave Taht
[not found] ` <CAA93jw5p2t1PWF=1M0pwn5BKo_12cyF8xq4nzToT6AQ-cbNYow@mail.gmail.com>
2017-12-15 0:58 ` [Cake] " Andy Furniss
0 siblings, 2 replies; 4+ messages in thread
From: Dave Taht @ 2017-11-23 17:33 UTC (permalink / raw)
To: cerowrt-devel, bloat, Cake List
It is my hope to get the cake qdisc into the Linux kernel in the next
release cycle. We could definitely use more testers! The version we
have here will compile against almost any kernel on any platform,
dating back as far as 3.10, and has been integrated into the
sqm-scripts and shipped in lede for ages...
But what I'm hoping for is for people to try it against their current
linux kernels on x86, arm, ppc64, arm64, etc hardware, and document
the kernel version, linux distribution, and any anomalies.
To build it you need to have installed your kernel headers, and:
git clone -b cobalt https://github.com/dtaht/sch_cake.git
git clone https://github.com/dtaht/iproute2-cake-next.git
cd sch_cake; make; sudo make install; cd ..
cd iproute2-cake-next; make;
# don't do a make install, instead something like export TC=`pwd`/tc/tc
$TC qdisc add dev your_device root cake bandwidth XXMbit ack-filter
And then pound it flat with whatever traffic types you like. In particular,
getting some videoconferencing and flooding results would be great.
There are a TON of features documented on the man page, several
(ack-filter, wash) are not, as I write.
Please comment via the cake@lists.bufferbloat.net mailing list. THX!
NOTE: flent has gained a lot of new features of late, including
support for the new go based irtt tool, which can measure one way
delay (which is also pretty nifty at the command line)
flent: https://github.com/tohojo/
irtt: https://github.com/peteheist/irtt
From the pending commit message:
Add Common Applications Kept Enhanced (sch_cake) qdisc
sch_cake is intended to squeeze the most bandwidth and lowest
latency out of even the slowest ISP links and routers, while
presenting an API simple enough that even an ISP can configure it.
Example of use on an ISP uplink:
tc qdisc add dev eth0 cake bandwidth 20Mbit nat docsis ack-filter
Cake can also be used in unlimited mode to drive packets at the
speed of the underlying link.
Cake is filled with:
* A hybrid Codel/Blue AQM algorithm, “Cobalt”, tied to an FQ_Codel
derived Flow Queuing system, which autoconfigures based on the bandwidth.
* A unique "triple-isolate" mode (the default) which balances per-flow
and per-host flow FQ even through NAT.
* An integral deficit based shaper with extensive dsl and docsis support
that can also be used in unlimited mode.
* 8 way set associative queuing to reduce flow collisions to a minimum.
* A reasonable interpretation of various diffserv latency/loss tradeoffs.
* Support for washing diffserv for entering and exiting traffic.
* Perfect support for interacting with Docsis 3.0 shapers.
* Extensive support for DSL framing types.
* (New) Support for ack filtering.
- 20 % better throughput at a 16x1 down/up ratio on the rrul test.
* Extensive statistics for measuring, loss, ecn markings, latency variation.
There are some features still considered experimental, notably the
ingress_autorate bandwidth estimator and cobalt itself.
sch_cake replaces a combination of iptables, tc filter, htb and fq_codel in
the sqm-scripts, with sane defaults and vastly easier configuration.
Cake's principal author is Jonathan Morton, with contributions from
Kevin Darbyshire-Bryant, Toke Høiland-Jørgensen, Sebastian Moeller,
Ryan Mounce, Dean Scarff, Guido Sarducci, Nils Andreas Svee, Dave
Täht, and Loganaden Velvindron.
--
Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Cake] Fwd: testers wanted for the "cake" qdisc on mainstream platforms
[not found] ` <20171124085133.6bdfc246@redhat.com>
@ 2017-11-24 21:28 ` Dave Taht
2017-11-24 22:35 ` Sebastian Moeller
0 siblings, 1 reply; 4+ messages in thread
From: Dave Taht @ 2017-11-24 21:28 UTC (permalink / raw)
To: Cake List
Jesper, unlike the rest of us, is regularly testing against 10Gig+ hw,
and did a brief test run against cake.
---------- Forwarded message ----------
From: Jesper Dangaard Brouer <brouer@redhat.com>
Date: Thu, Nov 23, 2017 at 11:51 PM
Subject: Re: testers wanted for the "cake" qdisc on mainstream platforms
To: Dave Taht <dave.taht@gmail.com>
Cc: brouer@redhat.com
On Thu, 23 Nov 2017 12:46:25 -0800
Dave Taht <dave.taht@gmail.com> wrote:
> you are the only person I know with 10GigE+ hardware.
>
> I'm kind of dying to know if ack filtering has any effect there, and
> how close we get to line rates these days, unshaped.
If unloading nf_conntrack, then Cake can do 10G with large 1514 bytes
packets ~820Kpps. With small packets (obviously) it cannot.
Details below
Quick eval on Broadwell
=======================
Host Broadwell: CPU E5-1650 v4 @ 3.60GHz
NICs: Intel ixgbe
Run pktgen on ingress ixgbe1, and forward packets egress ixgbe2.
Notice: 10G wirespeed PPS varies greatly with packet size:
- Smallest 64 bytes packets (+interframe gap + MAC preample = 84 bytes)
- @64 bytes = 14.88 Mpps
- @1538 bytes = 813 Kpps
pktgen01: single flow small packets
-----------------------------------
Install qdisc ::
sudo $TC qdisc add dev ixgbe2 root cake bandwidth 10000Mbit ack-filter
Pktgen script small 64 byte pkts::
./pktgen_sample03_burst_single_flow.sh -vi ixgbe2 \
x -m 00:1b:21:bb:9a:84 -d 10.10.10.1 -t3
Quick nstat::
$ nstat > /dev/null && sleep 1 && nstat
#kernel
IpInReceives 1584842 0.0
IpForwDatagrams 1584842 0.0
IpOutRequests 1584842 0.0
IpExtInOctets 72902318 0.0
IpExtOutOctets 139465304 0.0
IpExtInNoECTPkts 1584832 0.0
ethtool_stats::
$ ethtool_stats.pl --dev ixgbe1 --dev ixgbe2 --sec 3
[...]
Show adapter(s) (ixgbe1 ixgbe2) statistics (ONLY that changed!)
Ethtool(ixgbe1 ) stat: 9695260 ( 9,695,260) <= fdir_miss /sec
Ethtool(ixgbe1 ) stat: 94619772 ( 94,619,772) <= rx_bytes /sec
Ethtool(ixgbe1 ) stat: 952572687 ( 952,572,687) <= rx_bytes_nic /sec
Ethtool(ixgbe1 ) stat: 5245486 ( 5,245,486) <= rx_missed_errors /sec
Ethtool(ixgbe1 ) stat: 8061456 ( 8,061,456) <= rx_no_dma_resources /sec
Ethtool(ixgbe1 ) stat: 1576996 ( 1,576,996) <= rx_packets /sec
Ethtool(ixgbe1 ) stat: 9638460 ( 9,638,460) <= rx_pkts_nic /sec
Ethtool(ixgbe1 ) stat: 94619772 ( 94,619,772) <= rx_queue_4_bytes /sec
Ethtool(ixgbe1 ) stat: 1576996 ( 1,576,996) <= rx_queue_4_packets /sec
Ethtool(ixgbe2 ) stat: 91526011 ( 91,526,011) <= tx_bytes /sec
Ethtool(ixgbe2 ) stat: 100993190 ( 100,993,190) <= tx_bytes_nic /sec
Ethtool(ixgbe2 ) stat: 1578035 ( 1,578,035) <= tx_packets /sec
Ethtool(ixgbe2 ) stat: 1578019 ( 1,578,019) <= tx_pkts_nic /sec
Ethtool(ixgbe2 ) stat: 91526011 ( 91,526,011) <= tx_queue_4_bytes /sec
Ethtool(ixgbe2 ) stat: 1578035 ( 1,578,035) <= tx_queue_4_packets /sec
Perf report::
Samples: 72K of event 'cycles:ppp', Event count (approx.): 66265541749
Overhead CPU Shared Object Symbol
- 10.60% 004 [kernel.vmlinux] [k] _raw_spin_lock
- _raw_spin_lock
+ 10.04% sch_direct_xmit
+ 6.87% 004 [kernel.vmlinux] [k] flow_hash_from_keys
+ 6.26% 004 [kernel.vmlinux] [k] cake_dequeue
+ 6.12% 004 [kernel.vmlinux] [k] fib_table_lookup
+ 5.32% 004 [kernel.vmlinux] [k] ixgbe_poll
+ 4.62% 004 [kernel.vmlinux] [k] ixgbe_xmit_frame_ring
+ 3.17% 004 [kernel.vmlinux] [k] __skb_flow_dissect
+ 2.99% 004 [kernel.vmlinux] [k] cake_enqueue
+ 2.89% 004 [kernel.vmlinux] [k] ip_route_input_rcu
+ 2.68% 004 [kernel.vmlinux] [k] ktime_get
+ 2.47% 004 [kernel.vmlinux] [k] ip_rcv
+ 2.31% 004 [kernel.vmlinux] [k] ip_finish_output2
+ 2.23% 004 [kernel.vmlinux] [k] __netif_receive_skb_core
+ 2.14% 004 [kernel.vmlinux] [k] cake_hash
+ 2.04% 004 [kernel.vmlinux] [k] __dev_queue_xmit
+ 2.00% 004 [kernel.vmlinux] [k] inet_gro_receive
+ 1.87% 004 [kernel.vmlinux] [k] udp_v4_early_demux
+ 1.79% 004 [kernel.vmlinux] [k] dev_gro_receive
+ 1.59% 004 [kernel.vmlinux] [k] __qdisc_run
+ 1.51% 004 [kernel.vmlinux] [k] ip_rcv_finish
+ 1.48% 004 [kernel.vmlinux] [k] __build_skb
+ 1.41% 004 [kernel.vmlinux] [k] ip_forward
+ 1.21% 004 [kernel.vmlinux] [k] dev_hard_start_xmit
+ 1.13% 004 [kernel.vmlinux] [k] sch_direct_xmit
+ 1.08% 004 [kernel.vmlinux] [k] __local_bh_enable_ip
+ 1.05% 004 [kernel.vmlinux] [k] build_skb
+ 0.91% 004 [kernel.vmlinux] [k] fib_validate_source
+ 0.87% 004 [kernel.vmlinux] [k] ip_finish_output
+ 0.87% 004 [kernel.vmlinux] [k] read_tsc
+ 0.83% 004 [kernel.vmlinux] [k] kmem_cache_alloc
+ 0.81% 004 [kernel.vmlinux] [k] ___slab_alloc
+ 0.72% 004 [kernel.vmlinux] [k] kmem_cache_free_bulk
+ 0.67% 004 [kernel.vmlinux] [k] cake_advance_shaper
In the single flow test the overhead in _raw_spin_lock is FAKE. It is
actually caused by the NIC tailptr/doorbell on TX.
pktgen02: 8 x flows small packets
---------------------------------
Pktgen 8x flows::
./pktgen_sample05_flow_per_thread.sh -vi ixgbe2 \
-m 00:1b:21:bb:9a:84 -d 10.10.10.1 -t8
Quick nstat 8x flows::
$ nstat > /dev/null && sleep 1 && nstat
#kernel
IpInReceives 1163458 0.0
IpForwDatagrams 1163458 0.0
IpOutRequests 1163458 0.0
IpExtInOctets 53519160 0.0
IpExtOutOctets 102384480 0.0
IpExtInNoECTPkts 1163461 0.0
ethtool_stats 8x flows::
Show adapter(s) (ixgbe1 ixgbe2) statistics (ONLY that changed!)
Ethtool(ixgbe1 ) stat: 92911 ( 92,911) <= alloc_rx_page /sec
Ethtool(ixgbe1 ) stat: 9502884 ( 9,502,884) <= fdir_miss /sec
Ethtool(ixgbe1 ) stat: 66311206 ( 66,311,206) <= rx_bytes /sec
Ethtool(ixgbe1 ) stat: 956178211 ( 956,178,211) <= rx_bytes_nic /sec
Ethtool(ixgbe1 ) stat: 5440134 ( 5,440,134) <= rx_missed_errors /sec
Ethtool(ixgbe1 ) stat: 8394967 ( 8,394,967) <= rx_no_dma_resources /sec
Ethtool(ixgbe1 ) stat: 1105187 ( 1,105,187) <= rx_packets /sec
Ethtool(ixgbe1 ) stat: 9500151 ( 9,500,151) <= rx_pkts_nic /sec
Ethtool(ixgbe1 ) stat: 11149586 ( 11,149,586) <= rx_queue_0_bytes /sec
Ethtool(ixgbe1 ) stat: 185826 ( 185,826) <= rx_queue_0_packets /sec
Ethtool(ixgbe1 ) stat: 10937155 ( 10,937,155) <= rx_queue_1_bytes /sec
Ethtool(ixgbe1 ) stat: 182286 ( 182,286) <= rx_queue_1_packets /sec
Ethtool(ixgbe1 ) stat: 11119239 ( 11,119,239) <= rx_queue_2_bytes /sec
Ethtool(ixgbe1 ) stat: 185321 ( 185,321) <= rx_queue_2_packets /sec
Ethtool(ixgbe1 ) stat: 10950508 ( 10,950,508) <= rx_queue_3_bytes /sec
Ethtool(ixgbe1 ) stat: 182508 ( 182,508) <= rx_queue_3_packets /sec
Ethtool(ixgbe1 ) stat: 10950508 ( 10,950,508) <= rx_queue_4_bytes /sec
Ethtool(ixgbe1 ) stat: 182508 ( 182,508) <= rx_queue_4_packets /sec
Ethtool(ixgbe1 ) stat: 11204211 ( 11,204,211) <= rx_queue_5_bytes /sec
Ethtool(ixgbe1 ) stat: 186737 ( 186,737) <= rx_queue_5_packets /sec
Ethtool(ixgbe2 ) stat: 62216109 ( 62,216,109) <= tx_bytes /sec
Ethtool(ixgbe2 ) stat: 68653701 ( 68,653,701) <= tx_bytes_nic /sec
Ethtool(ixgbe2 ) stat: 1072692 ( 1,072,692) <= tx_packets /sec
Ethtool(ixgbe2 ) stat: 1072714 ( 1,072,714) <= tx_pkts_nic /sec
Ethtool(ixgbe2 ) stat: 8838570 ( 8,838,570) <= tx_queue_0_bytes /sec
Ethtool(ixgbe2 ) stat: 152389 ( 152,389) <= tx_queue_0_packets /sec
Ethtool(ixgbe2 ) stat: 10584083 ( 10,584,083) <= tx_queue_1_bytes /sec
Ethtool(ixgbe2 ) stat: 182484 ( 182,484) <= tx_queue_1_packets /sec
Ethtool(ixgbe2 ) stat: 10743376 ( 10,743,376) <= tx_queue_2_bytes /sec
Ethtool(ixgbe2 ) stat: 185231 ( 185,231) <= tx_queue_2_packets /sec
Ethtool(ixgbe2 ) stat: 10599318 ( 10,599,318) <= tx_queue_3_bytes /sec
Ethtool(ixgbe2 ) stat: 182747 ( 182,747) <= tx_queue_3_packets /sec
Ethtool(ixgbe2 ) stat: 10596994 ( 10,596,994) <= tx_queue_4_bytes /sec
Ethtool(ixgbe2 ) stat: 182707 ( 182,707) <= tx_queue_4_packets /sec
Ethtool(ixgbe2 ) stat: 10853767 ( 10,853,767) <= tx_queue_5_bytes /sec
Ethtool(ixgbe2 ) stat: 187134 ( 187,134) <= tx_queue_5_packets /sec
Perf report::
Samples: 733K of event 'cycles:ppp', Event count (approx.): 691576256580
Overhead CPU Shared Object Symbol
- 12.32% 005 [kernel.vmlinux] [k] queued_spin_lock_slowpath
- queued_spin_lock_slowpath
+ 11.68% __dev_queue_xmit
+ 0.62% sch_direct_xmit
+ 12.30% 004 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 12.19% 003 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 12.03% 000 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 11.98% 001 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 11.95% 002 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 0.78% 002 [kernel.vmlinux] [k] _raw_spin_lock
+ 0.76% 000 [kernel.vmlinux] [k] _raw_spin_lock
+ 0.74% 001 [kernel.vmlinux] [k] _raw_spin_lock
+ 0.70% 003 [kernel.vmlinux] [k] _raw_spin_lock
+ 0.65% 005 [kernel.vmlinux] [k] _raw_spin_lock
+ 0.64% 001 [kernel.vmlinux] [k] cake_dequeue
+ 0.63% 004 [kernel.vmlinux] [k] _raw_spin_lock
+ 0.53% 003 [kernel.vmlinux] [k] cake_dequeue
+ 0.50% 000 [kernel.vmlinux] [k] cake_dequeue
0.49% 002 [kernel.vmlinux] [k] cake_dequeue
0.48% 005 [kernel.vmlinux] [k] cake_dequeue
0.40% 004 [kernel.vmlinux] [k] cake_dequeue
0.32% 002 [kernel.vmlinux] [k] cake_enqueue
0.32% 004 [kernel.vmlinux] [k] cake_enqueue
0.31% 001 [kernel.vmlinux] [k] cake_enqueue
0.30% 000 [kernel.vmlinux] [k] cake_enqueue
0.29% 005 [kernel.vmlinux] [k] cake_enqueue
0.28% 000 [kernel.vmlinux] [k] __build_skb
0.27% 003 [kernel.vmlinux] [k] cake_enqueue
0.27% 004 [kernel.vmlinux] [k] __build_skb
0.25% 003 [kernel.vmlinux] [k] __build_skb
0.25% 002 [kernel.vmlinux] [k] __build_skb
0.24% 005 [kernel.vmlinux] [k] __build_skb
0.22% 002 [kernel.vmlinux] [k] __qdisc_run
0.22% 000 [kernel.vmlinux] [k] __qdisc_run
0.21% 004 [kernel.vmlinux] [k] cake_hash
0.20% 001 [kernel.vmlinux] [k] __build_skb
0.19% 005 [kernel.vmlinux] [k] cake_hash
0.19% 002 [kernel.vmlinux] [k] cake_hash
0.19% 004 [kernel.vmlinux] [k] ixgbe_poll
0.18% 003 [kernel.vmlinux] [k] cake_hash
0.18% 005 [kernel.vmlinux] [k] kmem_cache_alloc
0.17% 001 [kernel.vmlinux] [k] ixgbe_xmit_frame_ring
0.17% 001 [kernel.vmlinux] [k] cake_hash
0.17% 000 [kernel.vmlinux] [k] cake_hash
0.17% 004 [kernel.vmlinux] [k] __qdisc_run
0.17% 002 [kernel.vmlinux] [k] kmem_cache_alloc
In the multi-flow test the overhead/congestion on queued_spin_lock_slowpath
is real, and is simply due to cake being a single-queue qdisc.
pktgen03: 8 x flows large packets
---------------------------------
Pktgen 8x flows large packets::
./pktgen_sample05_flow_per_thread.sh -vi ixgbe2 \
-m 00:1b:21:bb:9a:84 -d 10.10.10.1 -t8 -s 1514
TX speed measured on generator: 821,361 <= tx_packets /sec
Quick nstat 8x flows large packets::
$ nstat > /dev/null && sleep 1 && nstat
#kernel
IpInReceives 824925 0.0
IpForwDatagrams 824925 0.0
IpOutRequests 824925 0.0
IpExtInOctets 1224145664 0.0
IpExtOutOctets 2448288360 0.0
IpExtInNoECTPkts 824895 0.0
Good result as we at 10G wirespeed with large packets.
ethtool_stats 8x flows large packets::
Show adapter(s) (ixgbe1 ixgbe2) statistics (ONLY that changed!)
Ethtool(ixgbe1 ) stat: 246451 ( 246,451) <= alloc_rx_page /sec
Ethtool(ixgbe1 ) stat: 820293 ( 820,293) <= fdir_miss /sec
Ethtool(ixgbe1 ) stat: 1226074253 (1,226,074,253) <= rx_bytes /sec
Ethtool(ixgbe1 ) stat: 1232080619 (1,232,080,619) <= rx_bytes_nic /sec
Ethtool(ixgbe1 ) stat: 1806 ( 1,806) <= rx_no_dma_resources /sec
Ethtool(ixgbe1 ) stat: 818474 ( 818,474) <= rx_packets /sec
Ethtool(ixgbe1 ) stat: 820286 ( 820,286) <= rx_pkts_nic /sec
Ethtool(ixgbe1 ) stat: 153599624 ( 153,599,624) <= rx_queue_0_bytes /sec
Ethtool(ixgbe1 ) stat: 102536 ( 102,536) <= rx_queue_0_packets /sec
Ethtool(ixgbe1 ) stat: 153600115 ( 153,600,115) <= rx_queue_1_bytes /sec
Ethtool(ixgbe1 ) stat: 102537 ( 102,537) <= rx_queue_1_packets /sec
Ethtool(ixgbe1 ) stat: 306006517 ( 306,006,517) <= rx_queue_2_bytes /sec
Ethtool(ixgbe1 ) stat: 204277 ( 204,277) <= rx_queue_2_packets /sec
Ethtool(ixgbe1 ) stat: 153601585 ( 153,601,585) <= rx_queue_3_bytes /sec
Ethtool(ixgbe1 ) stat: 102538 ( 102,538) <= rx_queue_3_packets /sec
Ethtool(ixgbe1 ) stat: 153603546 ( 153,603,546) <= rx_queue_4_bytes /sec
Ethtool(ixgbe1 ) stat: 102539 ( 102,539) <= rx_queue_4_packets /sec
Ethtool(ixgbe1 ) stat: 305662865 ( 305,662,865) <= rx_queue_5_bytes /sec
Ethtool(ixgbe1 ) stat: 204047 ( 204,047) <= rx_queue_5_packets /sec
Ethtool(ixgbe2 ) stat: 1223143065 (1,223,143,065) <= tx_bytes /sec
Ethtool(ixgbe2 ) stat: 1226420442 (1,226,420,442) <= tx_bytes_nic /sec
Ethtool(ixgbe2 ) stat: 816517 ( 816,517) <= tx_packets /sec
Ethtool(ixgbe2 ) stat: 816525 ( 816,525) <= tx_pkts_nic /sec
Ethtool(ixgbe2 ) stat: 152763623 ( 152,763,623) <= tx_queue_0_bytes /sec
Ethtool(ixgbe2 ) stat: 101978 ( 101,978) <= tx_queue_0_packets /sec
Ethtool(ixgbe2 ) stat: 152922976 ( 152,922,976) <= tx_queue_1_bytes /sec
Ethtool(ixgbe2 ) stat: 102085 ( 102,085) <= tx_queue_1_packets /sec
Ethtool(ixgbe2 ) stat: 305999422 ( 305,999,422) <= tx_queue_2_bytes /sec
Ethtool(ixgbe2 ) stat: 204272 ( 204,272) <= tx_queue_2_packets /sec
Ethtool(ixgbe2 ) stat: 152759210 ( 152,759,210) <= tx_queue_3_bytes /sec
Ethtool(ixgbe2 ) stat: 101975 ( 101,975) <= tx_queue_3_packets /sec
Ethtool(ixgbe2 ) stat: 152875415 ( 152,875,415) <= tx_queue_4_bytes /sec
Ethtool(ixgbe2 ) stat: 102053 ( 102,053) <= tx_queue_4_packets /sec
Ethtool(ixgbe2 ) stat: 305822417 ( 305,822,417) <= tx_queue_5_bytes /sec
Ethtool(ixgbe2 ) stat: 204154 ( 204,154) <= tx_queue_5_packets /sec
Perf report 8x flows large packets::
Samples: 136K of event 'cycles:ppp', Event count (approx.): 104319432585
Overhead CPU Shared Object Symbol
+ 23.06% 005 [kernel.vmlinux] [k] queued_spin_lock_slowpath
- queued_spin_lock_slowpath
+ 20.49% __dev_queue_xmit
+ 2.41% sch_direct_xmit
+ 22.16% 002 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 3.09% 005 [kernel.vmlinux] [k] cake_dequeue
+ 3.02% 005 [kernel.vmlinux] [k] _raw_spin_lock
+ 2.77% 002 [kernel.vmlinux] [k] cake_dequeue
+ 2.76% 002 [kernel.vmlinux] [k] _raw_spin_lock
+ 2.48% 001 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 2.03% 000 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 1.29% 003 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 1.20% 002 [kernel.vmlinux] [k] __build_skb
+ 1.12% 005 [kernel.vmlinux] [k] __build_skb
+ 1.08% 005 [kernel.vmlinux] [k] ixgbe_xmit_frame_ring
+ 0.91% 002 [kernel.vmlinux] [k] ixgbe_xmit_frame_ring
+ 0.91% 002 [kernel.vmlinux] [k] cake_enqueue
+ 0.85% 005 [kernel.vmlinux] [k] ixgbe_poll
+ 0.84% 005 [kernel.vmlinux] [k] cake_enqueue
+ 0.82% 002 [kernel.vmlinux] [k] ixgbe_poll
+ 0.70% 004 [kernel.vmlinux] [k] queued_spin_lock_slowpath
+ 0.68% 005 [kernel.vmlinux] [k] cake_dequeue_one
+ 0.59% 002 [kernel.vmlinux] [k] cake_dequeue_one
+ 0.53% 002 [kernel.vmlinux] [k] cake_hash
+ 0.52% 002 [kernel.vmlinux] [k] __dev_queue_xmit
+ 0.51% 005 [kernel.vmlinux] [k] prandom_u32_state
+ 0.50% 005 [kernel.vmlinux] [k] __qdisc_run
+ 0.50% 002 [kernel.vmlinux] [k] netif_skb_features
0.50% 002 [kernel.vmlinux] [k] prandom_u32_state
0.50% 002 [kernel.vmlinux] [k] flow_hash_from_keys
0.49% 005 [kernel.vmlinux] [k] __dev_queue_xmit
0.49% 002 [kernel.vmlinux] [k] __qdisc_run
0.48% 005 [kernel.vmlinux] [k] cake_hash
0.47% 005 [kernel.vmlinux] [k] flow_hash_from_keys
> ---------- Forwarded message ----------
> From: Dave Taht <dave.taht@gmail.com>
> Date: Thu, Nov 23, 2017 at 9:33 AM
> Subject: testers wanted for the "cake" qdisc on mainstream platforms
> To: "cerowrt-devel@lists.bufferbloat.net"
> <cerowrt-devel@lists.bufferbloat.net>, bloat
> <bloat@lists.bufferbloat.net>, Cake List <cake@lists.bufferbloat.net>
>
>
> It is my hope to get the cake qdisc into the Linux kernel in the next
> release cycle. We could definitely use more testers! The version we
> have here will compile against almost any kernel on any platform,
> dating back as far as 3.10, and has been integrated into the
> sqm-scripts and shipped in lede for ages...
>
> But what I'm hoping for is for people to try it against their current
> linux kernels on x86, arm, ppc64, arm64, etc hardware, and document
> the kernel version, linux distribution, and any anomalies.
>
> To build it you need to have installed your kernel headers, and:
>
> git clone -b cobalt https://github.com/dtaht/sch_cake.git
> git clone https://github.com/dtaht/iproute2-cake-next.git
> cd sch_cake; make; sudo make install; cd ..
> cd iproute2-cake-next; make;
> # don't do a make install, instead something like export TC=`pwd`/tc/tc
>
> $TC qdisc add dev your_device root cake bandwidth XXMbit ack-filter
>
> And then pound it flat with whatever traffic types you like. In particular,
> getting some videoconferencing and flooding results would be great.
>
> There are a TON of features documented on the man page, several
> (ack-filter, wash) are not, as I write.
>
> Please comment via the cake@lists.bufferbloat.net mailing list. THX!
>
> NOTE: flent has gained a lot of new features of late, including
> support for the new go based irtt tool, which can measure one way
> delay (which is also pretty nifty at the command line)
>
>
> flent: https://github.com/tohojo/
> irtt: https://github.com/peteheist/irtt
>
> From the pending commit message:
>
> Add Common Applications Kept Enhanced (sch_cake) qdisc
>
> sch_cake is intended to squeeze the most bandwidth and lowest
> latency out of even the slowest ISP links and routers, while
> presenting an API simple enough that even an ISP can configure it.
>
> Example of use on an ISP uplink:
>
> tc qdisc add dev eth0 cake bandwidth 20Mbit nat docsis ack-filter
>
> Cake can also be used in unlimited mode to drive packets at the
> speed of the underlying link.
>
> Cake is filled with:
>
> * A hybrid Codel/Blue AQM algorithm, “Cobalt”, tied to an FQ_Codel
> derived Flow Queuing system, which autoconfigures based on the bandwidth.
> * A unique "triple-isolate" mode (the default) which balances per-flow
> and per-host flow FQ even through NAT.
> * An integral deficit based shaper with extensive dsl and docsis support
> that can also be used in unlimited mode.
> * 8 way set associative queuing to reduce flow collisions to a minimum.
> * A reasonable interpretation of various diffserv latency/loss tradeoffs.
> * Support for washing diffserv for entering and exiting traffic.
> * Perfect support for interacting with Docsis 3.0 shapers.
> * Extensive support for DSL framing types.
> * (New) Support for ack filtering.
> - 20 % better throughput at a 16x1 down/up ratio on the rrul test.
> * Extensive statistics for measuring, loss, ecn markings, latency variation.
>
> There are some features still considered experimental, notably the
> ingress_autorate bandwidth estimator and cobalt itself.
>
> sch_cake replaces a combination of iptables, tc filter, htb and fq_codel in
> the sqm-scripts, with sane defaults and vastly easier configuration.
>
> Cake's principal author is Jonathan Morton, with contributions from
> Kevin Darbyshire-Bryant, Toke Høiland-Jørgensen, Sebastian Moeller,
> Ryan Mounce, Dean Scarff, Guido Sarducci, Nils Andreas Svee, Dave
> Täht, and Loganaden Velvindron.
>
>
>
>
>
> --
>
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619
>
>
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
--
Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Cake] Fwd: testers wanted for the "cake" qdisc on mainstream platforms
2017-11-24 21:28 ` [Cake] Fwd: " Dave Taht
@ 2017-11-24 22:35 ` Sebastian Moeller
0 siblings, 0 replies; 4+ messages in thread
From: Sebastian Moeller @ 2017-11-24 22:35 UTC (permalink / raw)
To: Dave Täht; +Cc: Cake List
Hi Dave hi Jesper,
I feel a bit bad, as Jesper knows way more about these things than I do, but...
> On Nov 24, 2017, at 22:28, Dave Taht <dave.taht@gmail.com> wrote:
>
> sudo $TC qdisc add dev ixgbe2 root cake bandwidth 10000Mbit ack-filter
I feel there is a missing "overhead 34 mpu 84" in that invocation, unless the goal is to have BQL get exercised...
I guess I am overlooking something somewhere.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Cake] testers wanted for the "cake" qdisc on mainstream platforms
2017-11-23 17:33 [Cake] testers wanted for the "cake" qdisc on mainstream platforms Dave Taht
[not found] ` <CAA93jw5p2t1PWF=1M0pwn5BKo_12cyF8xq4nzToT6AQ-cbNYow@mail.gmail.com>
@ 2017-12-15 0:58 ` Andy Furniss
1 sibling, 0 replies; 4+ messages in thread
From: Andy Furniss @ 2017-12-15 0:58 UTC (permalink / raw)
To: cake
Dave Taht wrote:
> It is my hope to get the cake qdisc into the Linux kernel in the next
> release cycle. We could definitely use more testers! The version we
> have here will compile against almost any kernel on any platform,
> dating back as far as 3.10, and has been integrated into the
> sqm-scripts and shipped in lede for ages...
Fails to build on on 4.1.46, needs
diff --git a/sch_cake.c b/sch_cake.c
index 021b215..d4f0c21 100644
--- a/sch_cake.c
+++ b/sch_cake.c
@@ -70,7 +70,7 @@
#include <net/netfilter/nf_conntrack.h>
#endif
-#if (KERNEL_VERSION(4, 4, 11) > LINUX_VERSION_CODE) ||
((KERNEL_VERSION(4, 5, 0) <= LINUX_VERSION_CODE) && (KERNEL_VERSION(4,
5, 5) > LINUX_VERSION_CODE))
+#if 0
#define qdisc_tree_reduce_backlog(_a, _b, _c)
qdisc_tree_decrease_qlen(_a, _b)
#endif
grepping kernel tree for qdisc_tree_reduce_backlog gets results.
grepping qdisc_tree_decrease_qlen doesn't.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-12-15 0:58 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-23 17:33 [Cake] testers wanted for the "cake" qdisc on mainstream platforms Dave Taht
[not found] ` <CAA93jw5p2t1PWF=1M0pwn5BKo_12cyF8xq4nzToT6AQ-cbNYow@mail.gmail.com>
[not found] ` <20171124085133.6bdfc246@redhat.com>
2017-11-24 21:28 ` [Cake] Fwd: " Dave Taht
2017-11-24 22:35 ` Sebastian Moeller
2017-12-15 0:58 ` [Cake] " Andy Furniss
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox