[Codel] fq_"codel" in apple's gear

Sebastian Moeller moeller0 at gmx.de
Sat Oct 14 15:44:16 EDT 2023


Hi Dave,


> On Oct 14, 2023, at 21:01, Dave Taht <dave.taht at gmail.com> wrote:
> 
> netstat -I en0 -qq

Here is a bit of output from macosx monterey 12.7 (on intel hardware). I know this is not really recent,  ut it is the most recent I have available...

This is on an realtek based USB3 ethernet dongle that used to support 1 Gbps, but at from macosx update on it is stuck on 100Mbps FastEthernet...


Here is the output after a flent test:

Out of my "--test-parameter markings=CS0,CS1,CS2,CS3,CS4,CS5,CS6,CS7" it seems only CS1 was honored and put into BK the rest all ended up in BE...
I have zero clue what macos uses to steer packets into the different classes, might be DSCP alone, but then at least using some more exotic ones than boring CS6/CS7
No idea what CTL is, the average packet size mist be around 67 bytes...


macbook:CAKE-autorate user$ netstat -I en3 -qq
en3:
     [ sched:  FQ_CODEL  qlength:    0/128 ]
     [ pkts:     522515  bytes:  334464372  dropped pkts:      0 bytes:      0 ]
=====================================================
     [ pri: CTL (0)	srv_cl: 0x480190	quantum: 605	drr_max: 8 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 292918	bytes: 19787064 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 0 compressed pkts: 0]
=====================================================
     [ pri: VO (1)	srv_cl: 0x400180	quantum: 605	drr_max: 8 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 0	bytes: 0 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 0 compressed pkts: 0]
=====================================================
     [ pri: VI (2)	srv_cl: 0x380100	quantum: 3028	drr_max: 6 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 0	bytes: 0 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 0 compressed pkts: 0]
=====================================================
     [ pri: RV (3)	srv_cl: 0x300110	quantum: 3028	drr_max: 6 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 0	bytes: 0 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 0 compressed pkts: 0]
=====================================================
     [ pri: AV (4)	srv_cl: 0x280120	quantum: 3028	drr_max: 6 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 0	bytes: 0 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 0 compressed pkts: 0]
=====================================================
     [ pri: OAM (5)	srv_cl: 0x200020	quantum: 1514	drr_max: 4 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 0	bytes: 0 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 0 compressed pkts: 0]
=====================================================
     [ pri: RD (6)	srv_cl: 0x180010	quantum: 1514	drr_max: 4 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 13	bytes: 6684 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 0 compressed pkts: 0]
=====================================================
     [ pri: BE (7)	srv_cl: 0x0	quantum: 1514	drr_max: 4 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 225368	bytes: 314071428 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 0 compressed pkts: 0]
=====================================================
     [ pri: BK (8)	srv_cl: 0x100080	quantum: 1514	drr_max: 2 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 3831	bytes: 576846 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 0 compressed pkts: 0]
=====================================================
     [ pri: BK_SYS (9)	srv_cl: 0x80090	quantum: 1514	drr_max: 2 ]
     [ queued pkts: 0	bytes: 0 ]
     [ dequeued pkts: 385	bytes: 22350 ]
     [ budget: 0	target qdelay: 10.00 msec	update interval:100.00 msec ]
     [ flow control: 0	feedback: 0	stalls: 0	failed: 0 	overwhelming: 0 ]
     [ drop overflow: 0	early: 0	memfail: 0	duprexmt:0 ]
     [ flows total: 0	new: 0	old: 0 ]
     [ throttle on: 0	off: 0	drop: 0 ]
     [ compressible pkts: 375 compressed pkts: 0]
macbook:CAKE-autorate user$ uname -a
Darwin macbook.local 21.6.0 Darwin Kernel Version 21.6.0: Fri Sep 15 16:17:23 PDT 2023; root:xnu-8020.240.18.703.5~1/RELEASE_X86_64 x86_64


Here is the flent output:

bash-3.2$ echo "IPv4" ; date ; ping -c 10 netperf-eu.bufferbloat.net ; ./run-flent --ipv4 -l 60 -H netperf-eu.bufferbloat.net rrul_var --remote-metadata=root at 192.168.42.1 --test-parameter=cpu_stats_hosts=root at 192.168.42.1 --step-size=.05 --test-parameter bidir_streams=8 --test-parameter markings=CS0,CS1,CS2,CS3,CS4,CS5,CS6,CS7 --test-parameter ping_hosts=1.1.1.1 -D . -t IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct45500of46719K-D090pct105000of116789K_hms-beagle2_en4TB2_TurrisOmnia-TurrisOS.6.3.2-pppoe-wan-eth2.7_2_bridged-FB7520-OpenWrt-r22256-14334c222e-OutdoorMSAN-VDSL100_2_netperf-eu.bufferbloat.net --log-file
IPv4
Sat Oct 14 21:19:24 CEST 2023
PING demo.tohojo.dk (193.10.227.30): 56 data bytes
64 bytes from 193.10.227.30: icmp_seq=0 ttl=48 time=24.259 ms
64 bytes from 193.10.227.30: icmp_seq=1 ttl=48 time=22.789 ms
64 bytes from 193.10.227.30: icmp_seq=2 ttl=48 time=24.097 ms
64 bytes from 193.10.227.30: icmp_seq=3 ttl=48 time=24.433 ms
64 bytes from 193.10.227.30: icmp_seq=4 ttl=48 time=24.342 ms
64 bytes from 193.10.227.30: icmp_seq=5 ttl=48 time=24.325 ms
64 bytes from 193.10.227.30: icmp_seq=6 ttl=48 time=23.456 ms
64 bytes from 193.10.227.30: icmp_seq=7 ttl=48 time=22.628 ms
64 bytes from 193.10.227.30: icmp_seq=8 ttl=48 time=24.396 ms
64 bytes from 193.10.227.30: icmp_seq=9 ttl=48 time=23.521 ms

--- demo.tohojo.dk ping statistics ---
10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 22.628/23.825/24.433/0.649 ms
Starting Flent 2.1.1+git.7b931915 using Python 3.10.8.
Starting rrul_var test. Expected run time: 70 seconds.
Data file written to ./rrul_var-2023-10-14T211934.512820.IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct45500of46719K-D090pct105000of116789K_hms-beagle2_en4TB2_TurrisOmnia-TurrisOS_6_3_2-pppoe-wan-eth2_7_2_bridged-FB7520-OpenWrt-r22256-14334c222e-OutdoorMSAN-VDSL100.flent.gz

Summary of rrul_var test run from 2023-10-14 19:19:34.512820
  Title: 'IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct45500of46719K-D090pct105000of116789K_hms-beagle2_en4TB2_TurrisOmnia-TurrisOS.6.3.2-pppoe-wan-eth2.7_2_bridged-FB7520-OpenWrt-r22256-14334c222e-OutdoorMSAN-VDSL100_2_netperf-eu.bufferbloat.net'

                                              avg       median       99th %          # data pts
 Ping (ms) ICMP 1.1.1.1 (extra)    :        13.10        12.90        17.40 ms             1396
 Ping (ms) avg                     :        27.07          N/A          N/A ms             1639
 Ping (ms)::ICMP                   :        26.15        26.10        30.70 ms             1399
 Ping (ms)::UDP 0 (CS0)            :        27.78        27.55        32.13 ms             1639
 Ping (ms)::UDP 1 (CS1)            :        29.34        29.51        35.09 ms             1639
 Ping (ms)::UDP 2 (CS2)            :        26.18        26.06        30.56 ms             1639
 Ping (ms)::UDP 3 (CS3)            :        26.22        26.06        30.77 ms             1639
 Ping (ms)::UDP 4 (CS4)            :        26.20        25.98        30.93 ms             1639
 Ping (ms)::UDP 5 (CS5)            :        26.20        26.06        31.33 ms             1639
 Ping (ms)::UDP 6 (CS6)            :        28.13        28.11        32.86 ms             1639
 Ping (ms)::UDP 7 (CS7)            :        26.51        26.44        31.39 ms             1639
 TCP download avg                  :        11.33          N/A          N/A Mbits/s        1639
 TCP download sum                  :        90.62          N/A          N/A Mbits/s        1639
 TCP download::0 (CS0)             :        11.39        11.25        17.48 Mbits/s        1639
 TCP download::1 (CS1)             :        10.66        10.21        28.97 Mbits/s        1639
 TCP download::2 (CS2)             :        12.17        11.99        18.87 Mbits/s        1639
 TCP download::3 (CS3)             :        11.29        11.06        18.27 Mbits/s        1639
 TCP download::4 (CS4)             :        11.87        11.34        23.51 Mbits/s        1639
 TCP download::5 (CS5)             :        10.51        10.49        16.58 Mbits/s        1639
 TCP download::6 (CS6)             :        11.37        11.15        18.32 Mbits/s        1639
 TCP download::7 (CS7)             :        11.36        11.25        16.99 Mbits/s        1639
 TCP totals                        :       130.00          N/A          N/A Mbits/s        1639
 TCP upload avg                    :         4.92          N/A          N/A Mbits/s        1639
 TCP upload sum                    :        39.38          N/A          N/A Mbits/s        1639
 TCP upload::0 (CS0)               :         5.50         5.48         7.94 Mbits/s        1639
 TCP upload::1 (CS1)               :         2.13         2.19         2.64 Mbits/s        1639
 TCP upload::2 (CS2)               :         5.47         5.47         7.13 Mbits/s        1639
 TCP upload::3 (CS3)               :         5.47         5.48         6.97 Mbits/s        1639
 TCP upload::4 (CS4)               :         5.47         5.49         7.23 Mbits/s        1639
 TCP upload::5 (CS5)               :         5.46         5.46         6.83 Mbits/s        1639
 TCP upload::6 (CS6)               :         4.96         4.95         6.25 Mbits/s        1639
 TCP upload::7 (CS7)               :         4.92         4.94         6.18 Mbits/s        1639
 cpu_stats_root at 192.168.42.1::load :         0.40         0.42         0.83                1243
bash-3.2$





More information about the Codel mailing list