* Re: [Cake] Need help troubleshooting cake
2021-03-04 20:44 [Cake] Need help troubleshooting cake Gustavo Lapido Loureiro
@ 2021-03-06 17:56 ` Gustavo Lapido Loureiro
0 siblings, 0 replies; 2+ messages in thread
From: Gustavo Lapido Loureiro @ 2021-03-06 17:56 UTC (permalink / raw)
To: cake
[-- Attachment #1.1: Type: text/plain, Size: 5985 bytes --]
Ok, I decided to step backwards, and start from the beginning.
My first goal will be to make sure the hardware is ok, mainly, the network
interfaces.
I reinstalled Debian 10, turned it into a barebones router, and ran the
*speedtest-netperf.sh* script from Openwrt
<https://github.com/openwrt/packages/blob/openwrt-19.07/net/speedtest-netperf/files/speedtest-netperf.sh>,
either from within the router, as well as from my 100/10 notebook.
The results seem to indicate everything is fine:
-
From the server, the test maxes out the Internet link;
-
Fromt the notebook, the test kinda maxes out its interface roof: close
to 90 Mbps in all four tests (I will assume this is because of poor
hardware quality)
I also switched WAN and LAN interfaces, just in case, and repeated the
test, which gave similar results as before.
PARTIAL CONCLUSION: So far so good.
Attached, both results.
*Atenciosamente,*
*Gustavo Lapido Loureiro*
Em qui., 4 de mar. de 2021 às 17:44, Gustavo Lapido Loureiro <
gustavo.lapido@gmail.com> escreveu:
> Hello, new member here.
> First, thank you all involved in making this great piece of software,
> which I have already experienced without issues during a couple of months.
> Back then, it was on Openwrt, in a TP-LINK Archer C-7 router, bandwidth
> capped to 60 Mbps (on the device's limits, but still making Internet usage
> much smoother to all users).
> Then I moved the router to a debian 10 Celeron J1900 mini-ITX, 4 GB of
> RAM, two entry level Gigabit cards.
> So far I couldn't get the stable A's in dslreports test I was getting with
> the previous setup.
> The point is that my Internet has been upgraded from 60 Mbps to 240/20
> Mbps. SInce most, if not all, devices are still 100 Mbps, I'm currently
> capping cake at 85/18.
> I basically "transported" the cake settings from Openwrt to debian,
> basically adjusting the speed limits.
> I would like to know where I start troubleshooting this, to help identify
> what is causing bufferbloat on this new setup.
>
> Follow cake settings, then tc, then network hardware info
>
> *CAKE*
>
> # Uplink and Downlink values are in kbps
> UPLINK=17500
> #DOWNLINK=216000
> DOWNLINK=90000
>
> # SQM recipe to use. For more information, see /usr/lib/sqm/*.help
> SCRIPT=piece_of_cake.qos
>
> # Optional/advanced config
>
> ENABLED=1
> QDISC=cake
>
> #LLAM=tc_stab
> LINKLAYER=ethernet
> OVERHEAD=22
> #STAB_MTU=2047
> #STAB_TSIZE=512
> #STAB_MPU=0
>
> #ILIMIT=
> #ELIMIT=
> #ITARGET=
> #ETARGET=
>
> # ECN ingress resp. egress. Values are ECN or NOECN.
> IECN=ECN
> EECN=NOECN
>
> # Extra qdisc options ingress resp. egress
> IQDISC_OPTS="nat dual-dsthost"
> EQDISC_OPTS="nat dual-srchost ack-filter"
>
> # CoDel target
> #TARGET=5ms
>
> #ZERO_DSCP_INGRESS=1
> #IGNORE_DSCP_INGRESS=1
>
> *TC SETTINGS:*
>
> qdisc noqueue 0: dev lo root refcnt 2
> qdisc pfifo_fast 0: dev enp2s0 root refcnt 2 bands 3 priomap 1 2 2 2 1 2
> 0 0 1 1 1 1 1 1 1 1
> qdisc cake 801b: dev enp3s0 root refcnt 2 bandwidth 17500Kbit besteffort
> dual-srchost nat nowash ack-filter split-gso rtt 100.0ms noatm overhead 22
> qdisc ingress ffff: dev enp3s0 parent ffff:fff1 ----------------
> qdisc cake 801c: dev ifb4enp3s0 root refcnt 2 bandwidth 90Mbit besteffort
> dual-dsthost nat wash no-ack-filter split-gso rtt 100.0ms noatm overhead 22
>
> *NETWORK HARDWARE:*
>
> SCSI
> *-network
> description: Ethernet interface
> product: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
> vendor: Realtek Semiconductor Co., Ltd.
> physical id: 0
> bus info: pci@0000:02:00.0
> logical name: enp2s0
> version: 06
> serial: c4:e9:84:00:1d:7b
> size: 1Gbit/s
> capacity: 1Gbit/s
> width: 64 bits
> clock: 33MHz
> capabilities: pm msi pciexpress msix vpd bus_master cap_list
> ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd 1000bt 1000bt-fd
> autonegotiation
> configuration: autonegotiation=on broadcast=yes driver=r8169
> duplex=full firmware=rtl_nic/rtl8168e-2.fw ip=192.168.1.1 latency=0
> link=yes multicast=yes port=MII speed=1Gbit/s
> resources: irq:17 ioport:e000(size=256) memory:d0704000-d0704fff
> memory:d0700000-d0703fff
> *-network
> description: Ethernet interface
> product: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
> vendor: Realtek Semiconductor Co., Ltd.
> physical id: 0
> bus info: pci@0000:03:00.0
> logical name: enp3s0
> version: 0c
> serial: 44:8a:5b:94:96:37
> size: 1Gbit/s
> capacity: 1Gbit/s
> width: 64 bits
> clock: 33MHz
> capabilities: pm msi pciexpress msix vpd bus_master cap_list
> ethernet physical tp aui bnc mii fibre 10bt 10bt-fd 100bt 100bt-fd
> 1000bt-fd autonegotiation
> configuration: autonegotiation=on broadcast=yes driver=r8169
> duplex=full firmware=rtl8168g-2_0.0.1 02/06/13 ip=179.210.1.62 latency=0
> link=yes multicast=yes port=MII speed=1Gbit/s
> resources: irq:18 ioport:d000(size=256) memory:d0604000-d0604fff
> memory:d0600000-d0603fff
> *-network:0
> description: Ethernet interface
> physical id: 1
> logical name: ifb4enp3s0
> serial: 76:6b:66:3c:6d:e9
> capabilities: ethernet physical
> configuration: broadcast=yes
> *-network:1 DISABLED
> description: Ethernet interface
> physical id: 2
> logical name: ifb1
> serial: 02:9e:3d:12:6e:16
> capabilities: ethernet physical
> configuration: broadcast=yes
> *-network:2 DISABLED
> description: Ethernet interface
> physical id: 3
> logical name: ifb0
> serial: 32:fe:93:2e:3a:1d
> capabilities: ethernet physical
> configuration: broadcast=yes
>
[-- Attachment #1.2: Type: text/html, Size: 8088 bytes --]
[-- Attachment #2: netperf before sqm from notebook --]
[-- Type: application/octet-stream, Size: 5666 bytes --]
ENP3S0 -> WAN
ENP2S0 -> LAN
gustavo@N4050:~/bin$ ./speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential
2021-03-05 18:35:13 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
..............................................................
Download: 89.00 Mbps
Latency: [in msec, 62 pings, 0,00% packet loss]
Min: 102,000
10pct: 109,000
Median: 46,000
Avg: 88,210
90pct: 89,000
Max: 99,000
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 54 samples]
cpu0: 7,4 +/- 2,3 @ 1013 MHz
cpu1: 8,4 +/- 12,2 @ 947 MHz
cpu2: 7,1 +/- 2,5 @ 998 MHz
cpu3: 24,6 +/- 4,3 @ 931 MHz
Overhead: [in % used of total CPU available]
netperf: 6,0
................................................................
Upload: 18.00 Mbps
Latency: [in msec, 65 pings, 0,00% packet loss]
Min: 11,000
10pct: 13,000
Median: 30,000
Avg: 27,677
90pct: 47,000
Max: 9,000
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 57 samples]
cpu0: 4,2 +/- 2,2 @ 931 MHz
cpu1: 3,2 +/- 1,5 @ 908 MHz
cpu2: 5,9 +/- 2,1 @ 978 MHz
cpu3: 7,7 +/- 2,2 @ 916 MHz
Overhead: [in % used of total CPU available]
netperf: 0,8
II
gustavo@N4050:~/bin$ ./speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential
2021-03-05 18:41:51 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
..............................................................
Download: 88.00 Mbps
Latency: [in msec, 62 pings, 0,00% packet loss]
Min: 102,000
10pct: 112,000
Median: 51,000
Avg: 87,806
90pct: 85,000
Max: 97,000
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 54 samples]
cpu0: 9,9 +/- 8,3 @ 1053 MHz
cpu1: 9,4 +/- 9,7 @ 1016 MHz
cpu2: 10,8 +/- 12,4 @ 1031 MHz
cpu3: 26,6 +/- 8,7 @ 975 MHz
Overhead: [in % used of total CPU available]
netperf: 6,0
................................................................
Upload: 18.00 Mbps
Latency: [in msec, 64 pings, 0,00% packet loss]
Min: 10,000
10pct: 12,000
Median: 18,000
Avg: 23,906
90pct: 43,000
Max: 7,000
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 56 samples]
cpu0: 4,3 +/- 2,4 @ 948 MHz
cpu1: 4,1 +/- 2,9 @ 928 MHz
cpu2: 6,5 +/- 2,8 @ 925 MHz
cpu3: 8,0 +/- 3,5 @ 981 MHz
Overhead: [in % used of total CPU available]
netperf: 0,8
ENP3S0 -> LAN
ENP2S0 -> WAN
I
gustavo@N4050:~/bin$ ./speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential
2021-03-06 13:55:02 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
..............................................................
Download: 89.00 Mbps
Latency: [in msec, 60 pings, 0,00% packet loss]
Min: 102,000
10pct: 110,000
Median: 135,000
Avg: 91,933
90pct: 90,000
Max: 96,000
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 54 samples]
cpu0: 5,8 +/- 2,2 @ 991 MHz
cpu1: 3,5 +/- 2,3 @ 906 MHz
cpu2: 2,9 +/- 2,0 @ 989 MHz
cpu3: 23,0 +/- 4,2 @ 878 MHz
Overhead: [in % used of total CPU available]
netperf: 6,1
.................................................................
Upload: 17.00 Mbps
Latency: [in msec, 65 pings, 0,00% packet loss]
Min: 10,000
10pct: 12,000
Median: 29,000
Avg: 28,308
90pct: 44,000
Max: 9,000
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 57 samples]
cpu0: 6,5 +/- 7,3 @ 1001 MHz
cpu1: 5,9 +/- 6,6 @ 1034 MHz
cpu2: 5,7 +/- 7,0 @ 1010 MHz
cpu3: 9,6 +/- 8,7 @ 1019 MHz
Overhead: [in % used of total CPU available]
netperf: 0,8
II
gustavo@N4050:~/bin$ ./speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential
2021-03-06 13:58:03 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
.............................................................
Download: 88.00 Mbps
Latency: [in msec, 59 pings, 0,00% packet loss]
Min: 101,000
10pct: 110,000
Median: 71,000
Avg: 84,017
90pct: 90,000
Max: 98,000
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 54 samples]
cpu0: 6,6 +/- 4,5 @ 1022 MHz
cpu1: 5,6 +/- 6,3 @ 966 MHz
cpu2: 5,2 +/- 7,7 @ 1011 MHz
cpu3: 23,5 +/- 5,0 @ 917 MHz
Overhead: [in % used of total CPU available]
netperf: 5,7
.................................................................
Upload: 18.00 Mbps
Latency: [in msec, 65 pings, 0,00% packet loss]
Min: 101,000
10pct: 11,000
Median: 20,000
Avg: 23,754
90pct: 44,000
Max: 9,000
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 57 samples]
cpu0: 1,7 +/- 1,4 @ 914 MHz
cpu1: 1,3 +/- 1,2 @ 885 MHz
cpu2: 1,5 +/- 1,2 @ 910 MHz
cpu3: 4,2 +/- 1,7 @ 905 MHz
Overhead: [in % used of total CPU available]
netperf: 0,8
[-- Attachment #3: netperf before sqm from router --]
[-- Type: application/octet-stream, Size: 5676 bytes --]
ENP3S0 -> WAN
ENP2S0 -> LAN
gustavo@srv:~/bin$ ./speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential
2021-03-05 18:29:43 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
..............................................................
Download: 202.45 Mbps
Latency: [in msec, 60 pings, 0.00% packet loss]
Min: 10.300
10pct: 14.200
Median: 5.840
Avg: 10.718
90pct: 9.200
Max: 14.100
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 54 samples]
cpu0: 2.4 +/- 1.9 @ 1333 MHz
cpu1: 1.9 +/- 2.1 @ 1333 MHz
cpu2: 10.0 +/- 3.7 @ 1733 MHz
cpu3: 39.0 +/- 10.2 @ 1846 MHz
Overhead: [in % used of total CPU available]
netperf: 5.5
.................................................................
Upload: 20.29 Mbps
Latency: [in msec, 65 pings, 0.00% packet loss]
Min: 10.200
10pct: 14.200
Median: 25.500
Avg: 25.806
90pct: 45.800
Max: 35.100
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 57 samples]
cpu0: 0.7 +/- 0.8 @ 1333 MHz
cpu1: 0.5 +/- 0.6 @ 1333 MHz
cpu2: 0.5 +/- 0.7 @ 1333 MHz
cpu3: 1.7 +/- 0.9 @ 1363 MHz
Overhead: [in % used of total CPU available]
netperf: 0.7
gustavo@srv:~/bin$
II
gustavo@srv:~/bin$ ./speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential
2021-03-05 18:39:00 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
..............................................................
Download: 214.91 Mbps
Latency: [in msec, 62 pings, 0.00% packet loss]
Min: 10.100
10pct: 11.500
Median: 5.710
Avg: 11.056
90pct: 8.680
Max: 22.100
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 54 samples]
cpu0: 1.7 +/- 1.6 @ 1333 MHz
cpu1: 1.6 +/- 1.3 @ 1333 MHz
cpu2: 10.8 +/- 3.9 @ 1743 MHz
cpu3: 43.9 +/- 9.9 @ 1807 MHz
Overhead: [in % used of total CPU available]
netperf: 5.5
.................................................................
Upload: 20.36 Mbps
Latency: [in msec, 65 pings, 0.00% packet loss]
Min: 11.900
10pct: 13.100
Median: 34.600
Avg: 28.831
90pct: 7.670
Max: 40.100
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 57 samples]
cpu0: 0.5 +/- 0.6 @ 1333 MHz
cpu1: 0.5 +/- 0.6 @ 1333 MHz
cpu2: 0.8 +/- 0.6 @ 1333 MHz
cpu3: 1.5 +/- 0.9 @ 1345 MHz
Overhead: [in % used of total CPU available]
netperf: 0.7
ENP3S0 -> LAN
ENP2S0 -> WAN
I
gustavo@srv:~/bin$ ./speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential
2021-03-06 13:48:27 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
.............................................................
Download: 209.78 Mbps
Latency: [in msec, 61 pings, 0.00% packet loss]
Min: 10.000
10pct: 12.700
Median: 6.220
Avg: 10.349
90pct: 8.450
Max: 9.790
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 54 samples]
cpu0: 1.7 +/- 1.7 @ 1333 MHz
cpu1: 1.6 +/- 1.4 @ 1336 MHz
cpu2: 11.1 +/- 3.8 @ 1708 MHz
cpu3: 40.1 +/- 8.5 @ 1788 MHz
Overhead: [in % used of total CPU available]
netperf: 5.7
.................................................................
Upload: 20.29 Mbps
Latency: [in msec, 65 pings, 0.00% packet loss]
Min: 10.200
10pct: 12.600
Median: 32.300
Avg: 28.973
90pct: 45.700
Max: 44.100
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 57 samples]
cpu0: 0.5 +/- 0.6 @ 1333 MHz
cpu1: 0.5 +/- 0.6 @ 1333 MHz
cpu2: 0.7 +/- 0.7 @ 1333 MHz
cpu3: 1.3 +/- 0.9 @ 1337 MHz
Overhead: [in % used of total CPU available]
netperf: 0.7
II
ustavo@srv:~/bin$ ./speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential
2021-03-06 13:51:47 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
.............................................................
Download: 186.45 Mbps
Latency: [in msec, 61 pings, 0.00% packet loss]
Min: 10.600
10pct: 14.900
Median: 6.290
Avg: 8.957
90pct: 8.950
Max: 27.100
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 54 samples]
cpu0: 1.9 +/- 1.4 @ 1333 MHz
cpu1: 1.4 +/- 1.3 @ 1334 MHz
cpu2: 11.0 +/- 4.2 @ 1648 MHz
cpu3: 37.7 +/- 8.7 @ 1742 MHz
Overhead: [in % used of total CPU available]
netperf: 5.3
................................................................
Upload: 19.89 Mbps
Latency: [in msec, 64 pings, 0.00% packet loss]
Min: 10.000
10pct: 12.900
Median: 30.400
Avg: 27.282
90pct: 46.500
Max: 40.100
CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 56 samples]
cpu0: 0.6 +/- 0.6 @ 1333 MHz
cpu1: 0.6 +/- 0.6 @ 1333 MHz
cpu2: 0.4 +/- 0.6 @ 1333 MHz
cpu3: 1.5 +/- 1.0 @ 1347 MHz
Overhead: [in % used of total CPU available]
netperf: 0.7
^ permalink raw reply [flat|nested] 2+ messages in thread