[Cake] some comprehensive arm64 w/cake results
Dave Taht
dave.taht at gmail.com
Tue Oct 24 18:06:41 EDT 2023
that is a really out of date doc on the bufferbloat site. Last I ran rg for
the network driver tree it was in the many hundreds depending on how you
count variants, but hundreds more left undone.
There is a guide to creating the six or so lines needed here:
http://www.taht.net/~d/broadcom_aug9_2018.pdf
Somewhere around on some preso of mine was the before/after for the
beaglebone black. It was NICE.
On Tue, Oct 24, 2023 at 2:35 PM dave seddon <dave.seddon.ca at gmail.com>
wrote:
> G'day Dave,
>
> Regarding the devices under test with Byte Queue Limits (BQL).
>
> lychee pi ( https://wiki.sipeed.com/hardware/en/lichee/th1520/lp4a.html )
> with driver "st_gmac" does NOT have BQL. ( not a great surprise )
>
> Driver is:
>
> https://github.com/randomizedcoder/cake/blob/main/device_info/lpi4a/ethtool_driver_end0
>
> das at lpi4a:~$ /usr/sbin/ethtool -i end0
> driver: st_gmac <-----------
> version: Jan_2016
> firmware-version:
> expansion-rom-version:
> bus-info:
> supports-statistics: yes
> supports-test: no
> supports-eeprom-access: no
> supports-register-dump: yes
> supports-priv-flags: no
>
> Jetson-nano has driver r8169 and is also not BQL enabled.
>
> https://github.com/randomizedcoder/cake/blob/main/device_info/jetson-nano/ethtool_driver_eth0
>
> Based on doc here:
> https://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Codel/,
> the current drivers with BQL are:
>
> das at t:~/Downloads/linux$ find drivers/net/ -name '*.c' -print | \
> xargs fgrep -l netdev_completed_queue
> drivers/net/can/spi/mcp251xfd/mcp251xfd-tef.c
> drivers/net/can/usb/etas_es58x/es58x_core.c
> drivers/net/can/dev/length.c
> drivers/net/ipa/ipa_gsi.c
> drivers/net/wan/fsl_ucc_hdlc.c
> drivers/net/ethernet/freescale/ucc_geth.c
> drivers/net/ethernet/marvell/skge.c
> drivers/net/ethernet/marvell/sky2.c
> drivers/net/ethernet/qualcomm/emac/emac-mac.c
> drivers/net/ethernet/socionext/netsec.c
> drivers/net/ethernet/atheros/ag71xx.c
> drivers/net/ethernet/mediatek/mtk_star_emac.c
> drivers/net/ethernet/3com/3c59x.c
> drivers/net/ethernet/broadcom/bcm4908_enet.c
> drivers/net/ethernet/broadcom/bgmac.c
> drivers/net/ethernet/broadcom/b44.c
> drivers/net/ethernet/broadcom/bcm63xx_enet.c
> drivers/net/ethernet/realtek/8139cp.c <------- close,
> but NOT r8169
> drivers/net/ethernet/lantiq_xrx200.c
> drivers/net/ethernet/via/via-rhine.c
> drivers/net/ethernet/hisilicon/hix5hd2_gmac.c
> drivers/net/ethernet/hisilicon/hip04_eth.c
> drivers/net/ethernet/hisilicon/hisi_femac.c
> drivers/net/ethernet/intel/e1000e/netdev.c
> drivers/net/ethernet/intel/e1000/e1000_main.c
> drivers/net/ethernet/nvidia/forcedeth.c
>
> The st_gmac driver can also be seen here:
>
> https://github.com/torvalds/linux/blob/4f82870119a46b0d04d91ef4697ac4977a255a9d/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c#L26
>
> I guess I can take a look at creating an MR or two (2).
>
> Regards,
> Dave Seddon
>
> On Tue, Oct 24, 2023 at 9:27 AM dave seddon <dave.seddon.ca at gmail.com>
> wrote:
>
>> G'day,
>>
>> Just to make sure the results are repeatable, I ran the flent tests again
>> for 600 seconds.
>>
>> Directory here
>>
>> https://github.com/randomizedcoder/qdisc_results/tree/main/qdisc/2023-10-24T05%3A16%3A21
>>
>> Example direct links to the flent files:
>>
>> https://github.com/randomizedcoder/qdisc_results/blob/main/qdisc/2023-10-24T05%3A16%3A21/pi4/cake20/flent/test/16_flent/rrul-2023-10-24T061153.944938.2023-10-24T05_16_21_pi4_cake20.flent.gz
>>
>> https://github.com/randomizedcoder/qdisc_results/blob/main/qdisc/2023-10-24T05%3A16%3A21/jetson-nano/fq_codel/flent/test/16_flent/rrul-2023-10-24T081332.189258.2023-10-24T05_16_21_jetson-nano_fq_codel.flent.gz
>>
>> I'm not experienced at reading these flent reports, but it looks to me
>> like fq_codel has similar latency, but higher Mb/s than cake.
>>
>> Jetson nano fq_codel
>> [image: image.png]
>>
>>
>> Jetson nano cake20
>> [image: image.png]
>>
>> The new network card arrived, so I need to work out how to integrate the
>> latency injection. ( I kind of wish I had multiple switches, so I could do
>> q-in-q, because this could dramatically simply the linux latency injecting
>> bridge machine's config. )
>>
>> Kind regards,
>> Dave Seddon
>>
>>
>>
>> On Mon, Oct 23, 2023 at 1:31 PM dave seddon <dave.seddon.ca at gmail.com>
>> wrote:
>>
>>> G'day,
>>>
>>> Dave Taht and I have had a couple of phone conversations now, and he's
>>> convinced me that rather than inserting the netem delay on each laptop,
>>> that latency should be added by a seperate device. To this end, I've got
>>> another little PC and a NIC coming, so that I can repeat all the tests with
>>> seperate latency injection.
>>>
>>> However, I've also completed the flent tests with the laptops adding
>>> latency at each end.
>>>
>>> Full test runs here:
>>>
>>> https://github.com/randomizedcoder/qdisc_results/tree/main/qdisc/2023-10-23T16%3A49%3A10
>>>
>>> You can find the actual rrul flent .tar.gz results for each test.
>>>
>>> e.g
>>> Pi4 fq is here:
>>>
>>> https://github.com/randomizedcoder/qdisc_results/blob/main/qdisc/2023-10-23T16%3A49%3A10/pi4/fq/flent/test/16_flent/rrul-2023-10-23T170016.068273.2023-10-23T16_49_10_pi4_fq.flent.gz
>>>
>>> Lychee Pi Risv with cake qdisc:
>>>
>>> https://github.com/randomizedcoder/qdisc_results/blob/main/qdisc/2023-10-23T16%3A49%3A10/lpi4a/cake20/flent/test/16_flent/rrul-2023-10-23T201354.818316.2023-10-23T16_49_10_lpi4a_cake20.flent.gz
>>>
>>> Just take these with a grain of salt until the new latency injection is
>>> in place.
>>>
>>> ... I'll see if I can script up the generation of all the pretty graphs
>>> soon
>>>
>>> Thanks,
>>> Dave Seddon
>>>
>>>
>>> On Sun, Oct 15, 2023 at 8:11 AM dave seddon <dave.seddon.ca at gmail.com>
>>> wrote:
>>>
>>>> G'day,
>>>>
>>>> I've put more work into a test framework around the qdisc tests, but
>>>> unfortunately flent doesn't work easily with Ubuntu LTS (
>>>> https://github.com/tohojo/flent/issues/232, which I think is an issue
>>>> with flent parsing the fping output ).
>>>>
>>>> Results and graphs in this sheet:
>>>>
>>>> https://docs.google.com/spreadsheets/d/1T59QwEdNwJFm4TgDFA_NY98gicOm8ABXKvDsSIMz9ag/edit#gid=1203641125
>>>>
>>>> Raw results of x2 test runs are here:
>>>>
>>>> https://github.com/randomizedcoder/qdisc_results/blob/main/qdisc/report.csv
>>>>
>>>> Each run:
>>>>
>>>> https://github.com/randomizedcoder/qdisc_results/blob/main/qdisc/2023-10-13T18%3A45%3A45/report.csv
>>>>
>>>> https://github.com/randomizedcoder/qdisc_results/blob/main/qdisc/2023-10-14T14%3A22%3A53/report.csv
>>>>
>>>> Full iperf outputs are available too, for example: https://github.com/randomizedcoder/qdisc_results/blob/main/qdisc/2023-10-13T18%3A45%3A45/nanopi-r2s/fq_codel/iperf/test/16_iperf/stdout
>>>>
>>>>
>>>> Logs for each run are also available, for example:
>>>> https://github.com/randomizedcoder/qdisc_results/blob/main/qdisc/2023-10-13T18%3A45%3A45/log.json
>>>>
>>>> The code repo updated here: https://github.com/randomizedcoder/cake ,
>>>> with thehttps://github.com/randomizedcoder/cake/blob/main/README.md
>>>> which explains how the test work.
>>>> Updated google doc is started here:
>>>> https://docs.google.com/document/d/1fYKj3BS89aB9drg_DsSq289xSdVQhn1zUJYCj0WuCs0/edit?usp=sharing
>>>>
>>>> Based on the questions on this list earlier, there is a folder with
>>>> device information for each of the devices
>>>> https://github.com/randomizedcoder/cake/tree/main/device_info
>>>>
>>>> For example, the Pi4 and the Lichee Pi (risc-v) hardware layout is here:
>>>> - https://github.com/randomizedcoder/cake/blob/main/device_info/pi4/hwloc-ls-pi4.png
>>>>
>>>> -
>>>> https://github.com/randomizedcoder/cake/blob/main/device_info/lpi4a/hwloc-ls-lpi4a.png
>>>>
>>>> The switch has also been upgraded to a Cisco 3750x, which I think based
>>>> on the "show interface" output has a max queue size of 40 frames. The test
>>>> process clears the counters before each test and gathers the "show
>>>> interface" output at the end.
>>>>
>>>> The Lichee Pi 4A doesn't look good (
>>>> https://wiki.sipeed.com/hardware/en/lichee/th1520/lp4a.html )
>>>>
>>>> [image: image.png]
>>>> I really wish the flent was working, so I'll probably see if I can work
>>>> out the parsing.
>>>>
>>>> Thanks,
>>>> Dave Seddon
>>>>
>>>> On Fri, Oct 13, 2023 at 10:25 AM dave seddon <dave.seddon.ca at gmail.com>
>>>> wrote:
>>>>
>>>>> My bad. There's a bug for this.... Looks like I have to downgrade
>>>>> fping
>>>>>
>>>>> https://github.com/tohojo/flent/issues/232
>>>>> https://github.com/schweikert/fping/issues/203
>>>>>
>>>>> On Fri, Oct 13, 2023 at 8:59 AM dave seddon <dave.seddon.ca at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> G'day,
>>>>>>
>>>>>> I've been working away on automation of the tests. Pretty close to
>>>>>> having much nicer tests with a lot more details. I've also got the risc-v
>>>>>> device working.
>>>>>>
>>>>>> However, I've run into something funny with flent. Flent is not
>>>>>> happy with fping or ping.
>>>>>>
>>>>>> das at 3rd:~/Downloads/cake/cmd/run_qdiscs_tests$ /usr/bin/sudo
>>>>>> /usr/sbin/ip netns exec network101 /usr/bin/flent rrul --output
>>>>>> /tmp/qdisc/2023-10-13T15:53:21/pi4/noqueue/flent/test/15_flent/flent_pi4_noqueue.png
>>>>>> --data-dir /tmp/qdisc/2023-10-13T15:53:21/pi4/noqueue/flent/test/15_flent/
>>>>>> --format summary --plot all_scaled --title-extra
>>>>>> 2023-10-13T15:53:21_pi4_noqueue --note 2023-10-13T15:53:21_pi4_noqueue
>>>>>> --extended-metadata --host 172.17.51.10 --length 60 --ipv4 --socket-stats
>>>>>> Starting Flent 2.0.1 using Python 3.10.12.
>>>>>> Starting rrul test. Expected run time: 70 seconds.
>>>>>> WARNING: Found fping, but couldn't parse its output. Not
>>>>>> using. <---------------- ???
>>>>>> ERROR: Runner Ping (ms) ICMP failed check: Cannot parse output of the
>>>>>> system ping binary (/usr/bin/ping). Please install fping v3.5+. <----- ??
>>>>>>
>>>>>> das at 3rd:~/Downloads/cake/cmd/run_qdiscs_tests$ dpkg --list | grep
>>>>>> ping
>>>>>> ii fping 5.1-1
>>>>>> amd64 sends ICMP ECHO_REQUEST packets to network hosts
>>>>>> ii iputils-ping 3:20211215-1
>>>>>> amd64 Tools to test the reachability of network hosts
>>>>>> ii kpartx 0.8.8-1ubuntu1.22.04.1
>>>>>> amd64 create device mappings for partitions
>>>>>> ii libharfbuzz0b:amd64 2.7.4-1ubuntu3.1
>>>>>> amd64 OpenType text shaping engine (shared library)
>>>>>> das at 3rd:~/Downloads/cake/cmd/run_qdiscs_tests$ fping --version
>>>>>> fping: Version 5.1
>>>>>> das at 3rd:~/Downloads/cake/cmd/run_qdiscs_tests$ ping -V
>>>>>> ping from iputils 20211215
>>>>>>
>>>>>> das at 3rd:~/Downloads/cake/cmd/run_qdiscs_tests$ cat /etc/lsb-release
>>>>>> DISTRIB_ID=Ubuntu
>>>>>> DISTRIB_RELEASE=22.04
>>>>>> DISTRIB_CODENAME=jammy
>>>>>> DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"
>>>>>>
>>>>>> I did install via "apt install fping"
>>>>>>
>>>>>> Any thoughts please?
>>>>>>
>>>>>> Kind regards,
>>>>>> Dave
>>>>>>
>>>>>> On Thu, Sep 28, 2023 at 6:27 AM Sebastian Moeller via Cake <
>>>>>> cake at lists.bufferbloat.net> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> > On Sep 28, 2023, at 15:19, David Lang <david at lang.hm> wrote:
>>>>>>> >
>>>>>>> > On Thu, 28 Sep 2023, Sebastian Moeller via Cake wrote:
>>>>>>> >
>>>>>>> >> P.S.: I am tempted, but will likely wait until they are available
>>>>>>> in quantity and hope that the street price comes down a bit before getting
>>>>>>> one ;)
>>>>>>> >
>>>>>>> > They aren't available at all yet, and it's not clear when they
>>>>>>> will be available.
>>>>>>>
>>>>>>> The announcement was end of October, but I think I could
>>>>>>> pre-order right now if I was feeling an urge. You are right though,
>>>>>>> announced != available or delivered.
>>>>>>>
>>>>>>> Regards
>>>>>>> Sebastian
>>>>>>>
>>>>>>> P.S.: I have a pi400 in use as "desktop" for my oldest kid, this is
>>>>>>> close to be actually generally usable, I would guess that changing a
>>>>>>> potential p500 from the pi400's 4GB to 8 GB together with the other
>>>>>>> imprivements the 5 brings might push it over the threshold into the truly
>>>>>>> useful category. Which probably means that either a potential pi500 will
>>>>>>> come late and probably with only 4 GB, but let's see how this works out now
>>>>>>> that the supply situation is less problematic.
>>>>>>> And I understand that there are other capable ARM based SoCs for
>>>>>>> homerouter/desktop duty, I just happen ot have a soft spot for the
>>>>>>> raspberry project ;)
>>>>>>>
>>>>>>> >
>>>>>>> > David Lang
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Cake mailing list
>>>>>>> Cake at lists.bufferbloat.net
>>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Dave Seddon
>>>>>> +1 415 857 5102
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Dave Seddon
>>>>> +1 415 857 5102
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Dave Seddon
>>>> +1 415 857 5102
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Dave Seddon
>>> +1 415 857 5102
>>>
>>
>>
>> --
>> Regards,
>> Dave Seddon
>> +1 415 857 5102
>>
>
>
> --
> Regards,
> Dave Seddon
> +1 415 857 5102
>
--
Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.html
Dave Täht CSO, LibreQos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20231024/bd42afaf/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 59679 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20231024/bd42afaf/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 378056 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20231024/bd42afaf/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 336208 bytes
Desc: not available
URL: <https://lists.bufferbloat.net/pipermail/cake/attachments/20231024/bd42afaf/attachment-0005.png>
More information about the Cake
mailing list