Re: [Cake] Cake on openwrt - falling behind

[Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk>]

[-- Attachment #1: Type: text/plain, Size: 3884 bytes --]



> On 1 Jul 2018, at 14:48, Pete Heist <pete@heistp.net> wrote:
> 
> Ok, my reboots were because I need to compile with om-watchdog support, otherwise the OM2P’s hardware watchdog reboots every few minutes. Thus, my sysupgrades were not _actually_ working (watchdog was interrupting them), thus the unknown symbols / dependency problems, etc, so that’s behind me. :/
> 
> And yes, I get no tin stats without the patch. With the patch, I get tin stats as expected. This is probably nothing new.
> 
> I also compiled on a Raspi 2 (32-bit ARM) running Debian 9.4, and it works as expected without the patch. This probably just confirms what George reported, because I think his wrt1900acs also uses 32-bit ARM.
> 
> So yes, MIPS 32-bit, both le and be are both fails. What next? We could test more archs, but it might be more productive to attempt to figure it out in the code when I can at least repro it on MIPS.
> 
> You mentioned earlier netlink_parse_nested doesn’t seem to handle 64-bit values but I’m not seeing where that is. I’m getting my head around how stats work so just writing out loud…looks like there are qstats from struct Qdisc then the tin stats which are probably custom for Cake and stored at some address obtained from qdisc_priv, which is right after the regular Qdisc struct. Interesting. I’m searching now for how this data ends up in tc’s hands, which looks like happens in cake_print_xstats with parse_rtattr_nested, then the printing gets underway, and at some point if st[TCA_CAKE_STATS_TIN_STATS] is true it gets into the tin printing business. What’s the condition that fails to make the tin stats not be printed? Best I know is to add debugging statements to figure it out, which I might give a try later today. Am I on the right track?

I think we’ve all been heading in a similar direction that’s for sure.  The symptoms are that netlink doesn’t appear to handle 64bit values on MIPS well because generally as soon as you put a 64 bit value in the netlink stream, things go a bit wonky.

I think this is some sort of alignment problem. e.g.  If I get cake kmod to insert a dummy 32bit pad netlink attribute at the top of ‘cake_dump’, then I see the tin stats magically appear, although some of the values ‘capacity’ ‘threshold’ ‘sent bytes’ are stupidly high, as if an endian issue has hit.

In openwrt world, mips is not defined as an architecture that can do ‘efficient unaligned access’, thus if you look in include/net/netlink.h you’ll see

static inline bool nla_need_padding_for_64bit(struct sk_buff *skb)
{
#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
        /* The nlattr header is 4 bytes in size, that's why we test
         * if the skb->data _is_ aligned.  A NOP attribute, plus
         * nlattr header for next attribute, will make nla_data()
         * 8-byte aligned.
         */
        if (IS_ALIGNED((unsigned long)skb_tail_pointer(skb), 8))
                return true;
#endif
        return false;
}

which looks to be designed to align netlink attributes in a friendly way.

I haven’t looked yet to see if working archs define ‘CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS’ and non working archs don’t….but to me it’s an obvious place to start poking, as we clearly have some/most archs wher the iproute/tc & cake code talk well….and one arch where they don’t…. to me that sort of rules out tc/cake doing anything silly.

There’s a ‘kmon-nlmon’ that allows you to create a virtual network interface for the netlink packets and thus capture (tcpdump) and export to wireshark for dissection.  I had a look at that a few weeks ago but my head exploded and have only just recovered enough to start looking at this problem again…. the EFFECIENT_UNALIGNED_ACCESS thing is something I discovered about 2 hours ago.

Kevin



[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]