[Cake] cake infinite loop(?) with hfsc on one-armed router
Pete Heist
pete at heistp.net
Sat Jan 5 06:06:44 EST 2019
Quick update to the trace because I had to apply the patch manually and missed one line to remove (qdisc_tree_reduce_backlog...), just so it doesn’t through off the addresses for you, but it still does the same thing:
root at apu1a:~/src/sch_cake# [ 697.089814] BUG: unable to handle kernel NULL pointer dereference at 00000008
[ 697.097009] IP: [<f9f39834>] adjust_parent_qlen+0x37/0xf08 [sch_cake]
[ 697.103491] *pde = 00000000
[ 697.106405] Oops: 0000 [#1] SMP
[ 697.109697] Modules linked in: em_meta cls_basic sch_hfsc sch_cake(O) xt_ACCOUNT(O) ipt_REJECT xt_recent iptable_mangle iptable_nat nfn
[ 697.160768] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O 3.16.7-ckt9-voyage #1
[ 697.168776] Hardware name: PC Engines APU/APU, BIOS 4.0 09/08/2014
[ 697.174957] task: f5c811d0 ti: f5c86000 task.ti: f5c86000
[ 697.180366] EIP: 0060:[<f9f39834>] EFLAGS: 00210206 CPU: 1
[ 697.185862] EIP is at adjust_parent_qlen+0x37/0xf08 [sch_cake]
[ 697.191701] EAX: f5cdd000 EBX: ffffffe8 ECX: 00000000 EDX: 00000003
[ 697.197977] ESI: 00010000 EDI: f2f00000 EBP: f5c99cf0 ESP: f5c99ce8
[ 697.204250] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
[ 697.209648] CR0: 8005003b CR2: 00000008 CR3: 305c6000 CR4: 00000790
[ 697.215913] Stack:
[ 697.217932] 000000ba ef8f0000 f5c99d84 f9f3a12a e0b706f9 0209090a 000011be 000017a8
[ 697.225879] 00000000 00000004 000000b9 f2f02e80 00200246 000000ba ef8f0100 c131996d
[ 697.233828] 93ceb5ff 0000009f f5486e80 f5486e80 f5c99dbc f5c99d54 0209090a 0209480a
[ 697.241776] Call Trace:
[ 697.244239] [<f9f3a12a>] adjust_parent_qlen+0x92d/0xf08 [sch_cake]
[ 697.250524] [<c131996d>] ? _raw_spin_unlock_bh+0x13/0x15
[ 697.255936] [<c129f91c>] ? tc_classify_compat+0x2f/0x5f
[ 697.261254] [<c12a091e>] ? tc_classify+0x1a/0x8b
[ 697.265967] [<f9f45a8f>] 0xf9f45a8e
[ 697.269564] [<c128b628>] __dev_queue_xmit+0x210/0x35f
[ 697.274715] [<c12af74e>] ? ip_fragment+0x79f/0x79f
[ 697.279601] [<c128b78b>] dev_queue_xmit+0xa/0xc
[ 697.284231] [<c12916f9>] neigh_resolve_output+0x12f/0x145
[ 697.289727] [<c12afaa0>] ip_finish_output+0x352/0x73d
[ 697.294872] [<c12b0e19>] ip_output+0x73/0xaf
[ 697.299240] [<c12ad986>] ip_forward_finish+0x66/0x6b
[ 697.304301] [<c12adc3b>] ip_forward+0x2b0/0x36d
[ 697.308930] [<c12ac467>] ip_rcv_finish+0x267/0x29a
[ 697.313817] [<c12aca4c>] ip_rcv+0x2b4/0x338
[ 697.318103] [<c12895dd>] __netif_receive_skb_core+0x467/0x4b6
[ 697.323944] [<c1289674>] __netif_receive_skb+0x48/0x59
[ 697.329176] [<c1289cb9>] netif_receive_skb_internal+0x59/0x85
[ 697.335017] [<c1289d6c>] napi_gro_complete+0x87/0x8c
[ 697.340080] [<c128a020>] napi_gro_flush+0x3e/0x53
[ 697.344880] [<c128a04c>] napi_complete+0x17/0x27
[ 697.349594] [<f81161a3>] 0xf81161a2
[ 697.353186] [<c128a0b2>] net_rx_action+0x56/0x10e
[ 697.357986] [<c102d689>] __do_softirq+0x91/0x175
[ 697.362701] [<c102d5f8>] ? __hrtimer_tasklet_trampoline+0x1a/0x1a
[ 697.368886] [<c10033c3>] do_softirq_own_stack+0x1d/0x23
[ 697.374203] <IRQ>
[ 697.376136] [<c102d8a9>] irq_exit+0x34/0x75
[ 697.380646] [<c1002f30>] do_IRQ+0x92/0xa6
[ 697.384755] [<c131a4ec>] common_interrupt+0x2c/0x40
[ 697.389733] [<c126add1>] ? cpuidle_enter_state+0x37/0x96
[ 697.395137] [<c126aee8>] cpuidle_enter+0xf/0x12
[ 697.399768] [<c1051e54>] cpu_startup_entry+0x135/0x1e1
[ 697.405002] [<c101d553>] start_secondary+0x1a6/0x1ab
[ 697.410056] Code: 70 24 85 f6 74 5e 66 31 f6 81 fe 00 00 ff ff 74 53 8b 40 34 8b 00 f6 40 08 01 75 05 39 70 20 74 16 8b 58 18 83 eb 183
[ 697.430491] EIP: [<f9f39834>] adjust_parent_qlen+0x37/0xf08 [sch_cake] SS:ESP 0068:f5c99ce8
[ 697.438891] CR2: 0000000000000008
[ 697.442220] ---[ end trace 4fdb119875d1f11d ]---
[ 697.446847] Kernel panic - not syncing: Fatal exception in interrupt
[ 697.453281] Kernel Offset: 0x0 from 0xc1000000 (relocation range: 0xc0000000-0xf7ffdfff)
[ 697.461426] Rebooting in 30 seconds..
> On Jan 5, 2019, at 11:59 AM, Pete Heist <pete at heistp.net> wrote:
>
>
>> On Jan 5, 2019, at 11:06 AM, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
>>
>> Hmm, try this version for 3.16 - probably doesn't work on later kernels.
>> I'll look into a proper backport once you've confirmed that it works :)
>
> Thanks! Quick reminder, I’ve only seen this happen with hfsc, not when cake is a leaf below htb, for whatever reason, but that aside...
>
> After the patch I was able to do an iperf3 upload through the one-armed router (receive on default VLAN and send on tagged VLAN), but when I ran iperf3 in reverse mode (receive on tagged VLAN and send on default VLAN), this happened right away (also see compile warnings below):
>
>
> root at apu1a:~# [ 341.268556] BUG: unable to handle kernel NULL pointer dereference at 00000008
> [ 341.275801] IP: [<fa0e8834>] adjust_parent_qlen+0x37/0xf1a [sch_cake]
> [ 341.282290] *pde = 00000000
> [ 341.285203] Oops: 0000 [#1] SMP
> [ 341.288496] Modules linked in: em_meta cls_basic sch_hfsc sch_cake(O) xt_ACCOUNT(O) ipt_REJECT xt_recent iptable_mangle iptable_nat nfi
> [ 341.339568] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O 3.16.7-ckt9-voyage #1
> [ 341.347576] Hardware name: PC Engines APU/APU, BIOS 4.0 09/08/2014
> [ 341.353765] task: f5c811d0 ti: f5c86000 task.ti: f5c86000
> [ 341.359173] EIP: 0060:[<fa0e8834>] EFLAGS: 00210206 CPU: 1
> [ 341.364669] EIP is at adjust_parent_qlen+0x37/0xf1a [sch_cake]
> [ 341.370508] EAX: f5d65000 EBX: ffffffe8 ECX: 00000000 EDX: 00000003
> [ 341.376774] ESI: 00010000 EDI: f2900000 EBP: f5c99cf0 ESP: f5c99ce8
> [ 341.383041] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
> [ 341.388447] CR0: 8005003b CR2: 00000008 CR3: 35cde000 CR4: 00000790
> [ 341.394713] Stack:
> [ 341.396734] 000001e1 f56f0000 f5c99d84 fa0e913c e0b706f9 0209090a f2907840 000017a8
> [ 341.404678] 00000000 00000004 000001e0 000001e1 f56f0100 f2a90000 000017a8 c131996d
> [ 341.412626] ca4edbd8 0000004d f54fca00 f54fca00 f5c99dbc f5c99d54 0209090a 0209480a
> [ 341.420573] Call Trace:
> [ 341.423037] [<fa0e913c>] adjust_parent_qlen+0x93f/0xf1a [sch_cake]
> [ 341.429322] [<c131996d>] ? _raw_spin_unlock_bh+0x13/0x15
> [ 341.434734] [<c129f91c>] ? tc_classify_compat+0x2f/0x5f
> [ 341.440054] [<c12a091e>] ? tc_classify+0x1a/0x8b
> [ 341.444767] [<fa0f4a8f>] 0xfa0f4a8e
> [ 341.448361] [<c128b628>] __dev_queue_xmit+0x210/0x35f
> [ 341.453504] [<c12af74e>] ? ip_fragment+0x79f/0x79f
> [ 341.458392] [<c128b78b>] dev_queue_xmit+0xa/0xc
> [ 341.463023] [<c12916f9>] neigh_resolve_output+0x12f/0x145
> [ 341.468517] [<c12afaa0>] ip_finish_output+0x352/0x73d
> [ 341.473664] [<c12b0e19>] ip_output+0x73/0xaf
> [ 341.478033] [<c12ad986>] ip_forward_finish+0x66/0x6b
> [ 341.483091] [<c12adc3b>] ip_forward+0x2b0/0x36d
> [ 341.487720] [<c12ac467>] ip_rcv_finish+0x267/0x29a
> [ 341.492607] [<c12aca4c>] ip_rcv+0x2b4/0x338
> [ 341.496894] [<c12895dd>] __netif_receive_skb_core+0x467/0x4b6
> [ 341.502741] [<c1289674>] __netif_receive_skb+0x48/0x59
> [ 341.507975] [<c1289cb9>] netif_receive_skb_internal+0x59/0x85
> [ 341.513818] [<c1289d6c>] napi_gro_complete+0x87/0x8c
> [ 341.518878] [<c128a020>] napi_gro_flush+0x3e/0x53
> [ 341.523680] [<c128a04c>] napi_complete+0x17/0x27
> [ 341.528394] [<f80361a3>] 0xf80361a2
> [ 341.531985] [<c128a0b2>] net_rx_action+0x56/0x10e
> [ 341.536785] [<c102d689>] __do_softirq+0x91/0x175
> [ 341.541501] [<c102d5f8>] ? __hrtimer_tasklet_trampoline+0x1a/0x1a
> [ 341.547685] [<c10033c3>] do_softirq_own_stack+0x1d/0x23
> [ 341.553002] <IRQ>
> [ 341.554933] [<c102d8a9>] irq_exit+0x34/0x75
> [ 341.559445] [<c1002f30>] do_IRQ+0x92/0xa6
> [ 341.563554] [<c131a4ec>] common_interrupt+0x2c/0x40
> [ 341.568531] [<c126add1>] ? cpuidle_enter_state+0x37/0x96
> [ 341.573936] [<c126aee8>] cpuidle_enter+0xf/0x12
> [ 341.578567] [<c1051e54>] cpu_startup_entry+0x135/0x1e1
> [ 341.583802] [<c101d553>] start_secondary+0x1a6/0x1ab
> [ 341.588856] Code: 70 24 85 f6 74 5e 66 31 f6 81 fe 00 00 ff ff 74 53 8b 40 34 8b 00 f6 40 08 01 75 05 39 70 20 74 16 8b 58 18 83 eb 18e
> [ 341.609292] EIP: [<fa0e8834>] adjust_parent_qlen+0x37/0xf1a [sch_cake] SS:ESP 0068:f5c99ce8
> [ 341.617689] CR2: 0000000000000008
> [ 341.621012] ---[ end trace db8ecd998020cc49 ]---
> [ 341.625639] Kernel panic - not syncing: Fatal exception in interrupt
> [ 341.632073] Kernel Offset: 0x0 from 0xc1000000 (relocation range: 0xc0000000-0xf7ffdfff)
> [ 341.640215] Rebooting in 30 seconds..
>
>
>
> Probably less critically, some compile warnings:
>
> root at apu1a:~/src/sch_cake# make clean
> make[1]: Entering directory '/usr/src/linux-headers-3.16.7-ckt9-voyage'
> CLEAN /root/src/sch_cake/.tmp_versions
> CLEAN /root/src/sch_cake/Module.symvers
> make[1]: Leaving directory '/usr/src/linux-headers-3.16.7-ckt9-voyage'
> root at apu1a:~/src/sch_cake# make
> make[1]: Entering directory '/usr/src/linux-headers-3.16.7-ckt9-voyage'
> CC [M] /root/src/sch_cake/sch_cake.o
> /root/src/sch_cake/sch_cake.c: In function ‘adjust_parent_qlen’:
> /root/src/sch_cake/sch_cake.c:1753:31: warning: passing argument 1 of ‘qdisc_match_from_root’ from incompatible pointer type
> sch = qdisc_match_from_root(qdisc_dev(sch), TC_H_MAJ(parentid));
> ^
> /root/src/sch_cake/sch_cake.c:1727:22: note: expected ‘struct Qdisc *’ but argument is of type ‘struct net_device *’
> static struct Qdisc *qdisc_match_from_root(struct Qdisc *root, u32 handle)
> ^
> Building modules, stage 2.
> MODPOST 1 modules
> CC /root/src/sch_cake/sch_cake.mod.o
> LD [M] /root/src/sch_cake/sch_cake.ko
> make[1]: Leaving directory '/usr/src/linux-headers-3.16.7-ckt9-voyage'
>
More information about the Cake
mailing list