[Cake] cake infinite loop(?) with hfsc on one-armed router
Toke Høiland-Jørgensen
toke at toke.dk
Sat Jan 5 06:18:52 EST 2019
Reverse, is that with an ingress qdisc?
-Toke
On 5 January 2019 12:06:44 CET, Pete Heist <pete at heistp.net> wrote:
>Quick update to the trace because I had to apply the patch manually and
>missed one line to remove (qdisc_tree_reduce_backlog...), just so it
>doesn’t through off the addresses for you, but it still does the same
>thing:
>
>root at apu1a:~/src/sch_cake# [ 697.089814] BUG: unable to handle kernel
>NULL pointer dereference at 00000008
>[ 697.097009] IP: [<f9f39834>] adjust_parent_qlen+0x37/0xf08
>[sch_cake]
>[ 697.103491] *pde = 00000000
>[ 697.106405] Oops: 0000 [#1] SMP
>[ 697.109697] Modules linked in: em_meta cls_basic sch_hfsc
>sch_cake(O) xt_ACCOUNT(O) ipt_REJECT xt_recent iptable_mangle
>iptable_nat nfn
>[ 697.160768] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O
>3.16.7-ckt9-voyage #1
>[ 697.168776] Hardware name: PC Engines APU/APU, BIOS 4.0 09/08/2014
>[ 697.174957] task: f5c811d0 ti: f5c86000 task.ti: f5c86000
>[ 697.180366] EIP: 0060:[<f9f39834>] EFLAGS: 00210206 CPU: 1
>[ 697.185862] EIP is at adjust_parent_qlen+0x37/0xf08 [sch_cake]
>[ 697.191701] EAX: f5cdd000 EBX: ffffffe8 ECX: 00000000 EDX: 00000003
>[ 697.197977] ESI: 00010000 EDI: f2f00000 EBP: f5c99cf0 ESP: f5c99ce8
>[ 697.204250] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
>[ 697.209648] CR0: 8005003b CR2: 00000008 CR3: 305c6000 CR4: 00000790
>[ 697.215913] Stack:
>[ 697.217932] 000000ba ef8f0000 f5c99d84 f9f3a12a e0b706f9 0209090a
>000011be 000017a8
>[ 697.225879] 00000000 00000004 000000b9 f2f02e80 00200246 000000ba
>ef8f0100 c131996d
>[ 697.233828] 93ceb5ff 0000009f f5486e80 f5486e80 f5c99dbc f5c99d54
>0209090a 0209480a
>[ 697.241776] Call Trace:
>[ 697.244239] [<f9f3a12a>] adjust_parent_qlen+0x92d/0xf08 [sch_cake]
>[ 697.250524] [<c131996d>] ? _raw_spin_unlock_bh+0x13/0x15
>[ 697.255936] [<c129f91c>] ? tc_classify_compat+0x2f/0x5f
>[ 697.261254] [<c12a091e>] ? tc_classify+0x1a/0x8b
>[ 697.265967] [<f9f45a8f>] 0xf9f45a8e
>[ 697.269564] [<c128b628>] __dev_queue_xmit+0x210/0x35f
>[ 697.274715] [<c12af74e>] ? ip_fragment+0x79f/0x79f
>[ 697.279601] [<c128b78b>] dev_queue_xmit+0xa/0xc
>[ 697.284231] [<c12916f9>] neigh_resolve_output+0x12f/0x145
>[ 697.289727] [<c12afaa0>] ip_finish_output+0x352/0x73d
>[ 697.294872] [<c12b0e19>] ip_output+0x73/0xaf
>[ 697.299240] [<c12ad986>] ip_forward_finish+0x66/0x6b
>[ 697.304301] [<c12adc3b>] ip_forward+0x2b0/0x36d
>[ 697.308930] [<c12ac467>] ip_rcv_finish+0x267/0x29a
>[ 697.313817] [<c12aca4c>] ip_rcv+0x2b4/0x338
>[ 697.318103] [<c12895dd>] __netif_receive_skb_core+0x467/0x4b6
>[ 697.323944] [<c1289674>] __netif_receive_skb+0x48/0x59
>[ 697.329176] [<c1289cb9>] netif_receive_skb_internal+0x59/0x85
>[ 697.335017] [<c1289d6c>] napi_gro_complete+0x87/0x8c
>[ 697.340080] [<c128a020>] napi_gro_flush+0x3e/0x53
>[ 697.344880] [<c128a04c>] napi_complete+0x17/0x27
>[ 697.349594] [<f81161a3>] 0xf81161a2
>[ 697.353186] [<c128a0b2>] net_rx_action+0x56/0x10e
>[ 697.357986] [<c102d689>] __do_softirq+0x91/0x175
>[ 697.362701] [<c102d5f8>] ? __hrtimer_tasklet_trampoline+0x1a/0x1a
>[ 697.368886] [<c10033c3>] do_softirq_own_stack+0x1d/0x23
>[ 697.374203] <IRQ>
>[ 697.376136] [<c102d8a9>] irq_exit+0x34/0x75
>[ 697.380646] [<c1002f30>] do_IRQ+0x92/0xa6
>[ 697.384755] [<c131a4ec>] common_interrupt+0x2c/0x40
>[ 697.389733] [<c126add1>] ? cpuidle_enter_state+0x37/0x96
>[ 697.395137] [<c126aee8>] cpuidle_enter+0xf/0x12
>[ 697.399768] [<c1051e54>] cpu_startup_entry+0x135/0x1e1
>[ 697.405002] [<c101d553>] start_secondary+0x1a6/0x1ab
>[ 697.410056] Code: 70 24 85 f6 74 5e 66 31 f6 81 fe 00 00 ff ff 74 53
>8b 40 34 8b 00 f6 40 08 01 75 05 39 70 20 74 16 8b 58 18 83 eb 183
>[ 697.430491] EIP: [<f9f39834>] adjust_parent_qlen+0x37/0xf08
>[sch_cake] SS:ESP 0068:f5c99ce8
>[ 697.438891] CR2: 0000000000000008
>[ 697.442220] ---[ end trace 4fdb119875d1f11d ]---
>[ 697.446847] Kernel panic - not syncing: Fatal exception in interrupt
>[ 697.453281] Kernel Offset: 0x0 from 0xc1000000 (relocation range:
>0xc0000000-0xf7ffdfff)
>[ 697.461426] Rebooting in 30 seconds..
>
>
>> On Jan 5, 2019, at 11:59 AM, Pete Heist <pete at heistp.net> wrote:
>>
>>
>>> On Jan 5, 2019, at 11:06 AM, Toke Høiland-Jørgensen <toke at toke.dk>
>wrote:
>>>
>>> Hmm, try this version for 3.16 - probably doesn't work on later
>kernels.
>>> I'll look into a proper backport once you've confirmed that it works
>:)
>>
>> Thanks! Quick reminder, I’ve only seen this happen with hfsc, not
>when cake is a leaf below htb, for whatever reason, but that aside...
>>
>> After the patch I was able to do an iperf3 upload through the
>one-armed router (receive on default VLAN and send on tagged VLAN), but
>when I ran iperf3 in reverse mode (receive on tagged VLAN and send on
>default VLAN), this happened right away (also see compile warnings
>below):
>>
>>
>> root at apu1a:~# [ 341.268556] BUG: unable to handle kernel NULL
>pointer dereference at 00000008
>> [ 341.275801] IP: [<fa0e8834>] adjust_parent_qlen+0x37/0xf1a
>[sch_cake]
>> [ 341.282290] *pde = 00000000
>> [ 341.285203] Oops: 0000 [#1] SMP
>> [ 341.288496] Modules linked in: em_meta cls_basic sch_hfsc
>sch_cake(O) xt_ACCOUNT(O) ipt_REJECT xt_recent iptable_mangle
>iptable_nat nfi
>> [ 341.339568] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O
>3.16.7-ckt9-voyage #1
>> [ 341.347576] Hardware name: PC Engines APU/APU, BIOS 4.0 09/08/2014
>> [ 341.353765] task: f5c811d0 ti: f5c86000 task.ti: f5c86000
>> [ 341.359173] EIP: 0060:[<fa0e8834>] EFLAGS: 00210206 CPU: 1
>> [ 341.364669] EIP is at adjust_parent_qlen+0x37/0xf1a [sch_cake]
>> [ 341.370508] EAX: f5d65000 EBX: ffffffe8 ECX: 00000000 EDX:
>00000003
>> [ 341.376774] ESI: 00010000 EDI: f2900000 EBP: f5c99cf0 ESP:
>f5c99ce8
>> [ 341.383041] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
>> [ 341.388447] CR0: 8005003b CR2: 00000008 CR3: 35cde000 CR4:
>00000790
>> [ 341.394713] Stack:
>> [ 341.396734] 000001e1 f56f0000 f5c99d84 fa0e913c e0b706f9 0209090a
>f2907840 000017a8
>> [ 341.404678] 00000000 00000004 000001e0 000001e1 f56f0100 f2a90000
>000017a8 c131996d
>> [ 341.412626] ca4edbd8 0000004d f54fca00 f54fca00 f5c99dbc f5c99d54
>0209090a 0209480a
>> [ 341.420573] Call Trace:
>> [ 341.423037] [<fa0e913c>] adjust_parent_qlen+0x93f/0xf1a
>[sch_cake]
>> [ 341.429322] [<c131996d>] ? _raw_spin_unlock_bh+0x13/0x15
>> [ 341.434734] [<c129f91c>] ? tc_classify_compat+0x2f/0x5f
>> [ 341.440054] [<c12a091e>] ? tc_classify+0x1a/0x8b
>> [ 341.444767] [<fa0f4a8f>] 0xfa0f4a8e
>> [ 341.448361] [<c128b628>] __dev_queue_xmit+0x210/0x35f
>> [ 341.453504] [<c12af74e>] ? ip_fragment+0x79f/0x79f
>> [ 341.458392] [<c128b78b>] dev_queue_xmit+0xa/0xc
>> [ 341.463023] [<c12916f9>] neigh_resolve_output+0x12f/0x145
>> [ 341.468517] [<c12afaa0>] ip_finish_output+0x352/0x73d
>> [ 341.473664] [<c12b0e19>] ip_output+0x73/0xaf
>> [ 341.478033] [<c12ad986>] ip_forward_finish+0x66/0x6b
>> [ 341.483091] [<c12adc3b>] ip_forward+0x2b0/0x36d
>> [ 341.487720] [<c12ac467>] ip_rcv_finish+0x267/0x29a
>> [ 341.492607] [<c12aca4c>] ip_rcv+0x2b4/0x338
>> [ 341.496894] [<c12895dd>] __netif_receive_skb_core+0x467/0x4b6
>> [ 341.502741] [<c1289674>] __netif_receive_skb+0x48/0x59
>> [ 341.507975] [<c1289cb9>] netif_receive_skb_internal+0x59/0x85
>> [ 341.513818] [<c1289d6c>] napi_gro_complete+0x87/0x8c
>> [ 341.518878] [<c128a020>] napi_gro_flush+0x3e/0x53
>> [ 341.523680] [<c128a04c>] napi_complete+0x17/0x27
>> [ 341.528394] [<f80361a3>] 0xf80361a2
>> [ 341.531985] [<c128a0b2>] net_rx_action+0x56/0x10e
>> [ 341.536785] [<c102d689>] __do_softirq+0x91/0x175
>> [ 341.541501] [<c102d5f8>] ? __hrtimer_tasklet_trampoline+0x1a/0x1a
>> [ 341.547685] [<c10033c3>] do_softirq_own_stack+0x1d/0x23
>> [ 341.553002] <IRQ>
>> [ 341.554933] [<c102d8a9>] irq_exit+0x34/0x75
>> [ 341.559445] [<c1002f30>] do_IRQ+0x92/0xa6
>> [ 341.563554] [<c131a4ec>] common_interrupt+0x2c/0x40
>> [ 341.568531] [<c126add1>] ? cpuidle_enter_state+0x37/0x96
>> [ 341.573936] [<c126aee8>] cpuidle_enter+0xf/0x12
>> [ 341.578567] [<c1051e54>] cpu_startup_entry+0x135/0x1e1
>> [ 341.583802] [<c101d553>] start_secondary+0x1a6/0x1ab
>> [ 341.588856] Code: 70 24 85 f6 74 5e 66 31 f6 81 fe 00 00 ff ff 74
>53 8b 40 34 8b 00 f6 40 08 01 75 05 39 70 20 74 16 8b 58 18 83 eb 18e
>> [ 341.609292] EIP: [<fa0e8834>] adjust_parent_qlen+0x37/0xf1a
>[sch_cake] SS:ESP 0068:f5c99ce8
>> [ 341.617689] CR2: 0000000000000008
>> [ 341.621012] ---[ end trace db8ecd998020cc49 ]---
>> [ 341.625639] Kernel panic - not syncing: Fatal exception in
>interrupt
>> [ 341.632073] Kernel Offset: 0x0 from 0xc1000000 (relocation range:
>0xc0000000-0xf7ffdfff)
>> [ 341.640215] Rebooting in 30 seconds..
>>
>>
>>
>> Probably less critically, some compile warnings:
>>
>> root at apu1a:~/src/sch_cake# make clean
>> make[1]: Entering directory
>'/usr/src/linux-headers-3.16.7-ckt9-voyage'
>> CLEAN /root/src/sch_cake/.tmp_versions
>> CLEAN /root/src/sch_cake/Module.symvers
>> make[1]: Leaving directory
>'/usr/src/linux-headers-3.16.7-ckt9-voyage'
>> root at apu1a:~/src/sch_cake# make
>> make[1]: Entering directory
>'/usr/src/linux-headers-3.16.7-ckt9-voyage'
>> CC [M] /root/src/sch_cake/sch_cake.o
>> /root/src/sch_cake/sch_cake.c: In function ‘adjust_parent_qlen’:
>> /root/src/sch_cake/sch_cake.c:1753:31: warning: passing argument 1 of
>‘qdisc_match_from_root’ from incompatible pointer type
>> sch = qdisc_match_from_root(qdisc_dev(sch), TC_H_MAJ(parentid));
>> ^
>> /root/src/sch_cake/sch_cake.c:1727:22: note: expected ‘struct Qdisc
>*’ but argument is of type ‘struct net_device *’
>> static struct Qdisc *qdisc_match_from_root(struct Qdisc *root, u32
>handle)
>> ^
>> Building modules, stage 2.
>> MODPOST 1 modules
>> CC /root/src/sch_cake/sch_cake.mod.o
>> LD [M] /root/src/sch_cake/sch_cake.ko
>> make[1]: Leaving directory
>'/usr/src/linux-headers-3.16.7-ckt9-voyage'
>>
More information about the Cake
mailing list