From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.toke.dk (mail.toke.dk [52.28.52.200]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AAC9F3BA8E for ; Sat, 5 Jan 2019 06:18:59 -0500 (EST) Date: Sat, 05 Jan 2019 12:18:52 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=toke.dk; s=20161023; t=1546687138; bh=KCyysn6tRcC7MbczcFJCwimqHqSS1BOKk5oDYUJemCk=; h=Date:In-Reply-To:References:Subject:To:CC:From:From; b=jZuIhwXa+DyV9R+oHxGHFtV8oxGrKPsku2n+k9thgQPOH88j/xYM7b7z6GFGYfWTW nknLxx+oClHnIS0siAUM+/zNC3xcRCH0r4wkaMjPMLu9ntSLoTYkgP4EphP7H8Gs10 cAMhDk2FrLpOt5MaFTttVYiqQAReha/8POKtGghFbVGjBzTzQDLhzkOj3DWJzqeHmD fEN90Pb4LUuPlqE015Fru9c/RBc6KeW/e3js37I+YgyWhDz4Ihf5x8XMPWxCO3cdIP 0ubnrTNCrB8jodo8M7xr0/hfepwCXD/ZNv7WMP9KweK3HCCoa0zuznMX/ZB1nFquuT Q7QmO01pnxB+Q== In-Reply-To: <49A6DCF8-BE98-47F4-9C66-6B4288390A58@heistp.net> References: <5482A3CA-9C36-4DDE-A858-24D8467F70C7@heistp.net> <8736q8yumt.fsf@toke.dk> <4C422792-7E51-4DBA-A229-FA7D3F987FB6@heistp.net> <87zhsgxdao.fsf@toke.dk> <87wonjxvss.fsf@toke.dk> <0077CC34-490F-4D76-82D3-BE37B27F2E1C@heistp.net> <49A6DCF8-BE98-47F4-9C66-6B4288390A58@heistp.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable To: Pete Heist CC: Cake List From: =?ISO-8859-1?Q?Toke_H=F8iland-J=F8rgensen?= X-Clacks-Overhead: GNU Terry Pratchett Message-ID: <52EF4FE7-0DF4-402F-A550-008A8F1DDDFA@toke.dk> Subject: Re: [Cake] cake infinite loop(?) with hfsc on one-armed router X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jan 2019 11:19:00 -0000 Reverse, is that with an ingress qdisc? -Toke On 5 January 2019 12:06:44 CET, Pete Heist wrote: >Quick update to the trace because I had to apply the patch manually and >missed one line to remove (qdisc_tree_reduce_backlog=2E=2E=2E), just so i= t >doesn=E2=80=99t through off the addresses for you, but it still does the = same >thing: > >root@apu1a:~/src/sch_cake# [ 697=2E089814] BUG: unable to handle kernel >NULL pointer dereference at 00000008 >[ 697=2E097009] IP: [] adjust_parent_qlen+0x37/0xf08 >[sch_cake] >[ 697=2E103491] *pde =3D 00000000=20 >[ 697=2E106405] Oops: 0000 [#1] SMP=20 >[ 697=2E109697] Modules linked in: em_meta cls_basic sch_hfsc >sch_cake(O) xt_ACCOUNT(O) ipt_REJECT xt_recent iptable_mangle >iptable_nat nfn >[ 697=2E160768] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O=20 >3=2E16=2E7-ckt9-voyage #1 >[ 697=2E168776] Hardware name: PC Engines APU/APU, BIOS 4=2E0 09/08/2014 >[ 697=2E174957] task: f5c811d0 ti: f5c86000 task=2Eti: f5c86000 >[ 697=2E180366] EIP: 0060:[] EFLAGS: 00210206 CPU: 1 >[ 697=2E185862] EIP is at adjust_parent_qlen+0x37/0xf08 [sch_cake] >[ 697=2E191701] EAX: f5cdd000 EBX: ffffffe8 ECX: 00000000 EDX: 00000003 >[ 697=2E197977] ESI: 00010000 EDI: f2f00000 EBP: f5c99cf0 ESP: f5c99ce8 >[ 697=2E204250] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 >[ 697=2E209648] CR0: 8005003b CR2: 00000008 CR3: 305c6000 CR4: 00000790 >[ 697=2E215913] Stack: >[ 697=2E217932] 000000ba ef8f0000 f5c99d84 f9f3a12a e0b706f9 0209090a >000011be 000017a8 >[ 697=2E225879] 00000000 00000004 000000b9 f2f02e80 00200246 000000ba >ef8f0100 c131996d >[ 697=2E233828] 93ceb5ff 0000009f f5486e80 f5486e80 f5c99dbc f5c99d54 >0209090a 0209480a >[ 697=2E241776] Call Trace: >[ 697=2E244239] [] adjust_parent_qlen+0x92d/0xf08 [sch_cake] >[ 697=2E250524] [] ? _raw_spin_unlock_bh+0x13/0x15 >[ 697=2E255936] [] ? tc_classify_compat+0x2f/0x5f >[ 697=2E261254] [] ? tc_classify+0x1a/0x8b >[ 697=2E265967] [] 0xf9f45a8e >[ 697=2E269564] [] __dev_queue_xmit+0x210/0x35f >[ 697=2E274715] [] ? ip_fragment+0x79f/0x79f >[ 697=2E279601] [] dev_queue_xmit+0xa/0xc >[ 697=2E284231] [] neigh_resolve_output+0x12f/0x145 >[ 697=2E289727] [] ip_finish_output+0x352/0x73d >[ 697=2E294872] [] ip_output+0x73/0xaf >[ 697=2E299240] [] ip_forward_finish+0x66/0x6b >[ 697=2E304301] [] ip_forward+0x2b0/0x36d >[ 697=2E308930] [] ip_rcv_finish+0x267/0x29a >[ 697=2E313817] [] ip_rcv+0x2b4/0x338 >[ 697=2E318103] [] __netif_receive_skb_core+0x467/0x4b6 >[ 697=2E323944] [] __netif_receive_skb+0x48/0x59 >[ 697=2E329176] [] netif_receive_skb_internal+0x59/0x85 >[ 697=2E335017] [] napi_gro_complete+0x87/0x8c >[ 697=2E340080] [] napi_gro_flush+0x3e/0x53 >[ 697=2E344880] [] napi_complete+0x17/0x27 >[ 697=2E349594] [] 0xf81161a2 >[ 697=2E353186] [] net_rx_action+0x56/0x10e >[ 697=2E357986] [] __do_softirq+0x91/0x175 >[ 697=2E362701] [] ? __hrtimer_tasklet_trampoline+0x1a/0x1a >[ 697=2E368886] [] do_softirq_own_stack+0x1d/0x23 >[ 697=2E374203] =20 >[ 697=2E376136] [] irq_exit+0x34/0x75 >[ 697=2E380646] [] do_IRQ+0x92/0xa6 >[ 697=2E384755] [] common_interrupt+0x2c/0x40 >[ 697=2E389733] [] ? cpuidle_enter_state+0x37/0x96 >[ 697=2E395137] [] cpuidle_enter+0xf/0x12 >[ 697=2E399768] [] cpu_startup_entry+0x135/0x1e1 >[ 697=2E405002] [] start_secondary+0x1a6/0x1ab >[ 697=2E410056] Code: 70 24 85 f6 74 5e 66 31 f6 81 fe 00 00 ff ff 74 53 >8b 40 34 8b 00 f6 40 08 01 75 05 39 70 20 74 16 8b 58 18 83 eb 183 >[ 697=2E430491] EIP: [] adjust_parent_qlen+0x37/0xf08 >[sch_cake] SS:ESP 0068:f5c99ce8 >[ 697=2E438891] CR2: 0000000000000008 >[ 697=2E442220] ---[ end trace 4fdb119875d1f11d ]--- >[ 697=2E446847] Kernel panic - not syncing: Fatal exception in interrupt >[ 697=2E453281] Kernel Offset: 0x0 from 0xc1000000 (relocation range: >0xc0000000-0xf7ffdfff) >[ 697=2E461426] Rebooting in 30 seconds=2E=2E > > >> On Jan 5, 2019, at 11:59 AM, Pete Heist wrote: >>=20 >>=20 >>> On Jan 5, 2019, at 11:06 AM, Toke H=C3=B8iland-J=C3=B8rgensen >wrote: >>>=20 >>> Hmm, try this version for 3=2E16 - probably doesn't work on later >kernels=2E >>> I'll look into a proper backport once you've confirmed that it works >:) >>=20 >> Thanks! Quick reminder, I=E2=80=99ve only seen this happen with hfsc, n= ot >when cake is a leaf below htb, for whatever reason, but that aside=2E=2E= =2E >>=20 >> After the patch I was able to do an iperf3 upload through the >one-armed router (receive on default VLAN and send on tagged VLAN), but >when I ran iperf3 in reverse mode (receive on tagged VLAN and send on >default VLAN), this happened right away (also see compile warnings >below): >>=20 >>=20 >> root@apu1a:~# [ 341=2E268556] BUG: unable to handle kernel NULL >pointer dereference at 00000008 >> [ 341=2E275801] IP: [] adjust_parent_qlen+0x37/0xf1a >[sch_cake] >> [ 341=2E282290] *pde =3D 00000000=20 >> [ 341=2E285203] Oops: 0000 [#1] SMP=20 >> [ 341=2E288496] Modules linked in: em_meta cls_basic sch_hfsc >sch_cake(O) xt_ACCOUNT(O) ipt_REJECT xt_recent iptable_mangle >iptable_nat nfi >> [ 341=2E339568] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O= =20 >3=2E16=2E7-ckt9-voyage #1 >> [ 341=2E347576] Hardware name: PC Engines APU/APU, BIOS 4=2E0 09/08/20= 14 >> [ 341=2E353765] task: f5c811d0 ti: f5c86000 task=2Eti: f5c86000 >> [ 341=2E359173] EIP: 0060:[] EFLAGS: 00210206 CPU: 1 >> [ 341=2E364669] EIP is at adjust_parent_qlen+0x37/0xf1a [sch_cake] >> [ 341=2E370508] EAX: f5d65000 EBX: ffffffe8 ECX: 00000000 EDX: >00000003 >> [ 341=2E376774] ESI: 00010000 EDI: f2900000 EBP: f5c99cf0 ESP: >f5c99ce8 >> [ 341=2E383041] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 >> [ 341=2E388447] CR0: 8005003b CR2: 00000008 CR3: 35cde000 CR4: >00000790 >> [ 341=2E394713] Stack: >> [ 341=2E396734] 000001e1 f56f0000 f5c99d84 fa0e913c e0b706f9 0209090a >f2907840 000017a8 >> [ 341=2E404678] 00000000 00000004 000001e0 000001e1 f56f0100 f2a90000 >000017a8 c131996d >> [ 341=2E412626] ca4edbd8 0000004d f54fca00 f54fca00 f5c99dbc f5c99d54 >0209090a 0209480a >> [ 341=2E420573] Call Trace: >> [ 341=2E423037] [] adjust_parent_qlen+0x93f/0xf1a >[sch_cake] >> [ 341=2E429322] [] ? _raw_spin_unlock_bh+0x13/0x15 >> [ 341=2E434734] [] ? tc_classify_compat+0x2f/0x5f >> [ 341=2E440054] [] ? tc_classify+0x1a/0x8b >> [ 341=2E444767] [] 0xfa0f4a8e >> [ 341=2E448361] [] __dev_queue_xmit+0x210/0x35f >> [ 341=2E453504] [] ? ip_fragment+0x79f/0x79f >> [ 341=2E458392] [] dev_queue_xmit+0xa/0xc >> [ 341=2E463023] [] neigh_resolve_output+0x12f/0x145 >> [ 341=2E468517] [] ip_finish_output+0x352/0x73d >> [ 341=2E473664] [] ip_output+0x73/0xaf >> [ 341=2E478033] [] ip_forward_finish+0x66/0x6b >> [ 341=2E483091] [] ip_forward+0x2b0/0x36d >> [ 341=2E487720] [] ip_rcv_finish+0x267/0x29a >> [ 341=2E492607] [] ip_rcv+0x2b4/0x338 >> [ 341=2E496894] [] __netif_receive_skb_core+0x467/0x4b6 >> [ 341=2E502741] [] __netif_receive_skb+0x48/0x59 >> [ 341=2E507975] [] netif_receive_skb_internal+0x59/0x85 >> [ 341=2E513818] [] napi_gro_complete+0x87/0x8c >> [ 341=2E518878] [] napi_gro_flush+0x3e/0x53 >> [ 341=2E523680] [] napi_complete+0x17/0x27 >> [ 341=2E528394] [] 0xf80361a2 >> [ 341=2E531985] [] net_rx_action+0x56/0x10e >> [ 341=2E536785] [] __do_softirq+0x91/0x175 >> [ 341=2E541501] [] ? __hrtimer_tasklet_trampoline+0x1a/0x1a >> [ 341=2E547685] [] do_softirq_own_stack+0x1d/0x23 >> [ 341=2E553002] =20 >> [ 341=2E554933] [] irq_exit+0x34/0x75 >> [ 341=2E559445] [] do_IRQ+0x92/0xa6 >> [ 341=2E563554] [] common_interrupt+0x2c/0x40 >> [ 341=2E568531] [] ? cpuidle_enter_state+0x37/0x96 >> [ 341=2E573936] [] cpuidle_enter+0xf/0x12 >> [ 341=2E578567] [] cpu_startup_entry+0x135/0x1e1 >> [ 341=2E583802] [] start_secondary+0x1a6/0x1ab >> [ 341=2E588856] Code: 70 24 85 f6 74 5e 66 31 f6 81 fe 00 00 ff ff 74 >53 8b 40 34 8b 00 f6 40 08 01 75 05 39 70 20 74 16 8b 58 18 83 eb 18e >> [ 341=2E609292] EIP: [] adjust_parent_qlen+0x37/0xf1a >[sch_cake] SS:ESP 0068:f5c99ce8 >> [ 341=2E617689] CR2: 0000000000000008 >> [ 341=2E621012] ---[ end trace db8ecd998020cc49 ]--- >> [ 341=2E625639] Kernel panic - not syncing: Fatal exception in >interrupt >> [ 341=2E632073] Kernel Offset: 0x0 from 0xc1000000 (relocation range: >0xc0000000-0xf7ffdfff) >> [ 341=2E640215] Rebooting in 30 seconds=2E=2E >>=20 >>=20 >>=20 >> Probably less critically, some compile warnings: >>=20 >> root@apu1a:~/src/sch_cake# make clean >> make[1]: Entering directory >'/usr/src/linux-headers-3=2E16=2E7-ckt9-voyage' >> CLEAN /root/src/sch_cake/=2Etmp_versions >> CLEAN /root/src/sch_cake/Module=2Esymvers >> make[1]: Leaving directory >'/usr/src/linux-headers-3=2E16=2E7-ckt9-voyage' >> root@apu1a:~/src/sch_cake# make >> make[1]: Entering directory >'/usr/src/linux-headers-3=2E16=2E7-ckt9-voyage' >> CC [M] /root/src/sch_cake/sch_cake=2Eo >> /root/src/sch_cake/sch_cake=2Ec: In function =E2=80=98adjust_parent_qle= n=E2=80=99: >> /root/src/sch_cake/sch_cake=2Ec:1753:31: warning: passing argument 1 of >=E2=80=98qdisc_match_from_root=E2=80=99 from incompatible pointer type >> sch =3D qdisc_match_from_root(qdisc_dev(sch), TC_H_MAJ(parentid)); >> ^ >> /root/src/sch_cake/sch_cake=2Ec:1727:22: note: expected =E2=80=98struct= Qdisc >*=E2=80=99 but argument is of type =E2=80=98struct net_device *=E2=80=99 >> static struct Qdisc *qdisc_match_from_root(struct Qdisc *root, u32 >handle) >> ^ >> Building modules, stage 2=2E >> MODPOST 1 modules >> CC /root/src/sch_cake/sch_cake=2Emod=2Eo >> LD [M] /root/src/sch_cake/sch_cake=2Eko >> make[1]: Leaving directory >'/usr/src/linux-headers-3=2E16=2E7-ckt9-voyage' >>=20