[Cake] Cake with Deep Packet Inspection

Outback Dingo outbackdingo at gmail.com
Fri Nov 11 12:38:28 EST 2016 

On Fri, Nov 11, 2016 at 10:56 PM, Outback Dingo <outbackdingo at gmail.com> wrote:
> On Fri, Nov 11, 2016 at 10:22 PM, Noah Causin <n0manletter at gmail.com> wrote:
>> What I would try is deleting your build source folder and start from
>> scratch.
>>
>> Then edit source/package/kernel/kmod-sched-cake/Makefile
>>
>> change the PKG_SOURCE_VERSION to 70169dba14daa6ef7907af9c1e922ef9f797993a
>>
>> That's the latest stable version of cake.
>>
>> https://github.com/dtaht/sch_cake/commit/70169dba14daa6ef7907af9c1e922ef9f797993a
>>
>>
>
> Appreciate the attempt to help, but it did this even prior to adding
> cake previously when just using loading sqm
> ill give it a go but i believe the issue is deeper then just cake.


like I stated made the changes and still seeing issues

[  149.490000] INFO: rcu_sched detected stalls on CPUs/tasks:
[  149.490000]  3-...: (0 ticks this GP) idle=b7c/0/0 softirq=3733/3733 fqs=0
[  149.500000]  (detected by 1, t=6003 jiffies, g=1039, c=1038, q=10655)
[  149.500000] Task dump for CPU 3:
[  149.510000] swapper/3       R running      0     0      1 0x00100000
[  149.510000] Stack : 00000000 00005ac7 8e049880 7ffdecec 00000000
80418a90 8045a2a4 80410000
[  149.510000]    8041275c 00000001 00000001 80412680 80412724
80410000 00000000 800132e8
[  149.510000]    00000000 00000001 8fc64000 8fc65ec0 80410000
8005c870 1100fc03 00000003
[  149.510000]    00000000 80410000 8045a2a4 8005c868 80410000
8001ade0 1100fc03 00000000
[  149.510000]    00000004 804124a0 000000a0 8001ade8 fefffbdb
aefffbfb fff77fe7 dfffffef
[  149.510000]    ...
[  149.550000] Call Trace:
[  149.550000] [<8000bba8>] __schedule+0x574/0x758
[  149.560000] [<800132e8>] r4k_wait_irqoff+0x0/0x20
[  149.560000]
[  149.560000] rcu_sched kthread starved for 6009 jiffies! g1039 c1038
f0x0 s3 ->state=0x1
[ 1326.670000] INFO: rcu_sched detected stalls on CPUs/tasks:
[ 1326.670000]  2-...: (0 ticks this GP) idle=998/0/0
softirq=56566/56566 fqs=0
[ 1326.680000]  (detected by 1, t=6003 jiffies, g=14603, c=14602, q=9455)
[ 1326.680000] Task dump for CPU 2:
[ 1326.690000] swapper/2       R running      0     0      1 0x00100000
[ 1326.690000] Stack : 00000000 00000001 0000000a 00000000 00000000
00000001 8045a2a4 80410000
[ 1326.690000]    8041275c 00000001 00000000 804125e0 80412724
80410000 00000000 800132e8
[ 1326.690000]    1100fc03 00000002 8fc62000 8fc63ec0 80410000
8005c870 1100fc03 00000002
[ 1326.690000]    00000000 80410000 8045a2a4 8005c868 80410000
8001ade0 1100fc03 00000000
[ 1326.690000]    00000004 804124a0 000000a0 8001ade8 fbffbfdf
7fff7b3b 7ffaeff8 67fbffff
[ 1326.690000]    ...
[ 1326.730000] Call Trace:
[ 1326.730000] [<8000bba8>] __schedule+0x574/0x758
[ 1326.740000] [<800132e8>] r4k_wait_irqoff+0x0/0x20
[ 1326.740000]
[ 1326.740000] rcu_sched kthread starved for 6009 jiffies! g14603
c14602 f0x0 s3 ->state=0x1
[ 1392.200000] INFO: rcu_sched detected stalls on CPUs/tasks:
[ 1392.200000]  2-...: (0 ticks this GP) idle=c80/0/0
softirq=56566/56566 fqs=0
[ 1392.210000]  (detected by 3, t=6003 jiffies, g=14669, c=14668, q=11973)
[ 1392.210000] Task dump for CPU 2:
[ 1392.220000] swapper/2       R running      0     0      1 0x00100000
[ 1392.220000] Stack : 00000000 4e88ab25 00000148 ffffffff 0000011d
773206f0 8045a2a4 80410000
[ 1392.220000]    8041275c 00000001 00000000 804125e0 80412724
80410000 000010d9 800132e8
[ 1392.220000]    1100fc03 00000002 8fc62000 8fc63ec0 80410000
8005c870 1100fc03 00000002
[ 1392.220000]    00000000 80410000 8045a2a4 8005c868 80410000
8001ade0 1100fc03 00000000
[ 1392.220000]    00000004 804124a0 000000a0 8001ade8 fbffbfdf
7fff7b3b 7ffaeff8 67fbffff
[ 1392.220000]    ...
[ 1392.260000] Call Trace:
[ 1392.260000] [<8000bba8>] __schedule+0x574/0x758
[ 1392.270000] [<800132e8>] r4k_wait_irqoff+0x0/0x20
[ 1392.270000]
[ 1392.270000] rcu_sched kthread starved for 6009 jiffies! g14669
c14668 f0x0 s3 ->state=0x1



>
>
>>
>> On 11/11/2016 9:08 AM, Outback Dingo wrote:
>>>
>>> On Fri, Nov 11, 2016 at 9:04 PM, Noah Causin <n0manletter at gmail.com>
>>> wrote:
>>>>
>>>> Do you build your firmware from the git repository?
>>>>
>>>>
>>> Yes it is built from a recent trunk, as of yesterday
>>>
>>>
>>>> On 11/10/2016 11:28 PM, Outback Dingo wrote:
>>>>>
>>>>> On Fri, Nov 11, 2016 at 10:52 AM, Outback Dingo <outbackdingo at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> On Fri, Nov 11, 2016 at 10:42 AM, Noah Causin <n0manletter at gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> I took an extra router I had and bridged two ports on the router's
>>>>>>> switch,
>>>>>>> so they just pass traffic. (eth0.2 and eth0.3)
>>>>>>>
>>>>>>> Clients >-[Main Router] --[Extra Router]--[Cable-Modem]
>>>>>>>
>>>>>>> The extra router is passive.  It acts like it's part of the Ethernet
>>>>>>> cable
>>>>>>> between the main router and cable modem.  It does not interfere.
>>>>>>>
>>>>>>> The extra router needs these packages:
>>>>>>>
>>>>>>> kmod-ebtables, kmod-ebtables-ipv4, kmod-crypto-pcompress and the two
>>>>>>> packages compiled from the Makefile I showed below (iptables-mod-ndpi
>>>>>>> and
>>>>>>> iptables-mod-ndpi)
>>>>>>>
>>>>>>> I edited /etc/sysctl.conf and change the last two lines to this:
>>>>>>>
>>>>>>> net.bridge.bridge-nf-call-ip6tables=1
>>>>>>> net.bridge.bridge-nf-call-iptables=1
>>>>>>>
>>>>>>> This enables bridge firewalling, so the traffic between the two ports
>>>>>>> can be
>>>>>>> marked.
>>>>>>>
>>>>>>> I then added firewall rules to LuCIs custom firewall rules tab.
>>>>>>>
>>>>>>> Example Rules, modify classes as desired:
>>>>>>>
>>>>>>> iptables -t mangle -A FORWARD -m ndpi --steam -m mac ! --mac-source
>>>>>>> Your
>>>>>>> Main Router's Mac Address -j DSCP --set-dscp-class cs0
>>>>>>> iptables -t mangle -A FORWARD -m ndpi --youtube -m mac ! --mac-source
>>>>>>> Your
>>>>>>> Main Router's Mac Address -j DSCP --set-dscp-class cs2
>>>>>>>
>>>>>>> iptables -t mangle -A FORWARD -m ndpi --netflix -m mac ! --mac-source
>>>>>>> Your
>>>>>>> Main Router's Mac Address -j DSCP --set-dscp-class cs3
>>>>>>> iptables -t mangle -A FORWARD -m ndpi --skype -m mac ! --mac-source
>>>>>>> Your
>>>>>>> Main Router's Mac Address -j DSCP --set-dscp-class cs4
>>>>>>>
>>>>>>> Have your main router use some form of DiffServ for both upload and
>>>>>>> download
>>>>>>> on its WAN interface.  For upload traffic, you just need the two
>>>>>>> packages
>>>>>>> from the makefile to be installed on the main router and create
>>>>>>> firewall
>>>>>>> rules like this:
>>>>>>>
>>>>>>> iptables -t mangle -A FORWARD -o eth2 -m ndpi --netflix -j DSCP
>>>>>>> --set-dscp-class cs3
>>>>>>> iptables -t mangle -A FORWARD -o eth2 -m ndpi --skype -j DSCP
>>>>>>> --set-dscp-class cs4
>>>>>>>
>>>>>>> If you need help building a custom firmware image, just let me know.
>>>>>>>
>>>>>> Ok got it, its in-line.... right now im fighting the GFW of China
>>>>>> using shadowsocks and chinadns so im pretty customized already,
>>>>>> i was having issues with sqm and kernel segfaults on 4.4.30 ill build
>>>>>> a cake and DPI image and see how that goes, maybe Ill try this
>>>>>> after i deem sqm is stable with cake on this build. It a ZBT-WG3526
>>>>>> router... so its ralink based
>>>>>
>>>>> like i stated on LEDE major issues when sqm is enabled.... even with
>>>>> cake... is nobody else seeing these issues?
>>>>>
>>>>> [  569.020000]  1-...: (6 GPs behind) idle=236/0/0 softirq=50607/50613
>>>>> fqs=1
>>>>> [  569.020000]  (detected by 0, t=6002 jiffies, g=5799, c=5798, q=573)
>>>>> [  569.020000] Task dump for CPU 1:
>>>>> [  569.020000] swapper/1       R running      0     0      1 0x00100000
>>>>> [  569.020000] Stack : 00000000 00000001 00000015 00000000 00000000
>>>>> 00000001 8045a2a4 80410000
>>>>> [  569.020000]    8041275c 00000001 00000001 80412540 80412724
>>>>> 80410000 00000000 800132e8
>>>>> [  569.020000]    1100fc03 00000001 8fc60000 8fc61ec0 80410000
>>>>> 8005c870 1100fc03 00000001
>>>>> [  569.020000]    00000000 80410000 8045a2a4 8005c868 80410000
>>>>> 8001ade0 1100fc03 00000000
>>>>> [  569.020000]    00000004 804124a0 000000a0 8001ade8 fe9ff9db
>>>>> aefffbff fff77fe7 dfffffef
>>>>> [  569.020000]    ...
>>>>> [  569.020000] Call Trace:
>>>>> [  569.020000] [<8000bba8>] __schedule+0x574/0x758
>>>>> [  569.020000] [<800132e8>] r4k_wait_irqoff+0x0/0x20
>>>>> [  569.020000]
>>>>> [  569.020000] rcu_sched kthread starved for 6001 jiffies! g5799 c5798
>>>>> f0x0 s3 ->state=0x1
>>>>> [  629.130000] INFO: rcu_sched detected stalls on CPUs/tasks:
>>>>> [  629.130000]  1-...: (8 GPs behind) idle=756/0/0 softirq=50607/50613
>>>>> fqs=1
>>>>> [  629.130000]  (detected by 0, t=6002 jiffies, g=5801, c=5800, q=653)
>>>>> [  629.130000] Task dump for CPU 1:
>>>>> [  629.130000] swapper/1       R running      0     0      1 0x00100000
>>>>> [  629.130000] Stack : 00000000 bfd47c67 00000094 ffffffff 00000081
>>>>> 00000011 8045a2a4 80410000
>>>>> [  629.130000]    8041275c 00000001 00000001 80412540 80412724
>>>>> 80410000 000010d9 800132e8
>>>>> [  629.130000]    1100fc03 00000001 8fc60000 8fc61ec0 80410000
>>>>> 8005c870 1100fc03 00000001
>>>>> [  629.130000]    00000000 80410000 8045a2a4 8005c868 80410000
>>>>> 8001ade0 1100fc03 00000000
>>>>> [  629.130000]    00000004 804124a0 000000a0 8001ade8 fe9ff9db
>>>>> aefffbff fff77fe7 dfffffef
>>>>> [  629.130000]    ...
>>>>> [  629.130000] Call Trace:
>>>>> [  629.130000] [<8000bba8>] __schedule+0x574/0x758
>>>>> [  629.130000] [<800132e8>] r4k_wait_irqoff+0x0/0x20
>>>>> [  629.130000]
>>>>> [  629.130000] rcu_sched kthread starved for 6001 jiffies! g5801 c5800
>>>>> f0x0 s3 ->state=0x1
>>>>> [  692.940000] INFO: rcu_sched detected stalls on CPUs/tasks:
>>>>> [  692.940000]  1-...: (0 ticks this GP) idle=4c2/0/0
>>>>> softirq=50613/50613 fqs=0
>>>>> [  692.940000]  2-...: (25 GPs behind) idle=55a/0/0 softirq=28459/28460
>>>>> fqs=0
>>>>> [  692.940000]  3-...: (25 GPs behind) idle=5c2/0/0 softirq=30711/30982
>>>>> fqs=0
>>>>> [  692.940000]  (detected by 0, t=6002 jiffies, g=5824, c=5823, q=3190)
>>>>> [  692.940000] Task dump for CPU 1:
>>>>> [  692.940000] swapper/1       R running      0     0      1 0x00100000
>>>>> [  692.940000] Stack : 00000000 3646fb94 000000af ffffffff 0000008f
>>>>> d2f1a9fc 8045a2a4 80410000
>>>>> [  692.940000]    8041275c 00000001 00000001 80412540 80412724
>>>>> 80410000 000010d9 800132e8
>>>>> [  692.940000]    00000000 00000001 8fc60000 8fc61ec0 80410000
>>>>> 8005c870 1100fc03 00000001
>>>>> [  692.940000]    00000000 80410000 8045a2a4 8005c868 80410000
>>>>> 8001ade0 1100fc03 00000000
>>>>> [  692.940000]    00000004 804124a0 000000a0 8001ade8 fe9ff9db
>>>>> aefffbff fff77fe7 dfffffef
>>>>> [  692.940000]    ...
>>>>> [  692.940000] Call Trace:
>>>>> [  692.940000] [<8000bba8>] __schedule+0x574/0x758
>>>>> [  692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20
>>>>> [  692.940000]
>>>>> [  692.940000] Task dump for CPU 2:
>>>>> [  692.940000] swapper/2       R running      0     0      1 0x00100000
>>>>> [  692.940000] Stack : 00000000 00000001 00000010 00000000 00000000
>>>>> 00010001 8045a2a4 80410000
>>>>> [  692.940000]    8041275c 00000001 00000000 804125e0 80412724
>>>>> 80410000 00000000 800132e8
>>>>> [  692.940000]    1100fc03 00000002 8fc62000 8fc63ec0 80410000
>>>>> 8005c870 1100fc03 00000002
>>>>> [  692.940000]    00000000 80410000 8045a2a4 8005c868 80410000
>>>>> 8001ade0 1100fc03 00000000
>>>>> [  692.940000]    00000004 804124a0 000000a0 8001ade8 fbffbfdf
>>>>> 7fff7b3b 7ffaeff8 67fbffff
>>>>> [  692.940000]    ...
>>>>> [  692.940000] Call Trace:
>>>>> [  692.940000] [<8000bba8>] __schedule+0x574/0x758
>>>>> [  692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20
>>>>> [  692.940000]
>>>>> [  692.940000] Task dump for CPU 3:
>>>>> [  692.940000] swapper/3       R running      0     0      1 0x00100000
>>>>> [  692.940000] Stack : 00000000 a69c5765 000000a3 ffffffff 0000008f
>>>>> 00000020 8045a2a4 80410000
>>>>> [  692.940000]    8041275c 00000001 00000001 80412680 80412724
>>>>> 80410000 000010d9 800132e8
>>>>> [  692.940000]    1100fc03 00000003 8fc64000 8fc65ec0 80410000
>>>>> 8005c870 1100fc03 00000003
>>>>> [  692.940000]    00000000 80410000 8045a2a4 8005c868 80410000
>>>>> 8001ade0 1100fc03 00000000
>>>>> [  692.940000]    00000004 804124a0 000000a0 8001ade8 fefffbdb
>>>>> aefffbfb fff77fe7 dfffffef
>>>>> [  692.940000]    ...
>>>>> [  692.940000] Call Trace:
>>>>> [  692.940000] [<8000bba8>] __schedule+0x574/0x758
>>>>> [  692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20
>>>>> [  692.940000]
>>>>> [  692.940000] rcu_sched kthread starved for 6002 jiffies! g5824 c5823
>>>>> f0x0 s3 ->state=0x1
>>>>> root at lede:~# packet_write_wait: Connection to 192.168.20.1 port 22:
>>>>> Broken
>>>>> pipe
>>>>>
>>>>>
>>>>>>> On 11/10/2016 9:06 PM, Outback Dingo wrote:
>>>>>>>
>>>>>>> On Fri, Nov 11, 2016 at 9:55 AM, Noah Causin <n0manletter at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> I finally got my project working.
>>>>>>>
>>>>>>> I integrated the NDPI Deep Packet Inspection engine into my LEDE
>>>>>>> build,
>>>>>>> so I
>>>>>>> could prioritize applications using Cake.
>>>>>>>
>>>>>>> http://www.ntop.org/products/deep-packet-inspection/ndpi/
>>>>>>>
>>>>>>> NDPI integrates into IPTables, which allows me to DSCP mark packets.
>>>>>>> Cake
>>>>>>> reads the DiffServ markings and puts the traffic into appropriate
>>>>>>> classes.
>>>>>>>
>>>>>>> I found a Makefile which successfully compiles:
>>>>>>>
>>>>>>>
>>>>>>> https://github.com/981213/lede_src/blob/0d344bc2958838dcbc547a8f0a3d8842e6f6d2f8/package/my_package/ndpi-netfilter/Makefile
>>>>>>>
>>>>>>> The system works very well.  Steam traffic is deprioritized to allow
>>>>>>> applications like YouTube, Netflix, and Skype to receive higher
>>>>>>> amounts
>>>>>>> of
>>>>>>> the available bandwidth.
>>>>>>>
>>>>>>> What I do for ingress is bridge two ports on an extra router, enable
>>>>>>> bridge
>>>>>>> firewalling, and create IPTables rules to mark downstream packets.
>>>>>>> The
>>>>>>> router I use is a D-Link DGL-5500, which is comparable to an Archer
>>>>>>> C7.
>>>>>>>
>>>>>>> IPv6 support is not available in this netfilter module, but the IPv4
>>>>>>> support
>>>>>>> is great.
>>>>>>>
>>>>>>> Noah Causin
>>>>>>>
>>>>>>> very nice.... however, can you describe a bit better how you
>>>>>>> configured for ingress ? a second router?  configuration file ? your
>>>>>>> high level description seems a bit confusing to me
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Cake mailing list
>>>>>>> Cake at lists.bufferbloat.net
>>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>>>>
>>>>>>>
>>

More information about the Cake mailing list