From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-x242.google.com (mail-it0-x242.google.com [IPv6:2607:f8b0:4001:c0b::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id B0D7E3B2A2 for ; Fri, 11 Nov 2016 09:09:33 -0500 (EST) Received: by mail-it0-x242.google.com with SMTP id n68so10795387itn.3 for ; Fri, 11 Nov 2016 06:09:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=eC6hlayuKCVcy6+8i6IObctD0vyzdQwsiIm+bXQu3hs=; b=Jy1sKRdJ17wMKJDugYVGiX9QJp5AHgksL/z133K4El1j9rxc3xIHJXe+V3I+J0l1yH g8NIdTm0gdf5srcmQAqEWHgoKJ6goI66ebPIb5wmCc6aBfMg52TuhgPE0235hh0k4NKh p4B4GYVbYyF3sKW0CvIcx/Gf0ysXslA15Y9zQHwH5v5BoBmVuERwHGdbVBcEYcwKG4XZ q7BpjhBAlzUmGa7sFzLADOHnsXxKrXOSEkVRPHc/fkuaTDUZ2JrUjaMqv/ZmuYhLSZwZ VNvGHxc5eGdkR07A6LXm/ZWGf39v2UnK7mphIgwz8HVkyeiQVHL3TClLJuwjG8fpaDZb 7olA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=eC6hlayuKCVcy6+8i6IObctD0vyzdQwsiIm+bXQu3hs=; b=UXDxJ3xMgbxghVeyS0/UP5o365ccGlgP24gFovl5QHQ3cRgzQJ7jYXdaH2y9qA6RM5 2UaDBDF/6Dst0etO1CoidvGxNuCLZc3axRfAaXeMe0eKR6s6umSLM4YiaJRYcXLcGdL0 vengT32GLTHpSKzGuEUUvChMOdSioQLG3Hqrmy9zZTARWyELd2aaauEUJZtQGdQizDJt ofrqc4rhYvAGvW7zJgG6djWGd1qHjbhhsuYFHfc27o+vpuw+pxFJ3IrOc3MByzIS5cEn /4oXRFZdbNjGWoFYlknq+Oj3eV0OW4VFN4tpqEGC/dWhVyt9E/z+Z4bRSlxWDi+o9OgE Woyg== X-Gm-Message-State: ABUngvfGYiTMaugZatAs48wlBhPJ6LzDecFqvIcHVmKJG7Z1egfl+VamqhFOYArbHgOBqsG2m/T1casISL4lpw== X-Received: by 10.36.237.3 with SMTP id r3mr8434089ith.76.1478873372829; Fri, 11 Nov 2016 06:09:32 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.22.80 with HTTP; Fri, 11 Nov 2016 06:08:52 -0800 (PST) In-Reply-To: References: <3e5942d2-6d6b-0e01-8aa6-98c3535c26ef@gmail.com> <46c0133b-b6f8-fe1f-4d2e-0cf6088e024d@gmail.com> From: Outback Dingo Date: Fri, 11 Nov 2016 22:08:52 +0800 Message-ID: To: Noah Causin Cc: cake@lists.bufferbloat.net Content-Type: text/plain; charset=UTF-8 Subject: Re: [Cake] Cake with Deep Packet Inspection X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Nov 2016 14:09:33 -0000 On Fri, Nov 11, 2016 at 9:04 PM, Noah Causin wrote: > Do you build your firmware from the git repository? > > Yes it is built from a recent trunk, as of yesterday > > On 11/10/2016 11:28 PM, Outback Dingo wrote: >> >> On Fri, Nov 11, 2016 at 10:52 AM, Outback Dingo >> wrote: >>> >>> On Fri, Nov 11, 2016 at 10:42 AM, Noah Causin >>> wrote: >>>> >>>> I took an extra router I had and bridged two ports on the router's >>>> switch, >>>> so they just pass traffic. (eth0.2 and eth0.3) >>>> >>>> Clients >-[Main Router] --[Extra Router]--[Cable-Modem] >>>> >>>> The extra router is passive. It acts like it's part of the Ethernet >>>> cable >>>> between the main router and cable modem. It does not interfere. >>>> >>>> The extra router needs these packages: >>>> >>>> kmod-ebtables, kmod-ebtables-ipv4, kmod-crypto-pcompress and the two >>>> packages compiled from the Makefile I showed below (iptables-mod-ndpi >>>> and >>>> iptables-mod-ndpi) >>>> >>>> I edited /etc/sysctl.conf and change the last two lines to this: >>>> >>>> net.bridge.bridge-nf-call-ip6tables=1 >>>> net.bridge.bridge-nf-call-iptables=1 >>>> >>>> This enables bridge firewalling, so the traffic between the two ports >>>> can be >>>> marked. >>>> >>>> I then added firewall rules to LuCIs custom firewall rules tab. >>>> >>>> Example Rules, modify classes as desired: >>>> >>>> iptables -t mangle -A FORWARD -m ndpi --steam -m mac ! --mac-source Your >>>> Main Router's Mac Address -j DSCP --set-dscp-class cs0 >>>> iptables -t mangle -A FORWARD -m ndpi --youtube -m mac ! --mac-source >>>> Your >>>> Main Router's Mac Address -j DSCP --set-dscp-class cs2 >>>> >>>> iptables -t mangle -A FORWARD -m ndpi --netflix -m mac ! --mac-source >>>> Your >>>> Main Router's Mac Address -j DSCP --set-dscp-class cs3 >>>> iptables -t mangle -A FORWARD -m ndpi --skype -m mac ! --mac-source Your >>>> Main Router's Mac Address -j DSCP --set-dscp-class cs4 >>>> >>>> Have your main router use some form of DiffServ for both upload and >>>> download >>>> on its WAN interface. For upload traffic, you just need the two >>>> packages >>>> from the makefile to be installed on the main router and create firewall >>>> rules like this: >>>> >>>> iptables -t mangle -A FORWARD -o eth2 -m ndpi --netflix -j DSCP >>>> --set-dscp-class cs3 >>>> iptables -t mangle -A FORWARD -o eth2 -m ndpi --skype -j DSCP >>>> --set-dscp-class cs4 >>>> >>>> If you need help building a custom firmware image, just let me know. >>>> >>> Ok got it, its in-line.... right now im fighting the GFW of China >>> using shadowsocks and chinadns so im pretty customized already, >>> i was having issues with sqm and kernel segfaults on 4.4.30 ill build >>> a cake and DPI image and see how that goes, maybe Ill try this >>> after i deem sqm is stable with cake on this build. It a ZBT-WG3526 >>> router... so its ralink based >> >> like i stated on LEDE major issues when sqm is enabled.... even with >> cake... is nobody else seeing these issues? >> >> [ 569.020000] 1-...: (6 GPs behind) idle=236/0/0 softirq=50607/50613 >> fqs=1 >> [ 569.020000] (detected by 0, t=6002 jiffies, g=5799, c=5798, q=573) >> [ 569.020000] Task dump for CPU 1: >> [ 569.020000] swapper/1 R running 0 0 1 0x00100000 >> [ 569.020000] Stack : 00000000 00000001 00000015 00000000 00000000 >> 00000001 8045a2a4 80410000 >> [ 569.020000] 8041275c 00000001 00000001 80412540 80412724 >> 80410000 00000000 800132e8 >> [ 569.020000] 1100fc03 00000001 8fc60000 8fc61ec0 80410000 >> 8005c870 1100fc03 00000001 >> [ 569.020000] 00000000 80410000 8045a2a4 8005c868 80410000 >> 8001ade0 1100fc03 00000000 >> [ 569.020000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db >> aefffbff fff77fe7 dfffffef >> [ 569.020000] ... >> [ 569.020000] Call Trace: >> [ 569.020000] [<8000bba8>] __schedule+0x574/0x758 >> [ 569.020000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >> [ 569.020000] >> [ 569.020000] rcu_sched kthread starved for 6001 jiffies! g5799 c5798 >> f0x0 s3 ->state=0x1 >> [ 629.130000] INFO: rcu_sched detected stalls on CPUs/tasks: >> [ 629.130000] 1-...: (8 GPs behind) idle=756/0/0 softirq=50607/50613 >> fqs=1 >> [ 629.130000] (detected by 0, t=6002 jiffies, g=5801, c=5800, q=653) >> [ 629.130000] Task dump for CPU 1: >> [ 629.130000] swapper/1 R running 0 0 1 0x00100000 >> [ 629.130000] Stack : 00000000 bfd47c67 00000094 ffffffff 00000081 >> 00000011 8045a2a4 80410000 >> [ 629.130000] 8041275c 00000001 00000001 80412540 80412724 >> 80410000 000010d9 800132e8 >> [ 629.130000] 1100fc03 00000001 8fc60000 8fc61ec0 80410000 >> 8005c870 1100fc03 00000001 >> [ 629.130000] 00000000 80410000 8045a2a4 8005c868 80410000 >> 8001ade0 1100fc03 00000000 >> [ 629.130000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db >> aefffbff fff77fe7 dfffffef >> [ 629.130000] ... >> [ 629.130000] Call Trace: >> [ 629.130000] [<8000bba8>] __schedule+0x574/0x758 >> [ 629.130000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >> [ 629.130000] >> [ 629.130000] rcu_sched kthread starved for 6001 jiffies! g5801 c5800 >> f0x0 s3 ->state=0x1 >> [ 692.940000] INFO: rcu_sched detected stalls on CPUs/tasks: >> [ 692.940000] 1-...: (0 ticks this GP) idle=4c2/0/0 >> softirq=50613/50613 fqs=0 >> [ 692.940000] 2-...: (25 GPs behind) idle=55a/0/0 softirq=28459/28460 >> fqs=0 >> [ 692.940000] 3-...: (25 GPs behind) idle=5c2/0/0 softirq=30711/30982 >> fqs=0 >> [ 692.940000] (detected by 0, t=6002 jiffies, g=5824, c=5823, q=3190) >> [ 692.940000] Task dump for CPU 1: >> [ 692.940000] swapper/1 R running 0 0 1 0x00100000 >> [ 692.940000] Stack : 00000000 3646fb94 000000af ffffffff 0000008f >> d2f1a9fc 8045a2a4 80410000 >> [ 692.940000] 8041275c 00000001 00000001 80412540 80412724 >> 80410000 000010d9 800132e8 >> [ 692.940000] 00000000 00000001 8fc60000 8fc61ec0 80410000 >> 8005c870 1100fc03 00000001 >> [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 >> 8001ade0 1100fc03 00000000 >> [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db >> aefffbff fff77fe7 dfffffef >> [ 692.940000] ... >> [ 692.940000] Call Trace: >> [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 >> [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >> [ 692.940000] >> [ 692.940000] Task dump for CPU 2: >> [ 692.940000] swapper/2 R running 0 0 1 0x00100000 >> [ 692.940000] Stack : 00000000 00000001 00000010 00000000 00000000 >> 00010001 8045a2a4 80410000 >> [ 692.940000] 8041275c 00000001 00000000 804125e0 80412724 >> 80410000 00000000 800132e8 >> [ 692.940000] 1100fc03 00000002 8fc62000 8fc63ec0 80410000 >> 8005c870 1100fc03 00000002 >> [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 >> 8001ade0 1100fc03 00000000 >> [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fbffbfdf >> 7fff7b3b 7ffaeff8 67fbffff >> [ 692.940000] ... >> [ 692.940000] Call Trace: >> [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 >> [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >> [ 692.940000] >> [ 692.940000] Task dump for CPU 3: >> [ 692.940000] swapper/3 R running 0 0 1 0x00100000 >> [ 692.940000] Stack : 00000000 a69c5765 000000a3 ffffffff 0000008f >> 00000020 8045a2a4 80410000 >> [ 692.940000] 8041275c 00000001 00000001 80412680 80412724 >> 80410000 000010d9 800132e8 >> [ 692.940000] 1100fc03 00000003 8fc64000 8fc65ec0 80410000 >> 8005c870 1100fc03 00000003 >> [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 >> 8001ade0 1100fc03 00000000 >> [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fefffbdb >> aefffbfb fff77fe7 dfffffef >> [ 692.940000] ... >> [ 692.940000] Call Trace: >> [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 >> [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >> [ 692.940000] >> [ 692.940000] rcu_sched kthread starved for 6002 jiffies! g5824 c5823 >> f0x0 s3 ->state=0x1 >> root@lede:~# packet_write_wait: Connection to 192.168.20.1 port 22: Broken >> pipe >> >> >>>> On 11/10/2016 9:06 PM, Outback Dingo wrote: >>>> >>>> On Fri, Nov 11, 2016 at 9:55 AM, Noah Causin >>>> wrote: >>>> >>>> I finally got my project working. >>>> >>>> I integrated the NDPI Deep Packet Inspection engine into my LEDE build, >>>> so I >>>> could prioritize applications using Cake. >>>> >>>> http://www.ntop.org/products/deep-packet-inspection/ndpi/ >>>> >>>> NDPI integrates into IPTables, which allows me to DSCP mark packets. >>>> Cake >>>> reads the DiffServ markings and puts the traffic into appropriate >>>> classes. >>>> >>>> I found a Makefile which successfully compiles: >>>> >>>> https://github.com/981213/lede_src/blob/0d344bc2958838dcbc547a8f0a3d8842e6f6d2f8/package/my_package/ndpi-netfilter/Makefile >>>> >>>> The system works very well. Steam traffic is deprioritized to allow >>>> applications like YouTube, Netflix, and Skype to receive higher amounts >>>> of >>>> the available bandwidth. >>>> >>>> What I do for ingress is bridge two ports on an extra router, enable >>>> bridge >>>> firewalling, and create IPTables rules to mark downstream packets. The >>>> router I use is a D-Link DGL-5500, which is comparable to an Archer C7. >>>> >>>> IPv6 support is not available in this netfilter module, but the IPv4 >>>> support >>>> is great. >>>> >>>> Noah Causin >>>> >>>> very nice.... however, can you describe a bit better how you >>>> configured for ingress ? a second router? configuration file ? your >>>> high level description seems a bit confusing to me >>>> >>>> >>>> _______________________________________________ >>>> Cake mailing list >>>> Cake@lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/cake >>>> >>>> >