From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-x243.google.com (mail-it0-x243.google.com [IPv6:2607:f8b0:4001:c0b::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 501DE3B2A2 for ; Fri, 11 Nov 2016 08:04:08 -0500 (EST) Received: by mail-it0-x243.google.com with SMTP id e187so10454028itc.0 for ; Fri, 11 Nov 2016 05:04:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=mrkhWnfg2XDzrxTy+TgkfbQaTQ2WgK0twHuxXvu4vkI=; b=A70mH/SD7SKzEU3x5kW706qw8R3xudgg3a4VvjLnVC3dq3qONC6Yc8Vg6JtaBoykl5 F+vSpcT1xRmE9k6NoSxQ1UQ8b3QfANEr5381pz38FG2M1OX7cf/8ov2Zw6sSTzg8rcac DVqtPNcN8YWUqox8tyApuwPNmlJdjJAwLLRJ0/m049HETS79HKYiG8g+3mX9Z3xKQ098 27sObunVzlDaFUr3GG3u1r19OxlCIRkUUboudzSDSqOZfQTIOb5tow4Xu0zdKF87cq3G XaR6Hc+qPos986gZHuKQhvGNugF5vwZ1wnkLdqiwLZ459osBnxAs3/QYbO7Ku6mT3qi2 ALSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=mrkhWnfg2XDzrxTy+TgkfbQaTQ2WgK0twHuxXvu4vkI=; b=DtJB/wyeg5oLm/3pKc47PCjAUxLsjQq1DptwHXA0JcC3DL3VCzvexgOAQRIsbkgfCn I4/4hAne40IXn6xDBam0ysAcqiBi1/NhqNM9b0yrPJWFjDFLXTZ8zaUjWBqZY3SJ5nkl 0AJfz0vFhokL6flpd6TzT14vqzZUANo0Q8q7c3LNpQl8bqYUucoIiYTaiT8uG7gpdWU3 xhqS+TCJdEcqqophH0G1bGN7qT94E3pmges/NatsbDMkEAIa5KKFZCrfiS9CoJkQNmBp qK8R1qdCl+YkAu3fsEY/U6rNtexNm9eQe6KH/+6+0qaiQ9bfWjqCTWROfs4JIEO7gnhC B7pA== X-Gm-Message-State: ABUngveQK9BYEUGK0YdhCzN8q+eJokH7/ZVEvC1ixWuSoRtF5H8yGPr9Wad3LAzo/s+gww== X-Received: by 10.36.51.76 with SMTP id k73mr23274244itk.22.1478869447318; Fri, 11 Nov 2016 05:04:07 -0800 (PST) Received: from [172.21.63.50] (hades.kettering.edu. [192.138.137.97]) by smtp.gmail.com with ESMTPSA id e72sm2191870iof.26.2016.11.11.05.04.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 11 Nov 2016 05:04:06 -0800 (PST) To: Outback Dingo References: <3e5942d2-6d6b-0e01-8aa6-98c3535c26ef@gmail.com> <46c0133b-b6f8-fe1f-4d2e-0cf6088e024d@gmail.com> Cc: cake@lists.bufferbloat.net From: Noah Causin Message-ID: Date: Fri, 11 Nov 2016 08:04:09 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Cake] Cake with Deep Packet Inspection X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Nov 2016 13:04:08 -0000 Do you build your firmware from the git repository? On 11/10/2016 11:28 PM, Outback Dingo wrote: > On Fri, Nov 11, 2016 at 10:52 AM, Outback Dingo wrote: >> On Fri, Nov 11, 2016 at 10:42 AM, Noah Causin wrote: >>> I took an extra router I had and bridged two ports on the router's switch, >>> so they just pass traffic. (eth0.2 and eth0.3) >>> >>> Clients >-[Main Router] --[Extra Router]--[Cable-Modem] >>> >>> The extra router is passive. It acts like it's part of the Ethernet cable >>> between the main router and cable modem. It does not interfere. >>> >>> The extra router needs these packages: >>> >>> kmod-ebtables, kmod-ebtables-ipv4, kmod-crypto-pcompress and the two >>> packages compiled from the Makefile I showed below (iptables-mod-ndpi and >>> iptables-mod-ndpi) >>> >>> I edited /etc/sysctl.conf and change the last two lines to this: >>> >>> net.bridge.bridge-nf-call-ip6tables=1 >>> net.bridge.bridge-nf-call-iptables=1 >>> >>> This enables bridge firewalling, so the traffic between the two ports can be >>> marked. >>> >>> I then added firewall rules to LuCIs custom firewall rules tab. >>> >>> Example Rules, modify classes as desired: >>> >>> iptables -t mangle -A FORWARD -m ndpi --steam -m mac ! --mac-source Your >>> Main Router's Mac Address -j DSCP --set-dscp-class cs0 >>> iptables -t mangle -A FORWARD -m ndpi --youtube -m mac ! --mac-source Your >>> Main Router's Mac Address -j DSCP --set-dscp-class cs2 >>> >>> iptables -t mangle -A FORWARD -m ndpi --netflix -m mac ! --mac-source Your >>> Main Router's Mac Address -j DSCP --set-dscp-class cs3 >>> iptables -t mangle -A FORWARD -m ndpi --skype -m mac ! --mac-source Your >>> Main Router's Mac Address -j DSCP --set-dscp-class cs4 >>> >>> Have your main router use some form of DiffServ for both upload and download >>> on its WAN interface. For upload traffic, you just need the two packages >>> from the makefile to be installed on the main router and create firewall >>> rules like this: >>> >>> iptables -t mangle -A FORWARD -o eth2 -m ndpi --netflix -j DSCP >>> --set-dscp-class cs3 >>> iptables -t mangle -A FORWARD -o eth2 -m ndpi --skype -j DSCP >>> --set-dscp-class cs4 >>> >>> If you need help building a custom firmware image, just let me know. >>> >> Ok got it, its in-line.... right now im fighting the GFW of China >> using shadowsocks and chinadns so im pretty customized already, >> i was having issues with sqm and kernel segfaults on 4.4.30 ill build >> a cake and DPI image and see how that goes, maybe Ill try this >> after i deem sqm is stable with cake on this build. It a ZBT-WG3526 >> router... so its ralink based > like i stated on LEDE major issues when sqm is enabled.... even with > cake... is nobody else seeing these issues? > > [ 569.020000] 1-...: (6 GPs behind) idle=236/0/0 softirq=50607/50613 fqs=1 > [ 569.020000] (detected by 0, t=6002 jiffies, g=5799, c=5798, q=573) > [ 569.020000] Task dump for CPU 1: > [ 569.020000] swapper/1 R running 0 0 1 0x00100000 > [ 569.020000] Stack : 00000000 00000001 00000015 00000000 00000000 > 00000001 8045a2a4 80410000 > [ 569.020000] 8041275c 00000001 00000001 80412540 80412724 > 80410000 00000000 800132e8 > [ 569.020000] 1100fc03 00000001 8fc60000 8fc61ec0 80410000 > 8005c870 1100fc03 00000001 > [ 569.020000] 00000000 80410000 8045a2a4 8005c868 80410000 > 8001ade0 1100fc03 00000000 > [ 569.020000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db > aefffbff fff77fe7 dfffffef > [ 569.020000] ... > [ 569.020000] Call Trace: > [ 569.020000] [<8000bba8>] __schedule+0x574/0x758 > [ 569.020000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 > [ 569.020000] > [ 569.020000] rcu_sched kthread starved for 6001 jiffies! g5799 c5798 > f0x0 s3 ->state=0x1 > [ 629.130000] INFO: rcu_sched detected stalls on CPUs/tasks: > [ 629.130000] 1-...: (8 GPs behind) idle=756/0/0 softirq=50607/50613 fqs=1 > [ 629.130000] (detected by 0, t=6002 jiffies, g=5801, c=5800, q=653) > [ 629.130000] Task dump for CPU 1: > [ 629.130000] swapper/1 R running 0 0 1 0x00100000 > [ 629.130000] Stack : 00000000 bfd47c67 00000094 ffffffff 00000081 > 00000011 8045a2a4 80410000 > [ 629.130000] 8041275c 00000001 00000001 80412540 80412724 > 80410000 000010d9 800132e8 > [ 629.130000] 1100fc03 00000001 8fc60000 8fc61ec0 80410000 > 8005c870 1100fc03 00000001 > [ 629.130000] 00000000 80410000 8045a2a4 8005c868 80410000 > 8001ade0 1100fc03 00000000 > [ 629.130000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db > aefffbff fff77fe7 dfffffef > [ 629.130000] ... > [ 629.130000] Call Trace: > [ 629.130000] [<8000bba8>] __schedule+0x574/0x758 > [ 629.130000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 > [ 629.130000] > [ 629.130000] rcu_sched kthread starved for 6001 jiffies! g5801 c5800 > f0x0 s3 ->state=0x1 > [ 692.940000] INFO: rcu_sched detected stalls on CPUs/tasks: > [ 692.940000] 1-...: (0 ticks this GP) idle=4c2/0/0 > softirq=50613/50613 fqs=0 > [ 692.940000] 2-...: (25 GPs behind) idle=55a/0/0 softirq=28459/28460 fqs=0 > [ 692.940000] 3-...: (25 GPs behind) idle=5c2/0/0 softirq=30711/30982 fqs=0 > [ 692.940000] (detected by 0, t=6002 jiffies, g=5824, c=5823, q=3190) > [ 692.940000] Task dump for CPU 1: > [ 692.940000] swapper/1 R running 0 0 1 0x00100000 > [ 692.940000] Stack : 00000000 3646fb94 000000af ffffffff 0000008f > d2f1a9fc 8045a2a4 80410000 > [ 692.940000] 8041275c 00000001 00000001 80412540 80412724 > 80410000 000010d9 800132e8 > [ 692.940000] 00000000 00000001 8fc60000 8fc61ec0 80410000 > 8005c870 1100fc03 00000001 > [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 > 8001ade0 1100fc03 00000000 > [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db > aefffbff fff77fe7 dfffffef > [ 692.940000] ... > [ 692.940000] Call Trace: > [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 > [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 > [ 692.940000] > [ 692.940000] Task dump for CPU 2: > [ 692.940000] swapper/2 R running 0 0 1 0x00100000 > [ 692.940000] Stack : 00000000 00000001 00000010 00000000 00000000 > 00010001 8045a2a4 80410000 > [ 692.940000] 8041275c 00000001 00000000 804125e0 80412724 > 80410000 00000000 800132e8 > [ 692.940000] 1100fc03 00000002 8fc62000 8fc63ec0 80410000 > 8005c870 1100fc03 00000002 > [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 > 8001ade0 1100fc03 00000000 > [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fbffbfdf > 7fff7b3b 7ffaeff8 67fbffff > [ 692.940000] ... > [ 692.940000] Call Trace: > [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 > [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 > [ 692.940000] > [ 692.940000] Task dump for CPU 3: > [ 692.940000] swapper/3 R running 0 0 1 0x00100000 > [ 692.940000] Stack : 00000000 a69c5765 000000a3 ffffffff 0000008f > 00000020 8045a2a4 80410000 > [ 692.940000] 8041275c 00000001 00000001 80412680 80412724 > 80410000 000010d9 800132e8 > [ 692.940000] 1100fc03 00000003 8fc64000 8fc65ec0 80410000 > 8005c870 1100fc03 00000003 > [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 > 8001ade0 1100fc03 00000000 > [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fefffbdb > aefffbfb fff77fe7 dfffffef > [ 692.940000] ... > [ 692.940000] Call Trace: > [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 > [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 > [ 692.940000] > [ 692.940000] rcu_sched kthread starved for 6002 jiffies! g5824 c5823 > f0x0 s3 ->state=0x1 > root@lede:~# packet_write_wait: Connection to 192.168.20.1 port 22: Broken pipe > > >>> On 11/10/2016 9:06 PM, Outback Dingo wrote: >>> >>> On Fri, Nov 11, 2016 at 9:55 AM, Noah Causin wrote: >>> >>> I finally got my project working. >>> >>> I integrated the NDPI Deep Packet Inspection engine into my LEDE build, so I >>> could prioritize applications using Cake. >>> >>> http://www.ntop.org/products/deep-packet-inspection/ndpi/ >>> >>> NDPI integrates into IPTables, which allows me to DSCP mark packets. Cake >>> reads the DiffServ markings and puts the traffic into appropriate classes. >>> >>> I found a Makefile which successfully compiles: >>> https://github.com/981213/lede_src/blob/0d344bc2958838dcbc547a8f0a3d8842e6f6d2f8/package/my_package/ndpi-netfilter/Makefile >>> >>> The system works very well. Steam traffic is deprioritized to allow >>> applications like YouTube, Netflix, and Skype to receive higher amounts of >>> the available bandwidth. >>> >>> What I do for ingress is bridge two ports on an extra router, enable bridge >>> firewalling, and create IPTables rules to mark downstream packets. The >>> router I use is a D-Link DGL-5500, which is comparable to an Archer C7. >>> >>> IPv6 support is not available in this netfilter module, but the IPv4 support >>> is great. >>> >>> Noah Causin >>> >>> very nice.... however, can you describe a bit better how you >>> configured for ingress ? a second router? configuration file ? your >>> high level description seems a bit confusing to me >>> >>> >>> _______________________________________________ >>> Cake mailing list >>> Cake@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/cake >>> >>>