From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-x244.google.com (mail-it0-x244.google.com [IPv6:2607:f8b0:4001:c0b::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 8DF3D3B260 for ; Thu, 10 Nov 2016 23:29:22 -0500 (EST) Received: by mail-it0-x244.google.com with SMTP id e187so8073004itc.0 for ; Thu, 10 Nov 2016 20:29:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=vO5lIJhs6cgGbwM7ZlZmKyHklI0A+ScwJ5MnG+2XJmc=; b=qopfJBx6BS2TJNa5A0wtDKohfQYIexpII408pibRxstdoZzuiOhTspcsV6wCvkRcH8 sKiLVsxdh4o9E0daawfAoCu4Rppg5GLNlqWXMFGz5GaRq+r3+wFc7P5sZehj/4WJ077D rphp13b/5ol0krblNuy2H+BqvThK7op+qHtVyezpsHfCLx1Pzi9S0Pm1yz38wDePNWs+ 9Esnz+aIO6N74ejkkgscfYB93FIN7b65K9ySEIl7bacvsMOHrFYkA9Kc6MheceowvRhv DY/lHY7NvJwShVHQewyPpFZncAkLKVsp4bqd/z9NynNP02/ZRdWa9r626FYMwNKSnSs9 LUbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=vO5lIJhs6cgGbwM7ZlZmKyHklI0A+ScwJ5MnG+2XJmc=; b=A87Ua4VgJgyJeWGj5m7JlveG2evJsVDDUKIdl6z5oKVOye+CE52RvftrhkwczpvuSb lq2nd0+1okZJhjIqk8KNI5jJLOb+UXT3ZZafQtJLiRKHIV7FfGI1LOTaHEvnRo82OoMo IRP+BFee5WUy7niqKJjV6z8LMe4+kBzP3Y4SdcvvenjRw3cFyCp4qzj6h9uHYu3FVswH BsAiB9SxCMwihPFDUCR94a/kaOxxu0EjQD3+BHEDiI/jdNgDlKKFouSl63XoVCa/1DCr Ar/zLCpA2e+BSnjAOVsz5WlG1JDQ3u4clvsOp9W4Gjm+NkiFH1qNoZjl+IU3iBnP9v02 jd9Q== X-Gm-Message-State: ABUngve2bTlYJyaeT3ICzJ/4322T7afP/K8oEGJU/b4QXmwvSYHGb9sF3tKHETBPAd2f20cVWyfRP+7kVP8c+A== X-Received: by 10.107.1.138 with SMTP id 132mr9634532iob.72.1478838561603; Thu, 10 Nov 2016 20:29:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.22.80 with HTTP; Thu, 10 Nov 2016 20:28:41 -0800 (PST) In-Reply-To: References: <3e5942d2-6d6b-0e01-8aa6-98c3535c26ef@gmail.com> <46c0133b-b6f8-fe1f-4d2e-0cf6088e024d@gmail.com> From: Outback Dingo Date: Fri, 11 Nov 2016 12:28:41 +0800 Message-ID: To: Noah Causin Cc: cake@lists.bufferbloat.net Content-Type: text/plain; charset=UTF-8 Subject: Re: [Cake] Cake with Deep Packet Inspection X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Nov 2016 04:29:22 -0000 On Fri, Nov 11, 2016 at 10:52 AM, Outback Dingo wrote: > On Fri, Nov 11, 2016 at 10:42 AM, Noah Causin wrote: >> I took an extra router I had and bridged two ports on the router's switch, >> so they just pass traffic. (eth0.2 and eth0.3) >> >> Clients >-[Main Router] --[Extra Router]--[Cable-Modem] >> >> The extra router is passive. It acts like it's part of the Ethernet cable >> between the main router and cable modem. It does not interfere. >> >> The extra router needs these packages: >> >> kmod-ebtables, kmod-ebtables-ipv4, kmod-crypto-pcompress and the two >> packages compiled from the Makefile I showed below (iptables-mod-ndpi and >> iptables-mod-ndpi) >> >> I edited /etc/sysctl.conf and change the last two lines to this: >> >> net.bridge.bridge-nf-call-ip6tables=1 >> net.bridge.bridge-nf-call-iptables=1 >> >> This enables bridge firewalling, so the traffic between the two ports can be >> marked. >> >> I then added firewall rules to LuCIs custom firewall rules tab. >> >> Example Rules, modify classes as desired: >> >> iptables -t mangle -A FORWARD -m ndpi --steam -m mac ! --mac-source Your >> Main Router's Mac Address -j DSCP --set-dscp-class cs0 >> iptables -t mangle -A FORWARD -m ndpi --youtube -m mac ! --mac-source Your >> Main Router's Mac Address -j DSCP --set-dscp-class cs2 >> >> iptables -t mangle -A FORWARD -m ndpi --netflix -m mac ! --mac-source Your >> Main Router's Mac Address -j DSCP --set-dscp-class cs3 >> iptables -t mangle -A FORWARD -m ndpi --skype -m mac ! --mac-source Your >> Main Router's Mac Address -j DSCP --set-dscp-class cs4 >> >> Have your main router use some form of DiffServ for both upload and download >> on its WAN interface. For upload traffic, you just need the two packages >> from the makefile to be installed on the main router and create firewall >> rules like this: >> >> iptables -t mangle -A FORWARD -o eth2 -m ndpi --netflix -j DSCP >> --set-dscp-class cs3 >> iptables -t mangle -A FORWARD -o eth2 -m ndpi --skype -j DSCP >> --set-dscp-class cs4 >> >> If you need help building a custom firmware image, just let me know. >> > > Ok got it, its in-line.... right now im fighting the GFW of China > using shadowsocks and chinadns so im pretty customized already, > i was having issues with sqm and kernel segfaults on 4.4.30 ill build > a cake and DPI image and see how that goes, maybe Ill try this > after i deem sqm is stable with cake on this build. It a ZBT-WG3526 > router... so its ralink based like i stated on LEDE major issues when sqm is enabled.... even with cake... is nobody else seeing these issues? [ 569.020000] 1-...: (6 GPs behind) idle=236/0/0 softirq=50607/50613 fqs=1 [ 569.020000] (detected by 0, t=6002 jiffies, g=5799, c=5798, q=573) [ 569.020000] Task dump for CPU 1: [ 569.020000] swapper/1 R running 0 0 1 0x00100000 [ 569.020000] Stack : 00000000 00000001 00000015 00000000 00000000 00000001 8045a2a4 80410000 [ 569.020000] 8041275c 00000001 00000001 80412540 80412724 80410000 00000000 800132e8 [ 569.020000] 1100fc03 00000001 8fc60000 8fc61ec0 80410000 8005c870 1100fc03 00000001 [ 569.020000] 00000000 80410000 8045a2a4 8005c868 80410000 8001ade0 1100fc03 00000000 [ 569.020000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db aefffbff fff77fe7 dfffffef [ 569.020000] ... [ 569.020000] Call Trace: [ 569.020000] [<8000bba8>] __schedule+0x574/0x758 [ 569.020000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 [ 569.020000] [ 569.020000] rcu_sched kthread starved for 6001 jiffies! g5799 c5798 f0x0 s3 ->state=0x1 [ 629.130000] INFO: rcu_sched detected stalls on CPUs/tasks: [ 629.130000] 1-...: (8 GPs behind) idle=756/0/0 softirq=50607/50613 fqs=1 [ 629.130000] (detected by 0, t=6002 jiffies, g=5801, c=5800, q=653) [ 629.130000] Task dump for CPU 1: [ 629.130000] swapper/1 R running 0 0 1 0x00100000 [ 629.130000] Stack : 00000000 bfd47c67 00000094 ffffffff 00000081 00000011 8045a2a4 80410000 [ 629.130000] 8041275c 00000001 00000001 80412540 80412724 80410000 000010d9 800132e8 [ 629.130000] 1100fc03 00000001 8fc60000 8fc61ec0 80410000 8005c870 1100fc03 00000001 [ 629.130000] 00000000 80410000 8045a2a4 8005c868 80410000 8001ade0 1100fc03 00000000 [ 629.130000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db aefffbff fff77fe7 dfffffef [ 629.130000] ... [ 629.130000] Call Trace: [ 629.130000] [<8000bba8>] __schedule+0x574/0x758 [ 629.130000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 [ 629.130000] [ 629.130000] rcu_sched kthread starved for 6001 jiffies! g5801 c5800 f0x0 s3 ->state=0x1 [ 692.940000] INFO: rcu_sched detected stalls on CPUs/tasks: [ 692.940000] 1-...: (0 ticks this GP) idle=4c2/0/0 softirq=50613/50613 fqs=0 [ 692.940000] 2-...: (25 GPs behind) idle=55a/0/0 softirq=28459/28460 fqs=0 [ 692.940000] 3-...: (25 GPs behind) idle=5c2/0/0 softirq=30711/30982 fqs=0 [ 692.940000] (detected by 0, t=6002 jiffies, g=5824, c=5823, q=3190) [ 692.940000] Task dump for CPU 1: [ 692.940000] swapper/1 R running 0 0 1 0x00100000 [ 692.940000] Stack : 00000000 3646fb94 000000af ffffffff 0000008f d2f1a9fc 8045a2a4 80410000 [ 692.940000] 8041275c 00000001 00000001 80412540 80412724 80410000 000010d9 800132e8 [ 692.940000] 00000000 00000001 8fc60000 8fc61ec0 80410000 8005c870 1100fc03 00000001 [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 8001ade0 1100fc03 00000000 [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db aefffbff fff77fe7 dfffffef [ 692.940000] ... [ 692.940000] Call Trace: [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 [ 692.940000] [ 692.940000] Task dump for CPU 2: [ 692.940000] swapper/2 R running 0 0 1 0x00100000 [ 692.940000] Stack : 00000000 00000001 00000010 00000000 00000000 00010001 8045a2a4 80410000 [ 692.940000] 8041275c 00000001 00000000 804125e0 80412724 80410000 00000000 800132e8 [ 692.940000] 1100fc03 00000002 8fc62000 8fc63ec0 80410000 8005c870 1100fc03 00000002 [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 8001ade0 1100fc03 00000000 [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fbffbfdf 7fff7b3b 7ffaeff8 67fbffff [ 692.940000] ... [ 692.940000] Call Trace: [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 [ 692.940000] [ 692.940000] Task dump for CPU 3: [ 692.940000] swapper/3 R running 0 0 1 0x00100000 [ 692.940000] Stack : 00000000 a69c5765 000000a3 ffffffff 0000008f 00000020 8045a2a4 80410000 [ 692.940000] 8041275c 00000001 00000001 80412680 80412724 80410000 000010d9 800132e8 [ 692.940000] 1100fc03 00000003 8fc64000 8fc65ec0 80410000 8005c870 1100fc03 00000003 [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 8001ade0 1100fc03 00000000 [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fefffbdb aefffbfb fff77fe7 dfffffef [ 692.940000] ... [ 692.940000] Call Trace: [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 [ 692.940000] [ 692.940000] rcu_sched kthread starved for 6002 jiffies! g5824 c5823 f0x0 s3 ->state=0x1 root@lede:~# packet_write_wait: Connection to 192.168.20.1 port 22: Broken pipe > >> >> On 11/10/2016 9:06 PM, Outback Dingo wrote: >> >> On Fri, Nov 11, 2016 at 9:55 AM, Noah Causin wrote: >> >> I finally got my project working. >> >> I integrated the NDPI Deep Packet Inspection engine into my LEDE build, so I >> could prioritize applications using Cake. >> >> http://www.ntop.org/products/deep-packet-inspection/ndpi/ >> >> NDPI integrates into IPTables, which allows me to DSCP mark packets. Cake >> reads the DiffServ markings and puts the traffic into appropriate classes. >> >> I found a Makefile which successfully compiles: >> https://github.com/981213/lede_src/blob/0d344bc2958838dcbc547a8f0a3d8842e6f6d2f8/package/my_package/ndpi-netfilter/Makefile >> >> The system works very well. Steam traffic is deprioritized to allow >> applications like YouTube, Netflix, and Skype to receive higher amounts of >> the available bandwidth. >> >> What I do for ingress is bridge two ports on an extra router, enable bridge >> firewalling, and create IPTables rules to mark downstream packets. The >> router I use is a D-Link DGL-5500, which is comparable to an Archer C7. >> >> IPv6 support is not available in this netfilter module, but the IPv4 support >> is great. >> >> Noah Causin >> >> very nice.... however, can you describe a bit better how you >> configured for ingress ? a second router? configuration file ? your >> high level description seems a bit confusing to me >> >> >> _______________________________________________ >> Cake mailing list >> Cake@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cake >> >>