From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-x241.google.com (mail-it0-x241.google.com [IPv6:2607:f8b0:4001:c0b::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E8CB53B2A2 for ; Fri, 11 Nov 2016 09:22:02 -0500 (EST) Received: by mail-it0-x241.google.com with SMTP id e187so10859153itc.0 for ; Fri, 11 Nov 2016 06:22:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=/B2hZwTrqUiEb+YOACn4eum5UlmnKWZuQ3mzVHelJ5s=; b=IER1Utwz+AtEuhovoydKZKykyxp7FmpsxN+r5TDkJ0MajPjQnFK9OwpTxeCbXD1+aH +zXNU5MwHF12fZ2Sh7SPkRvSQ/Hyhak3k6HhV85nYdQfK4pvqW52Srp8qOWBao8GopWb JWO2LYNjI8QeM28E/QcIxyLsPfsVHZYvhu+NXSjINFYTFrgJAxyMDItbE8a6ohtCV2Cx u6CqkAMmc0LXX48i/NDEP2QwEv+EAj0Ha1IKOcop2Zn5mfzMF2hufjdhUa+VkWJpGwVA D8QioActmaBnl+pXquF3cWpnSy5YnMwaa1BMm4Ci0jFQZjf8t1eQPkur1njgrvtLktBG dPGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=/B2hZwTrqUiEb+YOACn4eum5UlmnKWZuQ3mzVHelJ5s=; b=eML7sG1AOQNulsO2/pGjaQej8ikOWbOdWzgAeyxSjzILpcsHmJLOihD1q2jG1HXujG NBwEo6FfcPLANmeMFZX3q4rCTsxuthSO8fXEp8UKukMv1QuzKwQSJtmibxZ7k1J3i7uw 74JvblrS+AVkfyHBAfpClMfyzpWtq2Eu2DGecTk/9Ohpc9oY5CAA9ExuiYB+CurH0S6Z nMIjB/w2q5c6rbndIbh1LBs/10S7fXRItxVOwy9RB89N2jFNaCof7t3WqtfPyxNPokZz jhBSmDX+4r9RazZdWico8+CEOzA5mN/sWxa8R1c9eP+trSz4WHMBA5J/npneCFXPgGpF GRwg== X-Gm-Message-State: ABUngvcAo7eNfjJqoSIg6CvjLKT3WSJo6+ho1GrH2B/MWopgrEpUEkURNnjtXaUAu5+Arg== X-Received: by 10.107.7.200 with SMTP id g69mr12257748ioi.157.1478874121873; Fri, 11 Nov 2016 06:22:01 -0800 (PST) Received: from [10.10.0.2] (c-68-42-142-136.hsd1.mi.comcast.net. [68.42.142.136]) by smtp.gmail.com with ESMTPSA id n42sm3985734ioe.9.2016.11.11.06.22.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 11 Nov 2016 06:22:01 -0800 (PST) To: Outback Dingo References: <3e5942d2-6d6b-0e01-8aa6-98c3535c26ef@gmail.com> <46c0133b-b6f8-fe1f-4d2e-0cf6088e024d@gmail.com> Cc: cake@lists.bufferbloat.net From: Noah Causin Message-ID: Date: Fri, 11 Nov 2016 09:22:03 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Cake] Cake with Deep Packet Inspection X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Nov 2016 14:22:03 -0000 What I would try is deleting your build source folder and start from scratch. Then edit source/package/kernel/kmod-sched-cake/Makefile change the PKG_SOURCE_VERSION to 70169dba14daa6ef7907af9c1e922ef9f797993a That's the latest stable version of cake. https://github.com/dtaht/sch_cake/commit/70169dba14daa6ef7907af9c1e922ef9f797993a On 11/11/2016 9:08 AM, Outback Dingo wrote: > On Fri, Nov 11, 2016 at 9:04 PM, Noah Causin wrote: >> Do you build your firmware from the git repository? >> >> > Yes it is built from a recent trunk, as of yesterday > > >> On 11/10/2016 11:28 PM, Outback Dingo wrote: >>> On Fri, Nov 11, 2016 at 10:52 AM, Outback Dingo >>> wrote: >>>> On Fri, Nov 11, 2016 at 10:42 AM, Noah Causin >>>> wrote: >>>>> I took an extra router I had and bridged two ports on the router's >>>>> switch, >>>>> so they just pass traffic. (eth0.2 and eth0.3) >>>>> >>>>> Clients >-[Main Router] --[Extra Router]--[Cable-Modem] >>>>> >>>>> The extra router is passive. It acts like it's part of the Ethernet >>>>> cable >>>>> between the main router and cable modem. It does not interfere. >>>>> >>>>> The extra router needs these packages: >>>>> >>>>> kmod-ebtables, kmod-ebtables-ipv4, kmod-crypto-pcompress and the two >>>>> packages compiled from the Makefile I showed below (iptables-mod-ndpi >>>>> and >>>>> iptables-mod-ndpi) >>>>> >>>>> I edited /etc/sysctl.conf and change the last two lines to this: >>>>> >>>>> net.bridge.bridge-nf-call-ip6tables=1 >>>>> net.bridge.bridge-nf-call-iptables=1 >>>>> >>>>> This enables bridge firewalling, so the traffic between the two ports >>>>> can be >>>>> marked. >>>>> >>>>> I then added firewall rules to LuCIs custom firewall rules tab. >>>>> >>>>> Example Rules, modify classes as desired: >>>>> >>>>> iptables -t mangle -A FORWARD -m ndpi --steam -m mac ! --mac-source Your >>>>> Main Router's Mac Address -j DSCP --set-dscp-class cs0 >>>>> iptables -t mangle -A FORWARD -m ndpi --youtube -m mac ! --mac-source >>>>> Your >>>>> Main Router's Mac Address -j DSCP --set-dscp-class cs2 >>>>> >>>>> iptables -t mangle -A FORWARD -m ndpi --netflix -m mac ! --mac-source >>>>> Your >>>>> Main Router's Mac Address -j DSCP --set-dscp-class cs3 >>>>> iptables -t mangle -A FORWARD -m ndpi --skype -m mac ! --mac-source Your >>>>> Main Router's Mac Address -j DSCP --set-dscp-class cs4 >>>>> >>>>> Have your main router use some form of DiffServ for both upload and >>>>> download >>>>> on its WAN interface. For upload traffic, you just need the two >>>>> packages >>>>> from the makefile to be installed on the main router and create firewall >>>>> rules like this: >>>>> >>>>> iptables -t mangle -A FORWARD -o eth2 -m ndpi --netflix -j DSCP >>>>> --set-dscp-class cs3 >>>>> iptables -t mangle -A FORWARD -o eth2 -m ndpi --skype -j DSCP >>>>> --set-dscp-class cs4 >>>>> >>>>> If you need help building a custom firmware image, just let me know. >>>>> >>>> Ok got it, its in-line.... right now im fighting the GFW of China >>>> using shadowsocks and chinadns so im pretty customized already, >>>> i was having issues with sqm and kernel segfaults on 4.4.30 ill build >>>> a cake and DPI image and see how that goes, maybe Ill try this >>>> after i deem sqm is stable with cake on this build. It a ZBT-WG3526 >>>> router... so its ralink based >>> like i stated on LEDE major issues when sqm is enabled.... even with >>> cake... is nobody else seeing these issues? >>> >>> [ 569.020000] 1-...: (6 GPs behind) idle=236/0/0 softirq=50607/50613 >>> fqs=1 >>> [ 569.020000] (detected by 0, t=6002 jiffies, g=5799, c=5798, q=573) >>> [ 569.020000] Task dump for CPU 1: >>> [ 569.020000] swapper/1 R running 0 0 1 0x00100000 >>> [ 569.020000] Stack : 00000000 00000001 00000015 00000000 00000000 >>> 00000001 8045a2a4 80410000 >>> [ 569.020000] 8041275c 00000001 00000001 80412540 80412724 >>> 80410000 00000000 800132e8 >>> [ 569.020000] 1100fc03 00000001 8fc60000 8fc61ec0 80410000 >>> 8005c870 1100fc03 00000001 >>> [ 569.020000] 00000000 80410000 8045a2a4 8005c868 80410000 >>> 8001ade0 1100fc03 00000000 >>> [ 569.020000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db >>> aefffbff fff77fe7 dfffffef >>> [ 569.020000] ... >>> [ 569.020000] Call Trace: >>> [ 569.020000] [<8000bba8>] __schedule+0x574/0x758 >>> [ 569.020000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >>> [ 569.020000] >>> [ 569.020000] rcu_sched kthread starved for 6001 jiffies! g5799 c5798 >>> f0x0 s3 ->state=0x1 >>> [ 629.130000] INFO: rcu_sched detected stalls on CPUs/tasks: >>> [ 629.130000] 1-...: (8 GPs behind) idle=756/0/0 softirq=50607/50613 >>> fqs=1 >>> [ 629.130000] (detected by 0, t=6002 jiffies, g=5801, c=5800, q=653) >>> [ 629.130000] Task dump for CPU 1: >>> [ 629.130000] swapper/1 R running 0 0 1 0x00100000 >>> [ 629.130000] Stack : 00000000 bfd47c67 00000094 ffffffff 00000081 >>> 00000011 8045a2a4 80410000 >>> [ 629.130000] 8041275c 00000001 00000001 80412540 80412724 >>> 80410000 000010d9 800132e8 >>> [ 629.130000] 1100fc03 00000001 8fc60000 8fc61ec0 80410000 >>> 8005c870 1100fc03 00000001 >>> [ 629.130000] 00000000 80410000 8045a2a4 8005c868 80410000 >>> 8001ade0 1100fc03 00000000 >>> [ 629.130000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db >>> aefffbff fff77fe7 dfffffef >>> [ 629.130000] ... >>> [ 629.130000] Call Trace: >>> [ 629.130000] [<8000bba8>] __schedule+0x574/0x758 >>> [ 629.130000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >>> [ 629.130000] >>> [ 629.130000] rcu_sched kthread starved for 6001 jiffies! g5801 c5800 >>> f0x0 s3 ->state=0x1 >>> [ 692.940000] INFO: rcu_sched detected stalls on CPUs/tasks: >>> [ 692.940000] 1-...: (0 ticks this GP) idle=4c2/0/0 >>> softirq=50613/50613 fqs=0 >>> [ 692.940000] 2-...: (25 GPs behind) idle=55a/0/0 softirq=28459/28460 >>> fqs=0 >>> [ 692.940000] 3-...: (25 GPs behind) idle=5c2/0/0 softirq=30711/30982 >>> fqs=0 >>> [ 692.940000] (detected by 0, t=6002 jiffies, g=5824, c=5823, q=3190) >>> [ 692.940000] Task dump for CPU 1: >>> [ 692.940000] swapper/1 R running 0 0 1 0x00100000 >>> [ 692.940000] Stack : 00000000 3646fb94 000000af ffffffff 0000008f >>> d2f1a9fc 8045a2a4 80410000 >>> [ 692.940000] 8041275c 00000001 00000001 80412540 80412724 >>> 80410000 000010d9 800132e8 >>> [ 692.940000] 00000000 00000001 8fc60000 8fc61ec0 80410000 >>> 8005c870 1100fc03 00000001 >>> [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 >>> 8001ade0 1100fc03 00000000 >>> [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fe9ff9db >>> aefffbff fff77fe7 dfffffef >>> [ 692.940000] ... >>> [ 692.940000] Call Trace: >>> [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 >>> [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >>> [ 692.940000] >>> [ 692.940000] Task dump for CPU 2: >>> [ 692.940000] swapper/2 R running 0 0 1 0x00100000 >>> [ 692.940000] Stack : 00000000 00000001 00000010 00000000 00000000 >>> 00010001 8045a2a4 80410000 >>> [ 692.940000] 8041275c 00000001 00000000 804125e0 80412724 >>> 80410000 00000000 800132e8 >>> [ 692.940000] 1100fc03 00000002 8fc62000 8fc63ec0 80410000 >>> 8005c870 1100fc03 00000002 >>> [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 >>> 8001ade0 1100fc03 00000000 >>> [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fbffbfdf >>> 7fff7b3b 7ffaeff8 67fbffff >>> [ 692.940000] ... >>> [ 692.940000] Call Trace: >>> [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 >>> [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >>> [ 692.940000] >>> [ 692.940000] Task dump for CPU 3: >>> [ 692.940000] swapper/3 R running 0 0 1 0x00100000 >>> [ 692.940000] Stack : 00000000 a69c5765 000000a3 ffffffff 0000008f >>> 00000020 8045a2a4 80410000 >>> [ 692.940000] 8041275c 00000001 00000001 80412680 80412724 >>> 80410000 000010d9 800132e8 >>> [ 692.940000] 1100fc03 00000003 8fc64000 8fc65ec0 80410000 >>> 8005c870 1100fc03 00000003 >>> [ 692.940000] 00000000 80410000 8045a2a4 8005c868 80410000 >>> 8001ade0 1100fc03 00000000 >>> [ 692.940000] 00000004 804124a0 000000a0 8001ade8 fefffbdb >>> aefffbfb fff77fe7 dfffffef >>> [ 692.940000] ... >>> [ 692.940000] Call Trace: >>> [ 692.940000] [<8000bba8>] __schedule+0x574/0x758 >>> [ 692.940000] [<800132e8>] r4k_wait_irqoff+0x0/0x20 >>> [ 692.940000] >>> [ 692.940000] rcu_sched kthread starved for 6002 jiffies! g5824 c5823 >>> f0x0 s3 ->state=0x1 >>> root@lede:~# packet_write_wait: Connection to 192.168.20.1 port 22: Broken >>> pipe >>> >>> >>>>> On 11/10/2016 9:06 PM, Outback Dingo wrote: >>>>> >>>>> On Fri, Nov 11, 2016 at 9:55 AM, Noah Causin >>>>> wrote: >>>>> >>>>> I finally got my project working. >>>>> >>>>> I integrated the NDPI Deep Packet Inspection engine into my LEDE build, >>>>> so I >>>>> could prioritize applications using Cake. >>>>> >>>>> http://www.ntop.org/products/deep-packet-inspection/ndpi/ >>>>> >>>>> NDPI integrates into IPTables, which allows me to DSCP mark packets. >>>>> Cake >>>>> reads the DiffServ markings and puts the traffic into appropriate >>>>> classes. >>>>> >>>>> I found a Makefile which successfully compiles: >>>>> >>>>> https://github.com/981213/lede_src/blob/0d344bc2958838dcbc547a8f0a3d8842e6f6d2f8/package/my_package/ndpi-netfilter/Makefile >>>>> >>>>> The system works very well. Steam traffic is deprioritized to allow >>>>> applications like YouTube, Netflix, and Skype to receive higher amounts >>>>> of >>>>> the available bandwidth. >>>>> >>>>> What I do for ingress is bridge two ports on an extra router, enable >>>>> bridge >>>>> firewalling, and create IPTables rules to mark downstream packets. The >>>>> router I use is a D-Link DGL-5500, which is comparable to an Archer C7. >>>>> >>>>> IPv6 support is not available in this netfilter module, but the IPv4 >>>>> support >>>>> is great. >>>>> >>>>> Noah Causin >>>>> >>>>> very nice.... however, can you describe a bit better how you >>>>> configured for ingress ? a second router? configuration file ? your >>>>> high level description seems a bit confusing to me >>>>> >>>>> >>>>> _______________________________________________ >>>>> Cake mailing list >>>>> Cake@lists.bufferbloat.net >>>>> https://lists.bufferbloat.net/listinfo/cake >>>>> >>>>>