From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from webmail.newmedia-net.de (smtps.newmedia-net.de [IPv6:2a05:a1c0:0:de::167]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id A0D2C3CB36 for ; Tue, 17 Sep 2019 01:32:35 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=newmedia-net.de; s=mikd; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:To:Subject; bh=IBzePAnaa2HAb1RA1KfgE2J/2t10ktT34NQBpZc2FwU=; b=h02LlkjIR4nOPrls9lcmo7cC/WhKRluim8fLrgLyYsL2oKBI2QcgKgnPqH2oqs2QAqGJAQbHw5smqQofwA+q5IaQDbitRaNnx/CU7psRbcdpweaUATiOft/AU5zcjpvi8mSyR2w89zVCyjNjCF59H/Xrw4kYKvywkIntFLgCazU=; To: =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= , cake@lists.bufferbloat.net References: <2825CE14-2109-4580-A086-9701F4D3ADF0@gmail.com> <18b1c174-b88d-4664-9aa8-9c42925fc14c@www.fastmail.com> <9a90111b-2389-4dc6-8409-18c40f895540@www.fastmail.com> <43F02160-E691-4393-A0C0-8AB4AD962700@gmail.com> <057ea708-6797-23f5-ef01-9d3d7b002578@newmedia-net.de> <878sqomoj4.fsf@toke.dk> <87h85cl4qk.fsf@toke.dk> From: Sebastian Gottschall Message-ID: <65e68948-6464-23ac-5df0-91735fc298af@newmedia-net.de> Date: Tue, 17 Sep 2019 07:31:04 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.1.0 MIME-Version: 1.0 In-Reply-To: <87h85cl4qk.fsf@toke.dk> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Received: from [2003:c9:3f21:2600:e48d:ccf8:4911:ed41] by webmail.newmedia-net.de with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.72) (envelope-from ) id 1iA66U-0002nP-VG; Tue, 17 Sep 2019 07:32:42 +0200 Subject: Re: [Cake] cake memory consumption X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Sep 2019 05:32:35 -0000 according to the output there is a flaw/bug in the memory limit calculation cake_reconfigure     may set buffer_limit to  ~0 if no rate is set. the following line "min(buffer_limit, max(sch->limit * psched_mtu(qdisc_dec(sch), q->buffer_config_limit))" doesnt make it better since buffer_config_limit is not configured so we got a possible memory overuse here. my proposal -       q->buffer_limit = min(q->buffer_limit, -                             max(sch->limit * psched_mtu(qdisc_dev(sch)), -                                 q->buffer_config_limit)); +       if (q->buffer_config_limit) +               q->buffer_limit = min(q->buffer_limit, +                       max(sch->limit * psched_mtu(qdisc_dev(sch)), q->buffer_config_limit)); +       else +               q->buffer_limit = min(q->buffer_limit, +                       max(sch->limit * psched_mtu(qdisc_dev(sch)), 4U << 20)); Am 16.09.2019 um 16:01 schrieb Toke Høiland-Jørgensen: > Sebastian Gottschall writes: > >> Am 16.09.2019 um 14:08 schrieb Toke Høiland-Jørgensen: >>> Sebastian Gottschall writes: >>> >>>> after we found out serious out of memory issues on smaller embedded >>>> devices (128 mb ram) we made some benchmarks with different schedulers >>>> with the result that cake takes a serious amount of memory. we use the >>>> out of tree cake module and we use it class based since we have complex >>>> methods of doing qos per interface, per mac addresse or even per >>>> ip/network. so its not just simple cake on a single interface solution. >>>> we made some benchmarks with different schedulers. does anybody have a >>>> solution for making that better? >>>> >>>> HTB/FQ_CODEL ------- 62M >>>> HTB/SFQ ------- 62M >>>> HTB/PIE ------- 62M >>>> HTB/FQ_CODEL_FAST ------- 67M >>>> HTB/CAKE -------111M >>>> >>>> HFSC/FQ_CODEL_FAST ------- 47M >>>> HTB/PIE ------- 49M >>>> HTB/SFQ ------- 50M >>>> HFSC /FQ_CODEL ------- 52M >>>> HFSC/CAKE -------109M >>> How are you measuring the memory usage, and what is your full config for >>> each setup? :) >> me? nothing. i requested this test from a reporter and he uses just free >> / top. so there is a error tollerance. > Ah, I see. So this is just total system memory as reported by top. > >> but it shows a significant difference between cake and fq_codel etc. >> cake is doing a OOM at the end >> >> for the full report including config screenshots see this >> https://svn.dd-wrt.com/ticket/6798#comment:14. it shows also the qos >> setup which i can use to reproduce and to >> print out the full tc ruleset if required (which it surelly is for you). >> if you want i will recreate this setup and send the tc rules on this >> list > Yes, please do. The output of 'tc -s qdisc' would be useful as well to > see how much memory CAKE itself thinks it's using... > > Are you setting the memory_limit in your config or relying on CAKE's > default? > > -Toke > >