From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-xd43.google.com (mail-io1-xd43.google.com [IPv6:2607:f8b0:4864:20::d43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id A72A23B2A4 for ; Mon, 16 Sep 2019 08:51:12 -0400 (EDT) Received: by mail-io1-xd43.google.com with SMTP id h144so78233876iof.7 for ; Mon, 16 Sep 2019 05:51:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=ZuqNPLE9m/1AQLnuQ0qwq+HO/ALkaGocP/4LJitmTGs=; b=FQn230CyxNo8e99MKmbuSVuW9Up6XiQboajMAMDxoZjZVVRiV+rxtR+pyRRmJeMBXT cM9Son0B4UOdI9A3NkRU0aKoOE9jkRdZb3yXkJZ89H/Nrm5pRhHke7zBJ0GMLuQ75b+1 U6F5qQrRgNkq776n79YPST1KpjwWMJMpxaTjTJzgESKi5eJwsqByFQmVMFkSxlkYwtNW GIYreFcFmZ+XGVvaM5jCiqSyzBbUuBbDyq2ILI1/ejV/DV/HudcnzkohU92YrwPmlJGn DcyM+Mc4cxUw1qlQiZILF2w07UsKWdAbKVPKzPwCogtvr98unIIFK11PYSv1tJ2UZCC0 N8Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=ZuqNPLE9m/1AQLnuQ0qwq+HO/ALkaGocP/4LJitmTGs=; b=Zvj/W0EqLKrGydHvu3uRcT4C1gBULjj+bMRb00rixdX614+NyzABSdRiZhZtfoX3nq RFfcU5qJ12q3XCYOvrqH+MZAu/sktJ+goS6cE9xSXZzRB4BZ9yWPN8BYf+243gFJEAhj R1r5nRwivZ0GQlM8Yqhz5g8iDiirp52/rwVfQnfjl4Gdci1ZEM72rfOkrdu4n62cNlE0 TSmSv+vLi6U4DdtZX4A8rgl801MUk8BbXrw4vNIXaxIVNE+jFJpLLmfzBCRA/pk3eNfV w+xIeyPyNykuiAJVEHbLlwyo7vUjPN5vS5rn4dCITSTN5lZc+Yhkwo9aevPRysUIWlm8 VS5A== X-Gm-Message-State: APjAAAWFK65O8uhrz50+hIe4ceEZRCMjC681b0zVfnop73SGNvqnsV+L hjZbwvvNyUWcEPhXMNBnO/4D0TC8XX3UPoS/CAY= X-Google-Smtp-Source: APXvYqyuZzbnHeGoT47ap3lAD1CaAhSYMc4VtlgnrpJaNVNMMy40qg80BKEV4omMHvkFidnwy7H7YlE40f8caZI5bAs= X-Received: by 2002:a5d:9245:: with SMTP id e5mr5866175iol.45.1568638271994; Mon, 16 Sep 2019 05:51:11 -0700 (PDT) MIME-Version: 1.0 References: <2825CE14-2109-4580-A086-9701F4D3ADF0@gmail.com> <18b1c174-b88d-4664-9aa8-9c42925fc14c@www.fastmail.com> <9a90111b-2389-4dc6-8409-18c40f895540@www.fastmail.com> <43F02160-E691-4393-A0C0-8AB4AD962700@gmail.com> <057ea708-6797-23f5-ef01-9d3d7b002578@newmedia-net.de> In-Reply-To: From: Dave Taht Date: Mon, 16 Sep 2019 13:51:00 +0100 Message-ID: To: Sebastian Gottschall Cc: Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] cake memory consumption X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Sep 2019 12:51:12 -0000 Perhaps the differences in memory use are a memory leak of some kind? If you could run the same number of packets through each configuration and look at memory use, that might point somewhere. cake - with gso-splitting - should fragment memory more than the other alternatives, as will fq_codel_fast with sce enabled. On Mon, Sep 16, 2019 at 1:00 PM Dave Taht wrote: > > I am puzzled as to why fq_codel_fast would use more ram than fq_codel > would, was sce (gso-splotting) enabled? > > similarly, the differences between hfsc and htb are interesting. I > don't get that either. > > How many cake instances are being created? > > And for the sake of discussion, what does cake standalone consume? > > On Mon, Sep 16, 2019 at 11:22 AM Sebastian Gottschall > wrote: > > > > after we found out serious out of memory issues on smaller embedded dev= ices (128 mb ram) we made some benchmarks with different schedulers > > with the result that cake takes a serious amount of memory. we use the = out of tree cake module and we use it class based since we have complex met= hods of doing qos per interface, per mac addresse or even per > > I note that I often thought about having mac address functionality > might be a valuable mode for cake. > > >ip/network. so its not just simple cake on a single interface solution. = we made some benchmarks with different schedulers. does anybody have a solu= tion for making that better? > > With such complexity required I'd stick to hfsc + fq_X rather than > layer in cake. > > Understanding the model (sh -x the tc commands for, say, hfsc + > something and htb + something ) your users require, though, would be > helpful. We tried to design cake so that a jillion optimizations such > as ack prioritization, per network fq (instead per flow/per host) - > but we couldn't possibly cover all use cases in it with out more > feedback from the field. > > Still... such a big difference in memory use doesn't add up. Cake has > a larger fixed memory allocation > than fq_codel, but the rest is just packets which come from global memory= . > > Can you point to a build and a couple targets we could try? I am > presently travelling (in portugal) and won't > be back online until later this week. > > > > HTB/FQ_CODEL -------=EF=83=A0 62M > > HTB/SFQ -------=EF=83=A0 62M > > HTB/PIE -------=EF=83=A0 62M > > HTB/FQ_CODEL_FAST -------=EF=83=A0 67M > > HTB/CAKE -------=EF=83=A0111M > > > > HFSC/FQ_CODEL_FAST -------=EF=83=A0 47M > > HTB/PIE -------=EF=83=A0 49M > > HTB/SFQ -------=EF=83=A0 50M > > HFSC /FQ_CODEL -------=EF=83=A0 52M > > HFSC/CAKE -------=EF=83=A0109M > > > > > > consider that the benchmark doesnt show the real values. its system ove= rall and does not consider memory taken by the wireless driver for instance= which is about 45 mb of ram for ath10k > > so this makes all even more worse unfortunatly since there is not that = many ram left for cake. just about 70mb maybe. > > Am 08.09.2019 um 19:27 schrieb Jonathan Morton: > > > > You could also set it back to 'internet' and progressively reduce the > > bandwidth parameter, making the Cake shaper into the actual bottleneck. > > This is the correct fix for the problem, and you should notice an > > instant improvement as soon as the bandwidth parameter is correct. > > > > Hand tuning this one link is not a problem. I'm searching for a set of = settings that will provide generally good performance across a wide range o= f devices, links, and situations. > > > > From what you've indicated so far there's nothing as effective as a cor= rect bandwidth estimation if we consider the antenna (link) a black box. Ex= pecting the user to input expected throughput for every link and then manag= ing that information is essentially a non-starter. > > > > Radio tuning provides some improvement, but until ubiquiti starts shipp= ing with Codel on non-router devices I don't think there's a good solution = here. > > > > Any way to have the receiving device detect bloat and insert an ECN? > > > > That's what the qdisc itself is supposed to do. > > > > I don't think the time spent in the intermediate device is detectable a= t the kernel level but we keep track of latency for routing decisions and c= ould detect bloat with some accuracy, the problem is how to respond. > > > > As long as you can detect which link the bloat is on (and in which dire= ction), you can respond by reducing the bandwidth parameter on that half-li= nk by a small amount. Since you have a cooperating network, maintaining a = time standard on each node sufficient to observe one-way delays seems feasi= ble, as is establishing a normal baseline latency for each link. > > > > The characteristics of the bandwidth parameter being too high are easy = to observe. Not only will the one-way delay go up, but the received throug= hput in the same direction at the same time will be lower than configured. = You might use the latter as a hint as to how far you need to reduce the sh= aped bandwidth. > > > > Deciding when and by how much to *increase* bandwidth, which is presuma= bly desirable when link conditions improve, is a more difficult problem whe= n the link hardware doesn't cooperate by informing you of its status. (Thi= s is something you could reasonably ask Ubiquiti to address.) > > > > I would assume that link characteristics will change slowly, and run an= occasional explicit bandwidth probe to see if spare bandwidth is available= . If that probe comes through without exhibiting bloat, *and* the link is = otherwise loaded to capacity, then increase the shaper by an amount within = the probe's capacity of measurement - and schedule a repeat. > > > > A suitable probe might be 100x 1500b packets paced out over a second, b= ypassing the shaper. This will occupy just over 1Mbps of bandwidth, and ca= n be expected to induce 10ms of delay if injected into a saturated 100Mbps = link. Observe the delay experienced by each packet *and* the quantity of o= ther traffic that appears between them. Only if both are favourable can yo= u safely open the shaper, by 1Mbps. > > > > Since wireless links can be expected to change their capacity over time= , due to eg. weather and tree growth, this seems to be more generally usefu= l than a static guess. You could deploy a new link with a conservative "gu= ess" of say 10Mbps, and just probe from there. > > > > - Jonathan Morton > > _______________________________________________ > > Cake mailing list > > Cake@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/cake > > > > _______________________________________________ > > Cake mailing list > > Cake@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/cake > > > > -- > > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 --=20 Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740