From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id E4CAE3B2A4 for ; Thu, 24 Jun 2021 09:26:56 -0400 (EDT) Received: by mail-wm1-x32a.google.com with SMTP id p10-20020a05600c430ab02901df57d735f7so6242307wme.3 for ; Thu, 24 Jun 2021 06:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=lDfymRuWsiW8iyCb4al+bxvPRK2aXrgz7eHL69L7Wak=; b=ijF0tlloydDcWv7lkBI1UHUtDSXU/g3d2xP5VFCeI9zpXSSzPIgLgmmPR9UmuIQT1G NZ1q9hHTTPyI+KTZgMmidGxS9dHUMUA6IjIMoiS82MXGXkeQIpLyz7NR/0ujKEePknUw kzs55WmWjVToUlQl2z4BbkKgdCJ8Bjnn+jlgnwdpBrjPQkXvgz2T+7CM671wZSdArezP /hEGKoncGi0O6pOgpHUJebDIGi1bXAc95lwHWoDzOj7jLIDp2YWEvzQw63zj0jKogLTc PWnOL6sAdj3Z0xiXDa5bkwHtL7BUffQnXMBELxQgOQxaqqqhbKh7k6chCpiA378nz4zY 348w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=lDfymRuWsiW8iyCb4al+bxvPRK2aXrgz7eHL69L7Wak=; b=h3531Crl9GIkItyf/qPYyInpyPaYVKAweFfPTM6bcH5otFHhBvVdVUOf5i6bEgjZ4p 95PjBdvjjbTLQ6c9gKg/5vYWvD3JCUM7AlCJW2oJ8TVpx6pqxjLneBgLV+vmb304i9Yr NdvLdCRnNYN3iweDe2X3DAEsS6oiajz8CE777NCpLUwnIBQ9kuDiVZKM46VeBoV8DtZD eom5Vxyrbw/8YlDAE/2Vxef19WDyyEjRF4zO/lt6lT5zGGi42aLLwg9Y6JKCQe7grHuh UfS6O9NWTn7CUCdU1ZWWyGhwqjBWqKmRqxk75XNwdHbHa4UIKn7ooxOiKwwBuuTp+RMD zxUQ== X-Gm-Message-State: AOAM5333GaCNBGj19qU4WN+T4y5lBCkjtPMYfuaTSUpKldUdpkMqiUI6 b7h47IjbT+PPsl+lRIgDZCXC8v50BXBWsA== X-Google-Smtp-Source: ABdhPJylJaHLEOngEGC9m7RbZDmcgSkGWNvVS+baQv6O6VDB0AEWbdvUAB2kuLQtomMbdz3zPsdJ3A== X-Received: by 2002:a7b:c8d6:: with SMTP id f22mr4356845wml.87.1624541215774; Thu, 24 Jun 2021 06:26:55 -0700 (PDT) Received: from [10.72.0.88] (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id m67sm9179333wmm.17.2021.06.24.06.26.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 06:26:55 -0700 (PDT) Message-ID: From: Pete Heist To: Cake List Date: Thu, 24 Jun 2021 15:26:53 +0200 In-Reply-To: <3d529e0b78ef8ea54021d8f8e93285e9a474cba1.camel@heistp.net> References: <3d529e0b78ef8ea54021d8f8e93285e9a474cba1.camel@heistp.net> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.40.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: Re: [Cake] Cake memory usage after 1TB X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jun 2021 13:26:57 -0000 Hmm, since I see in the code that this could lead to drops, I've restarted it and upped the limit to 16M. I haven't had reports of widespread problems, but I'll keep an eye on this more closely. :) Pete On Thu, 2021-06-24 at 15:06 +0200, Pete Heist wrote: > I'm using Cake on an EdgeRouter-X at a site that does around 50- > 100GB/day, and we're quite happy with it so far. Egress and ingress > stats are below after ~1TB download and ~100GB upload. > > I use "memlimit 8M", and noticed that it's reporting 8389696 bytes > used > on egress, which is at the maximum (slightly over actually). Around > 6M > is used on ingress. Should I then raise these limits, or is it normal > for it to settle in around the maximum over time? > > Pete > > > Egress: > ------- > > qdisc cake 8008: dev eth4 root refcnt 2 bandwidth 8Mbit diffserv4 > dual- > srchost nat nowash no-ack-filter split-gso rtt 100.0ms raw overhead 0 > fwmark 0xf no-sce >  Sent 102655686381 bytes 386831397 pkt (dropped 377557, overlimits > 230324326 requeues 0) >  backlog 0b 0p requeues 0 >  memory used: 8389696b of 8Mb >  capacity estimate: 8Mbit >  min/max network layer size:           42 /    1514 >  min/max overhead-adjusted size:       42 /    1514 >  average network hdr offset:           14 > >                    Bulk  Best Effort        Video        Voice >   thresh        500Kbit        8Mbit        4Mbit        2Mbit >   target         36.3ms        5.0ms        5.0ms        9.1ms >   interval      131.3ms      100.0ms      100.0ms      104.1ms >   pk_delay       17.5ms        4.1ms        1.7ms        2.0ms >   av_delay        4.5ms        232us         82us        382us >   sp_delay         14us         23us          6us          7us >   backlog            0b           0b           0b           0b >   pkts           165781    324229040     61032588      1781545 >   bytes        45344666  98316181815   4518083349    291035866 >   way_inds           42     18492140      7782656        34167 >   way_miss         4289      6575635        77917        62650 >   way_cols            0          340            0            0 >   sce                 0            0            0            0 >   marks               7        19542            1            0 >   drops               8       377528           15            6 >   ack_drop            0            0            0            0 >   sp_flows            0            2            0            1 >   bk_flows            0            1            0            0 >   un_flows            0            0            0            0 >   max_len          1514         1514         1514         1514 >   quantum           300          300          300          300 > > Ingress: > -------- > > qdisc cake 1: dev ifb4eth4 root refcnt 2 bandwidth 40Mbit diffserv4 > dual-dsthost nat nowash no-ack-filter split-gso rtt 100.0ms raw > overhead 0 fwmark 0xf no-sce >  Sent 1081750548776 bytes 839670608 pkt (dropped 1616216, overlimits > 657093747 requeues 0) >  backlog 0b 0p requeues 0 >  memory used: 6027328b of 8Mb >  capacity estimate: 40Mbit >  min/max network layer size:           60 /    1514 >  min/max overhead-adjusted size:       60 /    1514 >  average network hdr offset:           14 > >                    Bulk  Best Effort        Video        Voice >   thresh       2500Kbit       40Mbit       20Mbit       10Mbit >   target          7.3ms        5.0ms        5.0ms        5.0ms >   interval      102.3ms      100.0ms      100.0ms      100.0ms >   pk_delay        2.0ms        6.5ms        692us        1.1ms >   av_delay        248us        2.6ms         88us        139us >   sp_delay         12us         16us          7us          6us >   backlog            0b           0b           0b           0b >   pkts            29132    682090182    155330794      3836716 >   bytes        25452377 850788764602 230875228776   2357107850 >   way_inds            3     37939194     21198287        71120 >   way_miss         9448      6525310        97141        51428 >   way_cols            0          107            0            0 >   sce                 0            0            0            0 >   marks               0       206496          203           30 >   drops              30      1534636        80809          741 >   ack_drop            0            0            0            0 >   sp_flows            0            2            0            0 >   bk_flows            0            1            0            0 >   un_flows            0            0            0            0 >   max_len          1484         1514         1514         1514 >   quantum           300         1220          610          305 > >