From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 76DCC3B2A4 for ; Tue, 15 Nov 2022 09:08:41 -0500 (EST) Received: by mail-pg1-x52d.google.com with SMTP id 62so5008807pgb.13 for ; Tue, 15 Nov 2022 06:08:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=iMrhPs/s2iv/8SLxSqmnOIkd+3EVdGhy6neEfc1wP/Y=; b=HljUDUOqgxpBw1VQeg45R60mWBPsB44E0MB0+UlQR9ne8AAi/R/GlDGu26PiMbjQZZ 0IVyIoV1gti2rUdEHo5CpDq+nyLjijYiV+KGYO1uwKYkXLXzmNqmBvm5VzJbQK8sicdQ 7czimSHdTNII+TUP4rf6MR+FK2QlU3t+euKnPijZdE/0PHgsQzGoq3rpxnDYAAGfoTQG iXqS9BsAGrPI0AVM96ShTxF5bc4RFVvJYYMyaqrbCt5xTjYzR/ZSlMyFLarF2lWuye0y k+VTrKosXwx33xxXN3cCZII54jFnPEB6vqwm8hn73whDdswNy7DW+GPq3dyjwOLUPpQ6 gyOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=iMrhPs/s2iv/8SLxSqmnOIkd+3EVdGhy6neEfc1wP/Y=; b=B6IpO+fzbEcDUpqpshLc1hFxLXehzBZCC6x5gz2FrMWOsTlHKF2qM7fmbnAoabsb3E e9LIKMyQd4v6Nt4jBcKzz9XeGtPQibXTsF429r7RIIl2UJHoVOvdWODzFtIyK1Gf71jM EVrIdRWlCDOUZARy8Q7HPV1ofIwdqkBFNQOcoLgU6+U2OhRXm4XRck+AUULT2WnhvAKk y0TVddwR6DDux7IbGUq8cjjrTWWRXTiPpciVVNY89NdTmBeGXt3tdEfaAiEsJ0p2xUz/ RhHvn6Sqv3nUO3rUova3kr+sOsRlu7Bhwee/kK6Y1/NFI0ajqwi0IHtp6jv40eW79jcB 6Y5Q== X-Gm-Message-State: ANoB5pk0v2H5jvDHc9249DHf5lktjwVNDEsLMvK0eTEQvYIz0ZIS3KO8 8c68czlhgUHQpLAOmvqL0659sXEmdktXIZYnoeVUiyryDXBKWw== X-Google-Smtp-Source: AA0mqf5ZYQmXGpgADDzIaFHEVnCB3QgO/bB26NCOnVY7cAPBBOtRu/MRk9/o6zKvbQBs/QzYqhBxwjNO52oz24zEsCE= X-Received: by 2002:a05:6a00:440b:b0:56d:9eed:61eb with SMTP id br11-20020a056a00440b00b0056d9eed61ebmr18532623pfb.4.1668521319979; Tue, 15 Nov 2022 06:08:39 -0800 (PST) MIME-Version: 1.0 From: Herbert Wolverson Date: Tue, 15 Nov 2022 08:08:29 -0600 Message-ID: To: libreqos Content-Type: multipart/alternative; boundary="00000000000093de5405ed82e400" Subject: [LibreQoS] cpumap-pping 1.0 final X-BeenThere: libreqos@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Many ISPs need the kinds of quality shaping cake can do List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Nov 2022 14:08:41 -0000 --00000000000093de5405ed82e400 Content-Type: text/plain; charset="UTF-8" Hey! Just wanted to announce that cpumap-pping is now tagged as the 1.0 release. The one major change from RC1 is that the xdp_pping utility no longer does any cleaning - it just reports statistics. That makes its execution a LOT faster, and removes a stalling issue that was showing up under heavy (5 gbit/s+) load. Under the hood, each "report flow" now carries a "recycle time" and a "fresh data" flag. They are also stored in an "LRU map" rather than a strict hash map - so if the buffer fills up, it automatically (in kernel space) evicts (replaces) the oldest member rather than pausing to report the event. This in turn allows for all memory allocation to be up-front (on load), so there are *no* memory allocations or reallocations within the running RTT tracking system. Once a "report flow" has achieved its target number of samples, it stops collecting data. 30 seconds (by default, it's configurable) later, the flow is "recycled" - its buffer is empties, "fresh data" is set to false, and the recycle time bumped to "now + 30 seconds". If data arrives, "fresh data" is set to true - and the reporting tool will show the new data. If no data arrives, the flow ages slowly to the bottom of the list and is replaced with no memory allocation/deallocation/reallocation. This has largely eliminated the occasional variance in execution time (it never spiked above 6000 nanoseconds to begin with; but now it's consistently under 0.0005 ms added ping time; you won't notice it). More importantly, it completely eradicated the high-load behavior of the xdp_pping sitting and spinning for long periods of times under high lock contention. -- Herbert --00000000000093de5405ed82e400 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hey!

Just wanted to announce= that cpumap-pping is now tagged as the 1.0 release.

The one major change from RC1 is that the xdp_pping utility no longer do= es any cleaning - it just reports statistics. That makes its execution a LO= T faster, and removes a stalling issue that was showing up under heavy (5 g= bit/s+) load.

Under the hood, each "report fl= ow" now carries a "recycle time" and a "fresh data"= ; flag. They are also stored in an "LRU map" rather than a strict= hash map - so if the buffer fills up, it automatically (in kernel space) e= victs (replaces) the oldest member rather than pausing to report the event.= This in turn allows for all memory allocation to be up-front (on load), so= there are no memory allocations or reallocations within the running= RTT tracking system.

Once a "report flow&quo= t; has achieved its target number of samples, it stops collecting data. 30 = seconds (by default, it's configurable) later, the flow is "recycl= ed" - its buffer is empties, "fresh data" is set to false, a= nd the recycle time bumped to "now + 30 seconds". If data arrives= , "fresh data" is set to true - and the reporting tool will show = the new data. If no data arrives, the flow ages slowly to the bottom of the= list and is replaced with no memory allocation/deallocation/reallocation.<= /div>

This has largely eliminated the occasional varianc= e in execution time (it never spiked above 6000 nanoseconds to begin with; = but now it's consistently under 0.0005 ms added ping time; you won'= t notice it). More importantly, it completely eradicated the high-load beha= vior of the xdp_pping sitting and spinning for long periods of times under = high lock contention.

-- Herbert
--00000000000093de5405ed82e400--