From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 566C33B2A4 for ; Mon, 25 Jan 2021 04:31:36 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611567095; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9lWp9dGz8vPUbO7BII+fJoCKfLt4JSG4/DypCVo0lEw=; b=YTSqjC9vffIyulVcAjZA5cYET82HeET6VAxL2qwAhn+1rN9KZ1PDk3fQexuApx9DvnUYso t+bf9k7V1offxYaIGl9mxKUjIs2T3kA28eSaZ1onVCQuvIyLPKfgb4J5q2lE9RKfey8WD1 15c2ga4o+1Nzc8dyYugr58lrpeP11QI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-10-jletTz2wPXiQVXiF04iZ5g-1; Mon, 25 Jan 2021 04:31:30 -0500 X-MC-Unique: jletTz2wPXiQVXiF04iZ5g-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 38645CC62B; Mon, 25 Jan 2021 09:31:29 +0000 (UTC) Received: from carbon (unknown [10.36.110.4]) by smtp.corp.redhat.com (Postfix) with ESMTP id 976A55F9B0; Mon, 25 Jan 2021 09:31:27 +0000 (UTC) Date: Mon, 25 Jan 2021 10:31:24 +0100 From: Jesper Dangaard Brouer To: Toke =?UTF-8?B?SMO4aWxhbmQtSsO4cmdlbnNlbg==?= via Bloat Cc: Toke =?UTF-8?B?SMO4aWxhbmQtSsO4cmdlbnNlbg==?= , Hal Murray Message-ID: <20210125103124.04943977@carbon> In-Reply-To: <87zh104hkk.fsf@toke.dk> References: <20210122233710.1B911406061@ip-64-139-1-69.sjc.megapath.net> <87zh104hkk.fsf@toke.dk> Organization: Red Hat Inc. MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jbrouer@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] Measuring CoDel X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2021 09:31:36 -0000 On Sat, 23 Jan 2021 01:34:19 +0100 Toke H=C3=B8iland-J=C3=B8rgensen via Bloat wr= ote: > Hal Murray writes: >=20 > > Toke said: =20 > >> Yeah, the overhead of CoDel itself (and even FQ-CoDel) is basically ni= l (as > >> in, we have not been able to measure it), when otherwise doing forward= ing > >> using the regular Linux stack. =20 > > > > I may be able to help with that. > > > > Are you familiar with Dick Sites' KUtrace? > > Stanford Seminar - KUtrace 2020 > > https://www.youtube.com/watch?v=3D2HE7tSZGna0 =20 >=20 > Nope - but from a quick glance it looks similar to what you can do with > 'perf'? :) Yes, but the 'perf' tool (mostly) uses sampling. I assume you want to catch latency outliers, right? I would probably recommend that you play with bpftrace[1], for processing all the events to catch the outliers. [1] https://github.com/iovisor/bpftrace As an example look at this bpftrace script[2], that I used for detecting latency issues, network hardIRQ-to-softirq latency. [2] https://github.com/xdp-project/xdp-project/blob/master/areas/latency/s= oftirq_net_latency.bt --=20 Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer