From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x234.google.com (mail-qt0-x234.google.com [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 4BC0B3B2A4; Wed, 1 Aug 2018 15:48:37 -0400 (EDT) Received: by mail-qt0-x234.google.com with SMTP id t5-v6so21288366qtn.3; Wed, 01 Aug 2018 12:48:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=mnQmNEDzSqf02Vj64wFxPyPoP7FuFOhmG27Uoc/u7VU=; b=kxmWuIoLEWshIB1lLwJF6tlxIJQhzeJjDQF3dbgGYY2QAY3yZSARvJznuuet1PAast Hj6JFeBLq8DpgmC+uxBaZZtwqVA2swqmMpkbfP/HWF6mXpyt2W7o2ZkoSIrh5zwHjjNm mV8PokA+Mj1cG5JaYQEDjfIUYAb7CUChucSfAhOMNbn13m/bcu1WDcLKQ3+6sPfmr6jO x2v4KSibKCI4CQo0RcGUtMULLcn2vcCqzO70hS5d3Ki2ud6seFKMh0SbD0ObojPaauMb zbLpw7f12EKcacwR1SyUkgWofjl/jk7shUJLsCJVzMLiw9Z4bmPE7vPguFwXjbruYXuh J34A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc :content-transfer-encoding; bh=mnQmNEDzSqf02Vj64wFxPyPoP7FuFOhmG27Uoc/u7VU=; b=D1AGjWNG4XwhxxiUHTG1Mx9xZxKWB5ne/uYOFmP3iQ01wYlCOE+y7kPh++yUvLnFG6 BZpIcd09dUGH0xaIOoH4q/856vhnHKg8Pl1RtBl5bPg5J3VmLNKtKkVuDTqMlzM7oYS5 br0NO2NRCQ8CKjjmW+kIIc5jrxU6QmX6IFrwK44xJWnQHssOWEPoRCAKfN1orxs5EVik Fz7eHd1YCNadDribee0iYp6pMXyYvDm44zVPg5E1G8mT42yLUe165Sn7hxSRFOq80nj+ mM8GhX4NRnO4MwiDoui4hKODx9GhDYhYA7+/aB0Q/biFN0Tzxa2vwawRBarsSSJpUgf8 hKfA== X-Gm-Message-State: AOUpUlEa0RknVTlpKun5WgJU7O/T8cUpkB5D1c4CuaWN7QTP7dXnwitI i+pzr0SqV3Io8hFjD5nVi9E/35EWa5yOsvA09QPyB/i/ X-Google-Smtp-Source: AAOMgpdz24dxIwptzhhZq3Y4vqcF5QnR1UOK4DghBvWtKO1aq9VuVwpOrCyv9nBD6t197pdJjUKxTgb6I/1J2j4zxdY= X-Received: by 2002:aed:2686:: with SMTP id q6-v6mr26622746qtd.199.1533152916437; Wed, 01 Aug 2018 12:48:36 -0700 (PDT) MIME-Version: 1.0 From: Dave Taht Date: Wed, 1 Aug 2018 12:48:58 -0700 Message-ID: To: cerowrt-devel@lists.bufferbloat.net, Cake List , codel@lists.bufferbloat.net Cc: Jakub Kicinski Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: [Cerowrt-devel] fq_codel on netronome's NICs? X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 19:48:37 -0000 Being kind of inspired by all the tricks https://homes.cs.washington.edu/~arvind/papers/afq.pdf used on the cavium, I went looking for other smart nics to play with. https://open-nfp.org/resources/ looked interesting so I pinged them... from netronome: "I think it would be feasible to implement fq_codel on the NFP. The hardware schedulers do not support fq_codel, so the schedulers would have to be implemented in one of the NFP firmware languages (e.g. micro-C or micro-code); the NFP hardware rings could be used for the queueing mechanism. Practically, this may be one way of making it work: The main worker threads could calculate the flow hash in order to select which ring should be used, and then issue the packet to a re-ordering thread. I believe the re-ordering thread can push the packets to the internal NFP rings instead of the wire. The scheduler thread could then make the scheduling decision, pop the packet from the corresponding ring, then send the packet to the hardware packet schedulers (or drop the packet if performing a head-drop), and also check the timestamp for the CoDel portion of the algorithm. The hardware packet schedulers should then transmit the packet. In terms of handling any rate-mismatch on the outgoing interface, you could have another thread monitor the NFP hardware packet scheduler queue levels. The scheduler thread can then throttle the packet rate being sent to the hardware packet schedulers (unless of course it is okay to tail-drop at the hardware packet scheduler queues). Finally, if the outgoing interface is not the natural point of congestion/rate mis-match (e.g. if the outgoing Ethernet interface is attached to a cable/DLS modem), the NFP hardware does have some support for rate-limiting the outgoing interface (e.g. limiting a 10 Gigabit Ethernet interface down to 600 Mbps outbound), so as to move the congestion/rate mis-match point to the NFP, so that fq_codel can take effect in terms of handling the buffer bloat." --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619