From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-x244.google.com (mail-lf0-x244.google.com [IPv6:2a00:1450:4010:c07::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 035523B2A4 for ; Sun, 15 Jul 2018 04:10:58 -0400 (EDT) Received: by mail-lf0-x244.google.com with SMTP id g6-v6so19407185lfb.11 for ; Sun, 15 Jul 2018 01:10:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3XV/rxSJQBPlcatrYjrFYTWvcARwSzdxd0qBOfD+0Fw=; b=C+kF8lCfLCpZXIQu4ISe5szY/TAUjFwdgmJR3YctY5f37DIYqPiRT9ONw0QvNtA7BP DQNtcCNF0UBQ2iNg7eHArFvqVXZac9j0bqImGzDE9ahOoEt6HJRjd+9wKhMLe1mdWSTj WstZSsk4d5940mDWjbaeWacWIB2kOd5fiGbFTcgN+xM8AhlI6Im2LhNDaQvdQDA0lq6v ZXdXXY+Ylg4yrByvbbxEFN2/lEgqnlPPWNfGx3hIbTLmRmcKs06xq9BH9yIKv84Qqoj8 atQx+eLNKcuJzFqS78tK4FbeI+57rXsKSfHwc1Z1QORf2rl2AQ0y+OCpcUO5NnuCQBZ9 oRwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=3XV/rxSJQBPlcatrYjrFYTWvcARwSzdxd0qBOfD+0Fw=; b=Plu1KU8dS39sIdSrFf5+DtfKI7xET11TtqaFcyfd/Jc72LKqjrMFQxW4cCmIorf5DJ Nwo0f3KhyE4TL+KxYOeVFkUezUliRnJ8k9V8sCwji/AU+kgaHaODgsiNaxa0MF/Mm1HZ gIeEeojqzJ1/aMcb0XQdh5YDmFpmlbdz/rhpIuZsrlRDq5+ZgW46XppM8cto1WP+Aw5J 8tHXne8XKLhCp9SXTx4ZDAv+XdreRqQQ+e/gtznSJWyvgn5fUnKjUlEKQkqrRDgnqjR6 /SfKT03cWV1OhoYcAkXWxi5bRgS1YoxQ3NKVF/gx9NR3++PMI2N4F+cNIBRydJTla6mF VrWg== X-Gm-Message-State: AOUpUlFO6OsDws8sVC06EZlRQbuNKlrsQvd+XIRsg2YKOkL5CzN/PXms f9s6ee6+Zb1o1AtJjzlOYS8= X-Google-Smtp-Source: AAOMgpeyX1H3PnAl8PYnbvcr2+2yh299D8FT6xh44XD1ofKTF2AlQJzcCcEX2c8MF4rjtLwLiNe0ew== X-Received: by 2002:a19:204f:: with SMTP id g76-v6mr8514114lfg.66.1531642257911; Sun, 15 Jul 2018 01:10:57 -0700 (PDT) Received: from [192.168.239.216] (83-245-233-27-nat-p.elisa-mobile.fi. [83.245.233.27]) by smtp.gmail.com with ESMTPSA id x11-v6sm5245209lfi.8.2018.07.15.01.10.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jul 2018 01:10:57 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.4 \(3445.8.2\)) From: Jonathan Morton In-Reply-To: Date: Sun, 15 Jul 2018 11:10:55 +0300 Cc: Cake@lists.bufferbloat.net Content-Transfer-Encoding: quoted-printable Message-Id: <3E2FE0BD-3A6C-4399-90FB-1334A7A0D962@gmail.com> References: To: dag dg X-Mailer: Apple Mail (2.3445.8.2) Subject: Re: [Cake] Multiple Hardware Queues X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jul 2018 08:10:59 -0000 > On 15 Jul, 2018, at 10:41 am, dag dg wrote: >=20 > In my box I have acting as a router I have an Intel i350-t2v2 nic that > has two gigabit ports(uplink/local). This card and its corresponding > driver supports multiple hardware-based transmit and receive queues > depending on the number of cores the system has up to 8. > qdisc cake 802c: dev enp2s0f0 root refcnt 9 bandwidth 23Mbit diffserv3 = triple-isolate split-gso rtt 100.0ms raw overhead 0 > qdisc cake 802d: dev ifb4enp2s0f0 root refcnt 2 bandwidth 330Mbit = besteffort triple-isolate wash split-gso rtt 100.0ms raw overhead 0 > Let me be clear that with cake and sqm I am seeing great results on > the dslreports speed test(A+) so this inquiry is less about solving a > problem and more along the lines of trying to take full advantage of > my available hardware. Any insight would be appreciated, and thanks > again for your contributions. At these bandwidths, you are not stressing your hardware at all - and I = don't even have to ask what CPU you have to know this. The NIC's = multiple queues would give you no benefit whatsoever. An Intel Atom or = an AMD E-450 can easily handle gigabit traffic through Cake, as long as = the NIC is attached via a bus capable of carrying that much bandwidth = (eg. PCIe). These are some of the least powerful 64-bit x86 CPUs that = ever reached the market. In any case, the MQ qdisc simply sorts packets into hardware queues = according to the CPU they were submitted from. This is useful for = something like a heavily loaded webserver, which has many worker = processes distributed evenly across all available CPUs, since it avoids = either passing data to a NIC-worker process on a fixed CPU, or = contending for a single NIC lock. But it's basically useless on a = desktop PC, or on a machine acting primarily as a router, since the = traffic is submitted from just one or two CPUs at a time, and usually = most of the CPUs are idle anyway. I have no idea what the hardware does = to coalesce those packets into a single stream to be sent over the wire. You can verify this for yourself by looking at your CPU load while = running a full bidirectional speed test. On any recent CPU, expect to = see the most loaded core with at least 90% idle - unless your web = browser happens to be occupying it. - Jonathan Morton