From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x235.google.com (mail-qt0-x235.google.com [IPv6:2607:f8b0:400d:c0d::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 89F683BA8E for ; Sat, 28 Jul 2018 12:51:33 -0400 (EDT) Received: by mail-qt0-x235.google.com with SMTP id t5-v6so8211277qtn.3 for ; Sat, 28 Jul 2018 09:51:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=SctfErhVopgFQxV4bD9cIh45dgB7y5fKELPETsWo6Is=; b=EZQYFvBGuLgC8zVnKEpXRycveb/XBruQwdOo40sKuPdqBRVTr7ZvdE8+IUi1YvSiEY IK8npcL/VboQ1MTsH5Y/axhrwZ+QcWMzis/LOlgXLv1QJURzXH1ffaqaAgMonqN+pTad ON10jAQbz6lCxyS5FHRiqldYu9rc+1oa/HvpBcatdusMHyAKPSLwlPtQP4BRRK3njnlB pGIa5B7SdclfaanKWea3Tl+AXm0Zvtkhp/bgo5xsm/CDqmtcbDTvtaaSmmGnhbQvMukW zq09jYao9mIhjrTJCWzy6+uEiqlKSOgvK7k8mIgJJ9xUDjygUaCau2nWIdPHPOjacTsK u2DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=SctfErhVopgFQxV4bD9cIh45dgB7y5fKELPETsWo6Is=; b=fLAwluvIhyDGL3iGdHNLmk30dDsUdB4Tq1+XQFn5iR7NEJYey8SQCU2DChgdLcV5T5 fw2nkd1ZbjBAX0x/ICbV0HUcptdIbmlxLlszXwDdcE7+Q+gw6JGDgnmAZNSdhJpxc9br m2EsY8W2LTSBKRQYPZt4DPdmUQUW0RwzwaA0ipa8hCCKvuwjLBYq6Rm1iimEWqFrUnQG nM0QOrQqEZjlRV1BvxZbW+hV6PmPnIxBVzPNBmpABWonbZd0fCDVNTVrAlU+cN7N9aHo CkATRU+dWeQGuJ2Fcf/HVfNkBVkG4YwmyycaU2PNbEw9GnSuzYkIK4hqG6JnMhKEOHbM vuBw== X-Gm-Message-State: AOUpUlECULaOkVY/s3VUoM0GMhInWAEdKpirRAJA7FvU7wRuhOj6aFo3 Gw1hoQHaU5dnlqi4c6kG8Bf6kST8T0/Toal5iYuUBQ== X-Google-Smtp-Source: AAOMgpeR/nN6pchtj5HrTrehz2+6PyIzwB1RNrhX1hdrEr4VXMHiOu/MitmIW2ryb0Y32y1LGZgvCpGP8Zndv8f8KzM= X-Received: by 2002:a0c:bfd8:: with SMTP id u24-v6mr9819530qvj.158.1532796692722; Sat, 28 Jul 2018 09:51:32 -0700 (PDT) MIME-Version: 1.0 From: Dave Taht Date: Sat, 28 Jul 2018 09:51:21 -0700 Message-ID: To: Cake List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: [Cake] cake, mq and bql on the apu2 X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Jul 2018 16:51:33 -0000 so, as I gradually built up my lab over the past few weeks I slammed ubuntu into one of my prized apu-2s to take a look at cake. This particular box has 4 hardware queues, so I made cake the default qdisc to see what happened at line rate. Running netserver locally on it (which is a dumb idea, I just wanted to see what happened) - It works at a gbit fine, but bql stablizes at about 60-100k per queue, (120-200 with gso enabled). I can set bql's limit_max to about 50k but start losing single stream performance below that (bql tends to be off by 2x in any scenario), for example, 20k for bql the best a single flow can do is 500mbit, so there is a lot more context switch overhead than I'd like compared to a fluid model. running cake as the sole qdisc, unshaped, bql stablizes a little higher, but throughput suffers (800k) also. In all cases observed latencies are much higher that desirable due to all the extra bql buffering, in 2-5ms range. If I shape to 100mbit, I get .36ms for ping on workloads from 1 to 16 streams. And also of course there's the usual birthday problem caused by so few queu= es. I'm going to go setup two boxes on the other side of it next. I've never liked bql's MIAD (rather than AIMD) scheme, I really hope that not running the test locally will reduce the automagically sized bql buffers smaller. Then I'll flash lede on another one and see what happens. Any ethtool settings for NAPI in particular worth trying? --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619