From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot0-x230.google.com (mail-ot0-x230.google.com [IPv6:2607:f8b0:4003:c0f::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id EA90B3B2A4 for ; Thu, 26 Jan 2017 13:38:22 -0500 (EST) Received: by mail-ot0-x230.google.com with SMTP id 73so181546347otj.0 for ; Thu, 26 Jan 2017 10:38:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=T+jHmF0ONln9ubvQRCHcvUTh6AtfbSp4HAqsl60CBts=; b=Ru8bV6MUDqV9ecqWbzGg5b6EQZfqPaCCQklP9nm6qXPzRdVCrUm/tXrKNXXHhROhaL XaYBQ5B676q28Gr+Td/SH4DOitAYWE1alFJW/LUl/amC1buK6aKuRbhweYONW4YKnl+7 zIkt/GQP/O3Y7+ewwza2QakPtvc1x1TMlzTdF8Poef0l8ruiKsul4CooMXXl0PZMwn0O asmYJa6HZvirzVJfSAkwwZa/448v+vxnnDertGsEMS8abbTjCFeYLbISxRjxWZAaoKts yKw9XUmfg92tFfxHLcRz/hnWvz/9a9lbgmdwF45PrBtxb/dt+iicTMjx1Ew8MZkmhfn3 sDOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=T+jHmF0ONln9ubvQRCHcvUTh6AtfbSp4HAqsl60CBts=; b=amnHvAtJdLMHkCh7L0vcNP5zQPk1fyAiSPwY5jROJrG9q3cSyJCEyKg0s14Jul7NQ0 UH1gsmlrmt7gm75sFF8h7sYsCD6ClOdU1qQFJGPujM5PGv0gu/O2oRNDHtVjC/WZnvjc 6gYKxbtanlwPlA1Yzawx8t4zY6136Vq+vfYHFympQT+r/223fTaHMmRczcHLIlNCvgRt AS1L5VjsxvEYBigeR/zJ7by5SR1DvXqNKnWavxFeVdOTbdh4wim9Ljhx5V0sb8m/6LSQ R4n8tpA0GdAWcRI+PYpI18FRMqozfG7NHMpbXYjv5T1wOZm1ySccBGW45OU0dJN6EJoy gunQ== X-Gm-Message-State: AIkVDXKxoT6PtSN+1AwUDhHsxYfDL+xwV1o20dG5z69GvG4SzhheE3OvVZKFFNb5mARkMJgjwRI3+jrVl7+0Zw== X-Received: by 10.157.4.1 with SMTP id 1mr1721517otc.205.1485455902238; Thu, 26 Jan 2017 10:38:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.1.21 with HTTP; Thu, 26 Jan 2017 10:38:21 -0800 (PST) In-Reply-To: References: From: Hans-Kristian Bakke Date: Thu, 26 Jan 2017 19:38:21 +0100 Message-ID: To: bloat Content-Type: multipart/alternative; boundary=001a113b136c61efa8054703a7f6 Subject: Re: [Bloat] Excessive throttling with fq X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2017 18:38:23 -0000 --001a113b136c61efa8054703a7f6 Content-Type: text/plain; charset=UTF-8 I can add that this is without BBR, just plain old kernel 4.8 cubic. On 26 January 2017 at 19:36, Hans-Kristian Bakke wrote: > Another day, another fq issue (or user error). > > I try to do the seeminlig simple task of downloading a single large file > over local gigabit LAN from a physical server running kernel 4.8 and > sch_fq on intel server NICs. > > For some reason it wouldn't go past around 25 MB/s. After having replaced > SSL with no SSL, replaced apache with nginx and verified that there is > plenty of bandwith available between my client and the server I tried to > change qdisc from fq to pfifo_fast. It instantly shot up to around the > expected 85-90 MB/s. The same happened with fq_codel in place of fq. > > I then checked the statistics for fq and the throttled counter is > increasing massively every second (eth0 and eth1 is LACPed using Linux > bonding so both is seen here): > > qdisc fq 8007: root refcnt 2 limit 10000p flow_limit 100p buckets 1024 > orphan_mask 1023 quantum 3028 initial_quantum 15140 refill_delay 40.0ms > Sent 787131797 bytes 520082 pkt (dropped 15, overlimits 0 requeues 0) > backlog 98410b 65p requeues 0 > 15 flows (14 inactive, 1 throttled) > 0 gc, 2 highprio, 259920 throttled, 15 flows_plimit > qdisc fq 8008: root refcnt 2 limit 10000p flow_limit 100p buckets 1024 > orphan_mask 1023 quantum 3028 initial_quantum 15140 refill_delay 40.0ms > Sent 2533167 bytes 6731 pkt (dropped 0, overlimits 0 requeues 0) > backlog 0b 0p requeues 0 > 24 flows (24 inactive, 0 throttled) > 0 gc, 2 highprio, 397 throttled > > Do you have any suggestions? > > Regards, > Hans-Kristian > --001a113b136c61efa8054703a7f6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I can add that this is without BBR, just plain old kernel 4.8 c= ubic.

= On 26 January 2017 at 19:36, Hans-Kristian Bakke <hkbakke@gmail.com>= ; wrote:
Another day, = another fq issue (or user error).

I try to do the seeminlig simple t= ask of downloading a single large file over local gigabit =C2=A0LAN from a = physical server running kernel 4.8 and sch_fq on intel server NICs.

Fo= r some reason it wouldn't go past around 25 MB/s. After having replaced= SSL with no SSL, replaced apache with nginx and verified that there is ple= nty of bandwith available between my client and the server I tried to chang= e qdisc from fq to pfifo_fast. It instantly shot up to around the expected = 85-90 MB/s. The same happened with fq_codel in place of fq.

I then che= cked the statistics for fq and the throttled counter is increasing massivel= y every second (eth0 and eth1 is LACPed using Linux bonding so both is seen= here):

qdisc fq 8007: root refcnt 2 limi= t 10000p flow_limit 100p buckets 1024 orphan_mask 1023 quantum 3028 initial= _quantum 15140 refill_delay 40.0ms
=C2=A0= Sent 787131797 bytes 520082 pkt (dropped 15, overlimits 0 requeues 0)
=
=C2=A0backlog 98410b 65p requeues 0
=C2=A0 15 flows (14 inactive, 1 throttled)
=C2=A0 0 gc, 2 highprio, 259920 throttled, 15 flo= ws_plimit
qdisc fq 8008: root refcnt 2 li= mit 10000p flow_limit 100p buckets 1024 orphan_mask 1023 quantum 3028 initi= al_quantum 15140 refill_delay 40.0ms
=C2= =A0Sent 2533167 bytes 6731 pkt (dropped 0, overlimits 0 requeues 0)
=C2=A0backlog 0b 0p requeues 0
=C2=A0 24 flows (24 inactive, 0 throttled)
=C2=A0 0 gc, 2 highprio, 397 throttled

<= /div>
Do you have any suggestions?

Regards,
Hans-Kristian

--001a113b136c61efa8054703a7f6--