From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-x236.google.com (mail-qt0-x236.google.com [IPv6:2607:f8b0:400d:c0d::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AF7863BA8E for ; Sat, 8 Sep 2018 11:53:42 -0400 (EDT) Received: by mail-qt0-x236.google.com with SMTP id o15-v6so19421442qtk.6 for ; Sat, 08 Sep 2018 08:53:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=Bi+MF2mmx+Iatc+9Gj65annFMCTjUGTUZyb6moTXV/8=; b=nQtFqGm3z6osOsyrZgDcSUZ4hKFa+YOQIm7l0IaLGS1mem+hRujZfF2ybxZOSAO5ep h7i9P3cz/2ejSUlkznMyXaBi6PlMbAp7gfh517owlIYDo+yZWTfyuEhrrWwYzRyJWR4F QUoyDmebQPvNo8Xu57pPztvItGSkMip5SE71L5y3jp3VLQ20vN02hAQRcypN1gC0c0jT PkMV4PaFDttmr+ZKlAtfRS5zXC481H4m2snQfgf04ZmZcy3WwgHB7VtqNhDEZBfvzITn qeEECB+Z+g3nrCXVTIwyWRyFjExPfDZY1hM6PMgqfRjVaGenxL80gSVUMPdOc0mlHZJY 4v+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=Bi+MF2mmx+Iatc+9Gj65annFMCTjUGTUZyb6moTXV/8=; b=uQ5oBdblI+O9Erc+D+vjNWB/bEWu/LCpPUtu2Q025JDczfwkV7JzcEAk9wAoSDkY66 xhbO9ZSYNU6bwr1FwEIPhH2z0aTJaZn73Uw2BCkELh0zLhlekf3aMfD+4casWA8JecD3 JEPXiF+hQZ2CQg1agU/2ju4+QUSlZBPvZoLuy+oNiPHX2uAOW890tmgDVY9Cfy9cvAme 8M+Br3jTXA/yJgO3Fz7kbN+zCIzWwva19quYkq/t+Z6wGCXFE6ClV+CdBr5sNQwuuiGZ xxyAXWuY4kL9r6cgYNBoEQd4/9Wst7+pSiP33im4GSpGCjUrCQlbI3TyqXMGb5OzTJQo bvqg== X-Gm-Message-State: APzg51CNUq2yT8LBIY8IxJIyGPlxszDFKyT6qW2+QHk9Sbnq7feuK2/i iM65ejGq/mi7eYv7CzF0DFORRjVvIkNm+oJolD4= X-Google-Smtp-Source: ANB0VdZMNKfuOB4UmGVjIQLx7A38swYVMD0X6Yrj/7VlaYlE3+tM89GLledlpSX6RJrBujd33wJ7YEL+ytparipcC9M= X-Received: by 2002:ac8:2db3:: with SMTP id p48-v6mr10337950qta.238.1536422022133; Sat, 08 Sep 2018 08:53:42 -0700 (PDT) MIME-Version: 1.0 References: <521EB406-0F8B-4043-A6ED-4C38B0B2B9D8@gmx.de> In-Reply-To: From: Dave Taht Date: Sat, 8 Sep 2018 08:54:36 -0700 Message-ID: To: Sebastian Moeller Cc: bloat Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Bloat] fq_codel on macosx X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2018 15:53:42 -0000 [ flow control: 86 feedback: 86 stalls: 4 failed: 0 ] [ drop overflow: 3116 early: 854 memfail: 0 duprexmt:0 ] Not sure what this means. I think they are using fairly short queues (which is fine given this is a wifi box), so they are taking advantage of the fq, but tail dropping a lot. "Early" would be my guess for codel itself engaging. As for flow control? ecn? Dunno. I've not looked at direct packet captures much on osx. Now I shall. I note I recently added something to increase the signal strength in fq_codel's bulk dropper, based on my observation that with openwrt's shorter queues (1000 packets) it still tail dropped a lot, and you could clearly see a bifurcation in flow behaviors when that started happening. https://github.com/dtaht/fq_codel_fast On Sat, Sep 8, 2018 at 8:24 AM Dave Taht wrote: > > On Sat, Sep 8, 2018 at 8:12 AM Sebastian Moeller wrote: > > > > Hi Dave, > > > > Jason Woringen just told me that macosx seems to include fq_codel now a= s well as some involved priority queues that I can not make much sense of. = It might simply have inherited this from its freebsd underpinnings... > > Wow. I had no idea. I know that they enabled tcp ecn universally last > year... but not fq_codel!! Holy cow! > > Same here on my air.... > > daves-Air-3:wiki d$ netstat -I en0 -qq > > en0: > > [ sched: FQ_CODEL qlength: 0/128 ] > > [ pkts: 0 bytes: 0 dropped pkts: 13262 > bytes: 8469105 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > > [ pri: VO (1) srv_cl: 0x400180 quantum: 600 drr_max: 8 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 380295 bytes: 47356766 ] > > [ budget: 0 target qdelay: 10.00 msec update interval:100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > > [ pri: VI (2) srv_cl: 0x380100 quantum: 3000 drr_max: 6 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 4339 bytes: 321840 ] > > [ budget: 0 target qdelay: 10.00 msec update interval:100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > > [ pri: BE (7) srv_cl: 0x0 quantum: 1500 drr_max: 4 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 40497473 bytes: 11269032336 ] > > [ budget: 0 target qdelay: 10.00 msec update interval:100.00 msec ] > > [ flow control: 86 feedback: 86 stalls: 4 failed: 0 ] > > [ drop overflow: 3116 early: 854 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > > [ pri: BK (8) srv_cl: 0x100080 quantum: 1500 drr_max: 2 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 1553687 bytes: 358969659 ] > > [ budget: 0 target qdelay: 10.00 msec update interval:100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > > > > en4 is 1Gbps ethernet > > macbook:~ user$ netstat -I en4 -qq > > en4: > > [ sched: FQ_CODEL qlength: 0/128 ] > > [ pkts: 100 bytes: 28920 dropped pkts: 1 bytes:= 342 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: CTL (0) srv_cl: 0x480190 quantum: 600 drr_max= : 8 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 91 bytes: 27706 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: VO (1) srv_cl: 0x400180 quantum: 600 drr_max= : 8 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 0 bytes: 0 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: VI (2) srv_cl: 0x380100 quantum: 3000 drr_max= : 6 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 0 bytes: 0 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: RV (3) srv_cl: 0x300110 quantum: 3000 drr_max= : 6 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 0 bytes: 0 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: AV (4) srv_cl: 0x280120 quantum: 3000 drr_max= : 6 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 0 bytes: 0 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: OAM (5) srv_cl: 0x200020 quantum: 1500 drr_max= : 4 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 0 bytes: 0 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: RD (6) srv_cl: 0x180010 quantum: 1500 drr_max= : 4 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 0 bytes: 0 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: BE (7) srv_cl: 0x0 quantum: 1500 drr_max: 4 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 9 bytes: 1214 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: BK (8) srv_cl: 0x100080 quantum: 1500 drr_max= : 2 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 0 bytes: 0 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: BK_SYS (9) srv_cl: 0x80090 quantum: 1500 drr_max: 2 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 0 bytes: 0 ] > > [ budget: 0 target qdelay: 5.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > macbook:~ user$ > > > > > > and on wifi (en0) > > macbook:~ user $ netstat -I en0 -qq > > en0: > > [ sched: FQ_CODEL qlength: 0/128 ] > > [ pkts: 0 bytes: 0 dropped pkts: 480 bytes:= 77809 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: VO (1) srv_cl: 0x400180 quantum: 600 drr_max= : 8 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 72537 bytes: 7015758 ] > > [ budget: 0 target qdelay: 10.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: VI (2) srv_cl: 0x380100 quantum: 3000 drr_max= : 6 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 1139670 bytes: 80872250 ] > > [ budget: 0 target qdelay: 10.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: BE (7) srv_cl: 0x0 quantum: 1500 drr_max: 4 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 6737285 bytes: 1608200668 ] > > [ budget: 0 target qdelay: 10.00 msec update interval= :100.00 msec ] > > [ flow control: 1 feedback: 1 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > [ pri: BK (8) srv_cl: 0x100080 quantum: 1500 drr_max= : 2 ] > > [ queued pkts: 0 bytes: 0 ] > > [ dequeued pkts: 1396402 bytes: 382861004 ] > > [ budget: 0 target qdelay: 10.00 msec update interval= :100.00 msec ] > > [ flow control: 0 feedback: 0 stalls: 0 failed: 0 ] > > [ drop overflow: 0 early: 0 memfail: 0 duprexmt:0 ] > > [ flows total: 0 new: 0 old: 0 ] > > [ throttle on: 0 off: 0 drop: 0 ] > > macbook:~ user $ > > > > > > > > > > > > > > > -- > > Dave T=C3=A4ht > CEO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-669-226-2619 --=20 Dave T=C3=A4ht CEO, TekLibre, LLC http://www.teklibre.com Tel: 1-669-226-2619