From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [IPv6:2607:f8b0:4864:20::82e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 1EC413CB36 for ; Sat, 6 Apr 2019 07:49:56 -0400 (EDT) Received: by mail-qt1-x82e.google.com with SMTP id w30so10387911qta.8 for ; Sat, 06 Apr 2019 04:49:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=LPtK0HKsriQ2KwuHi99bRdbUObWeeWckZ1r/mA9xsUw=; b=WqlMRZrr4EtiyLDW0VfbxB3TFPLfruSXPhFJ0tP+iUYJKFFdMlK2YPAu4+FIRhwY5K yV4W66Q1F2G5UCS2xTWP9loEL8J6u6eGeIKKddTdnn4FwXiXf+sF87e9JRABefMzGg05 FKPD51dL2m0zgMNgolWfv3xAImSU0syEqdjBUmpjYm/16oHgrt7wyKKqIWpVagA3vdt6 2pO0vEwUB2KrosdxArw6mJ/0+0J+NWc8RDVZS565HfndDMWjy5e1QOzAjDtKc9lps7zN Ii8NH4suWK9M1qLCP6MmOKIB1KStqqs/cBI/CL3X0RE6yWD9JdmEQtwMpzufb6Zfesg+ 2NYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LPtK0HKsriQ2KwuHi99bRdbUObWeeWckZ1r/mA9xsUw=; b=RiRWzDQNvCFfgls3b3ydjqOY3gMXgHPxhLLwSYqyF5xZ8iJooT+3ljBng/u/qiU5NN enSJqM9xcB4Hxo8d+YaX3wXAsn8OgnAA3RHNMjCNc3OSAszi4JxaqAiCFyhgFB9I3bZj 9ZgaEmnFO0kcxKd/kf42qmZ1Ci9soYLnXXr39GBcVjQVHo9gYOlsUxWeF94jC7+BZwUA ivKWp2/MIbfM7eKizxC9zRxJsHk/IIn0UukKRiQzU++tIfPB6D35k3f9FsOV4jubvp/i vaZFq7XU4FdUpf+cZs86jdcc+zrJxv/4PYnSAYKyPplX8WKvy3oslQqds6JA4ZgNSvDv SNCA== X-Gm-Message-State: APjAAAUQa5QXauxbaQOwYK+sveNyyhwRm55faa/XlHhMJLDtc3VYZA6g wikFTD6PjXGz/u33YR6D8++Idt90LDI5lYAbJmM= X-Google-Smtp-Source: APXvYqzT19rJAIkZ/6d3ru5XjFp2yx24q976k910iLUJlUnddekfbsJqSpnqk4ZJxM3n0J4KQ3QqkpV/fEJaDl8sz6Q= X-Received: by 2002:a0c:a322:: with SMTP id u31mr11592276qvu.207.1554551395591; Sat, 06 Apr 2019 04:49:55 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Sat, 6 Apr 2019 13:49:43 +0200 Message-ID: To: Neal Cardwell Cc: ECN-Sane , BBR Development , flent-users Content-Type: text/plain; charset="UTF-8" Subject: Re: [Ecn-sane] [bbr-dev] duplicating the BBRv2 tests at iccrg in flent? X-BeenThere: ecn-sane@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion of explicit congestion notification's impact on the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Apr 2019 11:49:56 -0000 > With fq_codel and the same ECN marking threshold (fq_codel ce_threshold 242us), we see slightly smoother fairness properties (not surprising) but with slightly higher latency. > > The basic summary: > > retransmits: 0 > flow throughput: [46.77 .. 51.48] > RTT samples at various percentiles: > % | RTT (ms) > ------+--------- > 0 1.009 > 50 1.334 > 60 1.416 > 70 1.493 > 80 1.569 > 90 1.655 > 95 1.725 > 99 1.902 > 99.9 2.328 > 100 6.414 I am still trying to decode the output of this tool. Perhaps you can check my math? At 0, I figure this is the bare minimum RTT latency measurement from the first flow started, basically a syn/syn ack pair, yielding 9us, of which gives a tiny baseline to/from the wire delay, the overhead of a tcp connection getting going, and the routing overheads (if you are using a route table?) of (last I looked) ~25ns per virtual hop (so *8) for ipv4 and something much larger if you are using ipv6. This is using ipv4? A perfect level of interleave of 20 flows, 2 large packets each, would yield a RTT measurement of around 532 extra us, but you are only seeing that at the 80th percentile... 100: The almost-worst case basic scenario is that 20 flows of 64k GRO/GSO bytes each are served in order. That's 13us * 42 * 20 = 10.920ms. It's potentially worse than that due to the laws of probability one flow could get scheduled more than once. 1) What is the qdisc on the server and client side? sch_fq? pfifo_fast? 2) What happens when you run the same test with gso/gro disabled?