From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ua1-x92e.google.com (mail-ua1-x92e.google.com [IPv6:2607:f8b0:4864:20::92e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 463593B29E for ; Mon, 16 Nov 2020 16:15:02 -0500 (EST) Received: by mail-ua1-x92e.google.com with SMTP id t15so5814643ual.6 for ; Mon, 16 Nov 2020 13:15:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=RxBnvp1qSSsqXKAfp8C8Q3Gy3EvLaTG7Fiz26mrwoGE=; b=a1Sl57w7pcOSHXQU0cr3zbq5mkSbcNxQLhkvifIIQMWhIuUho+OIopn/rCcBeUqQrR NtfVKGj4yQqhHFyX3CP7nqYDBKmORjpJmFNdyntcANH5Xt2PzdsOWF8qeuj95/LE04YK 2tJg6pl58Dk1k8MvjXRP6l9bkOCT7V5BGhyXAfR8fodkWHz0JMBfd6olrWvQ2V3lYgrw Trjk19Raqk60wNOjpNA3XKiKMo6qpILZj03p/nyZm8tUKBR5LYg5JYDwKfalLIbFM+aH ZjZQsv5ubMjW/RcaKKsJwHDm9yGDR1qNxHRrw0AZQpUP/1WdJxNLcxKB7LrhtTGr26wo 3/JA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=RxBnvp1qSSsqXKAfp8C8Q3Gy3EvLaTG7Fiz26mrwoGE=; b=dSferGm9CvQ1iNQSo0IcXO/O0vdEktw/CSQ7ko118xK8Tm+S1KJkANFp5hVwxoQmIs IWc3Hf0fWkRiOyRoDwcA+sdpiDUGPql77ZBqbwFwdeUsX6etKgoPQjVmsFIJslkkQfn/ OQ5ZnRf3jgdFoL/QDJZF0fYAIvqYimg5jjRx+OqoAA7+Pz+gZvtT/Da75s8c6Q2ToCp3 c3p/2w2nLtK3ShBfJuD1fDzG1lpN5aCZl3GaiF6a1GY4HDyQniwKsqB3X45cGoGhTrWJ JVljssyvu9aEc8sowT1IL7o14RSs+g4HmnmZXi9KdRZP2gT0ZRs7SZ8mWPijOjUzeuT7 CSPA== X-Gm-Message-State: AOAM530ik9Sr5CZLq0beitpKOdJWA0RiZL7MLbNAHtrR6+en3GeVR+zo /ZgVt0UBnPjRmc/6kfD9B1LEWcwi2H+8Yah939v10w== X-Google-Smtp-Source: ABdhPJyRK9Z+wQtenCVOjI1P2ngdUngXZvXISX+vFQzJmt2ckMbvpNC5mgtKopTK4YaKf6nBfjiTUbZfrC+3zO6JjS0= X-Received: by 2002:ab0:6355:: with SMTP id f21mr7683586uap.142.1605561301487; Mon, 16 Nov 2020 13:15:01 -0800 (PST) MIME-Version: 1.0 References: <1605540351240.98589@telenor.com> In-Reply-To: <1605540351240.98589@telenor.com> From: Neal Cardwell Date: Mon, 16 Nov 2020 16:14:45 -0500 Message-ID: To: erik.taraldsen@telenor.com Cc: bloat Content-Type: multipart/alternative; boundary="0000000000000af55805b43fdf6b" Subject: Re: [Bloat] BBR implementations, knobs to turn? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2020 21:15:02 -0000 --0000000000000af55805b43fdf6b Content-Type: text/plain; charset="UTF-8" A couple questions: - I guess this is Linux TCP BBRv1 ("bbr" module)? What's the OS distribution and exact kernel version ("uname -r")? - What do you mean when you say "The old server allows for more re-transmits"? - If BBRv1 is suffering throughput problems due to high retransmit rates, then usually the retransmit rate is around 15% or higher. If the retransmit rate is that high on a radio link that is being tested, then that radio link may be having issues that should be investigated separately? - Would you be able to take a tcpdump trace of the well-behaved and problematic traffic and share the pcap or a plot? https://github.com/google/bbr/blob/master/Documentation/bbr-faq.md#how-can-i-visualize-the-behavior-of-linux-tcp-bbr-connections - Would you be able to share the output of "ss -tin" from a recently built "ss" binary, near the end of a long-lived test flow, for the well-behaved and problematic cases? https://github.com/google/bbr/blob/master/Documentation/bbr-faq.md#how-can-i-monitor-linux-tcp-bbr-connections best, neal On Mon, Nov 16, 2020 at 10:25 AM wrote: > I'm in the process of replacing a throughput test server. The old server > is running a 1Gbit Ethernet card on a 1Gbit link and ubuntu. The new a > 10Gbit card on a 40Gbit link and centos. Both have low load and Xenon > processors. > > > The purpose is for field installers to verify the bandwidth sold to the > customers using known clients against known servers. (4G and 5G fixed > installations mainly). > > > What I'm finding is that the new server is consistently delivering > slightly lower throughput than the old server. The old server allows for > more re-transmits and has a slightly higher congestion window than the new > server. > > > Is there any way to tune bbr to allow for more re-transmits (which seems > to be the limiting factor)? Or other suggestions? > > > > (Frankly I think the old server is to aggressive for general purpose use. > It seems to starve out other tcp sessions more than the new server. So for > delivering regular content to users the new implementation seems more > balanced, but that is not the target here. We want to stress test the > radio link.) > > > Regards Erik > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --0000000000000af55805b43fdf6b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
A couple questions:

- I guess this is L= inux TCP BBRv1 ("bbr" module)? What's the OS distribution and= exact kernel version ("uname -r")?

- What do you mean w= hen you say "The old server allows for more re-transmits"?
- If BBRv1 is suffering throughput problems due to high retran= smit rates, then usually the retransmit rate is around 15% or higher. If th= e retransmit rate is that high on a radio link that is being tested, then t= hat radio link may be having issues that should be investigated separately?=

- Would you be able to take a tcpdump trace of the = well-behaved and problematic traffic and share the pcap or a plot?
=C2=A0h= ttps://github.com/google/bbr/blob/master/Documentation/bbr-faq.md#how-can-i= -visualize-the-behavior-of-linux-tcp-bbr-connections

- Would you be able to share the output=C2=A0of "ss -tin" = from a recently built "ss" binary, near the end of a long-lived t= est flow, for the well-behaved and problematic cases?

best,
neal


On Mon, Nov 16, = 2020 at 10:25 AM <erik.tar= aldsen@telenor.com> wrote:

I'm in the process of replacing a throughput test server.=C2=A0 The = old server is running a 1Gbit Ethernet card on a 1Gbit=C2=A0link and ubuntu= .=C2=A0 The new a 10Gbit card on a 40Gbit link and centos.=C2=A0 Both have = low load and Xenon processors.


The purpose is for field installers to verify the bandwidth sold to the = customers using known clients against known servers. =C2=A0(4G and 5G fixed= installations mainly).


What I'm finding is that the new server is consistently delivering s= lightly lower throughput than the old server.=C2=A0 The old server allows f= or more re-transmits and has a slightly higher congestion window than the n= ew server.


Is there any way to tune bbr to allow for more re-transmits (which seems= to be the limiting factor)?=C2=A0 Or other suggestions?



(Frankly I think the old server is to aggressive for general purpose use= .=C2=A0 It seems to starve out other tcp sessions more than the new server.= =C2=A0 So for delivering regular content to users the new implementation se= ems more balanced, but that is not the target here.=C2=A0 We want to stress test the radio link.)


Regards Erik

_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat
--0000000000000af55805b43fdf6b--