From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id AA8AC3CB35 for ; Thu, 19 Nov 2020 08:32:21 -0500 (EST) Received: by mail-wm1-x333.google.com with SMTP id c9so7197629wml.5 for ; Thu, 19 Nov 2020 05:32:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ieee.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=m9t9j+GqCDfW5M5AFD6oFHi/iXEz85jA7I1OrFWLXQY=; b=bl7QwyoyfOvLTBiWGBAToI+SYdoysdhcylamkbwkv7WXMwuD7E1fMTKCh4xpVIP+uA Hew45Ha7n6+9DJCv472AQ780zkSCBX9+ZANDTUDkBOA4LKnKypSDP9grawhYx6JzxA6+ JiasRoZjT+c7KarSAT39vSbmibp7Tdqk/jy8U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=m9t9j+GqCDfW5M5AFD6oFHi/iXEz85jA7I1OrFWLXQY=; b=hqIEjHqQB1TqRbfBH6DB5wu/MlzsaeImsDgLccZHXOjxu7qfv/o++Rcme6/AaoSH40 gBSQyfMQgp7HXqqCufyiHI0LsUenYb3nD2LvZqNCav0Ljj6AUh1Eggz5lJCSjuhL9DUx FJTRwvP4waomtM38nQ4RE2jJyrKAUoVo1sbJHFEuNGWlzs17BzCV72emA54SBARWdE+d 6UB2OSVCUDEIYzpvbzguLnF8kpUHbzoj97Bh5YL7JNa74S9TeFRmmbRj4hIbk/MX/GsD K86+XDJU/lHFXIRAP59Q/fmil05lmvVfzfSqlxssw1Ft5abAu4ogjehY96XJnpKW8JV5 MMVw== X-Gm-Message-State: AOAM531vSksvCLwA+o0FR7PG9A8f+9xx8yBkURLo1tZlwNFbyf0vI8dY u+n1GGnJNktXJ9yqH2H2AYiOZvwcpyZaAPXIAbYgDQ== X-Google-Smtp-Source: ABdhPJzCf6x2n9e+tOz3ubhDoKDGfIuC4pXwLbJ7l0gnjs56NYXXPSWNrEak4/PZDcgU1SIRCRci0C4zmN+A18ZuLJo= X-Received: by 2002:a1c:3d6:: with SMTP id 205mr4793762wmd.85.1605792740671; Thu, 19 Nov 2020 05:32:20 -0800 (PST) MIME-Version: 1.0 References: <1605540351240.98589@telenor.com> <1605607524611.20916@telenor.com> <20201117160744.395f108e@carbon> <1605772623976.41134@telenor.com> In-Reply-To: <1605772623976.41134@telenor.com> From: Luca Muscariello Date: Thu, 19 Nov 2020 14:32:09 +0100 Message-ID: To: erik.taraldsen@telenor.com Cc: Jesper Dangaard Brouer , priyarjha@google.com, bloat , Luca Muscariello Content-Type: multipart/alternative; boundary="000000000000e425b405b475c15b" Subject: Re: [Bloat] BBR implementations, knobs to turn? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2020 13:32:21 -0000 --000000000000e425b405b475c15b Content-Type: text/plain; charset="UTF-8" Hi Erick, one question about the PGW: is it a policer or a shaper that you have installed? Also, have you tried to run a ping session before and in parallel to the curl sessions? Luca On Thu, Nov 19, 2020 at 2:15 PM wrote: > Update: > The 5G router was connected to a new base station. Now the limiting > factor of throughput is the policer on the PGW in mobile core, not the > radio link itself. The SIM card used is limited to 30Mbit/s. This > scenario favours the new server. I have attached graphs comparing radio > link limited vs PGW policer results, and a zoomed in graph of the policer > > > We have Huawei RAN and Ericsson RAN, rate limited and not rate limited > subscriptions, 4G and 5G access, and we are migrating to a new core with > new PGW (policer). Starting to be a bit of a matrix to set up tests for. > > > -Erik > > > ________________________________________ > Fra: Jesper Dangaard Brouer > Sendt: 17. november 2020 16:07 > Til: Taraldsen Erik; Priyaranjan Jha > Kopi: brouer@redhat.com; ncardwell@google.com; bloat@lists.bufferbloat.net > Emne: Re: [Bloat] BBR implementations, knobs to turn? > > On Tue, 17 Nov 2020 10:05:24 +0000 wrote: > > > Thank you for the response Neal > > Yes. And it is impressive how many highly qualified people are on the > bufferbloat list. > > > old_hw # uname -r > > 5.3.0-64-generic > > (Ubuntu 19.10 on xenon workstation, integrated network card, 1Gbit > > GPON access. Used as proof of concept from the lab at work) > > > > > > new_hw # uname -r > > 4.18.0-193.19.1.el8_2.x86_64 > > (Centos 8.2 on xenon rack server, discrete 10Gbit network card, > > 40Gbit server farm link (low utilization on link), intended as fully > > supported and run service. Not possible to have newer kernel and > > still get service agreement in my organization) > > Let me help out here. The CentOS/RHEL8 kernels have a huge amount of > backports. I've attached a patch/diff of net/ipv4/tcp_bbr.c changes > missing in RHEL8. > > It looks like these patches are missing in CentOS/RHEL8: > [1] https://git.kernel.org/torvalds/c/78dc70ebaa38aa3 > [2] https://git.kernel.org/torvalds/c/a87c83d5ee25cf7 > > Could missing patch [1] result in the issue Erik is seeing? > (It explicitly mentions improvements for WiFi...) > > -- > Best regards, > Jesper Dangaard Brouer > MSc.CS, Principal Kernel Engineer at Red Hat > LinkedIn: http://www.linkedin.com/in/brouer > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --000000000000e425b405b475c15b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Erick,

= one question about the PGW: is it a policer or a shaper that you have insta= lled?
Als= o, have you tried to run a ping session before and in parallel=C2=A0to the = curl sessions?

Luca


On Thu, Nov 19, 2020 at 2:15 PM <erik.taraldsen@telenor.com> wrote:
Update:
The 5G router was connected to a new base station.=C2=A0 Now the limiting f= actor of throughput is the policer on the PGW in mobile core, not the radio= link itself.=C2=A0 The SIM card used is limited to 30Mbit/s.=C2=A0 This sc= enario favours the new server.=C2=A0 I have attached graphs comparing radio= link limited vs PGW policer results, and a zoomed in graph of the policer<= br>

We have Huawei RAN and Ericsson RAN, rate limited and not rate limited subs= criptions, 4G and 5G access, and we are migrating to a new core with new PG= W (policer).=C2=A0 Starting to be a bit of a matrix to set up tests for.

-Erik


________________________________________
Fra: Jesper Dangaard Brouer <brouer@redhat.com>
Sendt: 17. november 2020 16:07
Til: Taraldsen Erik; Priyaranjan Jha
Kopi: brouer@redhat.= com; ncardwel= l@google.com; bloat@lists.bufferbloat.net
Emne: Re: [Bloat] BBR implementations, knobs to turn?

On Tue, 17 Nov 2020 10:05:24 +0000 <erik.taraldsen@telenor.com> wrote:

> Thank you for the response Neal

Yes. And it is impressive how many highly qualified people are on the
bufferbloat list.

> old_hw # uname -r
> 5.3.0-64-generic
> (Ubuntu 19.10 on xenon workstation, integrated network card, 1Gbit
> GPON access.=C2=A0 Used as proof of concept from the lab at work)
>
>
> new_hw # uname -r
> 4.18.0-193.19.1.el8_2.x86_64
> (Centos 8.2 on xenon rack server, discrete 10Gbit network card,
> 40Gbit server farm link (low utilization on link), intended as fully > supported and run service.=C2=A0 Not possible to have newer kernel and=
> still get service agreement in my organization)

Let me help out here.=C2=A0 The CentOS/RHEL8 kernels have a huge amount of<= br> backports.=C2=A0 I've attached a patch/diff of net/ipv4/tcp_bbr.c chang= es
missing in RHEL8.

It looks like these patches are missing in CentOS/RHEL8:
=C2=A0[1] https://git.kernel.org/torvalds/c/78dc70e= baa38aa3
=C2=A0[2] https://git.kernel.org/torvalds/c/a87c83d= 5ee25cf7

Could missing patch [1] result in the issue Erik is seeing?
(It explicitly mentions improvements for WiFi...)

--
Best regards,
=C2=A0 Jesper Dangaard Brouer
=C2=A0 MSc.CS, Principal Kernel Engineer at Red Hat
=C2=A0 LinkedIn: http://www.linkedin.com/in/brouer
_______________________________________________
Bloat mailing list
Bloat@list= s.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat
--000000000000e425b405b475c15b--