From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.toke.dk (mail.toke.dk [45.145.95.4]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 16F763B29D for ; Fri, 20 Nov 2020 07:42:49 -0500 (EST) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=toke.dk; s=20161023; t=1605876167; bh=Pdfyn3al7Zc+3dDSEI9PEaIpyiZ6mjd5Og6ttAYF+5U=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=aaOOF5KOS3OfdWPrSjCuTgzZDmbC2lAl9BKlgUuQDFcA556puevG8wmTkafttPAud 7Tn3TU9aLRothaY22k/vvG8BLy5TX6b9ZUqVtQFts1F0zHGs52i06MoOPlQylSOGcL yumcLVQFawjtLB/wombmK8N1PzO62jDq1mD7oH/RjbB8z8rhKcihK3wuVaJkDUJrXx ByLSnsOKjaOGrztdpqmw/XRzknX4YpGpS+ntnmawNiSsrNP1+dCCwMHhKbexM3KVSF r3Lcebf+igS8PJ1LjSfr2HXaY9l6BJi8MtxOhXRsrTwOsnHzGUCHBMAOCYPvQBTNGR IaxvwdiYk02xA== To: Jesper Dangaard Brouer , erik.taraldsen@telenor.com Cc: muscariello@ieee.org, priyarjha@google.com, bloat@lists.bufferbloat.net, lumuscar@cisco.com, brouer@redhat.com, Marek Majkowski , Carlo Carraro In-Reply-To: <20201120121027.2a61b319@carbon> References: <1605540351240.98589@telenor.com> <1605607524611.20916@telenor.com> <20201117160744.395f108e@carbon> <1605772623976.41134@telenor.com> <1605796526806.72220@telenor.com> <20201120121027.2a61b319@carbon> Date: Fri, 20 Nov 2020 13:42:47 +0100 X-Clacks-Overhead: GNU Terry Pratchett Message-ID: <877dqgrygo.fsf@toke.dk> MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Bloat] BBR implementations, knobs to turn? X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Nov 2020 12:42:49 -0000 Jesper Dangaard Brouer writes: > Hi Erik, > > I really appreciate that you are reaching out to the bufferbloat community > for this real-life 5G mobile testing. Lets all help out Erik. Yes! FYI, I've been communicating off-list with Erik for quite some time, he's doing great work but fighting the usual up-hill battle to get others to recognise the issues; so +1, let's give him all the help we can :) > From your graphs, it does look like you are measuring latency > under-load, e.g. while the curl download/upload is running. This is > great as this is the first rule of bufferbloat measuring :-) (and Luca > hinted to this) > > The Huawei policer/shaper sounds scary. And 1000 packets deep queue > also sound like a recipe for bufferbloat. I would of-cause like to > re-write the Huawei policer/shaper with the knowledge and techniques we > know from our bufferbloat work in the Linux Kernel. (If only I knew > someone that coded on 5G solutions that could implement this on their > hardware solution, and provide a better product Cc. Carlo) > > Are you familiar with Toke's (cc) work/PhD on handling bufferbloat on > wireless networks? (Hint: Airtime fairness) > > Solving bufferbloat in wireless networks require more than applying > fq_codel on the bottleneck queue, it requires Airtime fairness. Doing > scheduling based Clients use of Radio-time and transmit-opportunities > (TXOP), instead of shaping based on bytes. (This is why it can (if you > are very careful) make sense to "holding back packets a bit" to > generate a packet aggregate that only consumes one TXOP). > > The culprit is that each Client/MobilePhone will be sending at > different rates, and scheduling based on bytes, will cause a Client with > a low rate to consume a too large part of the shared radio airtime. > That basically sums up Toke's PhD ;-) Much as I of course appreciate the call-out, airtime fairness itself is not actually much of an issue with mobile networks (LTE/5G/etc)... :) The reason being that they use TDMA scheduling enforced by the base station; so there's a central controller that enforces airtime usage built into the protocol, which ensures fairness (unless the operator explicitly configures it to be unfair for policy reasons). So the new insight in my PhD is not so much "airtime fairness is good for wireless links" as it is "we can achieve airtime fairness in CDMA/CS-scheduled networks like WiFi". Your other points about bloated queues etc, are spot on. Ideally, we could get operators to fix their gear, but working around the issues like Erik is doing can work in the meantime. And it's great to see that it seems like Telenor is starting to roll this out; as far as I can tell that has taken quite a bit of advocacy from Erik's side to get there! :) -Toke