[Bloat] BBR implementations, knobs to turn?

[Toke Høiland-Jørgensen]

Jesper Dangaard Brouer <brouer at redhat.com> writes:

> Hi Erik,
>
> I really appreciate that you are reaching out to the bufferbloat community
> for this real-life 5G mobile testing.  Lets all help out Erik.

Yes! FYI, I've been communicating off-list with Erik for quite some
time, he's doing great work but fighting the usual up-hill battle to get
others to recognise the issues; so +1, let's give him all the help we
can :)

> From your graphs, it does look like you are measuring latency
> under-load, e.g. while the curl download/upload is running.  This is
> great as this is the first rule of bufferbloat measuring :-)  (and Luca
> hinted to this)
>
> The Huawei policer/shaper sounds scary.  And 1000 packets deep queue
> also sound like a recipe for bufferbloat.  I would of-cause like to
> re-write the Huawei policer/shaper with the knowledge and techniques we
> know from our bufferbloat work in the Linux Kernel.  (If only I knew
> someone that coded on 5G solutions that could implement this on their
> hardware solution, and provide a better product Cc. Carlo)
>
> Are you familiar with Toke's (cc) work/PhD on handling bufferbloat on
> wireless networks?  (Hint: Airtime fairness)
>
> Solving bufferbloat in wireless networks require more than applying
> fq_codel on the bottleneck queue, it requires Airtime fairness.  Doing
> scheduling based Clients use of Radio-time and transmit-opportunities
> (TXOP), instead of shaping based on bytes. (This is why it can (if you
> are very careful) make sense to "holding back packets a bit" to
> generate a packet aggregate that only consumes one TXOP).
>
> The culprit is that each Client/MobilePhone will be sending at
> different rates, and scheduling based on bytes, will cause a Client with
> a low rate to consume a too large part of the shared radio airtime.
> That basically sums up Toke's PhD ;-)

Much as I of course appreciate the call-out, airtime fairness itself is
not actually much of an issue with mobile networks (LTE/5G/etc)... :)

The reason being that they use TDMA scheduling enforced by the base
station; so there's a central controller that enforces airtime usage
built into the protocol, which ensures fairness (unless the operator
explicitly configures it to be unfair for policy reasons). So the new
insight in my PhD is not so much "airtime fairness is good for wireless
links" as it is "we can achieve airtime fairness in CDMA/CS-scheduled
networks like WiFi".

Your other points about bloated queues etc, are spot on. Ideally, we
could get operators to fix their gear, but working around the issues
like Erik is doing can work in the meantime. And it's great to see that
it seems like Telenor is starting to roll this out; as far as I can tell
that has taken quite a bit of advocacy from Erik's side to get there! :)

-Toke