[Bloat] AQM and PPP on Linux

Wed Jul 29 09:41:41 EDT 2015

On Wed, Jul 29, 2015 at 3:15 PM, Stefan Alfredsson
<Stefan.Alfredsson at kau.se> wrote:
> * Quoting Dave Taht <dave.taht at gmail.com> [28 Jul-15 16:44]:
>> On Tue, Jul 28, 2015 at 3:09 PM, Juliusz Chroboczek
>> <jch at pps.univ-paris-diderot.fr> wrote:
>> > I'm currently away from home, and using a 3G modem for Internet access.
>> > I've found out that both NetworkManager and wvdial/pppd setup the
>> > interface to use pfifo_fast (with a qlen of a mere 3 packets!).  Setting
>> > fq_codel manually appears to work fine, but needs to be redone every time
>> > the modem has a hiccup.
>>
>> However, the netusb-related drivers, and the 3g devices themselves,
>> contain oft-huge buffers that reduce the effectiveness of any aqm or
>> fq system, sometimes to immeasurability.
>>
>> I would be very interested in flent benchmarks of your 3g device with
>> the 3 packet txqueue and with fq_codel, for the tcp_upload, rrul, and
>> rrul_be tests.
>
>
> I have a station with mobile broadband USB modems (4xHuawei E392),
> directly connected to a Linux box running kernel 3.16.3, with
> a (non-public) netperf server (same kernel) hosted on the well-connected
> Swedish university network.
>
> I also have a 3 packet txqueue, and ran the flent tcp_upload, rrul, rrul_be
> for 3.5G (HSPA+) and 4G (LTE) networks for one of the operators/modems.
> For comparison, I ran both pfifo_fast and fq_codel.
>
> The execution script, output and flent result files are available at
> https://www.dropbox.com/s/n5tc1dhtu9jdplz/bloatlist-ppp-150729.zip

Thank you SO MUCH for the raw flent data. That made poking through this a snap!

Of course, the news is mostly bad - a tiny fifo qdisc on LTE looks to
be presently best than any amount of queuing/aqm/fq at the layers
above the device driver and device.

Something like BQL is needed for usbnet, and the usb-cellmodem
driver... and some means of tracking completion interrupts sanely to
change this picture. We did a bit of work on both these layers a while
back but we lost steam at the complex interactions of the usb stack.

as an aside - and on the wifi front, recently toke and I did a string
of overnight tests on the wifi test rig we are taking to battlemesh,
which includes the latest minstrel blues and minstrel variance
patches. Sadly, also, mostly of note is the wide variabilty of results
from run to run, in the same wifi environment. There were only a few
radios we could "hear" to interfere... and I'd done much better with
the exact same radios in an environment fully under my control.

A tarball of all these flent testruns and tests is at

http://snapon.lab.bufferbloat.net/~d/long-running.tgz

Anyway... the variance from run to run:

http://snapon.lab.bufferbloat.net/~d/long-running/10testscompared.png

Trying to pull apart the apparently bi-modal interactions with rate
control (and *why*) is going to take some *work*.

http://snapon.lab.bufferbloat.net/~d/long-running/themessthatiswifi.png

This plot, randomly picked out from the others

http://snapon.lab.bufferbloat.net/~d/long-running/wifiratecontrol.png

shows clearly the current interaction between the current rate of the
wifi and the latency. (but I have no idea *why* it stayed stuck at the
lower rate for 20+ seconds!). Still, I believe we have developed means
to hold the latency low at nearly any bandwidth, we just have to roll
up our sleeves and make per station queuing work and then start
glomming on new algorithms. Or so I hope.

...

I happened to rather like this plot, from an artistic perspective,
showing the bands that each 802.11e queue fits into. Sort of.

http://snapon.lab.bufferbloat.net/~d/long-running/vangogh.png

But maybe I am thinking of the wrong artist?

> From flent --gui inspections, I'd say there is about a factor 10 increase
> from base to load latency (from around 40-50 ms to around 400-500 ms),
> for both pfifo_fast and fq_codel, for both link types,
> with some transients above 1000 ms.
>
>
> Juliusz: Regarding data usage, the 4G measurements consumed 1.4 Gbyte
> download and 357 Mbyte upload - but it all depends on the network
> throughput as the experiments are time bound (by default 60 seconds).
> For 3.5G 100 Mbyte data was sent, and guesstimately 500 Mbyte received
> (sorry, didn't capture the amount; but it can probably be found in
> the trace files if necessary. However it all depends on your link
> throughput which will be different from mine anyway).

Yes. One reason why I have not focused much on fixing LTE, is I refuse
to have them charge me for attempting to fix their technology. It is
bad enough throwing good money away to Netgear to fix their routers,
but to get charged through the **** for LTE on a bandwidth basis - and
I could see, eating a lot of bandwidth with various tests - ENOBUDGET.

Now, if we could, like, get 20 free unlimited use sims for anywhere in
the world, that might be an incentive.... ;)

>
>
> Cheers,
>  Stefan
>
> --
> Stefan Alfredsson, PhD         Tel: +46 (0) 54 700 1668
> Datavetenskap                  Kontor: 21E-414 (Hus Vanern)
> Karlstads universitet          PGP 0xB19B4B16
> SE-651 88 Karlstad             http://www.cs.kau.se/stefalfr/
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

-- 
Dave Täht
worldwide bufferbloat report:
http://www.dslreports.com/speedtest/results/bufferbloat
And:
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast