[Bloat] mobile broadband buffer bloat
Dave Taht
dave.taht at gmail.com
Thu Oct 31 21:32:32 EDT 2013
Just did telia. I really want 4G as good as this!!!
http://snapon.lab.bufferbloat.net/~d/telia-4g.svg
So you multiply the black line x4 to get a total for the down or up,
then insert a fudge factor for the amount of acks that probably went
into the down relative to the up and vice versa, and you sort of have
your bandwidth number each way.
I'm really impressed you can get ~72Mbit down and ~4Mbit up. (closer
to 8 up when you consider acks) Note also the relationship between
bandwidth and latency at T+22...
On Thu, Oct 31, 2013 at 6:09 PM, Dave Taht <dave.taht at gmail.com> wrote:
> On Thu, Oct 31, 2013 at 5:27 AM, Stefan Alfredsson
> <Stefan.Alfredsson at kau.se> wrote:
>> On 10/31/13 03:32 , Dave Taht wrote:
>>>
>>> On Thu, Oct 31, 2013 at 03:04:23AM +0100, Stefan Alfredsson wrote:
>>>>
>>>> Hi, Stephen, the list,
>>>>
>>>> We're running a 3G/4G measurement site at Karlstad university, with
>>>> mobile broadband subscriptions from the four major telecom operators
>>>> in Sweden. I was curious to see how measurements in our network
>>>> would compare to yours, so I did a measurement run with netalyzer.
>>>
>>> Groovy!
>>>
>>> Can I ask that you try something that can more likely stress
>>> out that bus than netanlyzer? The netperf-wrappers tools
>>> (notably the tcp_bidirectional and rrul test), use C, rather than java.
>>
>>
>> Sure! Actually we've been doing netperf-wrapper rrul measurements 24/4
>> during July to September, roughly 3-4 times per day, in total over 3000
>> individual tests.
>>
>> I think there's a lot of interesting information in there, but I have not
>> done aggregate analysis yet (eg. looking at time-of-day effects, long-term
>> trends, or the 10.000-students-arriving-in-august effect :-) )
>
> if you are interested in measuring one way delays, owamp + a gps is
> working out well.
>
>> I packed the rrul-dumps for one day (1 sept 2013) and put it here:
>> http://www.cs.kau.se/stefalfr/3G4G/rrul-tests-2013-09-01.tgz .
>>
>> Here's an overview of this set, showing the per-run mean RTT, download and
>> upload throughput:
>>
>> [alfs at sa-pc]:/tmp/n/2013-06> for f in */*/*/rrul*json.gz ; do test=$(echo $f
>> | cut -d / -f 1-2); netperf-wrapper -i $f -f stats|grep -A 4 'avg:'| awk -v
>> test=$test '/Mean:/ { if (NR==3) { rtt=$2 } if (NR==10) {dl=$2} if (NR==16)
>> {ul=$2; printf "%s:\tRTT: %.1f ms DL: %.1f Mbit/s UL: %.1f Mbit/s\n", test,
>> rtt, dl, ul}}'; done
>> Tele2/35G: RTT: 490.7 ms DL: 1.3 Mbit/s UL: 0.3 Mbit/s
>> Tele2/35G: RTT: 498.0 ms DL: 6.1 Mbit/s UL: 0.4 Mbit/s
>> Tele2/35G: RTT: 562.7 ms DL: 4.7 Mbit/s UL: 0.2 Mbit/s
>> Tele2/35G: RTT: 641.5 ms DL: 3.5 Mbit/s UL: 0.3 Mbit/s
>> Tele2/4G: RTT: 891.5 ms DL: 2.3 Mbit/s UL: 0.3 Mbit/s
>> Tele2/4G: RTT: 389.0 ms DL: 20.8 Mbit/s UL: 1.0 Mbit/s
>> Tele2/4G: RTT: 361.2 ms DL: 14.4 Mbit/s UL: 1.4 Mbit/s
>> Tele2/4G: RTT: 359.8 ms DL: 17.3 Mbit/s UL: 1.3 Mbit/s
>> Telenor/35G: RTT: 809.0 ms DL: 2.2 Mbit/s UL: 0.8 Mbit/s
>> Telenor/35G: RTT: 377.3 ms DL: 3.8 Mbit/s UL: 0.5 Mbit/s
>> Telenor/35G: RTT: 727.2 ms DL: 2.9 Mbit/s UL: 0.3 Mbit/s
>> Telenor/35G: RTT: 341.7 ms DL: 0.1 Mbit/s UL: 0.2 Mbit/s
>> Telenor/4G: RTT: 427.8 ms DL: 16.9 Mbit/s UL: 0.8 Mbit/s
>> Telenor/4G: RTT: 378.8 ms DL: 17.9 Mbit/s UL: 1.2 Mbit/s
>> Telenor/4G: RTT: 623.8 ms DL: 8.6 Mbit/s UL: 0.6 Mbit/s
>> Telenor/4G: RTT: 321.5 ms DL: 18.2 Mbit/s UL: 1.0 Mbit/s
>> Telia/35G: RTT: 686.9 ms DL: 4.8 Mbit/s UL: 0.2 Mbit/s
>> Telia/35G: RTT: 709.0 ms DL: 6.1 Mbit/s UL: 0.2 Mbit/s
>> Telia/35G: RTT: 546.9 ms DL: 3.8 Mbit/s UL: 0.2 Mbit/s
>> Telia/35G: RTT: 653.0 ms DL: 3.1 Mbit/s UL: 0.3 Mbit/s
>> Telia/4G: RTT: 233.6 ms DL: 14.1 Mbit/s UL: 1.1 Mbit/s
>> Telia/4G: RTT: 324.2 ms DL: 14.6 Mbit/s UL: 1.1 Mbit/s
>> Telia/4G: RTT: 381.5 ms DL: 15.2 Mbit/s UL: 1.2 Mbit/s
>> Tre/35G: RTT: 312.2 ms DL: 3.1 Mbit/s UL: 0.6 Mbit/s
>> Tre/35G: RTT: 376.1 ms DL: 4.1 Mbit/s UL: 0.8 Mbit/s
>> Tre/35G: RTT: 442.0 ms DL: 2.9 Mbit/s UL: 0.6 Mbit/s
>> Tre/35G: RTT: 294.0 ms DL: 2.9 Mbit/s UL: 0.6 Mbit/s
>> Tre/4G: RTT: 471.2 ms DL: 6.9 Mbit/s UL: 0.7 Mbit/s
>> Tre/4G: RTT: 396.2 ms DL: 6.8 Mbit/s UL: 0.7 Mbit/s
>> Tre/4G: RTT: 237.5 ms DL: 7.2 Mbit/s UL: 0.7 Mbit/s
>> Tre/4G: RTT: 321.5 ms DL: 6.9 Mbit/s UL: 0.7 Mbit/s
>
> Oh, that is wonderful data thank you! I confess to be more interested
> in outliers than averages, and induced delay over base delay.
>
> However, I'm concerned that you are not getting a correct result with
> your script above.
>
> I just took a quick look at telia-4g, using the 2100-2013-09-01T200038
> sample set, and looking at the raw (rrul) data: What I see here
>
> http://snapon.lab.bufferbloat.net/~d/telia-4g.svg
>
> is 60-80 ms of induced latency on this link, with 40ms of baseline
> latency. Not a RTT of > 300!
>
>
>>>> To summarize, the measured buffering is well below one second for
>>>> all tests, although the results are not directly comparable since we
>>>> are using directly attached USB modems (Huawei E392 with Ubuntu
>>>> 12.04)
>>>
>>> ubuntu has backported fq_codel as far as 12.04. What kernel are you using?
>>
>> On the "mobile terminal" we have 3.2.0-54-generic-pae #82-Ubuntu SMP
>
> I am delighted you've been collecting this data long term.
>
> I do have a problem in that the 3.2 kernel predates all the
> bufferbloat related fixes made in the 4 years since that release.
> we've been working our butts off here... :(
>
> Any chance you can update to 3.10.x or later for both this (driver and
> buffering fixes) and your fixed host? (tcp fixes). And it's generally
> been my hope people would try fq_codel (or sfq or pie or red) on such
> links, too...
>
> There has been very little done to optimize usb-net, although we
> tried. If this is using the ppp driver, that was improved
> substantially around 3.6?
>
>> The fixed host ("tptest") runs a self-compiled stock kernel 3.4.1 with
>> web10g patches, since we were interested in extracting TCP variable data not
>> available from packet traces.
>>
>>>> instead of a separate router box. However, the numbers might
>>>> be interesting in a more general perspective.
>>>
>>> Yes, they are. These are repeatable?
>>>
>>>
>>
>> For the netalyzer I've only done singular tests, but it can easily be
>> repeated if needed (probably not).
>> The mean of all mean RTTs for 4G, all operators, all times, is 334 ms, and
>> for 3.5G it's 573 ms.
>> This indicates that results are repeatable, and the set from 2013-09-01 is
>> representative.
>
> except that my plot shows differently...
>
> and you are running an ancient kernel, yes. Something of a major
> variable, that... and if you could let us know the usb path to the
> various underlying drivers we can poke into matters there?
>
>>
>> "below one second" was in relation to the >5 second RTT's, ideally it should
>> be much lower of course. As a side note, we also do measurements in unloaded
>> networks, and for 4G we get around 40-60 ms RTT (ICMP and TCP packets).
>>
>> Also, you might be interested in a paper we published in June, that reports
>> on measurements of bloat for short flows in relation to different congestion
>> control algorithms, in 3G, 3.5G and 4G (but for a single operator).
>> See http://urn.kb.se/resolve?urn=urn:nbn:se:kau:diva-27779
>> with fulltext at http://dx.doi.org/10.1109/WoWMoM.2013.6583408.
>> I have also put a local copy (for personal use etc) at
>> http://www.cs.kau.se/stefalfr/3G4G/alfredsson-bufferbloat-wowmom13.pdf
>>
>>
>> Regards,
>> Stefan
>>
>> --
>> Stefan Alfredsson, PhD Tel: +46 (0) 54 700 1668
>> Datavetenskap Kontor: 21F-425 (Hus Vanern)
>> Karlstads universitet PGP 0xB19B4B16
>> SE-651 88 Karlstad http://www.cs.kau.se/stefalfr/
>>
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat at lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>
>
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
--
Dave Täht
Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
More information about the Bloat
mailing list