From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client did not present a certificate) by huchra.bufferbloat.net (Postfix) with ESMTPS id 9ECBF21F208 for ; Thu, 31 Oct 2013 18:39:29 -0700 (PDT) Received: by mail-wi0-f169.google.com with SMTP id cb5so619381wib.2 for ; Thu, 31 Oct 2013 18:39:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=BDhqK/uYIubyEOJEb2ciWk9VUwEvERHjvOjGW0HkN2Q=; b=E+aqUn13WVy095n0wAHZDuEa9Z5g+EhLVijZixDaCbaehq1oOLXPfHVYOJastXUWjV HT9VnK9rEtjKHBz24COc+aO5KDC1Gqd2loEMnRmJeE3YwLqzzNPkDEm0hwAxP6AddjLY bsjHdTXguJT+zO2urzzzxTRxhKU/QQBvokZL49Q4rq4sRXq3qRxPVBSZhQkmawIgTCRF mJms4A4WC69s7CUofo8fkPeQqQyOzNHev4MWrlnQwKBUeBNes4jlQouJ8fve5Hw90uAE 2dtDnN91lLs+nGsr31nUESjZZJ/xEMy2vhfHuKelz3qs+OjAbFbzv1C8BS3aSfH/+rVX V2JA== MIME-Version: 1.0 X-Received: by 10.180.19.133 with SMTP id f5mr625027wie.60.1383269967634; Thu, 31 Oct 2013 18:39:27 -0700 (PDT) Received: by 10.217.51.5 with HTTP; Thu, 31 Oct 2013 18:39:27 -0700 (PDT) In-Reply-To: References: <20131030162534.29f34ada@nehalam.linuxnetplumber.net> <5271BAA7.2080905@kau.se> <20131031023257.GB3365@lists.bufferbloat.net> <52724CA9.4020606@kau.se> Date: Thu, 31 Oct 2013 18:39:27 -0700 Message-ID: From: Dave Taht To: Stefan Alfredsson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: bloat Subject: Re: [Bloat] mobile broadband buffer bloat X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Nov 2013 01:39:30 -0000 oops that was telenor I was commenting on http://snapon.lab.bufferbloat.net/~d/telenor-4g.svg On Thu, Oct 31, 2013 at 6:32 PM, Dave Taht wrote: > Just did telia. I really want 4G as good as this!!! > > http://snapon.lab.bufferbloat.net/~d/telia-4g.svg > > So you multiply the black line x4 to get a total for the down or up, > then insert a fudge factor for the amount of acks that probably went > into the down relative to the up and vice versa, and you sort of have > your bandwidth number each way. > > I'm really impressed you can get ~72Mbit down and ~4Mbit up. (closer > to 8 up when you consider acks) Note also the relationship between > bandwidth and latency at T+22... > > > On Thu, Oct 31, 2013 at 6:09 PM, Dave Taht wrote: >> On Thu, Oct 31, 2013 at 5:27 AM, Stefan Alfredsson >> wrote: >>> On 10/31/13 03:32 , Dave Taht wrote: >>>> >>>> On Thu, Oct 31, 2013 at 03:04:23AM +0100, Stefan Alfredsson wrote: >>>>> >>>>> Hi, Stephen, the list, >>>>> >>>>> We're running a 3G/4G measurement site at Karlstad university, with >>>>> mobile broadband subscriptions from the four major telecom operators >>>>> in Sweden. I was curious to see how measurements in our network >>>>> would compare to yours, so I did a measurement run with netalyzer. >>>> >>>> Groovy! >>>> >>>> Can I ask that you try something that can more likely stress >>>> out that bus than netanlyzer? The netperf-wrappers tools >>>> (notably the tcp_bidirectional and rrul test), use C, rather than java= . >>> >>> >>> Sure! Actually we've been doing netperf-wrapper rrul measurements 24/4 >>> during July to September, roughly 3-4 times per day, in total over 3000 >>> individual tests. >>> >>> I think there's a lot of interesting information in there, but I have n= ot >>> done aggregate analysis yet (eg. looking at time-of-day effects, long-t= erm >>> trends, or the 10.000-students-arriving-in-august effect :-) ) >> >> if you are interested in measuring one way delays, owamp + a gps is >> working out well. >> >>> I packed the rrul-dumps for one day (1 sept 2013) and put it here: >>> http://www.cs.kau.se/stefalfr/3G4G/rrul-tests-2013-09-01.tgz . >>> >>> Here's an overview of this set, showing the per-run mean RTT, download = and >>> upload throughput: >>> >>> [alfs@sa-pc]:/tmp/n/2013-06> for f in */*/*/rrul*json.gz ; do test=3D$(= echo $f >>> | cut -d / -f 1-2); netperf-wrapper -i $f -f stats|grep -A 4 'avg:'| aw= k -v >>> test=3D$test '/Mean:/ { if (NR=3D=3D3) { rtt=3D$2 } if (NR=3D=3D10) {dl= =3D$2} if (NR=3D=3D16) >>> {ul=3D$2; printf "%s:\tRTT: %.1f ms DL: %.1f Mbit/s UL: %.1f Mbit/s\n",= test, >>> rtt, dl, ul}}'; done >>> Tele2/35G: RTT: 490.7 ms DL: 1.3 Mbit/s UL: 0.3 Mbit/s >>> Tele2/35G: RTT: 498.0 ms DL: 6.1 Mbit/s UL: 0.4 Mbit/s >>> Tele2/35G: RTT: 562.7 ms DL: 4.7 Mbit/s UL: 0.2 Mbit/s >>> Tele2/35G: RTT: 641.5 ms DL: 3.5 Mbit/s UL: 0.3 Mbit/s >>> Tele2/4G: RTT: 891.5 ms DL: 2.3 Mbit/s UL: 0.3 Mbit/s >>> Tele2/4G: RTT: 389.0 ms DL: 20.8 Mbit/s UL: 1.0 Mbit/s >>> Tele2/4G: RTT: 361.2 ms DL: 14.4 Mbit/s UL: 1.4 Mbit/s >>> Tele2/4G: RTT: 359.8 ms DL: 17.3 Mbit/s UL: 1.3 Mbit/s >>> Telenor/35G: RTT: 809.0 ms DL: 2.2 Mbit/s UL: 0.8 Mbit/s >>> Telenor/35G: RTT: 377.3 ms DL: 3.8 Mbit/s UL: 0.5 Mbit/s >>> Telenor/35G: RTT: 727.2 ms DL: 2.9 Mbit/s UL: 0.3 Mbit/s >>> Telenor/35G: RTT: 341.7 ms DL: 0.1 Mbit/s UL: 0.2 Mbit/s >>> Telenor/4G: RTT: 427.8 ms DL: 16.9 Mbit/s UL: 0.8 Mbit/s >>> Telenor/4G: RTT: 378.8 ms DL: 17.9 Mbit/s UL: 1.2 Mbit/s >>> Telenor/4G: RTT: 623.8 ms DL: 8.6 Mbit/s UL: 0.6 Mbit/s >>> Telenor/4G: RTT: 321.5 ms DL: 18.2 Mbit/s UL: 1.0 Mbit/s >>> Telia/35G: RTT: 686.9 ms DL: 4.8 Mbit/s UL: 0.2 Mbit/s >>> Telia/35G: RTT: 709.0 ms DL: 6.1 Mbit/s UL: 0.2 Mbit/s >>> Telia/35G: RTT: 546.9 ms DL: 3.8 Mbit/s UL: 0.2 Mbit/s >>> Telia/35G: RTT: 653.0 ms DL: 3.1 Mbit/s UL: 0.3 Mbit/s >>> Telia/4G: RTT: 233.6 ms DL: 14.1 Mbit/s UL: 1.1 Mbit/s >>> Telia/4G: RTT: 324.2 ms DL: 14.6 Mbit/s UL: 1.1 Mbit/s >>> Telia/4G: RTT: 381.5 ms DL: 15.2 Mbit/s UL: 1.2 Mbit/s >>> Tre/35G: RTT: 312.2 ms DL: 3.1 Mbit/s UL: 0.6 Mbit/s >>> Tre/35G: RTT: 376.1 ms DL: 4.1 Mbit/s UL: 0.8 Mbit/s >>> Tre/35G: RTT: 442.0 ms DL: 2.9 Mbit/s UL: 0.6 Mbit/s >>> Tre/35G: RTT: 294.0 ms DL: 2.9 Mbit/s UL: 0.6 Mbit/s >>> Tre/4G: RTT: 471.2 ms DL: 6.9 Mbit/s UL: 0.7 Mbit/s >>> Tre/4G: RTT: 396.2 ms DL: 6.8 Mbit/s UL: 0.7 Mbit/s >>> Tre/4G: RTT: 237.5 ms DL: 7.2 Mbit/s UL: 0.7 Mbit/s >>> Tre/4G: RTT: 321.5 ms DL: 6.9 Mbit/s UL: 0.7 Mbit/s >> >> Oh, that is wonderful data thank you! I confess to be more interested >> in outliers than averages, and induced delay over base delay. >> >> However, I'm concerned that you are not getting a correct result with >> your script above. >> >> I just took a quick look at telia-4g, using the 2100-2013-09-01T200038 >> sample set, and looking at the raw (rrul) data: What I see here >> >> http://snapon.lab.bufferbloat.net/~d/telia-4g.svg >> >> is 60-80 ms of induced latency on this link, with 40ms of baseline >> latency. Not a RTT of > 300! >> >> >>>>> To summarize, the measured buffering is well below one second for >>>>> all tests, although the results are not directly comparable since we >>>>> are using directly attached USB modems (Huawei E392 with Ubuntu >>>>> 12.04) >>>> >>>> ubuntu has backported fq_codel as far as 12.04. What kernel are you us= ing? >>> >>> On the "mobile terminal" we have 3.2.0-54-generic-pae #82-Ubuntu SMP >> >> I am delighted you've been collecting this data long term. >> >> I do have a problem in that the 3.2 kernel predates all the >> bufferbloat related fixes made in the 4 years since that release. >> we've been working our butts off here... :( >> >> Any chance you can update to 3.10.x or later for both this (driver and >> buffering fixes) and your fixed host? (tcp fixes). And it's generally >> been my hope people would try fq_codel (or sfq or pie or red) on such >> links, too... >> >> There has been very little done to optimize usb-net, although we >> tried. If this is using the ppp driver, that was improved >> substantially around 3.6? >> >>> The fixed host ("tptest") runs a self-compiled stock kernel 3.4.1 with >>> web10g patches, since we were interested in extracting TCP variable dat= a not >>> available from packet traces. >>> >>>>> instead of a separate router box. However, the numbers might >>>>> be interesting in a more general perspective. >>>> >>>> Yes, they are. These are repeatable? >>>> >>>> >>> >>> For the netalyzer I've only done singular tests, but it can easily be >>> repeated if needed (probably not). >>> The mean of all mean RTTs for 4G, all operators, all times, is 334 ms, = and >>> for 3.5G it's 573 ms. >>> This indicates that results are repeatable, and the set from 2013-09-01= is >>> representative. >> >> except that my plot shows differently... >> >> and you are running an ancient kernel, yes. Something of a major >> variable, that... and if you could let us know the usb path to the >> various underlying drivers we can poke into matters there? >> >>> >>> "below one second" was in relation to the >5 second RTT's, ideally it s= hould >>> be much lower of course. As a side note, we also do measurements in unl= oaded >>> networks, and for 4G we get around 40-60 ms RTT (ICMP and TCP packets). >>> >>> Also, you might be interested in a paper we published in June, that rep= orts >>> on measurements of bloat for short flows in relation to different conge= stion >>> control algorithms, in 3G, 3.5G and 4G (but for a single operator). >>> See http://urn.kb.se/resolve?urn=3Durn:nbn:se:kau:diva-27779 >>> with fulltext at http://dx.doi.org/10.1109/WoWMoM.2013.6583408. >>> I have also put a local copy (for personal use etc) at >>> http://www.cs.kau.se/stefalfr/3G4G/alfredsson-bufferbloat-wowmom13.pdf >>> >>> >>> Regards, >>> Stefan >>> >>> -- >>> Stefan Alfredsson, PhD Tel: +46 (0) 54 700 1668 >>> Datavetenskap Kontor: 21F-425 (Hus Vanern) >>> Karlstads universitet PGP 0xB19B4B16 >>> SE-651 88 Karlstad http://www.cs.kau.se/stefalfr/ >>> >>> >>> _______________________________________________ >>> Bloat mailing list >>> Bloat@lists.bufferbloat.net >>> https://lists.bufferbloat.net/listinfo/bloat >> >> >> >> -- >> Dave T=E4ht >> >> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscri= be.html > > > > -- > Dave T=E4ht > > Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscrib= e.html --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.= html