From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-we0-x235.google.com (mail-we0-x235.google.com [IPv6:2a00:1450:400c:c03::235]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id E1EE421F208 for ; Thu, 31 Oct 2013 18:32:33 -0700 (PDT) Received: by mail-we0-f181.google.com with SMTP id t60so3343059wes.26 for ; Thu, 31 Oct 2013 18:32:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=4O4ePVr/yXOZw3153ZS8UF4IaZN1ysCesx0Xc0DTKxU=; b=dVuMHTjQo+xemREENdsRZOEG0kvJ5z7L7Cg+IP1UvvOuLOhdx44Z/NfmZgoJ6wJ6SK /P8sn7FDAiAi7dPkZ1nxwVpsWmFvo/nEMhI9XN9DFSWsjfh18rAqPmr/GMoBvwfi/iza XzSjL4L0lhzhToQPANwC7xINbLu+3jql2Dy2LhvWorHXts0M42gM+W0vVf09hW3Vup2W Nkmz/7XiF3UD/31rVMJfzDlzPj7L29D9NWc9rIEUmvD3y4HdSIL+UPhR2nllewTNZ0im nLo/O0+vMc7ZhXOXzzUwncZmXt4FhvDSul/p2I3n5ceG2L6p7QGndJwmUJcd9u2r5bXh 6JLg== MIME-Version: 1.0 X-Received: by 10.180.189.132 with SMTP id gi4mr579174wic.19.1383269552199; Thu, 31 Oct 2013 18:32:32 -0700 (PDT) Received: by 10.217.51.5 with HTTP; Thu, 31 Oct 2013 18:32:32 -0700 (PDT) In-Reply-To: References: <20131030162534.29f34ada@nehalam.linuxnetplumber.net> <5271BAA7.2080905@kau.se> <20131031023257.GB3365@lists.bufferbloat.net> <52724CA9.4020606@kau.se> Date: Thu, 31 Oct 2013 18:32:32 -0700 Message-ID: From: Dave Taht To: Stefan Alfredsson Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: bloat Subject: Re: [Bloat] mobile broadband buffer bloat X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Nov 2013 01:32:34 -0000 Just did telia. I really want 4G as good as this!!! http://snapon.lab.bufferbloat.net/~d/telia-4g.svg So you multiply the black line x4 to get a total for the down or up, then insert a fudge factor for the amount of acks that probably went into the down relative to the up and vice versa, and you sort of have your bandwidth number each way. I'm really impressed you can get ~72Mbit down and ~4Mbit up. (closer to 8 up when you consider acks) Note also the relationship between bandwidth and latency at T+22... On Thu, Oct 31, 2013 at 6:09 PM, Dave Taht wrote: > On Thu, Oct 31, 2013 at 5:27 AM, Stefan Alfredsson > wrote: >> On 10/31/13 03:32 , Dave Taht wrote: >>> >>> On Thu, Oct 31, 2013 at 03:04:23AM +0100, Stefan Alfredsson wrote: >>>> >>>> Hi, Stephen, the list, >>>> >>>> We're running a 3G/4G measurement site at Karlstad university, with >>>> mobile broadband subscriptions from the four major telecom operators >>>> in Sweden. I was curious to see how measurements in our network >>>> would compare to yours, so I did a measurement run with netalyzer. >>> >>> Groovy! >>> >>> Can I ask that you try something that can more likely stress >>> out that bus than netanlyzer? The netperf-wrappers tools >>> (notably the tcp_bidirectional and rrul test), use C, rather than java. >> >> >> Sure! Actually we've been doing netperf-wrapper rrul measurements 24/4 >> during July to September, roughly 3-4 times per day, in total over 3000 >> individual tests. >> >> I think there's a lot of interesting information in there, but I have no= t >> done aggregate analysis yet (eg. looking at time-of-day effects, long-te= rm >> trends, or the 10.000-students-arriving-in-august effect :-) ) > > if you are interested in measuring one way delays, owamp + a gps is > working out well. > >> I packed the rrul-dumps for one day (1 sept 2013) and put it here: >> http://www.cs.kau.se/stefalfr/3G4G/rrul-tests-2013-09-01.tgz . >> >> Here's an overview of this set, showing the per-run mean RTT, download a= nd >> upload throughput: >> >> [alfs@sa-pc]:/tmp/n/2013-06> for f in */*/*/rrul*json.gz ; do test=3D$(e= cho $f >> | cut -d / -f 1-2); netperf-wrapper -i $f -f stats|grep -A 4 'avg:'| awk= -v >> test=3D$test '/Mean:/ { if (NR=3D=3D3) { rtt=3D$2 } if (NR=3D=3D10) {dl= =3D$2} if (NR=3D=3D16) >> {ul=3D$2; printf "%s:\tRTT: %.1f ms DL: %.1f Mbit/s UL: %.1f Mbit/s\n", = test, >> rtt, dl, ul}}'; done >> Tele2/35G: RTT: 490.7 ms DL: 1.3 Mbit/s UL: 0.3 Mbit/s >> Tele2/35G: RTT: 498.0 ms DL: 6.1 Mbit/s UL: 0.4 Mbit/s >> Tele2/35G: RTT: 562.7 ms DL: 4.7 Mbit/s UL: 0.2 Mbit/s >> Tele2/35G: RTT: 641.5 ms DL: 3.5 Mbit/s UL: 0.3 Mbit/s >> Tele2/4G: RTT: 891.5 ms DL: 2.3 Mbit/s UL: 0.3 Mbit/s >> Tele2/4G: RTT: 389.0 ms DL: 20.8 Mbit/s UL: 1.0 Mbit/s >> Tele2/4G: RTT: 361.2 ms DL: 14.4 Mbit/s UL: 1.4 Mbit/s >> Tele2/4G: RTT: 359.8 ms DL: 17.3 Mbit/s UL: 1.3 Mbit/s >> Telenor/35G: RTT: 809.0 ms DL: 2.2 Mbit/s UL: 0.8 Mbit/s >> Telenor/35G: RTT: 377.3 ms DL: 3.8 Mbit/s UL: 0.5 Mbit/s >> Telenor/35G: RTT: 727.2 ms DL: 2.9 Mbit/s UL: 0.3 Mbit/s >> Telenor/35G: RTT: 341.7 ms DL: 0.1 Mbit/s UL: 0.2 Mbit/s >> Telenor/4G: RTT: 427.8 ms DL: 16.9 Mbit/s UL: 0.8 Mbit/s >> Telenor/4G: RTT: 378.8 ms DL: 17.9 Mbit/s UL: 1.2 Mbit/s >> Telenor/4G: RTT: 623.8 ms DL: 8.6 Mbit/s UL: 0.6 Mbit/s >> Telenor/4G: RTT: 321.5 ms DL: 18.2 Mbit/s UL: 1.0 Mbit/s >> Telia/35G: RTT: 686.9 ms DL: 4.8 Mbit/s UL: 0.2 Mbit/s >> Telia/35G: RTT: 709.0 ms DL: 6.1 Mbit/s UL: 0.2 Mbit/s >> Telia/35G: RTT: 546.9 ms DL: 3.8 Mbit/s UL: 0.2 Mbit/s >> Telia/35G: RTT: 653.0 ms DL: 3.1 Mbit/s UL: 0.3 Mbit/s >> Telia/4G: RTT: 233.6 ms DL: 14.1 Mbit/s UL: 1.1 Mbit/s >> Telia/4G: RTT: 324.2 ms DL: 14.6 Mbit/s UL: 1.1 Mbit/s >> Telia/4G: RTT: 381.5 ms DL: 15.2 Mbit/s UL: 1.2 Mbit/s >> Tre/35G: RTT: 312.2 ms DL: 3.1 Mbit/s UL: 0.6 Mbit/s >> Tre/35G: RTT: 376.1 ms DL: 4.1 Mbit/s UL: 0.8 Mbit/s >> Tre/35G: RTT: 442.0 ms DL: 2.9 Mbit/s UL: 0.6 Mbit/s >> Tre/35G: RTT: 294.0 ms DL: 2.9 Mbit/s UL: 0.6 Mbit/s >> Tre/4G: RTT: 471.2 ms DL: 6.9 Mbit/s UL: 0.7 Mbit/s >> Tre/4G: RTT: 396.2 ms DL: 6.8 Mbit/s UL: 0.7 Mbit/s >> Tre/4G: RTT: 237.5 ms DL: 7.2 Mbit/s UL: 0.7 Mbit/s >> Tre/4G: RTT: 321.5 ms DL: 6.9 Mbit/s UL: 0.7 Mbit/s > > Oh, that is wonderful data thank you! I confess to be more interested > in outliers than averages, and induced delay over base delay. > > However, I'm concerned that you are not getting a correct result with > your script above. > > I just took a quick look at telia-4g, using the 2100-2013-09-01T200038 > sample set, and looking at the raw (rrul) data: What I see here > > http://snapon.lab.bufferbloat.net/~d/telia-4g.svg > > is 60-80 ms of induced latency on this link, with 40ms of baseline > latency. Not a RTT of > 300! > > >>>> To summarize, the measured buffering is well below one second for >>>> all tests, although the results are not directly comparable since we >>>> are using directly attached USB modems (Huawei E392 with Ubuntu >>>> 12.04) >>> >>> ubuntu has backported fq_codel as far as 12.04. What kernel are you usi= ng? >> >> On the "mobile terminal" we have 3.2.0-54-generic-pae #82-Ubuntu SMP > > I am delighted you've been collecting this data long term. > > I do have a problem in that the 3.2 kernel predates all the > bufferbloat related fixes made in the 4 years since that release. > we've been working our butts off here... :( > > Any chance you can update to 3.10.x or later for both this (driver and > buffering fixes) and your fixed host? (tcp fixes). And it's generally > been my hope people would try fq_codel (or sfq or pie or red) on such > links, too... > > There has been very little done to optimize usb-net, although we > tried. If this is using the ppp driver, that was improved > substantially around 3.6? > >> The fixed host ("tptest") runs a self-compiled stock kernel 3.4.1 with >> web10g patches, since we were interested in extracting TCP variable data= not >> available from packet traces. >> >>>> instead of a separate router box. However, the numbers might >>>> be interesting in a more general perspective. >>> >>> Yes, they are. These are repeatable? >>> >>> >> >> For the netalyzer I've only done singular tests, but it can easily be >> repeated if needed (probably not). >> The mean of all mean RTTs for 4G, all operators, all times, is 334 ms, a= nd >> for 3.5G it's 573 ms. >> This indicates that results are repeatable, and the set from 2013-09-01 = is >> representative. > > except that my plot shows differently... > > and you are running an ancient kernel, yes. Something of a major > variable, that... and if you could let us know the usb path to the > various underlying drivers we can poke into matters there? > >> >> "below one second" was in relation to the >5 second RTT's, ideally it sh= ould >> be much lower of course. As a side note, we also do measurements in unlo= aded >> networks, and for 4G we get around 40-60 ms RTT (ICMP and TCP packets). >> >> Also, you might be interested in a paper we published in June, that repo= rts >> on measurements of bloat for short flows in relation to different conges= tion >> control algorithms, in 3G, 3.5G and 4G (but for a single operator). >> See http://urn.kb.se/resolve?urn=3Durn:nbn:se:kau:diva-27779 >> with fulltext at http://dx.doi.org/10.1109/WoWMoM.2013.6583408. >> I have also put a local copy (for personal use etc) at >> http://www.cs.kau.se/stefalfr/3G4G/alfredsson-bufferbloat-wowmom13.pdf >> >> >> Regards, >> Stefan >> >> -- >> Stefan Alfredsson, PhD Tel: +46 (0) 54 700 1668 >> Datavetenskap Kontor: 21F-425 (Hus Vanern) >> Karlstads universitet PGP 0xB19B4B16 >> SE-651 88 Karlstad http://www.cs.kau.se/stefalfr/ >> >> >> _______________________________________________ >> Bloat mailing list >> Bloat@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/bloat > > > > -- > Dave T=E4ht > > Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscrib= e.html --=20 Dave T=E4ht Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.= html