<font face="arial" size="2"><p style="margin:0;padding:0;">Why is the DNS PLR so high? 1% is pretty depressing.</p>
<p style="margin:0;padding:0;"> </p>
<p style="margin:0;padding:0;">Also, it seems odd to eliminate 19% of the content retrieval because the tail is fat and long rather than short. Wouldn't it be better to have 1000 servers?</p>
<p style="margin:0;padding:0;"> </p>
<p style="margin:0;padding:0;"> </p>
<!--WM_COMPOSE_SIGNATURE_START--><!--WM_COMPOSE_SIGNATURE_END-->
<p style="margin:0;padding:0;"><br /><br />On Friday, April 18, 2014 2:15pm, "Greg White" <g.white@CableLabs.com> said:<br /><br /></p>
<div id="SafeStyles1397846754">
<p style="margin:0;padding:0;">> Dave,<br />> <br />> We used the 25k object size for a short time back in 2012 until we had<br />> resources to build a more advanced model (appendix A). I did a bunch of<br />> captures of real web pages back in 2011 and compared the object size<br />> statistics to models that I'd seen published. Lognormal didn't seem to be<br />> *exactly* right, but it wasn't a bad fit to what I saw. I've attached a<br />> CDF.<br />> <br />> The choice of 4 servers was based somewhat on logistics, and also on a<br />> finding that across our data set, the average web page retrieved 81% of<br />> its resources from the top 4 servers. Increasing to 5 servers only<br />> increased that percentage to 84%.<br />> <br />> The choice of RTTs also came from the web traffic captures. I saw<br />> RTTmin=16ms, RTTmean=53.8ms, RTTmax=134ms.<br />> <br />> Much of this can be found in<br />> https://tools.ietf.org/html/draft-white-httpbis-spdy-analysis-00<br />> <br />> In many of the cases that we've simulated, the packet drop probability is<br />> less than 1% for DNS packets. In our web model, there are a total of 4<br />> servers, so 4 DNS lookups assuming none of the addresses are cached. If<br />> PLR = 1%, there would be a 3.9% chance of losing one or more DNS packets<br />> (with a resulting ~5 second additional delay on load time). I've probably<br />> oversimplified this, but Kathie N. and I made the call that it would be<br />> significantly easier to just do this math than to build a dns<br />> implementation in ns2. We've open sourced the web model (it's on Kathie's<br />> web page and will be part of ns2.36) with an encouragement to the<br />> community to improve on it. If you'd like to port it to ns3 and add a dns<br />> model, that would be fantastic.<br />> <br />> -Greg<br />> <br />> <br />> On 4/17/14, 3:07 PM, "Dave Taht" <dave.taht@gmail.com> wrote:<br />> <br />> >On Thu, Apr 17, 2014 at 12:01 PM, William Chan (陈智昌)<br />> ><willchan@chromium.org> wrote:<br />> >> Speaking as the primary Chromium developer in charge of this relevant<br />> >>code,<br />> >> I would like to caution putting too much trust in the numbers<br />> >>generated. Any<br />> >> statistical claims about the numbers are probably unreasonable to make.<br />> ><br />> >Sigh. Other benchmarks such as the apache ("ab") benchmark<br />> >are primarily designed as stress testers for web servers, not as realistic<br />> >traffic. Modern web traffic has such a high level of dynamicism in it,<br />> >that static web page loads along any distribution, seem insufficient,<br />> >passive analysis of aggregated traffic "feels" incorrect relative to the<br />> >sorts of home and small business traffic I've seen, and so on.<br />> ><br />> >Famous papers, such as this one:<br />> ><br />> >http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-leland.pdf<br />> ><br />> >Seem possibly irrelevant to draw conclusions from given the kind<br />> >of data they analysed and proceeding from an incorrect model or<br />> >gut feel for how the network behaves today seems to be foolish.<br />> ><br />> >Even the most basic of tools, such as httping, had three basic bugs<br />> >that I found in a few minutes of trying to come up with some basic<br />> >behaviors yesterday:<br />> ><br />> >https://lists.bufferbloat.net/pipermail/bloat/2014-April/001890.html<br />> ><br />> >Those are going to be a lot easier to fix than diving into the chromium<br />> >codebase!<br />> ><br />> >There are very few tools worth trusting, and I am always dubious<br />> >of papers that publish results with unavailable tools and data. The only<br />> >tools I have any faith in for network analysis are netperf,<br />> >netperf-wrapper,<br />> >tcpdump and xplot.org, and to a large extent wireshark. Toke and I have<br />> >been tearing apart d-itg and I hope to one day be able to add that to<br />> >my trustable list... but better tools are needed!<br />> ><br />> >Tools that I don't have a lot of faith in include that, iperf, anything<br />> >written<br />> >in java or other high level languages, speedtest.net, and things like<br />> >shaperprobe.<br />> ><br />> >Have very little faith in ns2, slightly more in ns3, and I've been meaning<br />> >to look over the mininet and other simulators whenever I got some spare<br />> >time; the mininet results stanford gets seem pretty reasonable and I<br />> >adore their reproducing results effort. Haven't explored ndt, keep meaning<br />> >to...<br />> ><br />> >> Reasons:<br />> >> * We don't actively maintain this code. It's behind the command line<br />> >>flags.<br />> >> They are broken. The fact that it still results in numbers on the<br />> >>benchmark<br />> >> extension is an example of where unmaintained code doesn't have the UI<br />> >> disabled, even though the internal workings of the code fail to<br />> >>guarantee<br />> >> correct operation. We haven't disabled it because, well, it's<br />> >>unmaintained.<br />> ><br />> >As I mentioned I was gearing up for a hacking run...<br />> ><br />> >The vast majority of results I look at are actually obtained via<br />> >looking at packet captures. I mostly use benchmarks as abstractions<br />> >to see if they make some sense relative to the captures and tend<br />> >to compare different benchmarks against each other.<br />> ><br />> >I realize others don't go into that level of detail, so you have given<br />> >fair warning! In our case we used the web page benchmarker as<br />> >a means to try and rapidly acquire some typical distributions of<br />> >get and tcp stream requests from things like the alexa top 1000,<br />> >and as a way to A/B different aqm/packet scheduling setups.<br />> ><br />> >... but the only easily publishable results were from the benchmark<br />> >itself,<br />> >and we (reluctantly) only published one graph from all the work that<br />> >went into it 2+ years back and used it as a test driver for the famous<br />> >ietf video, comparing two identical boxes running it at the same time<br />> >under different network conditions:<br />> ><br />> >https://www.bufferbloat.net/projects/cerowrt/wiki/Bloat-videos#IETF-demo-s<br />> >ide-by-side-of-a-normal-cable-modem-vs-fq_codel<br />> ><br />> >from what I fiddled with today, it is at least still useful for that?<br />> ><br />> >moving on...<br />> ><br />> >The web model in the cablelabs work doesn't look much like my captures,<br />> >in addition to not modeling dns at all, and using a smaller IW than google<br />> >it looks like this:<br />> ><br />> >>> Model single user web page download as follows:<br />> ><br />> >>> - Web page modeled as single HTML page + 100 objects spread evenly<br />> >>> across 4 servers. Web object sizes are currently fixed at 25 kB<br />> each,<br />> >>> whereas the initial HTML page is 100 kB. Appendix A provides an<br />> >>> alternative page model that may be explored in future work.<br />> ><br />> >Where what I see is a huge number of stuff that fits into a single<br />> >iw10 slow start episode and some level of pipelining on larger stuff, so<br />> >that a<br />> >large number of object sizes of less than 7k with a lightly tailed<br />> >distribution<br />> >outside of that makes more sense.<br />> ><br />> >(I'm not staring at appendix A right now, I'm under the impression<br />> > it was better)<br />> ><br />> >I certainly would like more suggestions for models and types<br />> >of web traffic, as well as simulation of https + pfs traffic,<br />> >spdy, quic, etc....<br />> ><br />> >>> - Server RTTs set as follows (20 ms, 30 ms, 50 ms, 100 ms).<br />> ><br />> >Server RTTs from my own web history tend to be lower than 50ms.<br />> ><br />> >>> - Initial HTTP GET to retrieve a moderately sized object (100 kB<br />> HTML<br />> >>> page) from server 1.<br />> ><br />> >An initial GET to google fits into iw10 - it's about 7k.<br />> ><br />> >>> - Once initial HTTP GET completes, initiate 24 simultaneous HTTP<br />> GETs<br />> >>> (via separate TCP connections), 6 connections each to 4 different<br />> >>> server nodes<br />> ><br />> >I usually don't see more than 15. and certainly not 25k sized objects.<br />> ><br />> > > - Once each individual HTTP GET completes, initiate a subsequent GET<br />> >> to the same server, until 25 objects have been retrieved from each<br />> >> server.<br />> ><br />> ><br />> >> * We don't make sure to flush all the network state in between runs, so<br />> >>if<br />> >> you're using that option, don't trust it to work.<br />> ><br />> >The typical scenario we used was a run against dozens or hundreds of urls,<br />> >capturing traffic, while varying network conditions.<br />> ><br />> >Regarded the first run as the most interesting.<br />> ><br />> >Can exit the browser and restart after a run like that.<br />> ><br />> >At moment, merely plan to use the tool primarily to survey various<br />> >web sites and load times while doing packet captures. Hope was<br />> >to get valid data from the network portion of the load, tho...<br />> ><br />> >> * If you have an advanced Chromium setup, this definitely does not<br />> >>work. I<br />> >> advise using the benchmark extension only with a separate Chromium<br />> >>profile<br />> >> for testing purposes. Our flushing of sockets, caches, etc does not<br />> >>actually<br />> >> work correctly when you use the Chromium multiprofile feature and also<br />> >>fails<br />> >> to flush lots of our other network caches.<br />> ><br />> >noted.<br />> ><br />> ><br />> >> * No one on Chromium really believes the time to paint numbers that we<br />> >> output :) It's complicated. Our graphics stack is complicated. The time<br />> >>from<br />> ><br />> >I actually care only about time-to-full layout as that's a core network<br />> >effect...<br />> ><br />> >> when Blink thinks it painted to when the GPU actually blits to the<br />> >>screen<br />> >> cannot currently be corroborated with any high degree of accuracy from<br />> >> within our code.<br />> ><br />> >> * It has not been maintained since 2010. It is quite likely there are<br />> >>many<br />> >> other subtle inaccuracies here.<br />> ><br />> >Grok.<br />> ><br />> >> In short, while you can expect it to give you a very high level<br />> >> understanding of performance issues, I advise against placing<br />> >>non-trivial<br />> >> confidence in the accuracy of the numbers generated by the benchmark<br />> >> extension. The fact that numbers are produced by the extension should<br />> >>not be<br />> >> treated as evidence that the extension actually functions correctly.<br />> ><br />> >OK, noted. Still delighted to be able to have a simple load generator<br />> >that exercises the browsers and generates some results, however<br />> >dubious.<br />> ><br />> >><br />> >> Cheers.<br />> >><br />> >><br />> >> On Thu, Apr 17, 2014 at 10:49 AM, Dave Taht <dave.taht@gmail.com><br />> wrote:<br />> >>><br />> >>> Getting a grip on real web page load time behavior in an age of<br />> >>> sharded websites,<br />> >>> dozens of dns lookups, javascript, and fairly random behavior in ad<br />> >>> services<br />> >>> and cdns against how a modern browsers behaves is very, very hard.<br />> >>><br />> >>> it turns out if you run<br />> >>><br />> >>> google-chrome --enable-benchmarking --enable-net-benchmarking<br />> >>><br />> >>> (Mac users have to embed these options in their startup script - see<br />> >>> http://www.chromium.org/developers/how-tos/run-chromium-with-flags<br />> )<br />> >>><br />> >>> enable developer options and install and run the chrome web page<br />> >>> benchmarker,<br />> >>> (<br />> >>><br />> >>>https://chrome.google.com/webstore/detail/page-benchmarker/channimfdomah<br />> >>>ekjcahlbpccbgaopjll?hl=en<br />> >>> )<br />> >>><br />> >>> that it works (at least for me, on a brief test of the latest<br />> chrome,<br />> >>>on<br />> >>> linux.<br />> >>> Can someone try windows and mac?)<br />> >>><br />> >>> You can then feed in a list of urls to test against, and post<br />> process<br />> >>> the resulting .csv file to your hearts content. We used to use this<br />> >>> benchmark a lot while trying to characterise typical web behaviors<br />> >>> under aqm and packet scheduling systems under load. Running<br />> >>> it simultaneously with a rrul test or one of the simpler tcp upload<br />> or<br />> >>> download<br />> >>> tests in the rrul suite was often quite interesting.<br />> >>><br />> >>> It turned out the doc has been wrong a while as to the name of the<br />> >>>second<br />> >>> command lnie option. I was gearing up mentally for having to look at<br />> >>> the source....<br />> >>><br />> >>> http://code.google.com/p/chromium/issues/detail?id=338705<br />> >>><br />> >>> /me happy<br />> >>><br />> >>> --<br />> >>> Dave Täht<br />> >>><br />> >>> Heartbleed POC on wifi campus networks with EAP auth:<br />> >>> http://www.eduroam.edu.au/advisory.html<br />> >>><br />> >>> _______________________________________________<br />> >>> aqm mailing list<br />> >>> aqm@ietf.org<br />> >>> https://www.ietf.org/mailman/listinfo/aqm<br />> >><br />> >><br />> ><br />> ><br />> ><br />> >--<br />> >Dave Täht<br />> ><br />> >NSFW:<br />> >https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indec<br />> >ent.article<br />> ><br />> >_______________________________________________<br />> >aqm mailing list<br />> >aqm@ietf.org<br />> >https://www.ietf.org/mailman/listinfo/aqm<br />> <br />> _______________________________________________<br />> Cerowrt-devel mailing list<br />> Cerowrt-devel@lists.bufferbloat.net<br />> https://lists.bufferbloat.net/listinfo/cerowrt-devel<br />></p>
</div></font>