From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-x22b.google.com (mail-la0-x22b.google.com [IPv6:2a00:1450:4010:c03::22b]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id D82FA21F209 for ; Sat, 13 Sep 2014 16:32:29 -0700 (PDT) Received: by mail-la0-f43.google.com with SMTP id gi9so2836410lab.2 for ; Sat, 13 Sep 2014 16:32:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=VNVWmPX9SL0gzxnjIsNPtE3bZX76RtVT0OYRUrdzrq4=; b=G6UKS1tv71y+RxySySWr7AwBXnRChwVOkgVdGmvjXvnK/f2MSQzyXcSFgHJn6g5DY7 EnqQCXz6JosbK0uZopAmQF04K48Ou2Q0SmWQbIUUi4V9ot1b+mxQ9bzD8n0BIhaZoU1i WujVMHLswxrfoRuCOacUPhq9N1fYdq2c5nRgTibhbNRHAw2r4iNEzP8LzQe6dfahwtg7 qbkny0v8qm/R3dr6BEHzav9nafBeZFJNN/StLW03D0cVbbMNFmbNXZxi3K6p18WzeXUK MwiUqVO6RZE5ljG1pMD/Uec0yy7bxjpF6Bc6R7XG+dSnpFH6b9o32RQD7mcKarrLMCiI 1TvQ== X-Received: by 10.152.21.42 with SMTP id s10mr18776268lae.61.1410651146433; Sat, 13 Sep 2014 16:32:26 -0700 (PDT) Received: from bass.home.chromatix.fi (87-93-10-22.bb.dnainternet.fi. [87.93.10.22]) by mx.google.com with ESMTPSA id n4sm2623461lah.2.2014.09.13.16.32.24 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 13 Sep 2014 16:32:25 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=windows-1252 From: Jonathan Morton In-Reply-To: <20418644-AB62-43AE-A09E-5F85ED42DBF4@gmx.de> Date: Sun, 14 Sep 2014 02:32:23 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140913194126.5B0D1406062@ip-64-139-1-69.sjc.megapath.net> <20418644-AB62-43AE-A09E-5F85ED42DBF4@gmx.de> To: Sebastian Moeller X-Mailer: Apple Mail (2.1085) Cc: Hal Murray , bloat@lists.bufferbloat.net Subject: Re: [Bloat] Measuring Latency X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Sep 2014 23:32:58 -0000 >>> When reading it, it strikes me, that you don't directly tell them = what to >>> do; e.g. add a latency test during upload and download. ... >>=20 >> Does round trip latency have enough info, or do you need to know how = much is=20 >> contributed by each direction? >=20 > RTT is fine, uni-directional transfer time would be too good to be = true ;). To expand on this: to measure one-way delay, you would need finely = synchronised clocks (to within a couple of ms) at both ends. The = average consumer doesn't have that sort of setup - not even if they = happen to use NTP. So it's not a goal worth pursuing at this level - = save it for scientific investigations, where the necessary effort and = equipment can be made available. >> If I gave you a large collection of latency data from a test run, how = do you=20 >> reduce it to something simple that a marketer could compare with the = results=20 >> from another test run? >=20 > I believe the added latency under load would be a marketable = number, but we had a discussion in the past where it was argued that = marketing wants a number which increases with goodness, so larger =3D = better, something the raw difference is not going to deliver=85 The obvious solution is to report the inverse value in Hz, a figure of = merit that gamers are familiar with (if not exactly in this context). For example, I occasionally play World Of Tanks, which has a latency = meter (in ms) in the top corner, and I can roughly invert that in my = head - if I set my shaper too optimistically, I get something like 500ms = if something is updating in the background, but this goes down = immediately to 100ms once I adjust the setting to a more conservative = value - it's a 3G connection, so it's not going to get much better than = that. The corresponding inverted readings would be 2Hz (where I miss = most of my shots) and 10Hz (where I can carry the team). It's probably = worth reporting to one decimal place. WoT isn't exactly the "twitchiest" of online games, either - have you = any idea how long it takes to aim a tank gun? Even so, when some tanks = can move at over 30km/h, a half-second difference in position is a whole = tank length, so with the slower response I no longer know *where* or = *when* to fire, unless the target is a sitting duck. Even though my = framerate is at 30Hz or more and appears smooth, my performance as a = player is dependent on the Internet's response frequency, because that = is lower. So here's the outline of a step-by-step methodology: - Prepare space for a list of latency measurements. Each measurement = needs to be tagged with information about what else was going on at the = same time, ie. idle/upload/download/both. Some latency probes may be = lost, and this fact should also be recorded on a per-tag basis. - Start taking latency measurements, tagging as idle to begin with. = Keep on doing so continuously, changing the tag as required, until = several seconds after the bandwidth measurements are complete. - Latency measurements should be taken sufficiently frequently (several = times a second is okay) that there will be at least a hundred samples = with each tag, and the frequency of sampling should not change during = the test. Each probe must be tagged with a unique ID, so that losses or = re-ordering of packets can be detected and don't confuse the = measurement. - The latency probes should use UDP, not ICMP. They should also use the = same Diffserv/TOS tag as the bandwidth measurement traffic; the default = "best-effort" tag is fine. - To satisfy the above requirements, the latency tester must *not* wait = for a previous reply to return before sending the next one. It should = send at regular intervals based on wall-clock time. But don't send so = many probes that they themselves clog the link. - Once several seconds of "idle" samples are recorded, start the = download test. Change the tag to "download" at this point. - The download test is complete when all the data sent by the server has = reached the client. Change the tag back to "idle" at this moment. - Repeat the previous two steps for the upload measurement, using the = "upload" tag. - Repeat again, but perform upload and download tests at the same time = (predicting, if necessary, that the bandwidth in each direction should = be similar to that previously measured), and use the "both" tag. = Uploads and downloads tend to interfere with each other when the loaded = response frequency is poor, so don't simply assume that the results will = be the same as in the individual tests - *measure* it. - Once a few seconds of "idle" samples have been taken, stop measuring = and start analysis. - Separate the list of latency samples by tag, and sort the four = resulting lists in ascending order. - In each list, find the sample nearest 98% of the way through the list. = This is the "98th percentile", a good way of finding the "highest" = value while discarding irrelevant outliers. The highest latency is the = one that users will notice. Typically poor results: idle 50ms, download = 250ms, upload 500ms, both 800ms. - Correct the 98-percentile latencies for packet loss, by multiplying it = by the number of probes *sent* with the appropriate tag, and then = dividing it by the number of probes *received* with that tag. It is not = necessary to report packet loss in any other way, *except* for the = "idle" tag. - Convert the corrected 98-percentile latencies into "response = frequencies" by dividing one second by them. The typical figures above = would become: idle 20.0 Hz, download 4.0 Hz, upload 2.0 Hz, both 1.25 Hz = - assuming there was no packet loss. These figures are comparable in = meaning and importance to "frames per second" figures in games. - Report these response frequencies, to a precision of at least one = decimal place, alongside and with equal visual importance to, the = bandwidth figures. For example: IDLE: Response 20.0 Hz Packet loss 0.00 % DOWNLOAD: Response 4.0 Hz Bandwidth 20.00 Mbps UPLOAD: Response 2.0 Hz Bandwidth 2.56 Mbps BIDIRECT: Response 1.3 Hz Bandwidth 15.23 / 2.35 Mbps - Improving the response figures in the loaded condition will probably = also improve the *bidirectional* bandwidth figures as a side-effect, = while having a minimal effect on the *unidirectional* figures. A = connection with such qualities can legitimately be described as = supporting multiple activities at the same time. A connection with the = example figures shown above can *not*. The next trick is getting across the importance of acknowledging that = more than one person in the average household wants to use the Internet = at the same time these days, and they often want to do different things = from each other. In this case, the simplest valid argument probably has = a lot going for it. An illustration might help to get the concept across. A household with = four users in different rooms: father in the study downloading a video, = mother in the kitchen on the (VoIP) phone, son in his bedroom playing a = game, daughter in her bedroom uploading photos. All via a single = black-box modem and connection. Captions should emphasise that mother = and son both need low latency (high response frequency), while father = and daughter need high bandwidth (in opposite directions!), and that = they're doing all these things at the same time. - Jonathan Morton