From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from eu1sys200aog101.obsmtp.com (eu1sys200aog101.obsmtp.com [207.126.144.111]) by huchra.bufferbloat.net (Postfix) with SMTP id CAAC021F25B for ; Thu, 10 Apr 2014 13:50:05 -0700 (PDT) Received: from mail-pb0-f42.google.com ([209.85.160.42]) (using TLSv1) by eu1sys200aob101.postini.com ([207.126.147.11]) with SMTP ID DSNKU0cD+iNzaXJS/fImABX14SBDxNNGKdEt@postini.com; Thu, 10 Apr 2014 20:50:06 UTC Received: by mail-pb0-f42.google.com with SMTP id rr13so4491883pbb.1 for ; Thu, 10 Apr 2014 13:50:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=FwkVagaZpu8pLTN4KtBoWOWQb9sVuqxKCLPSVXIqbGA=; b=QH21raMELZ097sFhvd+tfOu/ne/ub2WL49bd5ZCu+ZMk82oihVGBOzIaTUPrBQHKMI RXfxnM4CEdUWIc4E/JkJ63+0YknFNVcaGVkZFl7pag4zu0mRcLjw/6szhHUvZP3Jw+in FfJG0wbhUBX/EelHWCOB6iiolN5wXulDwMIj15Ruc2kWv7cMJwSQSU1+DKusobEQvnql R+PgeIR+4t2+ro63wD6LKhuo/ZyrMs4UYB0fXUH28Q/rISMyBMjjyHsXirOyo8EhslmC 0Hoa4dMarhUOeZHHV3cY2UAoj4grqGp1/BaiutYEVI3JOdxt9mxeJ+TkcX4fy7i9/yKy DFqQ== X-Gm-Message-State: ALoCoQk3/BwAcSLeNJe58FLCW3VYLSnOEQYe0m/AQ8hQYIe2Rf40ALoIYHSLcSWKCmi/P9cFk9DDqurjXnyQ8xfLWV+7PIwJXW3oDsed2/g25demxghjuqfRFtz+8vXSba2PAdkfbtK/6zPXl+A75Hf3l94ZZiCMXA== X-Received: by 10.68.194.65 with SMTP id hu1mr664981pbc.151.1397163001218; Thu, 10 Apr 2014 13:50:01 -0700 (PDT) X-Received: by 10.68.194.65 with SMTP id hu1mr664972pbc.151.1397163001096; Thu, 10 Apr 2014 13:50:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.70.60.196 with HTTP; Thu, 10 Apr 2014 13:49:40 -0700 (PDT) In-Reply-To: References: From: Martin Geddes Date: Thu, 10 Apr 2014 21:49:40 +0100 Message-ID: To: bloat Content-Type: multipart/alternative; boundary=047d7b15a9595fbce804f6b65c24 Subject: Re: [Bloat] modest scale local measurements X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 20:50:07 -0000 --047d7b15a9595fbce804f6b65c24 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable With some encouragement from Dave, I'm posting a link to a slide deck with a relevant set of metrics that many of you may find of interest. "Will my Netflix/Lync/whatever work?" is a question that many would like to know in advance of committing to a multi-year contract for some piece of network access. We perform that kind of analysis for large fixed and mobile operators as our day job. The relevant slides are 8 and 9 in this presentation . It's an annotated version of a technical sales deck that was designed to be talked through, but should make reasonable sense just being read. The *QoE breach metric* (red crosses) is a synthetic one that: - takes the *probability distribution* of loss and delay - compares this measurement against a (stochastic) requirement "contract" (*quality transport agreement*) to quantify the *performance hazard* - and as result, the chart indicates the likelihood of an application falling below (or above) its required QoE. This process generalises the predictive PESQ type of work done for voice to any application. Red crosses mean the QoE of the application is at risk. (In this case it happens to be voice over a 100Mbps fibre link in a collector arc carrying multiple types of traffic as a single class of service). The green line is the daily usage cycle measured using 5 minute (IIRC) average link utilisation data. There is a correlation, but not a causative relationship. What makes this data interesting is the robustness of the measurement and mathematics behind this QoE measure. The metric used (=CE=94Q) is the only (possible) network measure that is also a strong QoE proxy (as relates to application performance). In this case, we were measuring for a client whether over-provisioning was an efficient and effective approach to dealing with QoE issues. The result reflects similar measures from other fixed and mobile operators: over-provisioning is not an effective means of resolving scheduling issues. This should not be any news to anyone involved in the AQM/bufferbloat space; scheduling choices really do matter, and getting them wrong does affect the customer experience. Hence the client is engaging in improving their scheduling choices, rather than spending capex on capacity that doesn't fix the problem. The tools used to generate this data are described in How to X-ray a telecoms network. Critically, you need to measure the *same test stream packet* passing *multiple points*, and produce the *distribution*, and not use averages (e.g. "jitter" or "bandwidth"). How we temporally decompose the data is described in thispresentation. I have a very long deck I'm still working on that introduces the =CE=94Q resource model, predictive calculus, and network performance science. The terms used are unfamiliar because producing this kind of data has required the development of a fundamental new branch of mathematics to describe the improper random variables and their composite behaviour. Please don't hesitate to ask questions. I hope the above is of interest and value, since it represents (to the best of our knowledge) the state of the art in measurement and performance analysis techniques. For more information on them, see www.pnsol.com. Martin Geddes www.martingeddes.com LinkedIn Twitter Mobile: +44 7957 499219 Skype: mgeddes On Mon, Apr 7, 2014 at 7:18 PM, Dave Taht wrote: > This weekend for the first time ever I saw netflix become unusable for > several hours on a friday night. On DSL line capable of over 6mbits > down, it slowed below 1.2Mbit and frequently stopped. > > For a change I was NOT interested in leaping up from the couch to > capture traffic (read a book instead!) to see what was up, but it did > set me to thinking about monitoring congestion and delay on local > network segments, like those you find in cable, where you frequently > see arp requests go by anyway and thus can build > an partial picture of the rest of the wire. > > I was seriously impressed and somewhat scared by the power of zmap: > > https://zmap.io/ > > But it would be interesting to be able to collect a periodic picture > of the universe one hop beyond the local gateway. > > -- > Dave T=C3=A4ht > > NSFW: > https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_inde= cent.article > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --047d7b15a9595fbce804f6b65c24 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
With some encouragement from Dave, I'm posting a link = to a slide deck with a relevant set of metrics that many of you may find of= interest. "Will my Netflix/Lync/whatever work?" is a question th= at many would like to know in advance of committing to a multi-year contrac= t for some piece of network access. We perform that kind of analysis for la= rge fixed and mobile operators as our day job.

The relevant slides are 8 and 9 in=C2=A0this presentation.=C2=A0It's an annotated version of a technical s= ales deck that was designed to be talked through, but should make reasonabl= e sense just being read.

The QoE breach metric=C2=A0(red crosses) is a sy= nthetic one that:
- takes the probability distribution of = loss and delay
- compares this measurement against a (stochastic)= requirement "contract" (quality transport agreement)=C2= =A0to quantify the=C2=A0performance hazard=C2=A0
- and as result, the chart indicates the likelihood of an application = falling below (or above) its required QoE.

This pr= ocess generalises the predictive PESQ type of work done for voice to any ap= plication.

Red crosses mean the QoE of the application is at= risk. (In this case it happens to be voice over a 100Mbps fibre link in a = collector arc carrying multiple types of traffic as a single class of servi= ce). The green line is the daily usage cycle measured using 5 minute=C2=A0(= IIRC)=C2=A0average link utilisation data. There is a correlation, but not a= causative relationship.

What makes this data interesting is the robustness of t= he measurement and mathematics behind this QoE measure. The metric used (= =CE=94Q) is the only (possible) network measure that is also a strong QoE p= roxy (as relates to application performance).

In this case, we were measuring for a client whet= her over-provisioning was an efficient and effective approach to dealing wi= th QoE issues. The result reflects similar measures from other fixed and mo= bile operators: over-provisioning is not an effective means of resolving sc= heduling issues. This should not be any news to anyone involved in the AQM/= bufferbloat space; scheduling choices really do matter, and getting them wr= ong does affect the customer experience. Hence the client is engaging in im= proving their scheduling choices, rather than spending capex on capacity th= at doesn't fix the problem.

The tools used to generate this data are described in= =C2=A0How to X-ray a telecoms network. Critically,= you need to measure the same test stream packet=C2=A0passing=C2=A0<= i>multiple=C2=A0points, and produce the distribution, and not us= e averages (e.g. "jitter" or "bandwidth"). How we tempo= rally decompose the data is described in this presentation.

I have a very long deck I'm still working on that i= ntroduces the =CE=94Q resource model, predictive calculus, and network perf= ormance science. The terms used are unfamiliar because producing this kind = of data has required the development of a fundamental new branch of mathema= tics to describe the improper random variables and their composite behaviou= r.

Martin Geddes
www.martingeddes.com=C2=A0Li= nkedIn=C2=A0Twitter= =C2=A0M= obile: +44 7957 499219=C2=A0Skype: mgeddes=C2=A0



On Mon, Apr 7, 2014 at 7:18 PM, Dave Tah= t <dave.taht@gmail.com> wrote:
This weekend for the first time ever I saw netflix become unusable for
several hours on a friday night. On DSL line capable of over 6mbits
down, it slowed below 1.2Mbit and frequently stopped.

For a change I was NOT interested in leaping up from the couch to
capture traffic (read a book instead!) to see what was up, but it did
set me to thinking about monitoring congestion and delay on local
network segments, like those you find in cable, where you frequently
see arp requests go by anyway and thus can build
an partial picture of the rest of the wire.

I was seriously impressed and somewhat scared by the power of zmap:

https://zmap.io/

But it would be interesting to be able to collect a periodic picture
of the universe one hop beyond the local gateway.

--
Dave T=C3=A4ht

NSFW: https://w2.eff.org/Censorshi= p/Internet_censorship_bills/russell_0296_indecent.article
_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net
= https://lists.bufferbloat.net/listinfo/bloat

--047d7b15a9595fbce804f6b65c24--