[Bloat] modest scale local measurements

[Martin Geddes]

With some encouragement from Dave, I'm posting a link to a slide deck with
a relevant set of metrics that many of you may find of interest. "Will my
Netflix/Lync/whatever work?" is a question that many would like to know in
advance of committing to a multi-year contract for some piece of network
access. We perform that kind of analysis for large fixed and mobile
operators as our day job.

The relevant slides are 8 and 9 in this
presentation<http://www.slideshare.net/mgeddes/network-performance-optimisation-using-highfidelity-measures>
. It's an annotated version of a technical sales deck that was designed to
be talked through, but should make reasonable sense just being read.

The *QoE breach metric* (red crosses) is a synthetic one that:
- takes the *probability distribution* of loss and delay
- compares this measurement against a (stochastic) requirement
"contract" (*quality
transport agreement*) to quantify the *performance hazard*
- and as result, the chart indicates the likelihood of an application
falling below (or above) its required QoE.

This process generalises the predictive PESQ type of work done for voice to
any application.

Red crosses mean the QoE of the application is at risk. (In this case it
happens to be voice over a 100Mbps fibre link in a collector arc carrying
multiple types of traffic as a single class of service). The green line is
the daily usage cycle measured using 5 minute (IIRC) average link
utilisation data. There is a correlation, but not a causative relationship.

What makes this data interesting is the robustness of the measurement and
mathematics behind this QoE measure. The metric used (ΔQ) is the only
(possible) network measure that is also a strong QoE proxy (as relates to
application performance).

In this case, we were measuring for a client whether over-provisioning was
an efficient and effective approach to dealing with QoE issues. The result
reflects similar measures from other fixed and mobile operators:
over-provisioning is not an effective means of resolving scheduling issues.
This should not be any news to anyone involved in the AQM/bufferbloat
space; scheduling choices really do matter, and getting them wrong does
affect the customer experience. Hence the client is engaging in improving
their scheduling choices, rather than spending capex on capacity that
doesn't fix the problem.

The tools used to generate this data are described in How to X-ray a
telecoms network<http://us1.campaign-archive1.com/?u=f105fd56904428bca9da44a82&id=b3f51d6467>.
Critically, you need to measure the *same test stream packet* passing
*multiple points*, and produce the *distribution*, and not use averages
(e.g. "jitter" or "bandwidth"). How we temporally decompose the data is
described in this<http://www.slideshare.net/mgeddes/fundamentals-of-network-performance-engineering-18548490>presentation.

I have a very long deck I'm still working on that introduces the ΔQ
resource model, predictive calculus, and network performance science. The
terms used are unfamiliar because producing this kind of data has required
the development of a fundamental new branch of mathematics to describe the
improper random variables and their composite behaviour.

Please don't hesitate to ask questions. I hope the above is of interest and
value, since it represents (to the best of our knowledge) the state of the
art in measurement and performance analysis techniques. For more
information on them, see www.pnsol.com.

Martin Geddes
www.martingeddes.com LinkedIn <http://www.linkedin.com/in/mgeddes>
Twitter<https://twitter.com/martingeddes>
 Mobile: +44 7957 499219 Skype: mgeddes



On Mon, Apr 7, 2014 at 7:18 PM, Dave Taht <dave.taht at gmail.com> wrote:

> This weekend for the first time ever I saw netflix become unusable for
> several hours on a friday night. On DSL line capable of over 6mbits
> down, it slowed below 1.2Mbit and frequently stopped.
>
> For a change I was NOT interested in leaping up from the couch to
> capture traffic (read a book instead!) to see what was up, but it did
> set me to thinking about monitoring congestion and delay on local
> network segments, like those you find in cable, where you frequently
> see arp requests go by anyway and thus can build
> an partial picture of the rest of the wire.
>
> I was seriously impressed and somewhat scared by the power of zmap:
>
> https://zmap.io/
>
> But it would be interesting to be able to collect a periodic picture
> of the universe one hop beyond the local gateway.
>
> --
> Dave Täht
>
> NSFW:
> https://w2.eff.org/Censorship/Internet_censorship_bills/russell_0296_indecent.article
> _______________________________________________
> Bloat mailing list
> Bloat at lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.bufferbloat.net/pipermail/bloat/attachments/20140410/1cb87c7a/attachment-0003.html>