From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id CBED621F25D for ; Wed, 6 May 2015 08:30:40 -0700 (PDT) Received: by wgyo15 with SMTP id o15so15753842wgy.2 for ; Wed, 06 May 2015 08:30:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ntcUhn885VxLQRwsS/lY0Dnvf1CVlEBYlG8ni0kb2bU=; b=B9Adkp8JmjuH2K3SREGXaCBV7JzDLjbxH1joq1BkNr7+c6Tcx2AtCNYn1PdNMjdIEl fcBTtLi6wlh7Apkzk6KCHIpuIz7l7UgFsV86I2WUB6/IuaSR0CxwOb06xfRZXYGyjhT0 do5lhJ1rQfP2uIO5sWwnrN3+LUVvuzQk1wmpRk5t30//Qjh0MVBMzr3hpAm33/+ycVkd tTwtYSyCyHHUKwXsHlVJJVpVVGnUaRCtg0V32+rLjYKtGyTbqOQoxAZ4RMt4zMI2aipR ySMvdQNgqoFlYXuRuTIOtbeNbsO+4eKCYFAsoblAFZr9M85B/WFdHvpxBSD4yW6Yx5x7 ibvA== MIME-Version: 1.0 X-Received: by 10.180.85.9 with SMTP id d9mr6191568wiz.28.1430926222791; Wed, 06 May 2015 08:30:22 -0700 (PDT) Sender: gettysjim@gmail.com Received: by 10.194.118.166 with HTTP; Wed, 6 May 2015 08:30:22 -0700 (PDT) In-Reply-To: References: <25065_1429716388_5537BDA4_25065_2328_1_63pyislbvtjf653k3qt8gw2c.1429715929544@email.android.com> <1429717468.18561.90.camel@edumazet-glaptop2.roam.corp.google.com> <5537CDB7.60301@orange.com> <1429722979.18561.112.camel@edumazet-glaptop2.roam.corp.google.com> <5537DA20.1090008@orange.com> <5537DE4D.8090100@orange.com> <553882D7.4020301@orange.com> <1429771718.22254.32.camel@edumazet-glaptop2.roam.corp.google.com> <6C0D04CF-53AA-4D18-A4E4-B746AF6487C7@gmx.de> <87wq123p5r.fsf@toke.dk> <2288B614-B415-4017-A842-76E8F5DFDE4C@gmx.de> <553B06CE.1050209@superduper.net> <14ceed3c818.27f7.e972a4f4d859b00521b2b659602cb2f9@superduper.net> <0C930D43-A05B-48E2-BC01-792CAA72CAD1@gmx.de> <5549A1B8.50005@superduper.net> Date: Wed, 6 May 2015 11:30:22 -0400 X-Google-Sender-Auth: Q6YQPA_sPrDyZFuNqgrUMuZ39V8 Message-ID: From: Jim Gettys To: Sebastian Moeller Content-Type: multipart/alternative; boundary=f46d04426abc359be205156b7901 Cc: bloat Subject: Re: [Bloat] DSLReports Speed Test has latency measurement built-in X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 May 2015 15:31:20 -0000 --f46d04426abc359be205156b7901 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, May 6, 2015 at 4:50 AM, Sebastian Moeller wrote: > Hi Simon, > > On May 6, 2015, at 07:08 , Simon Barber wrote: > > > Hi Sebastian, > > > > My numbers are what I've personally come up with after working for many > years with VoIP - they have no other basis. > > I did not intend to be-little such numbers at all, I just wanted > to propose that we either use generally accepted scientifically measured > numbers or make such measurements our self. > > > One thing is that you have to compare apples to apples - the ITU number= s > are for acoustic one way delay. > > True, and this is why we easily can estimate the delay cost of different > stages of the whole voip one-way pipeline to deuce how much latent budget > we have for the network (aka buffer bloat on the way), but still bases ou= r > numbers on some reference for mouth-to-ear-delay. I think we can > conservatively estimate the latency cost of the sampling, sender processi= ng > and receiver processing (outside of the de-jitter buffering) seem harder = to > estimate reliably, to my untrained eye. > > > The poor state of jitter buffer implementations that almost every VoIP > app or device has means that to hit these acoustic delay numbers you need > significantly lower network delays. > > I fully agree, and if we can estimate this I think we can justify > deductions from the mouth-to-ear budget. I would as first approximation > assume that what we call latency under load increase to be tightly > correlated with jitter, so we could take our =E2=80=9Cbloat-measurement= =E2=80=9D in ms an > directly deduct it from the budget (or if we want to accept occasional > voice degradation we can pick a sufficiently high percentile, but that is > implementation detail). > > > Also note that these numbers are worst case, which must include trip > halfway around the globe - if you can hit the numbers with half globe > propagation then you will hit much better numbers for 'local calls=E2=80= =99. > > We could turn this around by estimating to what distance voip > quality will be good/decent/acceptable/lughable=E2=80=A6 > > =E2=80=8B > =E2=80=8BMean RTT is almost useless for VOIP and teleconferencing. What ma= tters is the RTT + jitter; a VOIP or teleconferencing application cannot function at a given latency unless the "drop outs" caused by late packets is low enough to not be obnoxious to human perception; there are a number of techniques to hide such late packet dropouts but all of them (short of FEC) damage the audio stream. So ideally, not only do you measure the delay, you also measure the jitter to be able to figure out a realistic operating point for such applications. - Jim =E2=80=8B > =E2=80=8B > > > > > > Simon > > > > > > On 4/24/2015 11:03 PM, Sebastian Moeller wrote: > >> Hi Simon, hi List > >> > >> On Apr 25, 2015, at 06:26 , Simon Barber wrote: > >> > >>> Certainly the VoIP numbers are for peak total latency, and while > Justin is measuring total latency because he is only taking a few samples > the peak values will be a little higher. > >> If your voip number are for peak total latency they need > literature citations to back them up, as they are way shorter than what t= he > ITU recommends for one-way-latency (see ITU-T G.114, Fig. 1). I am not > "married=E2=80=9D to the ITU numbers but I think we should use generally = accepted > numbers here and not bake our own thresholds (and for all I know your > numbers are fine, I just don=E2=80=99t know where they are coming from ;)= ) > >> > >> Best Regards > >> Sebastian > >> > >> > >>> Simon > >>> > >>> Sent with AquaMail for Android > >>> http://www.aqua-mail.com > >>> > >>> > >>> On April 24, 2015 9:04:45 PM Dave Taht wrote: > >>> > >>>> simon all your numbers are too large by at least a factor of 2. I > >>>> think also you are thinking about total latency, rather than induced > >>>> latency and jitter. > >>>> > >>>> Please see my earlier email laying out the bands. And gettys' > manifesto. > >>>> > >>>> If you are thinking in terms of voip, less than 30ms *jitter* is wha= t > >>>> you want, and a latency increase of 30ms is a proxy for also holding > >>>> jitter that low. > >>>> > >>>> > >>>> On Fri, Apr 24, 2015 at 8:15 PM, Simon Barber > wrote: > >>>>> I think it might be useful to have a 'latency guide' for users. It > would say > >>>>> things like > >>>>> > >>>>> 100ms - VoIP applications work well > >>>>> 250ms - VoIP applications - conversation is not as natural as it > could be, > >>>>> although users may not notice this. > >> The only way to detect whether a conversation is natural is if > users notice, I would say... > >> > >>>>> 500ms - VoIP applications begin to have awkward pauses in > conversation. > >>>>> 1000ms - VoIP applications have significant annoying pauses in > conversation. > >>>>> 2000ms - VoIP unusable for most interactive conversations. > >>>>> > >>>>> 0-50ms - web pages load snappily > >>>>> 250ms - web pages can often take an extra second to appear, even on > the > >>>>> highest bandwidth links > >>>>> 1000ms - web pages load significantly slower than they should, taki= ng > >>>>> several extra seconds to appear, even on the highest bandwidth link= s > >>>>> 2000ms+ - web browsing is heavily slowed, with many seconds or even > 10s of > >>>>> seconds of delays for pages to load, even on the highest bandwidth > links. > >>>>> > >>>>> Gaming.... some kind of guide here.... > >>>>> > >>>>> Simon > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On 4/24/2015 1:55 AM, Sebastian Moeller wrote: > >>>>>> Hi Toke, > >>>>>> > >>>>>> On Apr 24, 2015, at 10:29 , Toke H=C3=B8iland-J=C3=B8rgensen > wrote: > >>>>>> > >>>>>>> Sebastian Moeller writes: > >>>>>>> > >>>>>>>> I know this is not perfect and the numbers will probably require > >>>>>>>> severe "bike-shedding=E2=80=9D > >>>>>>> Since you're literally asking for it... ;) > >>>>>>> > >>>>>>> > >>>>>>> In this case we're talking about *added* latency. So the ambition > should > >>>>>>> be zero, or so close to it as to be indiscernible. Furthermore, w= e > know > >>>>>>> that proper application of a good queue management algorithm can > keep it > >>>>>>> pretty close to this. Certainly under 20-30 ms of added latency. > So from > >>>>>>> this, IMO the 'green' or 'excellent' score should be from zero to > 30 ms. > >>>>>> Oh, I can get behind that easily, I just thought basing th= e > limits > >>>>>> on externally relevant total latency thresholds would directly tel= l > the user > >>>>>> which applications might run well on his link. Sure this means tha= t > people > >>>>>> on a satellite link most likely will miss out the acceptable voip > threshold > >>>>>> by their base-latency alone, but guess what telephony via satellit= e > leaves > >>>>>> something to be desired. That said if the alternative is no > telephony I > >>>>>> would take 1 second one-way delay any day ;). > >>>>>> What I liked about fixed thresholds is that the test would > give a > >>>>>> good indication what kind of uses are going to work well on the > link under > >>>>>> load, given that during load both base and induced latency come > into play. I > >>>>>> agree that 300ms as first threshold is rather unambiguous though > (and I am > >>>>>> certain that remote X11 will require a massively lower RTT unless > one likes > >>>>>> to think of remote desktop as an oil tanker simulator ;) ) > >>>>>> > >>>>>>> The other increments I have less opinions about, but 100 ms does > seem to > >>>>>>> be a nice round number, so do yellow from 30-100 ms, then start > with the > >>>>>>> reds somewhere above that, and range up into the deep red / purpl= e > / > >>>>>>> black with skulls and fiery death as we go nearer and above one > second? > >>>>>>> > >>>>>>> > >>>>>>> I very much think that raising peoples expectations and being qui= te > >>>>>>> ambitious about what to expect is an important part of this. Of > course > >>>>>>> the base latency is going to vary, but the added latency > shouldn't. And > >>>>>>> sine we have the technology to make sure it doesn't, calling out > bad > >>>>>>> results when we see them is reasonable! > >>>>>> Okay so this would turn into: > >>>>>> > >>>>>> base latency to base latency + 30 ms: > green > >>>>>> base latency + 31 ms to base latency + 100 ms: yellow > >>>>>> base latency + 101 ms to base latency + 200 ms: orange? > >>>>>> base latency + 201 ms to base latency + 500 ms: red > >>>>>> base latency + 501 ms to base latency + 1000 ms: fire > >>>>>> base latency + 1001 ms to infinity: > >>>>>> fire & brimstone > >>>>>> > >>>>>> correct? > >>>>>> > >>>>>> > >>>>>>> -Toke > >>>>>> _______________________________________________ > >>>>>> Bloat mailing list > >>>>>> Bloat@lists.bufferbloat.net > >>>>>> https://lists.bufferbloat.net/listinfo/bloat > >>>>> > >>>>> _______________________________________________ > >>>>> Bloat mailing list > >>>>> Bloat@lists.bufferbloat.net > >>>>> https://lists.bufferbloat.net/listinfo/bloat > >>>> > >>>> > >>>> -- > >>>> Dave T=C3=A4ht > >>>> Open Networking needs **Open Source Hardware** > >>>> > >>>> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67 > >>> > >>> _______________________________________________ > >>> Bloat mailing list > >>> Bloat@lists.bufferbloat.net > >>> https://lists.bufferbloat.net/listinfo/bloat > > > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > --f46d04426abc359be205156b7901 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, Ma= y 6, 2015 at 4:50 AM, Sebastian Moeller <moeller0@gmx.de> wrot= e:
Hi Simon,

On May 6, 2015, at 07:08 , Simon Barber <simon@superduper.net> wrote:

> Hi Sebastian,
>
> My numbers are what I've personally come up with after working for= many years with VoIP - they have no other basis.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 I did not intend to be-little such numbe= rs at all, I just wanted to propose that we either use generally accepted s= cientifically measured numbers or make such measurements our self.

> One thing is that you have to compare apples to apples - the ITU numbe= rs are for acoustic one way delay.

True, and this is why we easily can estimate the delay cost of diffe= rent stages of the whole voip one-way pipeline to deuce how much latent bud= get we have for the network (aka buffer bloat on the way), but still bases = our numbers on some reference for mouth-to-ear-delay. I think we can conser= vatively estimate the latency cost of the sampling, sender processing and r= eceiver processing (outside of the de-jitter buffering) seem harder to esti= mate reliably, to my untrained eye.

> The poor state of jitter buffer implementations that almost every VoIP= app or device has means that to hit these acoustic delay numbers you need = significantly lower network delays.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 I fully agree, and if we can estimate th= is I think we can justify deductions from the mouth-to-ear budget. I would = as first approximation assume that what we call latency under load increase= to be tightly correlated with jitter, so we could take our =E2=80=9Cbloat-= measurement=E2=80=9D in ms an directly deduct it from the budget (or if we = want to accept occasional voice degradation we can pick a sufficiently high= percentile, but that is implementation detail).

> Also note that these numbers are worst case, which must include trip h= alfway around the globe - if you can hit the numbers with half globe propag= ation then you will hit much better numbers for 'local calls=E2=80=99.<= br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 We=C2=A0 could turn this around by estim= ating to what distance voip quality will be good/decent/acceptable/lughable= =E2=80=A6

=E2=80=8B
=

=E2=80=8BMean RTT is almost useless for VOIP an= d teleconferencing.=C2=A0 What matters is the RTT + jitter; a VOIP or telec= onferencing application cannot function at a given latency unless the "= ;drop outs" caused by late packets is low enough to not be obnoxious t= o human perception; there are a number of techniques to hide such late pack= et dropouts but all of them (short of FEC) damage the audio stream.

So ideally, not only do you measure the delay, you also= measure the jitter to be able to figure out a realistic operating point fo= r such applications.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 - Jim


=
=E2=80=8B
=C2=A0
=C2=A0
=E2=80=8B


>
> Simon
>
>
> On 4/24/2015 11:03 PM, Sebastian Moeller wrote:
>> Hi Simon, hi List
>>
>> On Apr 25, 2015, at 06:26 , Simon Barber <simon@superduper.net> wrote:
>>
>>> Certainly the VoIP numbers are for peak total latency, and whi= le Justin is measuring total latency because he is only taking a few sample= s the peak values will be a little higher.
>>=C2=A0 =C2=A0 =C2=A0 If your voip number are for peak total latency= they need literature citations to back them up, as they are way shorter th= an what the ITU recommends for one-way-latency (see ITU-T G.114, Fig. 1). I= am not "married=E2=80=9D to the ITU numbers but I think we should use= generally accepted numbers here and not bake our own thresholds (and for a= ll I know your numbers are fine, I just don=E2=80=99t know where they are c= oming from ;) )
>>
>> Best Regards
>>=C2=A0 =C2=A0 =C2=A0 Sebastian
>>
>>
>>> Simon
>>>
>>> Sent with AquaMail for Android
>>> http://= www.aqua-mail.com
>>>
>>>
>>> On April 24, 2015 9:04:45 PM Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>>> simon all your numbers are too large by at least a factor = of 2. I
>>>> think also you are thinking about total latency, rather th= an induced
>>>> latency and jitter.
>>>>
>>>> Please see my earlier email laying out the bands. And gett= ys' manifesto.
>>>>
>>>> If you are thinking in terms of voip, less than 30ms *jitt= er* is what
>>>> you want, and a latency increase of 30ms is a proxy for al= so holding
>>>> jitter that low.
>>>>
>>>>
>>>> On Fri, Apr 24, 2015 at 8:15 PM, Simon Barber <simon@superduper.net> wrote:
>>>>> I think it might be useful to have a 'latency guid= e' for users. It would say
>>>>> things like
>>>>>
>>>>> 100ms - VoIP applications work well
>>>>> 250ms - VoIP applications - conversation is not as nat= ural as it could be,
>>>>> although users may not notice this.
>>=C2=A0 =C2=A0 =C2=A0 The only way to detect whether a conversation = is natural is if users notice, I would say...
>>
>>>>> 500ms - VoIP applications begin to have awkward pauses= in conversation.
>>>>> 1000ms - VoIP applications have significant annoying p= auses in conversation.
>>>>> 2000ms - VoIP unusable for most interactive conversati= ons.
>>>>>
>>>>> 0-50ms - web pages load snappily
>>>>> 250ms - web pages can often take an extra second to ap= pear, even on the
>>>>> highest bandwidth links
>>>>> 1000ms - web pages load significantly slower than they= should, taking
>>>>> several extra seconds to appear, even on the highest b= andwidth links
>>>>> 2000ms+ - web browsing is heavily slowed, with many se= conds or even 10s of
>>>>> seconds of delays for pages to load, even on the highe= st bandwidth links.
>>>>>
>>>>> Gaming.... some kind of guide here....
>>>>>
>>>>> Simon
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 4/24/2015 1:55 AM, Sebastian Moeller wrote:
>>>>>> Hi Toke,
>>>>>>
>>>>>> On Apr 24, 2015, at 10:29 , Toke H=C3=B8iland-J=C3= =B8rgensen <toke@toke.dk> wrote:<= br> >>>>>>
>>>>>>> Sebastian Moeller <moeller0@gmx.de> writes:
>>>>>>>
>>>>>>>> I know this is not perfect and the numbers= will probably require
>>>>>>>> severe "bike-shedding=E2=80=9D
>>>>>>> Since you're literally asking for it... ;)=
>>>>>>>
>>>>>>>
>>>>>>> In this case we're talking about *added* l= atency. So the ambition should
>>>>>>> be zero, or so close to it as to be indiscerni= ble. Furthermore, we know
>>>>>>> that proper application of a good queue manage= ment algorithm can keep it
>>>>>>> pretty close to this. Certainly under 20-30 ms= of added latency. So from
>>>>>>> this, IMO the 'green' or 'excellen= t' score should be from zero to 30 ms.
>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Oh, I can get beh= ind that easily, I just thought basing the limits
>>>>>> on externally relevant total latency thresholds wo= uld directly tell the user
>>>>>> which applications might run well on his link. Sur= e this means that people
>>>>>> on a satellite link most likely will miss out the = acceptable voip threshold
>>>>>> by their base-latency alone, but guess what teleph= ony via satellite leaves
>>>>>> something to be desired. That said if the alternat= ive is no telephony I
>>>>>> would take 1 second one-way delay any day ;).
>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0What I liked abou= t fixed thresholds is that the test would give a
>>>>>> good indication what kind of uses are going to wor= k well on the link under
>>>>>> load, given that during load both base and induced= latency come into play. I
>>>>>> agree that 300ms as first threshold is rather unam= biguous though (and I am
>>>>>> certain that remote X11 will require a massively l= ower RTT unless one likes
>>>>>> to think of remote desktop as an oil tanker simula= tor ;) )
>>>>>>
>>>>>>> The other increments I have less opinions abou= t, but 100 ms does seem to
>>>>>>> be a nice round number, so do yellow from 30-1= 00 ms, then start with the
>>>>>>> reds somewhere above that, and range up into t= he deep red / purple /
>>>>>>> black with skulls and fiery death as we go nea= rer and above one second?
>>>>>>>
>>>>>>>
>>>>>>> I very much think that raising peoples expecta= tions and being quite
>>>>>>> ambitious about what to expect is an important= part of this. Of course
>>>>>>> the base latency is going to vary, but the add= ed latency shouldn't. And
>>>>>>> sine we have the technology to make sure it do= esn't, calling out bad
>>>>>>> results when we see them is reasonable!
>>>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Okay so this woul= d turn into:
>>>>>>
>>>>>> base latency to base latency + 30 ms:=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0green
>>>>>> base latency + 31 ms to base latency + 100 ms:=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 yellow
>>>>>> base latency + 101 ms to base latency + 200 ms:=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0orange?
>>>>>> base latency + 201 ms to base latency + 500 ms:=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0red
>>>>>> base latency + 501 ms to base latency + 1000 ms:= =C2=A0 =C2=A0 =C2=A0 =C2=A0 fire
>>>>>> base latency + 1001 ms to infinity:
>>>>>> fire & brimstone
>>>>>>
>>>>>> correct?
>>>>>>
>>>>>>
>>>>>>> -Toke
>>>>>> _______________________________________________ >>>>>> Bloat mailing list
>>>>>> Blo= at@lists.bufferbloat.net
>>>>>> https://lists.bufferbloat.net/listinfo/bloat >>>>>
>>>>> _______________________________________________
>>>>> Bloat mailing list
>>>>> Bloat@l= ists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/bloat
>>>>
>>>>
>>>> --
>>>> Dave T=C3=A4ht
>>>> Open Networking needs **Open Source Hardware**
>>>>
>>>> https://plus.google.com/u/0/+EricRaymond/pos= ts/JqxCe2pFr67
>>>
>>> _______________________________________________
>>> Bloat mailing list
>>> Bloat@lists.buf= ferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
>

_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net<= /a>
= https://lists.bufferbloat.net/listinfo/bloat

--f46d04426abc359be205156b7901--