[Bloat] Speed tests - attribution of latency to relevant network hops

[Bloat] Speed tests - attribution of latency to relevant network hops

[Jan Ceuleers]

Dear list,

Now that dslreports has set the benchmark on combining speed testing
with latency measurements I was thinking about what the next steps could
be. Here's what I came up with:

Would it be useful to try and attribute the latency to certain relevant
network hops, like so:

First hop: round-trip latency of the link connecting the user's machine
to their local network. This could be wifi or Ethernet in a home or
office environment; it could be wifi or cellular in a public
environment, etc.

Local network: round-trip latency to the first upstream port that has a
public IP address. Of course this is only useful if the user's machine
doesn't already have a routable IP address.

ISP: round-trip latency to the second upstream port that has a public IP
address. So this would be the DSLAM/BRAS/CMTS or whatever access
concentrator the ISP uses.

This would probably need to be based on ping. Which IP addresses to ping
would initially need to be found out through traceroute-like methods.

If we can then get users to tell us not only what Internet access
technology they use (i.e. cable, DSL, GPON etc) but also what local
access technology (i.e. wifi, Ethernet etc) we'd know how to attribute
the above numbers to technologies.

Jan

Re: [Bloat] Speed tests - attribution of latency to relevant network hops

[David Lang]

On Wed, 29 Jul 2015, Jan Ceuleers wrote:

> Dear list,
>
> Now that dslreports has set the benchmark on combining speed testing
> with latency measurements I was thinking about what the next steps could
> be. Here's what I came up with:
>
> Would it be useful to try and attribute the latency to certain relevant
> network hops, like so:
>
> First hop: round-trip latency of the link connecting the user's machine
> to their local network. This could be wifi or Ethernet in a home or
> office environment; it could be wifi or cellular in a public
> environment, etc.
>
> Local network: round-trip latency to the first upstream port that has a
> public IP address. Of course this is only useful if the user's machine
> doesn't already have a routable IP address.
>
> ISP: round-trip latency to the second upstream port that has a public IP
> address. So this would be the DSLAM/BRAS/CMTS or whatever access
> concentrator the ISP uses.
>
> This would probably need to be based on ping. Which IP addresses to ping
> would initially need to be found out through traceroute-like methods.
>
> If we can then get users to tell us not only what Internet access
> technology they use (i.e. cable, DSL, GPON etc) but also what local
> access technology (i.e. wifi, Ethernet etc) we'd know how to attribute
> the above numbers to technologies.

unless you measure it per hop, how are you going to attribute it to each hop? 
and unless you have a server at that layer to talk to, how do you know what the 
latency or bandwidth is?

David Lang

Re: [Bloat] Speed tests - attribution of latency to relevant network hops

[Mikael Abrahamsson]

On Wed, 29 Jul 2015, David Lang wrote:

> unless you measure it per hop, how are you going to attribute it to each 
> hop? and unless you have a server at that layer to talk to, how do you 
> know what the latency or bandwidth is?

Measuring latency is doable (using the same mechanism that traceroute with 
for instance max-ttl 5), but I don't know how much of this is available to 
your web application?

If you sent 5 packets with TTL 1-5 and measured the time to get back the 
ttl-expired-in-transit ICMP, you could get an indication where the latency 
increase was happening.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] Speed tests - attribution of latency to relevant network hops

[Jan Ceuleers]

On 30/07/15 06:52, Mikael Abrahamsson wrote:
> On Wed, 29 Jul 2015, David Lang wrote:
> 
>> unless you measure it per hop, how are you going to attribute it to
>> each hop? and unless you have a server at that layer to talk to, how
>> do you know what the latency or bandwidth is?
> 
> Measuring latency is doable (using the same mechanism that traceroute
> with for instance max-ttl 5), but I don't know how much of this is
> available to your web application?
> 
> If you sent 5 packets with TTL 1-5 and measured the time to get back the
> ttl-expired-in-transit ICMP, you could get an indication where the
> latency increase was happening.

Yes, that's what I had in mind.

Re: [Bloat] Speed tests - attribution of latency to relevant network hops

[Bill Ver Steeg (versteb)]

You do have to be careful using ttl tricks and/or ICMP packets for fine-grained timing measurements. On many middleboxes, there is a "fast path" that is done in a highly optimized manner (often in silicon) and a "slow path". In a non-zero number of cases, you will be in the fast path for normal packets and in the slow path for unusual packets. 

What you describe will work for the vast majority of cases, but there will be situations in which you will get misleading results. You would need to handle the timing differences when the slow path was MUCH slower than the fast path and it distorted the results. You would also have to handle some weird edge conditions..... The worst case that springs to mind would be a bloated buffer somewhere in a device's fast path (due to lots of "normal" packets), but the slow path on the same device is not congested and thus actually has less delay for this single packet.

Bvs
  

-----Original Message-----
From: bloat-bounces@lists.bufferbloat.net [mailto:bloat-bounces@lists.bufferbloat.net] On Behalf Of Jan Ceuleers
Sent: Thursday, July 30, 2015 1:18 AM
To: Mikael Abrahamsson; David Lang
Cc: bloat
Subject: Re: [Bloat] Speed tests - attribution of latency to relevant network hops

On 30/07/15 06:52, Mikael Abrahamsson wrote:
> On Wed, 29 Jul 2015, David Lang wrote:
> 
>> unless you measure it per hop, how are you going to attribute it to 
>> each hop? and unless you have a server at that layer to talk to, how 
>> do you know what the latency or bandwidth is?
> 
> Measuring latency is doable (using the same mechanism that traceroute 
> with for instance max-ttl 5), but I don't know how much of this is 
> available to your web application?
> 
> If you sent 5 packets with TTL 1-5 and measured the time to get back 
> the ttl-expired-in-transit ICMP, you could get an indication where the 
> latency increase was happening.

Yes, that's what I had in mind.
_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Speed tests - attribution of latency to relevant network hops

[Jan Ceuleers]

On 30/07/15 15:04, Bill Ver Steeg (versteb) wrote:
> You do have to be careful using ttl tricks and/or ICMP packets for fine-grained timing measurements. On many middleboxes, there is a "fast path" that is done in a highly optimized manner (often in silicon) and a "slow path". In a non-zero number of cases, you will be in the fast path for normal packets and in the slow path for unusual packets. 

Yes, understood and agreed.

But I think it would already be very enlightening to lots of users (and
also assist the "make wifi faster" efforts) to understand how much of
the bloat sits in the part of the network that they can fix themselves,
how much of it they can hold their service provided accountable for and
how much of it they essentially have to put up with because it's
inherent to the way they're plumbed into the interweb.

So perhaps it's not possible to break the dslreports bloat graphs down
into the three segments that I described, because all we have available
is ICMP and the TTL field, but even a graphical representation of a
traceroute-while-idle and a traceroute-under-load would be a step forward.

> What you describe will work for the vast majority of cases, but there will be situations in which you will get misleading results. You would need to handle the timing differences when the slow path was MUCH slower than the fast path and it distorted the results. You would also have to handle some weird edge conditions..... The worst case that springs to mind would be a bloated buffer somewhere in a device's fast path (due to lots of "normal" packets), but the slow path on the same device is not congested and thus actually has less delay for this single packet.

My first thought was that I'd expect that the fast path queues in which
packets might spend a non-negligible amount of time would sit on the
egress side of the box rather than on the ingress side, and that it is
only the ingress queue that would affect both fast-path and slow-path
traffic. But I guess that this depends heavily on the architecture of
the box, so you're probably right.

Jan

Re: [Bloat] Speed tests - attribution of latency to relevant network hops

[Dave Taht]

The tool for looking at this is called "mtr", which is available in
various forms for linux, osx, and windows.

It was originally part of the rrul spec, but parsing it's output is a
little difficult (or was - there is csv support for it now in linux
mainline), and the biggest reason left (besides lack of time to
implement it - patches gladly accepted!) we don't use it yet is that
it imparts enough traffic to measurably jigger the other results at
lower bandwidths.

Certainly, instead of blending together the outputs of multiple
separate tools as we currently do with flent, it would be better to
have one specific tool that measured everything we care about all at
the same time and (for example) saw, rather than estimated, acks - but
that requires a scale of development effort and funding that we have
never had.

I do have hope that one day webrtc could be leveraged to give us udp
measurements from within a browser. The current dslreports ping over
tcp method is subject to a multiplicity of problems. I would not mind
at all if browsers gained the ability to set the ttl, also.