[Bloat] Fwd: [Babel-users] RTT stability inside a GRE tunnel

[Bloat] Fwd: [Babel-users] RTT stability inside a GRE tunnel

[Juliusz Chroboczek]

[-- Attachment #0: Type: message/rfc822, Size: 4216 bytes --]

[-- Attachment #1.1.1: Type: text/plain, Size: 617 bytes --]

Hi,

While experimenting with the RTT-aware branch of babeld, I ran into an
odd but interesting behaviour: between two given hosts reachable over
the Internet, the RTT is more stable inside a GRE tunnel than outside.

I've measured the RTT between the two hosts (with a simple continuous
ping), both inside and outside the GRE tunnel.

The result is quite odd: the RTT has way more jitter inside the tunnel
(and is also somewhat lower) than outside.  See an excerpt of the data:

  http://ze.polyno.me/babel/ping-tunnel-oddity.png

I have absolutely no clue as to what could cause this behaviour; any
idea?

Baptiste

[-- Attachment #1.1.2: Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #1.2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
Babel-users mailing list
Babel-users@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users

Re: [Bloat] [Babel-users] RTT stability inside a GRE tunnel

[Juliusz Chroboczek]

> While experimenting with the RTT-aware branch of babeld, I ran into an
> odd but interesting behaviour: between two given hosts reachable over
> the Internet, the RTT is more stable inside a GRE tunnel than outside.

The RTT is around 300 ms for both kinds of packets.  GRE packets have
virtually no jitter, while ICMP packets have some 15ms jitter, which
would appear to be completely random.

My guess would be that there might be a router in the way that handles
GRE packets in hardware, and ICMP packets in software.  The extra 15
ms is the CPU time spent on processing the ICMP packets.  Still, the
nondeterminacy is weird.

Anybody seen anything similar?

-- Juliusz

Re: [Bloat] [Babel-users] RTT stability inside a GRE tunnel

[Mikael Abrahamsson]

On Tue, 2 Jul 2013, Juliusz Chroboczek wrote:

> Anybody seen anything similar?

I don't know if this is the cause, but older gear from Extreme Networks 
(their end-of-90ties and beginning of 2000s), will do ICMP in slow-path 
when routing (even packets going through the device) and "everything else" 
in fast-path. This could cause latency spikes and PDV approximately 
similar to what's seen in the picture.

The seen behaviour could also perhaps be caused by per-packet QoS settings 
in the path, where ICMP might have lower priority than GRE. I don't think 
this explanation is likely though.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] [Babel-users] RTT stability inside a GRE tunnel

[Baptiste Jonglez]

[-- Attachment #1: Type: text/plain, Size: 837 bytes --]

On Tue, Jul 02, 2013 at 06:48:14AM +0200, Mikael Abrahamsson wrote:
> On Tue, 2 Jul 2013, Juliusz Chroboczek wrote:
> 
> >Anybody seen anything similar?
> 
> I don't know if this is the cause, but older gear from Extreme
> Networks (their end-of-90ties and beginning of 2000s), will do ICMP
> in slow-path when routing (even packets going through the device) and
> "everything else" in fast-path. This could cause latency spikes and
> PDV approximately similar to what's seen in the picture.

This looks like a plausible explanation, thanks.  They may not be the
only router make to exhibit this behaviour.

Is there a reason for this slow-path routing of ICMP?  I've seen
routers applying some kind of rate-limiting to ICMP packets, but as
far as I know, it's only limited to ICMP packets originating from the
router.

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [Bloat] [Babel-users] RTT stability inside a GRE tunnel

[Mikael Abrahamsson]

On Tue, 2 Jul 2013, Baptiste Jonglez wrote:

> Is there a reason for this slow-path routing of ICMP?

They thought they needed to inspect/act on some/all ICMP traffic at the 
time they designed the forwarding ASIC, so they just punted all ICMP to 
CPU.

These platforms have other shortcomings from being designed in the late 
90:ties, but I know people who still have them in production because if 
you don't have too many flows (they have a per-destination based 
forwarding cache), they still work fairly well.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se