Re: [Babel-users] Follow-up on the new babel-rtt branch: µs resolution

Re: [Babel-users] Follow-up on the new babel-rtt branch: µs resolution

[Dave Taht]

On Wed, Oct 30, 2013 at 11:36 AM, Baptiste Jonglez
<baptiste.jonglez@ens-lyon.fr> wrote:
> Dear Babel hackers,
>
> I'm currently testing my new babel-rtt code, which uses 32-bits
> timestamps to achieve a µs-resolution on RTT.
>
> A graph showing the RTT between two directly-connected hosts (Gigabit
> Ethernet) can be found in [1].
>
> The two hosts are laptops, directly connected with an Ethernet cable.
> Both have Gigabit interfaces.  One has a fast Intel Core2 duo CPU (2.4
> GHz), while the other has a slow AMD E2-1800 APU (1.7 GHz).
>
> Do note that the RTT is expressed in µs, not ms.  A few comments,
> prompted by a discussion with Juliusz and Matthieu:

I am pretty curious if you can tell the difference between 10GigE,
GigE, and 100Mbit under various loads.

> - the ping6 result is surprising, as two groups of points clearly
>   appear, separated by about 200 µs.  It seems that interrupts might
>   be responsible for this, but it's still somewhat unclear.

It could be caused by napi. There is also some parameter to ethtool
(?) to tell it to return pings more immediately (that I recall was
specific to the e1000e network card). I'm pretty sure it was the
--coalesce some_option

You might find it easier to verify your measurement with owamp and gps
synced clocks.

I'm delighted to see this measurement as collected by babel as my
sekret plan was to be able to measure heavy traffic benchmarks vs
various qdiscs like the new "fq" and older fq_codel ones...

> - Babel sees a RTT that is 400 µs higher than ping6.  The babel-rtt
>   implementation timestamps outgoing message as late as possible, and
>   timestamps incoming messages as early as possible, but it's not
>   perfect.

Did you try hardware timestamping on rx? (this is a kernel compile option)

> Context-switching and syscalls can probably account for
>   these 400 µs: a quick test, on the slower laptop, showed that
>   calling the gettimeofday() syscall takes about 100 µs.

Kernel version? This was sped up in some fairly recent kernel version (?)

>
> Further comments are welcome.
>
> [1] http://ze.polyno.me/babel/32bits-rtt-ethernet.pdf
>
> _______________________________________________
> Babel-users mailing list
> Babel-users@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Babel-users] Follow-up on the new babel-rtt branch: µs resolution

[Juliusz Chroboczek]

Baptiste:

> - Babel sees a RTT that is 400 µs higher than ping6.  The babel-rtt
>   implementation timestamps outgoing message as late as possible, and
>   timestamps incoming messages as early as possible, but it's not
>   perfect.

The good news, of course, is that the offset is pretty constant, so
the samples computed by babeld are good enough to be input to the
metric computation.  The smoothed samples would appear to have
a precision of roughly 200us, which should be good enough for almost
any application.

Baptiste, I'm wondering if you can count the number of Ethernet
switches that way.

> I'm delighted to see this measurement as collected by babel

I knew you would be :-)

> my sekret plan was to be able to measure heavy traffic benchmarks vs
> various qdiscs like the new "fq" and older fq_codel ones...

Well, the reason we decided it's worthwile to improve the precision of
our measurements is that, according to Jim, you had a secret plan to
use RTT to measure link-layer congestion in Wifi meshes.  How many
more secret plans do you have?

-- Juliusz

Re: [Babel-users] Follow-up on the new babel-rtt branch: µs resolution

[Dave Taht]

On Wed, Oct 30, 2013 at 5:15 PM, Juliusz Chroboczek
<jch@pps.univ-paris-diderot.fr> wrote:
> Baptiste:
>
>> - Babel sees a RTT that is 400 µs higher than ping6.  The babel-rtt
>>   implementation timestamps outgoing message as late as possible, and
>>   timestamps incoming messages as early as possible, but it's not
>>   perfect.
>
> The good news, of course, is that the offset is pretty constant, so
> the samples computed by babeld are good enough to be input to the
> metric computation.  The smoothed samples would appear to have
> a precision of roughly 200us, which should be good enough for almost
> any application.

I will try to get an equivalent measurement on vastly weaker hardware
(ar71xx/ath9k) on my next set of builds (after ietf), ~2 weeks.

I would like measurements under various loads...

> Baptiste, I'm wondering if you can count the number of Ethernet
> switches that way.

In cut-through mode this switch claims 400ns or so latency.

http://ark.intel.com/products/76302/Intel-Ethernet-Switch-FM5224

Older models of this were better - 300ns or less.

>
>> I'm delighted to see this measurement as collected by babel
>
> I knew you would be :-)
>
>> my sekret plan was to be able to measure heavy traffic benchmarks vs
>> various qdiscs like the new "fq" and older fq_codel ones...
>
> Well, the reason we decided it's worthwile to improve the precision of
> our measurements is that, according to Jim, you had a secret plan to
> use RTT to measure link-layer congestion in Wifi meshes.

Yes, ~200us precision is nicely less than a txop. This will help.

> How many
> more secret plans do you have?

One or two. ;)

> -- Juliusz
>
> _______________________________________________
> Babel-users mailing list
> Babel-users@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html