Re: [Bloat] Bufferbloat Paper

Re: [Bloat] Bufferbloat Paper

[Mark Allman]

[-- Attachment #1: Type: text/plain, Size: 1102 bytes --]


> > Note the paper does not work in units of *connections* in section 2, but
> > rather in terms of *RTT samples*.  So, nearly 5% of the RTT samples add
> >>= 400msec to the base delay measured for the given remote (in the
> > "residential" case).
> 
> Hmm, yes, I was wondering about this and was unable to fully grok it:
> what, exactly, is an RTT sample? :)

One RTT measurement between the CCZ monitoring point and the remote end
host. 

> Incidentally, are the data extraction scripts available somewhere?
> Might be worthwhile to distribute them as some kind of tool that
> people with interesting vantage points could apply to get useful data?

Well, they are not readily available.  I used a non-public extension to
Bro (that is not mine) to get the RTT samples.  So, that is a sticking
point.  And, then there is a ball of goop to analyze those.  If folks
have a place they can monitor and are interested in doing so, please
contact me.  I can probably get this in shape enough to give you.  But,
I doubt I'll be able to somehow package this for general consumption any
time soon.

allman




[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

Re: [Bloat] Bufferbloat Paper

[David Lang]

 On Tue, 08 Jan 2013 22:39:20 -0500, Mark Allman wrote:
>> > Note the paper does not work in units of *connections* in section 
>> 2, but
>> > rather in terms of *RTT samples*.  So, nearly 5% of the RTT 
>> samples add
>> >>= 400msec to the base delay measured for the given remote (in the
>> > "residential" case).
>>
>> Hmm, yes, I was wondering about this and was unable to fully grok 
>> it:
>> what, exactly, is an RTT sample? :)
>
> One RTT measurement between the CCZ monitoring point and the remote 
> end
> host.
>
>> Incidentally, are the data extraction scripts available somewhere?
>> Might be worthwhile to distribute them as some kind of tool that
>> people with interesting vantage points could apply to get useful 
>> data?
>
> Well, they are not readily available.  I used a non-public extension 
> to
> Bro (that is not mine) to get the RTT samples.  So, that is a 
> sticking
> point.  And, then there is a ball of goop to analyze those.  If folks
> have a place they can monitor and are interested in doing so, please
> contact me.  I can probably get this in shape enough to give you.  
> But,
> I doubt I'll be able to somehow package this for general consumption 
> any
> time soon.

 I really like the idea of trying to measure latency by sniffing the 
 network and watching the time for responses.

 If this can work then I think a lot of people would be willing to put a 
 sniffer inline in their datacenter to measure this.

 How specialized is what you are running? can it be made into a 
 single-use tool that just measures and reports latency?

 David Lang

[Bloat] Passive RTT measurements [was: Re: Bufferbloat Paper]

[Simon Leinen]

[CC'ing Shawn as the author of tcptrace]

Mark Allman writes:
[Toke Høiland-Jørgensen:]
>> Incidentally, are the data extraction scripts available somewhere?
>> Might be worthwhile to distribute them as some kind of tool that
>> people with interesting vantage points could apply to get useful data?

> Well, they are not readily available.  I used a non-public extension
> to Bro (that is not mine) to get the RTT samples.

I have found that with "tcptrace -Z -lrn", I can extract lots of RTT
samples and also some per-connection statistics (which are useful but
not as detailed as I'd like to or as in your paper).  This even works in
finite time, like a couple minutes per 100Mpacket trace of mostly HTTP
traffic taken near a popular archive server.  As a bonus, it supports IPv6.

So it seems that tcptrace (6.6.7) does *almost* what we need.  But you
probably have reasons why you haven't extended Bro rather than using
tcptrace (possibly after improving it somehow).  Care to elaborate?

Maybe we can add the necessary improvements to tcptrace and make it even
more awesome.  (I'm sure Bro is awesome too, I'm just not as familiar
with it.)

> So, that is a sticking point.  And, then there is a ball of goop to
> analyze those.  If folks have a place they can monitor and are
> interested in doing so, please contact me.

* raises hand *

> I can probably get this in shape enough to give you.  But, I doubt
> I'll be able to somehow package this for general consumption any time
> soon.

Understood.
-- 
Simon.

[Bloat] Passive RTT measurements [was: Re: Bufferbloat Paper]

[Simon Leinen]

Simon Leinen writes:
> [...] But you probably have reasons why you haven't extended Bro
> rather than using tcptrace (possibly after improving it somehow).

Sorry, gibberish - I meant "...reasons for extending Bro rather
than using tcptrace".

> Care to elaborate?

> Maybe we can add the necessary improvements to tcptrace and make it even
> more awesome.  (I'm sure Bro is awesome too, I'm just not as familiar
> with it.)
[...]
-- 
Simon.

Re: [Bloat] Bufferbloat Paper

[grenville armitage]

On 01/09/2013 16:02, David Lang wrote:
	.[.]
> I really like the idea of trying to measure latency by sniffing the network and watching the time for responses.

Probably tangential, but http://caia.swin.edu.au/tools/spp/ has proven useful to our group for measuring RTT between two arbitrary packet capture points, for symmetric or asymmetric UDP or TCP traffic flows.

Even more tangential, http://dx.doi.org/10.1109/LCN.2005.101 ("Passive TCP Stream Estimation of RTT and Jitter Parameters") might be an interesting algorithm to implement for estimating RTT of TCP flows seen at a single capture point. (This 2005 paper describes an extension of a technique used by tstat at the time.)

cheers,
gja

Re: [Bloat] Passive RTT measurements [was: Re: Bufferbloat Paper]

[Hagen Paul Pfeifer]

* Simon Leinen | 2013-01-17 15:03:55 [+0100]:

>I have found that with "tcptrace -Z -lrn", I can extract lots of RTT
>samples and also some per-connection statistics (which are useful but
>not as detailed as I'd like to or as in your paper).  This even works in
>finite time, like a couple minutes per 100Mpacket trace of mostly HTTP
>traffic taken near a popular archive server.  As a bonus, it supports IPv6.

Hey Simon,

some years ago I started to develop a clean and easy expendable python
application similar to tcptrace, called captcp:

http://research.protocollabs.com/captcp/


Probably you should take a look at the source code, here the throughput module:

https://github.com/hgn/captcp/blob/master/captcp.py#L2239


Simple python code, TCP streams are already splitted, the packet is also
already parsed (all IP/TCP values) - you simple have to build your logic
around and hook into pre_process_packet() or process_packet().


Cheers, Hagen