[Bloat] Dealing with P2P traffic in modern networks - measurement, identification, and control

[Bloat] Dealing with P2P traffic in modern networks - measurement, identification, and control

[Dave Taht]

I am presently wading through the following thesis by a researcher
here at lincs.

http://perso.telecom-paristech.fr/~valenti/pmwiki/pmwiki.php?n=Main.PhDDefense

The portions regarding LEDBAT (chapter 8 onwards) are particularly interesting.

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://the-edge.blogspot.com

Re: [Bloat] Dealing with P2P traffic in modern networks - measurement, identification, and control

[Jesper Dangaard Brouer]

Thanks Dave,

I have always had the dream of implementing a behavioural based traffic
classification Netfilter module.  But I have been unable to find some
good research in this area, this might be the answer :-)

If anybody else on the list have links/articles relating to behavioral
traffic classification, I'm interested! :-)


On Wed, 2011-09-28 at 08:56 -0700, Dave Taht wrote:
> I am presently wading through the following thesis by a researcher
> here at lincs.
> 
> http://perso.telecom-paristech.fr/~valenti/pmwiki/pmwiki.php?n=Main.PhDDefense
> 
> The portions regarding LEDBAT (chapter 8 onwards) are particularly interesting.

-- 
Best regards,
  Jesper Dangaard Brouer
  ComX Networks A/S
  Linux Network Kernel Developer
  Cand. Scient Datalog / MSc.CS
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Re: [Bloat] Dealing with P2P traffic in modern networks - measurement, identification, and control

[Jesper Louis Andersen]

On Wed, Sep 28, 2011 at 19:40, Jesper Dangaard Brouer <hawk@comx.dk> wrote:

> If anybody else on the list have links/articles relating to behavioral
> traffic classification, I'm interested! :-)

My eTorrent uTP branch almost but not quite implement uTP in Erlang.
It uses a LEDBAT classifier in its base, and the idea is definitely
alluring. I have not had time testing it yet though, so I don't know
anything about how it fares or don't. The "almost but not quite" part
is that I skipped over some things in uTP to make other parts work
first. I do have the LEDBAT scheduler but miss important things like
Nagles algorithm, etc. The thesis Dave linked is quite interesting,
but I have not yet even skimmed it.

-- 
J.

Re: [Bloat] Dealing with P2P traffic in modern networks - measurement, identification, and control

[grenville armitage]

On 09/29/2011 03:40, Jesper Dangaard Brouer wrote:
>
> Thanks Dave,
>
> I have always had the dream of implementing a behavioural based traffic
> classification Netfilter module.  But I have been unable to find some
> good research in this area, this might be the answer :-)
>
> If anybody else on the list have links/articles relating to behavioral
> traffic classification, I'm interested! :-)

If by "behavior" you're referring to the statistical patterns within flows
(packet length variations, inter arrival times, etc) you might be interested
in our DIFFUSE (http://caia.swin.edu.au/urp/diffuse) work. We've extended
FreeBSD's ipfw firewall code so that it can recognise traffic based on statistical
characteristics, and use this (rather than direct packet inspection) to trigger
e.g. rate shaping, etc. Although our prototype code was initially developed
for FreeBSD, we've got a preliminary Linux port too. The website contains an
overview description, docs and patch files against FreeBSD and Linux source.

cheers,
gja

Re: [Bloat] Dealing with P2P traffic in modern networks - measurement, identification, and control

[Jesper Dangaard Brouer]

On Thu, 2011-09-29 at 06:51 +1000, grenville armitage wrote:
> 
> On 09/29/2011 03:40, Jesper Dangaard Brouer wrote:
> >
> > Thanks Dave,
> >
> > I have always had the dream of implementing a behavioural based traffic
> > classification Netfilter module.  But I have been unable to find some
> > good research in this area, this might be the answer :-)
> >
> > If anybody else on the list have links/articles relating to behavioral
> > traffic classification, I'm interested! :-)
> 
> If by "behavior" you're referring to the statistical patterns within flows
> (packet length variations, inter arrival times, etc) you might be interested
> in our DIFFUSE (http://caia.swin.edu.au/urp/diffuse) work. We've extended
> FreeBSD's ipfw firewall code so that it can recognise traffic based on statistical
> characteristics, and use this (rather than direct packet inspection) to trigger
> e.g. rate shaping, etc. Although our prototype code was initially developed
> for FreeBSD, we've got a preliminary Linux port too. The website contains an
> overview description, docs and patch files against FreeBSD and Linux source.

Thanks, it looks really interesting and it seem to be what I have been
looking for :-)

I have only skimmed the code, but it looks like you have
implemented/ported ipfw to Linux in-order to run your module on top of
that. An interesting approach.

-- 
Best regards,
  Jesper Dangaard Brouer
  ComX Networks A/S
  Linux Network Kernel Developer
  Cand. Scient Datalog / MSc.CS
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Re: [Bloat] Dealing with P2P traffic in modern networks - measurement, identification, and control

[grenville armitage]

On 09/30/2011 07:58, Jesper Dangaard Brouer wrote:
> On Thu, 2011-09-29 at 06:51 +1000, grenville armitage wrote:
>>
>> On 09/29/2011 03:40, Jesper Dangaard Brouer wrote:
>>>
>>> Thanks Dave,
>>>
>>> I have always had the dream of implementing a behavioural based traffic
>>> classification Netfilter module.  But I have been unable to find some
>>> good research in this area, this might be the answer :-)
>>>
>>> If anybody else on the list have links/articles relating to behavioral
>>> traffic classification, I'm interested! :-)
>>
>> If by "behavior" you're referring to the statistical patterns within flows
>> (packet length variations, inter arrival times, etc) you might be interested
>> in our DIFFUSE (http://caia.swin.edu.au/urp/diffuse) work. We've extended
>> FreeBSD's ipfw firewall code so that it can recognise traffic based on statistical
>> characteristics, and use this (rather than direct packet inspection) to trigger
>> e.g. rate shaping, etc. Although our prototype code was initially developed
>> for FreeBSD, we've got a preliminary Linux port too. The website contains an
>> overview description, docs and patch files against FreeBSD and Linux source.
>
> Thanks, it looks really interesting and it seem to be what I have been
> looking for :-)

Cool :)

>
> I have only skimmed the code, but it looks like you have
> implemented/ported ipfw to Linux in-order to run your module on top of
> that. An interesting approach.

To give credit where it is due, ipfw & dummynet were ported to Linux by
others (Luigi Rizzo, I believe, http://info.iet.unipi.it/~luigi/dummynet/)
and we've just tweaked our DIFFUSE patches to ipfw so they work in the
Linux context as well. For prototype-level definitions of "work", naturally ;)

cheers,
gja

Re: [Bloat] Dealing with P2P traffic in modern networks - measurement, identification, and control

[David Täht]

On 09/28/2011 11:03 AM, Jesper Louis Andersen wrote:
> On Wed, Sep 28, 2011 at 19:40, Jesper Dangaard Brouer <hawk@comx.dk> wrote:
>
>> If anybody else on the list have links/articles relating to behavioral
>> traffic classification, I'm interested! :-)
> My eTorrent uTP branch almost but not quite implement uTP in Erlang.
> It uses a LEDBAT classifier in its base, and the idea is definitely
> alluring. I have not had time testing it yet though, so I don't know
> anything about how it fares or don't. The "almost but not quite" part
> is that I skipped over some things in uTP to make other parts work
> first. I do have the LEDBAT scheduler but miss important things like
> Nagles algorithm, etc. The thesis Dave linked is quite interesting,
> but I have not yet even skimmed it.
>
I finished reading the paper today, finally, on the train from mantes la
jolie into Paris....

Aside from admiration for the scope and breadth of the analysis
(particuarly chapter 8 forward - I'm rather dubious about most
classification teniques)...

My principal question on all that work... All the models/sims were over
ethernet and assumed fairly high bandwidth connectivity.

I would really like to take a recent model showing 'fairness' of a
network protocol such as LEDBAT, and apply it to a set of wireless
connections that exhibiting the kinds of behaviors we've been seeing
(exorbitant buffering and retries)  The results should be... interesting.

Perhaps the models used here - or the real implementation(s) - can be
fed buffer sizes in excess of 1064, and delays in excess of 2 seconds
... over fairly long paths.

Re: [Bloat] Dealing with P2P traffic in modern networks - measurement, identification, and control

[grenville armitage]

Apologies for re-animating an old thread, but please see
below for something possibly of tangential interest to
the list.

On 09/29/2011 06:51, grenville armitage wrote:
>
>
> On 09/29/2011 03:40, Jesper Dangaard Brouer wrote:
>>
>> Thanks Dave,
>>
>> I have always had the dream of implementing a behavioural based traffic
>> classification Netfilter module. But I have been unable to find some
>> good research in this area, this might be the answer :-)
>>
>> If anybody else on the list have links/articles relating to behavioral
>> traffic classification, I'm interested! :-)
>
> If by "behavior" you're referring to the statistical patterns within flows
> (packet length variations, inter arrival times, etc) you might be interested
> in our DIFFUSE (http://caia.swin.edu.au/urp/diffuse) work. We've extended
> FreeBSD's ipfw firewall code so that it can recognise traffic based on statistical
> characteristics, and use this (rather than direct packet inspection) to trigger
> e.g. rate shaping, etc. Although our prototype code was initially developed
> for FreeBSD, we've got a preliminary Linux port too. The website contains an
> overview description, docs and patch files against FreeBSD and Linux source.

After some poking, prodding and hair-pulling over the past few months
we've released a prototype implementation of our DIFFUSE system ported
to OpenWRT -- http://caia.swin.edu.au/urp/diffuse/openwrt/

We have specifically implemented and tested DIFFUSE running on a
TP-Link WR1043ND using OpenWRT, and documented the work in tech report
(http://caia.swin.edu.au/reports/120412A/CAIA-TR-120412A.pdf)
In particular, we demonstrated the ability of a DIFFUSE-enabled WR1043ND
to detect & protect certain online game traffic (without knowing ports and
addresses a priori) from queuing delays usually induced by bulk TCP cross
traffic, and do so at line rates quite suitable for typical ADSL2+ environments.

This is prototype work, not rigorously debugged nor read for prime time. But
we hope it is nevertheless of some interest!

cheers,
gja