From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id B382321F0F0; Tue, 9 Jul 2013 00:30:03 -0700 (PDT) Received: by mail-la0-f50.google.com with SMTP id ep20so148575lab.37 for ; Tue, 09 Jul 2013 00:30:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Xs1RgeRUDeiE1NBSV5YNQ4odZPdiReRCdBFRwpj6CFg=; b=qBTBsP46SBzeKq90CpCtQ68jO1bB4vL2W7FFPpVO9oDQOiqKoETgpcgWAheTLkYpqc aLqa1OdvGXce5huoHT3KCHmqkOMRc6kaWZJ/pl1LE6kZIzGjBa1UBrwgXOiEPsohsBGx tBMKotERIiSMzPLc6RsQ9bWC9WN4KTmXkHdSh8kMsxpmB2B/5eBlzimg5ZkifMrrNWou u2GZF+jiF4NZYFrd5BZZpwNMS7HKvb5SVf2oRXi4scle+e7otyiVohrXhuIR7+k+fcuS HwRmcM44FgtWj9a6+vbeuY2IVedsLlmNbLgSMWGZN7HpOhRxs1RBpM2Dsom/InC/rQnY XMIw== MIME-Version: 1.0 X-Received: by 10.152.5.197 with SMTP id u5mr12080480lau.59.1373355000422; Tue, 09 Jul 2013 00:30:00 -0700 (PDT) Received: by 10.112.51.41 with HTTP; Tue, 9 Jul 2013 00:30:00 -0700 (PDT) In-Reply-To: References: <1373223178.486913695@apps.rackspace.com> <871u79x9kb.fsf@toke.dk> Date: Tue, 9 Jul 2013 17:30:00 +1000 Message-ID: From: Andrew McGregor To: Dave Taht Content-Type: multipart/alternative; boundary=089e013d1002f365ea04e10f20ec Cc: =?ISO-8859-1?Q?Toke_H=F8iland=2DJ=F8rgensen?= , codel@lists.bufferbloat.net, cerowrt-devel@lists.bufferbloat.net Subject: Re: [Cerowrt-devel] [Codel] happy 4th! X-BeenThere: cerowrt-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: Development issues regarding the cerowrt test router project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jul 2013 07:30:04 -0000 --089e013d1002f365ea04e10f20ec Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Possibly a better simulation environment than netem would be ns-3's NSC (network simulation cradle), which lets you connect up multiple VMs over an emulated network in userspace... obviously, you better have a multicore system with plenty of resources available, but it works very nicely and needs no physical network at all. ns-3 virtual network nodes also speak real protocols, so you can talk to them with real tools as well (netcat to a ns-3 virtual node, for example, or ping them). I suppose it would be possible also to bridge one of the TAP devices ns-3 is talking on with a real interface. On Tue, Jul 9, 2013 at 4:32 PM, Dave Taht wrote: > this really, really, really is the wrong list for this dialog. cc-ing cod= el > > On Mon, Jul 8, 2013 at 11:04 PM, Mikael Abrahamsson > wrote: > > On Mon, 8 Jul 2013, Toke H=F8iland-J=F8rgensen wrote: > > > >> Did a few test runs on my setup. Here are some figures (can't go highe= r > >> than 100mbit with the hardware I have, sorry). > > > > > > Thanks, much appreciated! > > > > > >> Note that I haven't done tests at 100mbit on this setup before, so can= 't > >> say whether something weird is going on there. I'm a little bit puzzle= d > >> as to why the flows don't seem to get going at all in one direction fo= r > >> the rrul test. I'm guessing it has something to do with TSQ. > > > > > > For me, it shows that FQ_CODEL indeed affects TCP performance negativel= y > for > > long links, however it looks like the impact is only about 20-30%. > > I would be extremely reluctant to draw any conclusions from any test > derived from netem's results at this point. (netem is a qdisc that can > insert delay and loss into a stream) I did a lot of netem testing in > the beginning of the bufferbloat effort and the results differed so > much from what I'd got in the "real world" that I gave up and stuck > with the real world for most of the past couple years. There were in > particular, major problems with combining netem with any other > qdisc... > > > https://www.bufferbloat.net/projects/codel/wiki/Best_practices_for_benchm= arking_Codel_and_FQ_Codel > > One of the simplest problems with netem is that by default it delays > all packets, including things like arp and nd, which are kind of > needed in ethernet... > > That said, now that more problems are understood, toke and I, and > maybe matt mathis are trying to take it on... > > The simulated results with ns2 codel were very good in the range > 2-300ms, but that's not the version of codel in linux. It worked well > up to about 1sec, actually, but fell off afterwards. I have a set of > more ns2-like patches for the ns2 model in cerowrt and as part of my > 3.10 builds that I should release as a deb soon. > > Recently a few major bugs in htb have come to light and been fixed in > the 3.10 series. > > There have also been so many changes to the TCP stack that I'd > distrust comparing tcp results between any given kernel version. The > TSQ addition is not well understood, and I think, but am not sure, > it's both too big for low bandwidths and not big enough for larger > ones... > > and... unlike in the past where tcp was being optimized for > supercomputer center to supercomputer center, the vast majority of tcp > related work is now coming out of google, who are optimizing for short > transfers over short rtts. > > It would be nice to have access to internet2 for more real world testing. > > > > > What's stranger is that latency only goes up to around 230ms from its > 200ms > > "floor" with FIFO, I had expected a bigger increase in buffering with > FIFO. > > TSQ, here, probably. > > > Have you done any TCP tuning? > > Not recently, aside from turning up tsq to higher defaults and lower > defaults without definitive results. > > > Would it be easy for you to do tests with the streams that "loads up th= e > > link" being 200ms RTT, and the realtime flows only having 30-40ms RTT, > > simulating downloads from a high RTT server and doing interactive thing= s > to > > a more local web server. > > It would be a useful workload. Higher on my list is emulating > cablelab's latest tests, which is about the same thing only closer > statistically to what a real web page might look like - except > cablelabs tests don't have the redirects or dns lookups most web pages > do. > > > > > > > > -- > > Mikael Abrahamsson email: swmike@swm.pp.se > > > > _______________________________________________ > > Cerowrt-devel mailing list > > Cerowrt-devel@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/cerowrt-devel > > > > > > -- > Dave T=E4ht > > Fixing bufferbloat with cerowrt: > http://www.teklibre.com/cerowrt/subscribe.html > _______________________________________________ > Codel mailing list > Codel@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/codel > --089e013d1002f365ea04e10f20ec Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Possibly a better simulation environment than netem would = be ns-3's NSC (network simulation cradle), which lets you connect up mu= ltiple VMs over an emulated network in userspace... obviously, you better h= ave a multicore system with plenty of resources available, but it works ver= y nicely and needs no physical network at all. =A0ns-3 virtual network node= s also speak real protocols, so you can talk to them with real tools as wel= l (netcat to a ns-3 virtual node, for example, or ping them). =A0I suppose = it would be possible also to bridge one of the TAP devices ns-3 is talking = on with a real interface.


On Tue, Jul 9= , 2013 at 4:32 PM, Dave Taht <dave.taht@gmail.com> wrote:<= br>
this really, really, really is the wrong list for this dialog. cc-ing codel=

On Mon, Jul 8, 2013 at 11:04 PM, Mikael Abrahamsson <swmike@swm.pp.se> wrote:
> On Mon, 8 Jul 2013, Toke H=F8iland-J=F8rgensen wrote:
>
>> Did a few test runs on my setup. Here are some figures (can't = go higher
>> than 100mbit with the hardware I have, sorry).
>
>
> Thanks, much appreciated!
>
>
>> Note that I haven't done tests at 100mbit on this setup before= , so can't
>> say whether something weird is going on there. I'm a little bi= t puzzled
>> as to why the flows don't seem to get going at all in one dire= ction for
>> the rrul test. I'm guessing it has something to do with TSQ. >
>
> For me, it shows that FQ_CODEL indeed affects TCP performance negative= ly for
> long links, however it looks like the impact is only about 20-30%.

I would be extremely reluctant to draw any conclusions from any test
derived from netem's results at this point. (netem is a qdisc that can<= br> insert delay and loss into a stream) I did a lot of netem testing in
the beginning of the bufferbloat effort and the results differed so
much from what I'd got in the "real world" that I gave up and= stuck
with the real world for most of the past couple years. There were in
particular, major problems with combining netem with any other
qdisc...

https://www.bufferblo= at.net/projects/codel/wiki/Best_practices_for_benchmarking_Codel_and_FQ_Cod= el

One of the simplest problems with netem is that by default it delays
all packets, including things like arp and nd, which are kind of
needed in ethernet...

That said, now that more problems are understood, toke and I, and
maybe matt mathis are trying to take it on...

The simulated results with ns2 codel were very good in the range
2-300ms, but that's not the version of codel in linux. It worked well up to about 1sec, actually, but fell off afterwards. I have a set of
more ns2-like patches for the ns2 model in cerowrt and as part of my
3.10 builds that I should release as a deb soon.

Recently a few major bugs in htb have come to light and been fixed in
the 3.10 series.

There have also been so many changes to the TCP stack that I'd
distrust comparing tcp results between any given kernel version. The
TSQ addition is not well understood, and I think, but am not sure,
it's both too big for low bandwidths and not big enough for larger
ones...

and... unlike in the past where tcp was being optimized for
supercomputer center to supercomputer center, the vast majority of tcp
related work is now coming out of google, who are optimizing for short
transfers over short rtts.

It would be nice to have access to internet2 for more real world testing.
>
> What's stranger is that latency only goes up to around 230ms from = its 200ms
> "floor" with FIFO, I had expected a bigger increase in buffe= ring with FIFO.

TSQ, here, probably.

> Have you done any TCP tuning?

Not recently, aside from turning up tsq to higher defaults and lower
defaults without definitive results.

> Would it be easy for you to do tests with the streams that "loads= up the
> link" being 200ms RTT, and the realtime flows only having 30-40ms= RTT,
> simulating downloads from a high RTT server and doing interactive thin= gs to
> a more local web server.

It would be a useful workload. Higher on my list is emulating
cablelab's latest tests, which is about the same thing only closer
statistically to what a real web page might look like - except
cablelabs tests don't have the redirects or dns lookups most web pages<= br> do.


>
>
> --
> Mikael Abrahamsson =A0 =A0email: s= wmike@swm.pp.se
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@l= ists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>



--
Dave T=E4ht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscrib= e.html
_______________________________________________
Codel mailing list
Codel@lists.bufferbloat.net<= /a>
= https://lists.bufferbloat.net/listinfo/codel

--089e013d1002f365ea04e10f20ec--