From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-iy0-f171.google.com (mail-iy0-f171.google.com [209.85.210.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 1A43E201745 for ; Fri, 13 May 2011 13:13:24 -0700 (PDT) Received: by iyi20 with SMTP id 20so3674634iyi.16 for ; Fri, 13 May 2011 13:21:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=vxAKdt4TekqKxFTln6EIBgFfmQqwm67hEJrJ3/p3jaE=; b=gngupQrOJHlTsWLVuRnbPwDWGhi5K291E/9uKj2FVXqTM3ibhLa1gRg1BJby/eiq+U kPAq4/4Xy2/6o1YiOrC+4786gKlMZncy8BQkMJuXQT5gfTNkfHNRiUfyr1kzNDTAt69v +4zeixYhN3ZvqDHBOVU09yvMaWklzcCwo65OQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=tpZYjQMVPlcYR4fKP20I9azodmzFsUKQaq/l2FxlmvCqfAd1oNEQbpZMH5XQf10jKz AFi2BkMvDW6ys1zgea3vMh7H8TQpm59LFTLA7kkI1wZTE20n0T98rKLDH1MTpQiUFvPD +u5GSfIspnOgu2WsApYSTc7VLMiLDYMYif3bE= MIME-Version: 1.0 Received: by 10.231.215.3 with SMTP id hc3mr1449467ibb.156.1305318106278; Fri, 13 May 2011 13:21:46 -0700 (PDT) Received: by 10.231.31.201 with HTTP; Fri, 13 May 2011 13:21:46 -0700 (PDT) In-Reply-To: <-4629065256951087821@unknownmsgid> References: <4DB70FDA.6000507@mti-systems.com> <4DC2C9D2.8040703@freedesktop.org> <20110505091046.3c73e067@nehalam> <6E25D2CF-D0F0-4C41-BABC-4AB0C00862A6@pnsol.com> <35D8AC71C7BF46E29CC3118AACD97FA6@srichardlxp2> <1304964368.8149.202.camel@tardy> <4DD9A464-8845-49AA-ADC4-A0D36D91AAEC@cisco.com> <1305297321.8149.549.camel@tardy> <-4629065256951087821@unknownmsgid> Date: Fri, 13 May 2011 14:21:46 -0600 Message-ID: From: Dave Taht To: Kevin Gross Content-Type: multipart/alternative; boundary=000e0cd4d4dc0b2a8804a32e0e5f Cc: bloat@lists.bufferbloat.net Subject: Re: [Bloat] Jumbo frames and LAN buffers (was: RE: Burst Loss) X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2011 20:13:25 -0000 --000e0cd4d4dc0b2a8804a32e0e5f Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Fri, May 13, 2011 at 2:03 PM, Kevin Gross wrote: > Do we think that bufferbloat is just a WAN problem? I work on live media > applications for LANs and campus networks. I'm seeing what I think could = be > characterized as bufferbloat in LAN equipment. The timescales on 1 Gb > Ethernet are orders of magnitude shorter and the performance problems cau= sed > are in many cases a bit different but root cause and potential solutions > are, I'm hoping, very similar. > > > > Keeping the frame byte size small while the frame time has shrunk maintai= ns > the overhead at the same level. Again, this has been a conscious decision > not a stubborn relic. Ethernet improvements have increased bandwidth by > orders of magnitude. Do we really need to increase it by a couple percent= age > points more by reducing overhead for large payloads? > > > > The cost of that improved marginal bandwidth efficiency is a 6x increase = in > latency. Many applications would not notice an increase from 12 us to 72 = us > for a Gigabit switch hop. But on a large network it adds up, some > applications are absolutely that sensitive (transaction processing, clust= er > computing, SANs) and (I thought I'd be preaching to the choir here) there= 's > no way to ever recover the lost performance. > > > You are preaching to the choir here, but I note several things: Large frame sizes on 10GigE networks to other 10GigE networks is less of a problem than 10GigE to 10Mbit networks. I would hope/expect that frame woul= d fragment in that case. Getting to where latencies are less than 10ms in the general case makes voi= p feasible again. I'm still at well over 300ms on bismark. Enabling higher speed stock market trades and live music exchange over a la= n would be next on my list after getting below 10ms on the local switch/wireless interface! A lot of research points to widely enabling some form of fair queuing at th= e servers and switches to distribute the load at sane levels. (nagle, 89) I think few gig+e vendors are doing that in hardware, and it would be good to know who is and who isn't. For example, the switch I'm using on bismark has all sorts of wonderful QoS features such as fair queuing, but as best as I can tell they are not enabled, and I'm seeing buffering in the switch at well above 20ms.... It is astonishing that a switch chip this capable has reached the consumer marketplace... http://realtek.info/pdf/rtl8366s_8366sr_datasheet_vpre-1.4_20071022.pdf And depressing that so few of it's capabilities have software to configure them. > Kevin Gross > > > > *From:* Dave Taht [mailto:dave.taht@gmail.com] > *Sent:* Friday, May 13, 2011 8:54 AM > *To:* rick.jones2@hp.com > *Cc:* Kevin Gross; bloat@lists.bufferbloat.net > *Subject:* Re: [Bloat] Burst Loss > > > > > > On Fri, May 13, 2011 at 8:35 AM, Rick Jones wrote: > > On Thu, 2011-05-12 at 23:00 -0600, Kevin Gross wrote: > > One of the principal reasons jumbo frames have not been standardized > > is due to latency concerns. I assume this group can appreciate the > > IEEE holding ground on this. > > Thusfar at least, bloaters are fighting to eliminate 10s of milliseconds > of queuing delay. I don't think this list is worrying about the tens of > microseconds difference between the transmission time of a 9000 byte > frame at 1 GbE vs a 1500 byte frame, or the single digit microseconds > difference at 10 GbE. > > > Heh. With the first iteration of the bismark project I'm trying to get t= o > where I have less than 30ms latency under load and have far larger proble= ms > to worry about than jumbo frames. I'll be lucky to manage 1/10th that > (300ms) at this point. > > Not, incidentally that I mind the idea of jumbo frames. It seems silly to > be saddled with default frame sizes that made sense in the 70s, and in an > age where we will be seeing ever more packet encapsulation, reducing the > header size as a ratio to data size strikes me as a very worthy goal. > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat > > --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 http://the-edge.blogspot.com --000e0cd4d4dc0b2a8804a32e0e5f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Fri, May 13, 2011 at 2:03 PM, Kevin G= ross <kevin.g= ross@avanw.com> wrote:

Do we think t= hat bufferbloat is just a WAN problem? I work on live media applications fo= r LANs and campus networks. I'm seeing what I think could be characteri= zed as bufferbloat in LAN equipment. The timescales on 1 Gb Ethernet are or= ders of magnitude shorter and the performance problems caused are in many c= ases a bit different but root cause and potential solutions are, I'm ho= ping, very similar.

=A0

Keeping the frame byte size small while the frame= time has shrunk maintains the overhead at the same level. Again, this has = been a conscious decision not a stubborn relic. Ethernet improvements have = increased bandwidth by orders of magnitude. Do we really need to increase i= t by a couple percentage points more by reducing overhead for large payload= s?

=A0

The cost of that im= proved marginal bandwidth efficiency is a 6x increase in latency. Many appl= ications would not notice an increase from 12 us to 72 us for a Gigabit swi= tch hop. But on a large network it adds up, some applications are absolutel= y that sensitive (transaction processing, cluster computing, SANs) and (I t= hought I'd be preaching to the choir here) there's no way to ever r= ecover the lost performance.

=A0


You are preaching to = the choir here, but I note several things:

Large frame sizes on 10Gi= gE networks to other 10GigE networks is less of a problem than 10GigE to 10= Mbit networks. I would hope/expect that frame would fragment in that case.<= br>
Getting to where latencies are less than 10ms in the general case makes= voip feasible again. I'm still at well over 300ms on bismark.

E= nabling higher speed stock market trades and live music exchange over a lan= would be next on my list after getting below 10ms on the local switch/wire= less interface!

A lot of research points to widely enabling some form of fair queuing a= t the servers and switches to distribute the load at sane levels. (nagle, 8= 9) I think few gig+e vendors are doing that in hardware, and it would be go= od to know who is and who isn't.

For example, the switch I'm using on bismark has all sorts of wonde= rful QoS features such as fair queuing, but as best as I can tell they are = not enabled, and I'm seeing buffering in the switch at well above 20ms.= ...

It is astonishing that a switch chip this capable has reached the consu= mer marketplace...

http://realtek.info/pdf/rtl8366s_8366sr_= datasheet_vpre-1.4_20071022.pdf

And depressing that so few of it's capabilities have software to co= nfigure them.

Kevin Gross

=A0

From: Dave Taht [mailt= o:dave.taht@gmail.= com]
Sent: Friday, May 13, 2011 8:54 AM
To: rick.jones2@hp.com
Cc: K= evin Gross; bloat@lists.bufferbloat.net
Subject: Re: [Bloat] Burst Loss

=A0

=A0

On Fri, May 13, 2011 at 8:35 AM, Rick Jones <rick.jones2@hp.com= > wrote:

On Thu, 2011-05-= 12 at 23:00 -0600, Kevin Gross wrote:
> One of the principal reasons = jumbo frames have not been standardized
> is due to latency concerns.= I assume this group can appreciate the
> IEEE holding ground on this.

Thusfar a= t least, bloaters are fighting to eliminate 10s of milliseconds
of queui= ng delay. =A0I don't think this list is worrying about the tens of
m= icroseconds difference between the transmission time of a 9000 byte
frame at 1 GbE vs a 1500 byte frame, or the single digit microseconds
di= fference at 10 GbE.


Heh.=A0 With the first iteration of the bismark project I'm = trying to get to where I have less than 30ms latency under load and have fa= r larger problems to worry about than jumbo frames. I'll be lucky to ma= nage 1/10th that (300ms) at this point.

Not, incidentally that I mind the idea of jumbo frames. It seems silly = to be saddled with default frame sizes that made sense in the 70s, and in a= n age where we will be seeing ever more packet encapsulation, reducing the = header size as a ratio to data size strikes me as a very worthy goal.


_______________________________________________=
Bloat mailing list
Bloat@lists.bufferbloat.net<= /a>
= https://lists.bufferbloat.net/listinfo/bloat




--
Dave T=E4ht
SKYP= E: davetaht
US Tel: 1-239-829-5608
http://the-edge.blogspot.com
--000e0cd4d4dc0b2a8804a32e0e5f--