Re: [Bloat] Jumbo frames and LAN buffers

Re: [Bloat] Jumbo frames and LAN buffers

[Richard Scheffenegger]

Also found this:
http://www.stanford.edu/~balaji/papers/QCN.pdf

Jim, you may notice that the congestion feedback probability function looks 
just like the basic RED marking function :)

Regards,
  Richard

----- Original Message ----- 
From: "Richard Scheffenegger" <rscheff@gmx.at>
To: "Kevin Gross" <kevin.gross@avanw.com>; <bloat@lists.bufferbloat.net>
Sent: Monday, May 16, 2011 8:36 PM
Subject: Re: [Bloat] Jumbo frames and LAN buffers


> Kevin,
>
>> My understanding is that 802.1au, "lossless Ethernet", was designed
>> primarily to allow Fibre Channel to be carried over 10 GbE so that SAN 
>> and
>> LAN can share a common infrastructure in datacenters. I don't believe 
>> anyone
>> intends for it to be enabled for traffic classes carrying TCP.
>
> Well, QCN requires a L2 MAC sender, network and receiver cooperation (thus 
> you need fancy "CNA" converged network adapters, to start using it - these 
> would be reaction/reflection points; plus the congestion points - 
> switches - would need HW support too; nothing one can buy today; 
> higher-grade (carrier?) switches may have the reaction/reflection points 
> built into them, and could use legacy 802.3x signalling outside the 
> 802.1Qau cloud).
>
> The following may be too simplistic
>
> Once the hardware has a reaction point support, it classifies traffic, and 
> calculates the per flow congestion of the path (with flow really being the 
> classification rules by the sender), the intermediates / receiver sample 
> the flow and return the congestion back to the sender - and within the 
> sender, a token bucket-like rate limiter will adjust the sending rate of 
> the appropriate flow(s) to adjust to the observed network conditions.
>
> http://www.stanford.edu/~balaji/presentations/au-prabhakar-qcn-description.pdf
> http://www.ieee802.org/1/files/public/docs2007/au-pan-qcn-details-053007.pdf
>
> The congestion control loop has a lot of similarities to TCP CC as you 
> will note...
>
> Also, I haven't found out how fine-grained the classification is supposed 
> to be (per L2 address pair? Group of flows? Which hashing then to use for 
> mapping L2 flows into those groups between reaction/congestion/reflection 
> points...).
>
>
> Anyway, for the here and now, this is pretty much esoteric stuff not 
> relevant in this context :)
>
> Best regards,
>  Richard
>
> ----- Original Message ----- 
> From: "Kevin Gross" <kevin.gross@avanw.com>
> To: <bloat@lists.bufferbloat.net>
> Sent: Monday, May 16, 2011 3:15 PM
> Subject: Re: [Bloat] Jumbo frames and LAN buffers
>
>
>> All the stand-alone switches I've looked at recently either do not 
>> support
>> 802.3x or support it in the (desireable) manner described in the last
>> paragraph of the linked blog post. I don't believe Ethernet flow control 
>> is
>> a factor in current LANs. I'd be interested to know the specifics if 
>> anyone
>> sees it differently.
>>
>> My understanding is that 802.1au, "lossless Ethernet", was designed
>> primarily to allow Fibre Channel to be carried over 10 GbE so that SAN 
>> and
>> LAN can share a common infrastructure in datacenters. I don't believe 
>> anyone
>> intends for it to be enabled for traffic classes carrying TCP.
>>
>> Kevin Gross
>>
>> -----Original Message-----
>> From: bloat-bounces@lists.bufferbloat.net
>> [mailto:bloat-bounces@lists.bufferbloat.net] On Behalf Of Jim Gettys
>> Sent: Monday, May 16, 2011 5:24 AM
>> To: bloat@lists.bufferbloat.net
>> Subject: Re: [Bloat] Jumbo frames and LAN buffers
>>
>> Not necessarily out of knowledge or desire (since it isn't usually
>> controllable in the small switches you buy for home).  It can cause
>> trouble even in small environments as your house.
>>
>> http://virtualthreads.blogspot.com/2006/02/beware-ethernet-flow-control.html
>>
>> I know I'm at least three consumer switches deep, and it's not by choice.
>>                     - Jim
>>
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
> 

Re: [Bloat] Jumbo frames and LAN buffers

[Richard Scheffenegger]

Kevin,

> My understanding is that 802.1au, "lossless Ethernet", was designed
> primarily to allow Fibre Channel to be carried over 10 GbE so that SAN and
> LAN can share a common infrastructure in datacenters. I don't believe 
> anyone
> intends for it to be enabled for traffic classes carrying TCP.

Well, QCN requires a L2 MAC sender, network and receiver cooperation (thus 
you need fancy "CNA" converged network adapters, to start using it - these 
would be reaction/reflection points; plus the congestion points - switches - 
would need HW support too; nothing one can buy today; higher-grade 
(carrier?) switches may have the reaction/reflection points built into them, 
and could use legacy 802.3x signalling outside the 802.1Qau cloud).

The following may be too simplistic

Once the hardware has a reaction point support, it classifies traffic, and 
calculates the per flow congestion of the path (with flow really being the 
classification rules by the sender), the intermediates / receiver sample the 
flow and return the congestion back to the sender - and within the sender, a 
token bucket-like rate limiter will adjust the sending rate of the 
appropriate flow(s) to adjust to the observed network conditions.

http://www.stanford.edu/~balaji/presentations/au-prabhakar-qcn-description.pdf
http://www.ieee802.org/1/files/public/docs2007/au-pan-qcn-details-053007.pdf

The congestion control loop has a lot of similarities to TCP CC as you will 
note...

Also, I haven't found out how fine-grained the classification is supposed to 
be (per L2 address pair? Group of flows? Which hashing then to use for 
mapping L2 flows into those groups between reaction/congestion/reflection 
points...).


Anyway, for the here and now, this is pretty much esoteric stuff not 
relevant in this context :)

Best regards,
  Richard

----- Original Message ----- 
From: "Kevin Gross" <kevin.gross@avanw.com>
To: <bloat@lists.bufferbloat.net>
Sent: Monday, May 16, 2011 3:15 PM
Subject: Re: [Bloat] Jumbo frames and LAN buffers


> All the stand-alone switches I've looked at recently either do not support
> 802.3x or support it in the (desireable) manner described in the last
> paragraph of the linked blog post. I don't believe Ethernet flow control 
> is
> a factor in current LANs. I'd be interested to know the specifics if 
> anyone
> sees it differently.
>
> My understanding is that 802.1au, "lossless Ethernet", was designed
> primarily to allow Fibre Channel to be carried over 10 GbE so that SAN and
> LAN can share a common infrastructure in datacenters. I don't believe 
> anyone
> intends for it to be enabled for traffic classes carrying TCP.
>
> Kevin Gross
>
> -----Original Message-----
> From: bloat-bounces@lists.bufferbloat.net
> [mailto:bloat-bounces@lists.bufferbloat.net] On Behalf Of Jim Gettys
> Sent: Monday, May 16, 2011 5:24 AM
> To: bloat@lists.bufferbloat.net
> Subject: Re: [Bloat] Jumbo frames and LAN buffers
>
> Not necessarily out of knowledge or desire (since it isn't usually
> controllable in the small switches you buy for home).  It can cause
> trouble even in small environments as your house.
>
> http://virtualthreads.blogspot.com/2006/02/beware-ethernet-flow-control.html
>
> I know I'm at least three consumer switches deep, and it's not by choice.
>                     - Jim
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
> 

Re: [Bloat] Jumbo frames and LAN buffers

[Jim Gettys]

On 05/16/2011 09:42 AM, Kevin Gross wrote:
> I would like to try this. Can you suggest specific equipment to look at. Due
> to integration and low port count, most of the cheap consumer stuff has
> surprisingly good layer-2 performance. I've tested a bunch of Linksys and
> other small/medium business 5 to 24 port gigabit switches. Since I measure
> latency, I expect I would have noticed if flow control were kicking in.

I think I was using a D-Link DGS2208. (8 port consumer switch).

I then went and looked at the spec sheets of some of the other consumer 
kit out there and found they all had the "feature" of 802.3 flow control.

I may have been using iperf to tickle it, rather than ssh.

I was also playing around with an old 100Mbps switch, as documented in 
my blog; I don't remember if I saw it there.
                         - Jim

> Kevin Gross
>
> -----Original Message-----
> From: bloat-bounces@lists.bufferbloat.net
> [mailto:bloat-bounces@lists.bufferbloat.net] On Behalf Of Jim Gettys
> Sent: Monday, May 16, 2011 7:23 AM
> To: bloat@lists.bufferbloat.net
> Subject: Re: [Bloat] Jumbo frames and LAN buffers
>
> On 05/16/2011 09:15 AM, Kevin Gross wrote:
>> All the stand-alone switches I've looked at recently either do not support
>> 802.3x or support it in the (desireable) manner described in the last
>> paragraph of the linked blog post. I don't believe Ethernet flow control
> is
>> a factor in current LANs. I'd be interested to know the specifics if
> anyone
>> sees it differently.
> Heh.  Plug wireshark into current off the shelf cheap consumer switches
> intended for the home.  You won't like what you see.  And you have no
> way to manage them.  I was quite surprised last fall when doing my home
> experiments to see 802.3 frames; I had been blissfully unaware of its
> existence, and had to go read up on it as a result.
>
> I don't think any of the enterprise switches are so brain damaged.  So i
> suspect it's mostly lurking to cause trouble in home and small office
> environments, exactly where no-one will know what's going on.
>                           - Jim
>
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Jumbo frames and LAN buffers

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 2196 bytes --]

On Mon, May 16, 2011 at 7:42 AM, Kevin Gross <kevin.gross@avanw.com> wrote:

> I would like to try this. Can you suggest specific equipment to look at.
> Due
> to integration and low port count, most of the cheap consumer stuff has
> surprisingly good layer-2 performance. I've tested a bunch of Linksys and
> other small/medium business 5 to 24 port gigabit switches. Since I measure
> latency, I expect I would have noticed if flow control were kicking in.
>

I would certainly appreciate more people looking at the switch in the
wndr3700v2 we're using on the bismark project.

I'm seeing some pretty deep buffering on it

>
> Kevin Gross
>
> -----Original Message-----
> From: bloat-bounces@lists.bufferbloat.net
> [mailto:bloat-bounces@lists.bufferbloat.net] On Behalf Of Jim Gettys
> Sent: Monday, May 16, 2011 7:23 AM
> To: bloat@lists.bufferbloat.net
> Subject: Re: [Bloat] Jumbo frames and LAN buffers
>
> On 05/16/2011 09:15 AM, Kevin Gross wrote:
> > All the stand-alone switches I've looked at recently either do not
> support
> > 802.3x or support it in the (desireable) manner described in the last
> > paragraph of the linked blog post. I don't believe Ethernet flow control
> is
> > a factor in current LANs. I'd be interested to know the specifics if
> anyone
> > sees it differently.
>
> Heh.  Plug wireshark into current off the shelf cheap consumer switches
> intended for the home.  You won't like what you see.  And you have no
> way to manage them.  I was quite surprised last fall when doing my home
> experiments to see 802.3 frames; I had been blissfully unaware of its
> existence, and had to go read up on it as a result.
>
> I don't think any of the enterprise switches are so brain damaged.  So i
> suspect it's mostly lurking to cause trouble in home and small office
> environments, exactly where no-one will know what's going on.
>                         - Jim
>
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>



-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://the-edge.blogspot.com

[-- Attachment #2: Type: text/html, Size: 3233 bytes --]

Re: [Bloat] Jumbo frames and LAN buffers

[Kevin Gross]

I would like to try this. Can you suggest specific equipment to look at. Due
to integration and low port count, most of the cheap consumer stuff has
surprisingly good layer-2 performance. I've tested a bunch of Linksys and
other small/medium business 5 to 24 port gigabit switches. Since I measure
latency, I expect I would have noticed if flow control were kicking in.

Kevin Gross

-----Original Message-----
From: bloat-bounces@lists.bufferbloat.net
[mailto:bloat-bounces@lists.bufferbloat.net] On Behalf Of Jim Gettys
Sent: Monday, May 16, 2011 7:23 AM
To: bloat@lists.bufferbloat.net
Subject: Re: [Bloat] Jumbo frames and LAN buffers

On 05/16/2011 09:15 AM, Kevin Gross wrote:
> All the stand-alone switches I've looked at recently either do not support
> 802.3x or support it in the (desireable) manner described in the last
> paragraph of the linked blog post. I don't believe Ethernet flow control
is
> a factor in current LANs. I'd be interested to know the specifics if
anyone
> sees it differently.

Heh.  Plug wireshark into current off the shelf cheap consumer switches 
intended for the home.  You won't like what you see.  And you have no 
way to manage them.  I was quite surprised last fall when doing my home 
experiments to see 802.3 frames; I had been blissfully unaware of its 
existence, and had to go read up on it as a result.

I don't think any of the enterprise switches are so brain damaged.  So i 
suspect it's mostly lurking to cause trouble in home and small office 
environments, exactly where no-one will know what's going on.
                         - Jim

Re: [Bloat] Jumbo frames and LAN buffers

[Jim Gettys]

On 05/16/2011 09:15 AM, Kevin Gross wrote:
> All the stand-alone switches I've looked at recently either do not support
> 802.3x or support it in the (desireable) manner described in the last
> paragraph of the linked blog post. I don't believe Ethernet flow control is
> a factor in current LANs. I'd be interested to know the specifics if anyone
> sees it differently.

Heh.  Plug wireshark into current off the shelf cheap consumer switches 
intended for the home.  You won't like what you see.  And you have no 
way to manage them.  I was quite surprised last fall when doing my home 
experiments to see 802.3 frames; I had been blissfully unaware of its 
existence, and had to go read up on it as a result.

I don't think any of the enterprise switches are so brain damaged.  So i 
suspect it's mostly lurking to cause trouble in home and small office 
environments, exactly where no-one will know what's going on.
                         - Jim

> My understanding is that 802.1au, "lossless Ethernet", was designed
> primarily to allow Fibre Channel to be carried over 10 GbE so that SAN and
> LAN can share a common infrastructure in datacenters. I don't believe anyone
> intends for it to be enabled for traffic classes carrying TCP.
>
> Kevin Gross
>
> -----Original Message-----
> From: bloat-bounces@lists.bufferbloat.net
> [mailto:bloat-bounces@lists.bufferbloat.net] On Behalf Of Jim Gettys
> Sent: Monday, May 16, 2011 5:24 AM
> To: bloat@lists.bufferbloat.net
> Subject: Re: [Bloat] Jumbo frames and LAN buffers
>
> Not necessarily out of knowledge or desire (since it isn't usually
> controllable in the small switches you buy for home).  It can cause
> trouble even in small environments as your house.
>
> http://virtualthreads.blogspot.com/2006/02/beware-ethernet-flow-control.html
>
> I know I'm at least three consumer switches deep, and it's not by choice.
>                       - Jim
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Jumbo frames and LAN buffers

[Kevin Gross]

All the stand-alone switches I've looked at recently either do not support
802.3x or support it in the (desireable) manner described in the last
paragraph of the linked blog post. I don't believe Ethernet flow control is
a factor in current LANs. I'd be interested to know the specifics if anyone
sees it differently.

My understanding is that 802.1au, "lossless Ethernet", was designed
primarily to allow Fibre Channel to be carried over 10 GbE so that SAN and
LAN can share a common infrastructure in datacenters. I don't believe anyone
intends for it to be enabled for traffic classes carrying TCP.

Kevin Gross

-----Original Message-----
From: bloat-bounces@lists.bufferbloat.net
[mailto:bloat-bounces@lists.bufferbloat.net] On Behalf Of Jim Gettys
Sent: Monday, May 16, 2011 5:24 AM
To: bloat@lists.bufferbloat.net
Subject: Re: [Bloat] Jumbo frames and LAN buffers

Not necessarily out of knowledge or desire (since it isn't usually 
controllable in the small switches you buy for home).  It can cause 
trouble even in small environments as your house.

http://virtualthreads.blogspot.com/2006/02/beware-ethernet-flow-control.html

I know I'm at least three consumer switches deep, and it's not by choice.
                     - Jim

Re: [Bloat] Jumbo frames and LAN buffers

[Jim Gettys]

On 05/16/2011 05:49 AM, Fred Baker wrote:
> On May 16, 2011, at 9:51 AM, Richard Scheffenegger wrote:
>
>> Second, you wouldn't want to deploy basic 802.3x to any network consisting of more than a single switch.
> actually, it's pretty common practice. Three layers, even. People build backbones, and then ring them with workgroup switches, and then put small switches on their desks.
>
Not necessarily out of knowledge or desire (since it isn't usually 
controllable in the small switches you buy for home).  It can cause 
trouble even in small environments as your house.

http://virtualthreads.blogspot.com/2006/02/beware-ethernet-flow-control.html

I know I'm at least three consumer switches deep, and it's not by choice.
                     - Jim