[Codel] question about "data center" models

[Codel] question about "data center" models

[Kathleen Nichols]

So, I thought I'd see what happens with a 1G bottleneck and a 5ms RTT.
I'm still checking
it out, but I thought I'd try to understand the concern.  So, under a
constant heavy load,
codel will accept a 5ms standing queue. I got the impression this was a
concern. I'm
not exactly certain why. How much buffer is normally in the bottleneck?
If it's enough
for a "nominal" RTT of 100 ms, with drop tail it will just fill up and
stay there under
load. So, 100ms of delay. If it's at a RTT of 5ms, then the buffer will
fill up and give 5ms
of delay without any ability to absorb bursts.  What is the normal
configuration?

	Kathie

Re: [Codel] question about "data center" models

[Andrew McGregor]

[-- Attachment #1: Type: text/plain, Size: 1718 bytes --]

A normal configuration is more like a 300 microsecond RTT with early marking RED and/or switches configured to backpressure sources with fairly short queues.

There may be enormous amounts of buffer in a switch, but it is usually divided among a rather large number of queues (ingress and egress queues per port, plus perhaps several in the forwarder).

Data center operators are very concerned about latency because in most cases the metric they are tracking is the completion time of the last short flow to complete in a transaction (that will be composed of, say, 20-odd 500kB or thereabouts TCP connections).  There is often a hard time limit on transaction components; one I've heard of is that all data for a query must be at the aggregation node in 8ms, or it will be dropped and the query result quality will go down.

On 4/05/2012, at 11:23 AM, Kathleen Nichols wrote:

> 
> So, I thought I'd see what happens with a 1G bottleneck and a 5ms RTT.
> I'm still checking
> it out, but I thought I'd try to understand the concern.  So, under a
> constant heavy load,
> codel will accept a 5ms standing queue. I got the impression this was a
> concern. I'm
> not exactly certain why. How much buffer is normally in the bottleneck?
> If it's enough
> for a "nominal" RTT of 100 ms, with drop tail it will just fill up and
> stay there under
> load. So, 100ms of delay. If it's at a RTT of 5ms, then the buffer will
> fill up and give 5ms
> of delay without any ability to absorb bursts.  What is the normal
> configuration?
> 
> 	Kathie
> _______________________________________________
> Codel mailing list
> Codel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/codel


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 2330 bytes --]

Re: [Codel] question about "data center" models

[Dave Taht]

The DCB work is on networks that run with a native
RTT the width of a data center at the speed of light in the medium.

Projected speed ranges are in the 100GB/sec or higher range.

While there was early work on congestion control for that,
it seemed to have stalled out.

http://www.ieee802.org/1/pages/802.1au.html

overview: (can't find a better one right now)

http://www.ieee802.org/1/pages/dcbridges.html

http://www.ieee802.org/1/pages/802.1az.html

While there is underlying support for lossless protocols via
various forms of hardware pause frames, there is an intent to run tcp over it.


On Thu, May 3, 2012 at 4:36 PM, Andrew McGregor <andrewmcgr@gmail.com> wrote:
> A normal configuration is more like a 300 microsecond RTT with early marking RED and/or switches configured to backpressure sources with fairly short queues.
>
> There may be enormous amounts of buffer in a switch, but it is usually divided among a rather large number of queues (ingress and egress queues per port, plus perhaps several in the forwarder).
>
> Data center operators are very concerned about latency because in most cases the metric they are tracking is the completion time of the last short flow to complete in a transaction (that will be composed of, say, 20-odd 500kB or thereabouts TCP connections).  There is often a hard time limit on transaction components; one I've heard of is that all data for a query must be at the aggregation node in 8ms, or it will be dropped and the query result quality will go down.
>
> On 4/05/2012, at 11:23 AM, Kathleen Nichols wrote:
>
>>
>> So, I thought I'd see what happens with a 1G bottleneck and a 5ms RTT.
>> I'm still checking
>> it out, but I thought I'd try to understand the concern.  So, under a
>> constant heavy load,
>> codel will accept a 5ms standing queue. I got the impression this was a
>> concern. I'm
>> not exactly certain why. How much buffer is normally in the bottleneck?
>> If it's enough
>> for a "nominal" RTT of 100 ms, with drop tail it will just fill up and
>> stay there under
>> load. So, 100ms of delay. If it's at a RTT of 5ms, then the buffer will
>> fill up and give 5ms
>> of delay without any ability to absorb bursts.  What is the normal
>> configuration?
>>
>>       Kathie
>> _______________________________________________
>> Codel mailing list
>> Codel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/codel
>
>
> _______________________________________________
> Codel mailing list
> Codel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/codel
>



-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://www.bufferbloat.net