[Codel] Fwd: Re: [tsvwg] new draft on CoDel AQM

[Codel] Fwd: Re: [tsvwg] new draft on CoDel AQM

[Kathleen Nichols]

Fred,

My reply all to your message bounced from the Cisco list...on
the assumption that perhaps some of those folks are on the
bufferbloat codel list, I'll forward there. Also, your points
are probably of general interest.

	Kathie


-------- Original Message --------
Subject: Re: [tsvwg] new draft on CoDel AQM
Date: Fri, 13 Jul 2012 18:49:43 -0700
From: Kathleen Nichols <nichols@pollere.com>
To: Fred Baker (fred) <fred@cisco.com>
CC: van@parc.com <van@parc.com>,  buffer_bloat(mailer list)
<buffer_bloat@cisco.com>

On 7/12/12 10:36 PM, Fred Baker (fred) wrote:
>> 3.2. About the target value
>> 
>> The target value constant is the maximum acceptable standing queue 
>> delay under a prolonged overload. Our initial focus with CoDel was
>> on devices for the ordinary open internet where bottleneck
>> standing queues of a few milliseconds are acceptable for ordinary
>> internet traffic. We experimented with values between 1 and 20
>> milliseconds to determine a target that gives a consistently high
>> utilization while controlling delay across a range of bandwidths,
>> RTTs, and traffic loads. Below a target of 5ms, utilization suffers
>> for some conditions and traffic loads, above 5ms we saw very little
>> or no improvement in utilization. Thus target SHOULD be set to
>> 5ms.
> 
> A humble question from a mere engineer, if you please. I am an
> egg...
> 
> The Sprint paper on jitter in INFOCOMM 2004 would tend to agree with
> that; within the 90th percentile, they measured variation in queuing
> delay on the order of 1 ms and peaks (which they described as
> "frequent") to perhaps 10 ms. They were also dealing with a 2.5 GBPS
> network, arguably one of the best-engineered in the world.
> 
> If I look at a 64 KBPS or T-1/E-1 link, which are still common
> outside the broadband zone comprizing eastern Asta and ANZ,
> Anglophone North America, and Western Europe, 5 ms is smaller than
> the duration of a 1 MTU packet (~8*1500 = 12000 bit times). At that
> point, I find myself questioning this limit.
>
> Question: would you object to making that be something like max(5, N
> * 1000/(8 * MTU / bit rate)) ms, for some appropriate value of N?
> What would you set N to? Kathleen, your simulation studies at Cisco
> circa 1998 suggested 30 ms, which would be on the order of N=3...


Yes, I sloppily did not put a complete discussion of target in the
"about target" section.
Which is why an internet draft is a "work in progress"!  The first
paragraph of section 3.1 does cover this, but it does need to be in 3.2.
I'm proposing to add the following paragraph to section 3.2 in the next
revision of this draft:

"By inhibiting drops when there is less than an (outbound link) MTU
worth of bytes in the buffer, CoDel adapts to very low bandwidth links.
This is shown in [CODEL2012] and interested parties should see the
discussion of results there. Unpublished studies were carried out down
to 64Kbps."

This was one of the findings that I found really exciting - this seemed
to work well, but
of course further studies should be carried out. But, at this point, I
*would* object to
the tie to "some appropriate value of N" - unless, of course, there is
data suggesting
otherwise.

Ah. That 30ms value came from analysis backed up with simulation
...BUT...it was for a
completely different approach, for "REDlight". For CoDel, I didn't see
any gains in utilization above about 8ms, and the gains between 5 and
8ms were largely non-existent
and not across the board. So...that was for 1998 apples, not 2012 oranges.

> 
>> CoDel has to see sojourn times that remain above target for an
>> entire interval in order to enter the drop state.
> 
> or mark? I found ECN mentioned once in your document but not in your
> pseudocode.

Yes, you could mark rather than drop, but my studies didn't cover that
and there will be
different properties in the effect on delay. Further, for the open
internet, I have concerns
about the interaction of ECN traffic with non-ECN traffic and how to
ensure that ECN
is being used properly. So, I can't make a definitive statement on how
something I didn't
test will work.

> 
>> 3.3. About the interval
>> 
>> The interval constant is loosely related to RTT since it is chosen
>> to give endpoints time to react without being so long that
>> response times suffer.  A setting of 100ms works well across a
>> range of RTTs from 10ms to 1 second (excellent performance is
>> achieved in the range from 10 ms to 300ms). For devices intended
>> for the normal, terrestrial internet interval SHOULD have the value
>> of 100ms.
> 
> For very low RTT environments, such as data centers, or very high
> delay environments such as multi-hop satcom, is "interval" a
> configuration constant?
> 
> 

Well, there are words about that in Section 5. Testing and analysis can
determine
appropriate settings for those circumstances.

This seems to just be cross-posted to a Cisco internal list? This was
not meant for IETF
discussion?

Re: [Codel] Fwd: Re: [tsvwg] new draft on CoDel AQM

[Dave Taht]

I'm not sure if fred is on the codel list...

On Fri, Jul 13, 2012 at 9:53 PM, Kathleen Nichols <nichols@pollere.com> wrote:

>>
>>> 3.3. About the interval
>>>
>>> The interval constant is loosely related to RTT since it is chosen
>>> to give endpoints time to react without being so long that
>>> response times suffer.  A setting of 100ms works well across a
>>> range of RTTs from 10ms to 1 second (excellent performance is
>>> achieved in the range from 10 ms to 300ms). For devices intended
>>> for the normal, terrestrial internet interval SHOULD have the value
>>> of 100ms.
>>
>> For very low RTT environments, such as data centers, or very high
>> delay environments such as multi-hop satcom, is "interval" a
>> configuration constant?

Eric Dumazet has run many of his (targetted at data center) 10GigE
benchmarks with a target delay of 500us and an interval of 20, with
the default packet limit of 10k packets.

5ms is an eternity at those speeds. Overrunning the packet limit
results in "interesting" drop tail behavior, too.

The reference linux implementation allows for setting target,
interval, ecn enablement and packet limits. Additionally the fq_codel
implementation allows for alternate quantums and number of flows in
the hashing algorithm.


Having not read this draft... I do have some real world cautions to make.

IF an ethernet driver (under linux) isn't soft limited (htb or hfsc)
or has does NOT have BQL enabled, the underlying buffering in the
stack can/will defeat codel/fq_codel's management utterly.

On wifi, which features wildly random delays in the .2-20 second
range, I am presently getting poor utilization at 13ms and 27ms
targets in fq_codel.  I frankly didn't expect much at this stage of
the game there (only got one of the core patches last week) - there's
too much work left to be done at the wifi driver layer to draw any
conclusions from the data points I have so far.

I think the ecn implementation of codel in linux needs to be thunk on harder.




>>
>>
>
> Well, there are words about that in Section 5. Testing and analysis can
> determine
> appropriate settings for those circumstances.
>
> This seems to just be cross-posted to a Cisco internal list? This was
> not meant for IETF
> discussion?
>
>
> _______________________________________________
> Codel mailing list
> Codel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/codel



-- 
Dave Täht
http://www.bufferbloat.net/projects/cerowrt/wiki - "3.3.8-6 is out
with fq_codel!"