Re: [Codel] [Bloat] Slightly off-topic: What might be wrong with the diagrams in "Controlling Queue Delay"?

[Jim Gettys <jg@freedesktop.org>]

David,

There is a codel@lists.bufferbloat.net list these days, where both
Kathie and Van hang out among others.  I don't know if they are
subscribed to the bloat@lists.bufferbloat.net list.
                    - Jim


On 05/14/2012 12:00 PM, David Collier-Brown wrote:
> Management Summary
>
>     In figures 5 and 6, the eye can't see the trend. 8 and 9 aren't as
>     bad, but it would be A Good Thing to make the trend explicit.
>
>
> In a blog discussion with Jim Gettys, I noted that at least two of the
> figures in /Controlling Queue Delay,/ by Kathleen Nichols and Van
> Jacobson, were mysteriously unclear.   Jim, not unexpectedly,  asked
> me to be specific (:-)) 
>
> This was hard because I had no idea /why/ I had reacted so badly to
> two diagrams which I understood. After a considerable time, I realized
> what was wrong:I knew what the diagrams were about!  Thus this
> message, to see if the community saw the same thing.  I've written it
> as if I were addressing Van Jacobson...
>
> I knew that 5A, 6A and 7A were all comparisons of delay versus load,
> so I understood that they should have the form I saw.  I already
> expected CoDel to be "better", so I expected the diagrams to
> illustrate that.
>
> However, if I had not already known what to expect, the diagrams would
> have been much less clear.  Two things would have stood out
>
>      1. The trend as the bandwidth increased would not have been obvious
>      2. The behaviour of RED would have looked better than that of CoDel.
>
>
> When I look at any of the "A" diagrams without preconceptions, I see
> two separate pairs of data, separated by a blank space. To my eyes,
> that looks like there is something to compare between each of the data
> in those pairs, or perhaps between the pairs. This is wildly different
> from what the diagrams seek to illustrate!  They really describe a
> curve, with known margins of error, relating packet delay to
> bandwidth.  That's a continuous curve, not four disparate points.
>
> It's unclear why there should be only four points: with this spacing,
> there arguably should be eight, to make the diagram look continuous. 
> Since it's non-linear, perhaps one should consider making the X axis
> logarithmic?
>
>  In fact, it's unclear why there should be points at all and not
> lines.  The useful information is that delay falls in a non-linear
> fashion as bandwidth increases. The box plots tell me about the margin
> of error, and therefor suggest to me that the margin of error's
> important.  After all, that's what's emphasized!  [I had a 5A.gif here]
>
> To make the trend important, it needs to be emphasized.  A curve drawn
> through the medians would probably suffice, but explicitly drawing the
> diagram as a "river" the width of the box, plus a center-line, would
> be much clearer.  {I had a pair of pencil-sketches here]
>
> Finally, when comparing RED to CoDel, it's best to plot them both on
> one graph, so one can see the improvement, and not have to mentally
> subtract one diagram from another.
>
> The second consideration is that the diagrams give the the impression
> that RED is better, as it yields less delay at every point except 3
> MB/S.  Somehow, that seems the very**/opposite/** of what we're trying
> to demonstrate. Are these values reversed? If not, do they support our
> thesis that CoDel is better, or should we seek a better metric?
>
> Similarly, all the "B" diagrams need at least a trend line drawn. 
> Fortunately the CoDel results are clearly better than RED's when
> considering utilization.
>
> Figures 8 and 9 are much better at illustrating the trend of each of
> their data, but could use at least a trend line, if not a redrawing
> into "rivers". 
>
> Figure 7 I simply skipped over: the others were puzzling enough to the
> eye that I deferred trying to puzzle it out until now.
>
> To accompany this, even for an audience of mathematicians, the body of
> the article should say what the diagrams are intended to demonstrate,
> and why that information supports the thesis of the article.
>
> Now, just so you don't think you've blown everything, figures 1 and 2
> clearly illustrate what the text describes, and shows a practitioner
> how this change fixes the "bufferbloat" problem.  They're really
> rather brilliant*, and allow the reader to understand how the fix works.
>
> If they were the only thing the article had shown, it would still have
> made its point.
>
> --dave
> [* I'm a Canadian, and engage in understatement (:-)]
> -- 
> David Collier-Brown,         | Always do right. This will gratify
> System Programmer and Author | some people and astonish the rest
> davecb@spamcop.net           |                      -- Mark Twain
> (416) 223-8968
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat