[Cake] cake infinite loop(?) with hfsc on one-armed router

[Dave Taht]

On Tue, Jan 8, 2019 at 2:01 PM Pete Heist <pete at heistp.net> wrote:
>
> I should have done that: https://www.heistp.net/downloads/htb_split_gso_patched/
>
> Note that I changed the names in the plots to match the convention of my first email, but it should be clear which is which and I left all plots in. The text output is there too as I sometimes like to open several up in different browser tabs and switch between tabs to compare values.
>
> It looks like about 100 usec to me. Throughput also looks consistently about 0.3 Mbit higher (~1.3%) in the split results.

My guess is with ecn on would have the highest latency and the same
throughput. ?

Since we started this effort in an era when seconds of added latency
was common, a mere 100us improvement seems insignificant, except that
that's a 10% improvement over the present-day baseline, and *that's
worth it*. ;) This is also a function of the number of flows, kernel
scheduling time, etc, etc. Theoretically,
were there no other delays in the system, we seem to actually be at
the minimal RTT achievable (4*130ms in each direction for the fat
flows = 1040us) but given the fast/slow queue abstraction the best
possible result would be 10s of us for the measurement flows.

I have noticed that BQL's values can get quite large with cake doing
the shaping, btw, much larger than they do with htb.

> > On Jan 8, 2019, at 9:44 PM, Dave Taht <dave.taht at gmail.com> wrote:
> >
> > On Tue, Jan 8, 2019 at 12:03 PM Pete Heist <pete at heistp.net> wrote:
> >>
> >> Here’s the re-test with the patched version and HTB. Looks like success, nice work!
> >
> > I note that I'm big on having the flent.gz files around also. In this
> > case, by eyeball, split-gso appears to have about 130us less latency,
> > but a cdf comparison of split vs no-split woud show that more easily.
> >
> >> Split GSO on:
> >>
> >> https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_split1.svg
> >> https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_split2.svg
> >> https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_split3.svg
> >>
> >> Split GSO off:
> >>
> >> https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_no_split1.svg
> >> https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_no_split2.svg
> >> https://www.heistp.net/downloads/htb_split_gso_patched/htb_cakep_no_split3.svg
> >>
> >> Your patch in the latest kernels looks simpler. Bringing the patch back to prior kernel versions would be appreciated, but I can understand how 3.16 becomes less and less relevant as time goes on, although, it’s not at end of life yet. :)
> >>
> >> Interesting how download rate control in each of the graphs with GSO splitting on looks accurate to the point where flent’s throughput graph scale is at 0.02 Mbit per step, and one can see that values coming back from netperf are probably quantized to 0.01 Mbit...
> >>
> >>> On Jan 7, 2019, at 4:07 PM, Pete Heist <pete at heistp.net> wrote:
> >>>
> >>> Sorry, that’s without the patch, will give that a try when I have a chance and post the results, probably tomorrow...
> >>>
> >>>> On Jan 7, 2019, at 12:30 PM, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
> >>>>
> >>>> Pete Heist <pete at heistp.net> writes:
> >>>>
> >>>>>> On Jan 6, 2019, at 9:56 PM, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
> >>>>>>
> >>>>>> Pete Heist <pete at heistp.net> writes:
> >>>>>>
> >>>>>>> Lastly, is using cake as a leaf to htb risky until a fix is made? I’ve
> >>>>>>> been doing that for a while without any apparent issues, though I’m
> >>>>>>> hesitating now to try that in a production environment.
> >>>>>>
> >>>>>> Hmm, that's a good question. I would expect so; but I would also expect
> >>>>>> the issue to show up pretty much straight away, so if you haven't hit it
> >>>>>> yet, I may be wrong. I'll do some more digging... Should probably also
> >>>>>> try to replicate all this stuff on my own machine :)
> >>>>>
> >>>>>
> >>>>> Ok, after what I’m seeing on my APU1 tests on 3.16.7, I’m definitely
> >>>>> not putting split GSO into production. I just turned it on and off
> >>>>> three times and here’s what I got:
> >>>>>
> >>>>> Split GSO on:
> >>>>>
> >>>>> https://www.heistp.net/downloads/htb_split_gso/htb_cake_split_gso.svg
> >>>>> https://www.heistp.net/downloads/htb_split_gso/htb_cake_split_gso2.svg
> >>>>> https://www.heistp.net/downloads/htb_split_gso/htb_cake_split_gso3.svg
> >>>>>
> >>>>> Split GSO off:
> >>>>>
> >>>>> https://www.heistp.net/downloads/htb_split_gso/htb_cake_no_split_gso.svg
> >>>>> https://www.heistp.net/downloads/htb_split_gso/htb_cake_no_split_gso2.svg
> >>>>> https://www.heistp.net/downloads/htb_split_gso/htb_cake_no_split_gso3.svg
> >>>>>
> >>>>> I’ve seen these square waves before with htb and wondered where they
> >>>>> came from, and I think we may finally have an answer! What manner of
> >>>>> thing causes this I don’t know, but there’s a chance you may end up
> >>>>> finding out… :)
> >>>>
> >>>> Is this without the patch to CAKE that adjusts the qlen? And have you
> >>>> tried running with that patch (with HTB)?
> >>>>
> >>>> -Toke
> >>>
> >>
> >> _______________________________________________
> >> Cake mailing list
> >> Cake at lists.bufferbloat.net
> >> https://lists.bufferbloat.net/listinfo/cake
> >
> >
> >
> > --
> >
> > Dave Täht
> > CTO, TekLibre, LLC
> > http://www.teklibre.com
> > Tel: 1-831-205-9740
>


-- 

Dave Täht
CTO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-831-205-9740