Re: [Cake] [PATCH 0/3] Add Common Applications Kept Enhanced (sch_cake) qdisc

[Sebastian Moeller <moeller0@gmx.de>]

Hi Dave,


> On Nov 22, 2017, at 19:03, Dave Taht <dave.taht@gmail.com> wrote:
> 
> at what commit it worked properly?

That is a good question, I will need to research that (and I also should consider the possibilirty that maybe this issue is more related to my ubuntu 16.4 LTS's Linux kernel: work-horse 4.10.0-38-generic #42~16.04.1-Ubuntu SMP Tue Oct 10 16:32:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux). Now it shows that I am quite a git novice, but I guess "git bisect" should be my friend here, assuming I can divine a initial good commit.. For that could you help me out with which command I need to use to "checkout/rewind to" a specific git commit, so I can easily confirm I start with a known good state?
You might be able to tell this is all new territory for me, so it might take longer than I would wish...

Best Regards


> 
> On Wed, Nov 22, 2017 at 3:02 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>> Hi All,
>> 
>> Tl;dr: Automatic accounting of the kernel added dev->hard_header_len seems currently broken at least in the cobalt branch...
>> 
>> 
>> as far as I can tell overhead accounting is still busted:
>> 
>> 
>> user@work-horse:~/CODE/sch_cake$ sudo ../tc-adv/tc/tc qdisc delete dev eth0 root; sudo ../tc-adv/tc/tc qdisc replace dev eth0 root cake raw overhead 5; ../tc-adv/tc/tc -s qdisc show dev eth0
>> sudo: unable to resolve host work-horse
>> sudo: unable to resolve host work-horse
>> qdisc cake 8014: root refcnt 6 unlimited diffserv3 triple-isolate rtt 100.0ms noatm overhead 19 via-ethernet total_overhead 19 hard_header_len 14
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> memory used: 0b of 15140Kb
>> capacity estimate: 0bit
>>                 Bulk   Best Effort      Voice
>>  thresh          0bit        0bit        0bit
>>  target         5.0ms       5.0ms       5.0ms
>>  interval     100.0ms     100.0ms      10.0ms
>>  pk_delay         0us         0us         0us
>>  av_delay         0us         0us         0us
>>  sp_delay         0us         0us         0us
>>  pkts               0           0           0
>>  bytes              0           0           0
>>  way_inds           0           0           0
>>  way_miss           0           0           0
>>  way_cols           0           0           0
>>  drops              0           0           0
>>  marks              0           0           0
>>  ack_drop           0           0           0
>>  sp_flows           0           0           0
>>  bk_flows           0           0           0
>>  un_flows           0           0           0
>>  max_len            0           0           0
>> 
>> user@work-horse:~/CODE/sch_cake$ sudo ../tc-adv/tc/tc qdisc delete dev eth0 root; sudo ../tc-adv/tc/tc qdisc replace dev eth0 root cake overhead 5; ../tc-adv/tc/tc -s qdisc show dev eth0
>> sudo: unable to resolve host work-horse
>> sudo: unable to resolve host work-horse
>> qdisc cake 8015: root refcnt 6 unlimited diffserv3 triple-isolate rtt 100.0ms raw total_overhead 14 hard_header_len 14
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> memory used: 0b of 15140Kb
>> capacity estimate: 0bit
>>                 Bulk   Best Effort      Voice
>>  thresh          0bit        0bit        0bit
>>  target         5.0ms       5.0ms       5.0ms
>>  interval     100.0ms     100.0ms      10.0ms
>>  pk_delay         0us         0us         0us
>>  av_delay         0us         0us         0us
>>  sp_delay         0us         0us         0us
>>  pkts               0           0           0
>>  bytes              0           0           0
>>  way_inds           0           0           0
>>  way_miss           0           0           0
>>  way_cols           0           0           0
>>  drops              0           0           0
>>  marks              0           0           0
>>  ack_drop           0           0           0
>>  sp_flows           0           0           0
>>  bk_flows           0           0           0
>>  un_flows           0           0           0
>>  max_len            0           0           0
>> 
>> user@work-horse:~/CODE/sch_cake$
>> 
>> 
>> The first seems okay, but the second should not have reported "raw" but rather total_overhead 5 hard_header_len 14 if I understand the intention of the overhead keyword correctly. This also affects the keywords like conservative:
>> 
>> user@work-horse:~/CODE/sch_cake$ sudo ../tc-adv/tc/tc qdisc delete dev eth0 root; sudo ../tc-adv/tc/tc qdisc replace dev eth0 root cake conservative; ../tc-adv/tc/tc -s qdisc show dev eth0
>> sudo: unable to resolve host work-horse
>> sudo: unable to resolve host work-horse
>> qdisc cake 8019: root refcnt 6 unlimited diffserv3 triple-isolate rtt 100.0ms atm overhead 14 via-ethernet total_overhead 14 hard_header_len 14
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> memory used: 0b of 15140Kb
>> capacity estimate: 0bit
>>                 Bulk   Best Effort      Voice
>>  thresh          0bit        0bit        0bit
>>  target         5.0ms       5.0ms       5.0ms
>>  interval     100.0ms     100.0ms      10.0ms
>>  pk_delay         0us         0us         0us
>>  av_delay         0us         0us         0us
>>  sp_delay         0us         0us         0us
>>  pkts               0           0           0
>>  bytes              0           0           0
>>  way_inds           0           0           0
>>  way_miss           0           0           0
>>  way_cols           0           0           0
>>  drops              0           0           0
>>  marks              0           0           0
>>  ack_drop           0           0           0
>>  sp_flows           0           0           0
>>  bk_flows           0           0           0
>>  un_flows           0           0           0
>>  max_len            0           0           0
>> 
>> user@work-horse:~/CODE/sch_cake$ sudo ../tc-adv/tc/tc qdisc delete dev eth0 root; sudo ../tc-adv/tc/tc qdisc replace dev eth0 root cake raw conservative; ../tc-adv/tc/tc -s qdisc show dev eth0
>> sudo: unable to resolve host work-horse
>> sudo: unable to resolve host work-horse
>> qdisc cake 801a: root refcnt 6 unlimited diffserv3 triple-isolate rtt 100.0ms atm overhead 62 via-ethernet total_overhead 62 hard_header_len 14
>> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
>> backlog 0b 0p requeues 0
>> memory used: 0b of 15140Kb
>> capacity estimate: 0bit
>>                 Bulk   Best Effort      Voice
>>  thresh          0bit        0bit        0bit
>>  target         5.0ms       5.0ms       5.0ms
>>  interval     100.0ms     100.0ms      10.0ms
>>  pk_delay         0us         0us         0us
>>  av_delay         0us         0us         0us
>>  sp_delay         0us         0us         0us
>>  pkts               0           0           0
>>  bytes              0           0           0
>>  way_inds           0           0           0
>>  way_miss           0           0           0
>>  way_cols           0           0           0
>>  drops              0           0           0
>>  marks              0           0           0
>>  ack_drop           0           0           0
>>  sp_flows           0           0           0
>>  bk_flows           0           0           0
>>  un_flows           0           0           0
>>  max_len            0           0           0
>> 
>> user@work-horse:~/CODE/sch_cake$
>> 
>> 
>> So in other words currently the automatic accounting for the kernel added hard_header_len seems broken as the raw keyword disables that functionality. I will try to look deeper into that, but I guess with my limited C skills this might do more harm than good...
>> 
>> Best Regards
>>        Sebastian
>> 
>> 
>>> On Nov 22, 2017, at 01:09, Dave Taht <dave.taht@gmail.com> wrote:
>>> 
>>> sch_cake is intended to squeeze the most bandwidth and lowest latency out
>>> of even the slowest ISP links and routers, while presenting an API simple
>>> enough that even an ISP can configure it.
>>> 
>>> Example of use on an ISP uplink:
>>> 
>>> tc qdisc add dev eth0 cake bandwidth 20Mbit nat docsis ack-filter
>>> 
>>> Cake can also be used in unlimited mode to drive packets at the speed
>>> of the underlying link.
>>> 
>>> Cake is filled with:
>>> 
>>> * A hybrid Codel/Blue AQM algorithm, “Cobalt”, tied to an FQ_Codel
>>> derived Flow Queuing system, which autoconfigures based on the bandwidth.
>>> * A unique "triple-isolate" mode (the default) which balances per-flow
>>> and per-host flow FQ even through NAT.
>>> * An integral deficit based shaper with extensive dsl and docsis support
>>> that can also be used in unlimited mode.
>>> * 8 way set associative queuing to reduce flow collisions to a minimum.
>>> * A reasonable interpretation of various diffserv latency/loss tradeoffs.
>>> * Support for washing diffserv for entering and exiting traffic.
>>> * Perfect support for interacting with Docsis 3.0 shapers.
>>> * Extensive support for DSL framing types.
>>> * (New) Support for ack filtering.
>>> - 20 % better throughput at a 16x1 down/up ratio on the rrul test.
>>> * Extensive statistics for measuring loss, ecn markings, latency variation.
>>> 
>>> There are some features still considered experimental, notably the
>>> ingress_autorate bandwidth estimator and cobalt itself.
>>> 
>>> Various versions shipping have been available as an out of tree build for
>>> kernel versions going back to 3.10, as the embedded router world has been
>>> running a few years behind mainline Linux. A stable version has been
>>> generally available on lede-17.01 and later.
>>> 
>>> sch_cake replaces a combination of iptables, tc filter, htb and fq_codel in
>>> the sqm-scripts, with sane defaults and vastly easier configuration.
>>> 
>>> Cake's principal author is Jonathan Morton, with contributions from
>>> Kevin Darbyshire-Bryant, Toke Høiland-Jørgensen, Sebastian Moeller,
>>> Ryan Mounce, Dean Scarff, Guido Sarducci, Nils Andreas Svee, Dave Täht, and
>>> Loganaden Velvindron.
>>> 
>>> Dave Taht (3):
>>> pkt_sched.h: add support for sch_cake API
>>> Add Common Applications Kept Enhanced (sch_cake) qdisc
>>> Add support for building the new cake qdisc
>>> 
>>> include/net/cobalt.h           |  152 +++
>>> include/uapi/linux/pkt_sched.h |   58 +
>>> net/sched/Kconfig              |   11 +
>>> net/sched/Makefile             |    1 +
>>> net/sched/sch_cake.c           | 2551 ++++++++++++++++++++++++++++++++++++++++
>>> 5 files changed, 2773 insertions(+)
>>> create mode 100644 include/net/cobalt.h
>>> create mode 100644 net/sched/sch_cake.c
>>> 
>>> --
>>> 2.7.4
>>> 
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
>> 
> 
> 
> 
> -- 
> 
> Dave Täht
> CEO, TekLibre, LLC
> http://www.teklibre.com
> Tel: 1-669-226-2619