[Cake] options for shrinking cake xstats

[Cake] options for shrinking cake xstats

[Dave Taht]

as 1400+ bytes on the parisc stack, is a bit much. That said, I don't
see much possibility for shrinkage overall.

A) if we resort to having a struct new_cake_xstats[q->tin_cnt], we
still end up with a big stack on the diffserv8 case (that won't get
caught by a static checker)

B) I'm on record as disliking any statistics calculation that is not
directly needed by the algorithm. Putting my asbestos suit on, that's
peak_delay_us, avge_delay_us, base_delay_us, way_indirect_hits,
way_misses, way_collisions, bulk_flow_count, and a few others.

C) Can we return a tuple to tc saying "we have 8 queues, poll me for each"?

D) Put the extended stats in sysfs, instead?

E) ??

struct tc_cake_traffic_stats {
        __u32 packets;
        __u32 link_ms; // essentially unused, but we could promote
packets to u64.
        __u64 bytes;
};

#define TC_CAKE_MAX_TINS (8)
struct tc_cake_xstats {
        __u16 version;  /* == 5, increments when struct extended */
        __u8  max_tins; /* == TC_CAKE_MAX_TINS */
        __u8  tin_cnt;  /* <= TC_CAKE_MAX_TINS */

        __u32 threshold_rate[TC_CAKE_MAX_TINS];
        __u32 target_us[TC_CAKE_MAX_TINS];
        struct tc_cake_traffic_stats sent[TC_CAKE_MAX_TINS];
        struct tc_cake_traffic_stats dropped[TC_CAKE_MAX_TINS];
        struct tc_cake_traffic_stats ecn_marked[TC_CAKE_MAX_TINS];
        struct tc_cake_traffic_stats backlog[TC_CAKE_MAX_TINS];
        __u32 interval_us[TC_CAKE_MAX_TINS];
        __u32 way_indirect_hits[TC_CAKE_MAX_TINS];
        __u32 way_misses[TC_CAKE_MAX_TINS];
        __u32 way_collisions[TC_CAKE_MAX_TINS];
        __u32 peak_delay_us[TC_CAKE_MAX_TINS]; /* ~= bulk flow delay */
        __u32 avge_delay_us[TC_CAKE_MAX_TINS];
        __u32 base_delay_us[TC_CAKE_MAX_TINS]; /* ~= sparse flows delay */
        __u16 sparse_flows[TC_CAKE_MAX_TINS];
        __u16 bulk_flows[TC_CAKE_MAX_TINS];
        __u16 unresponse_flows[TC_CAKE_MAX_TINS]; /* v4 - was u32 last_len */
        __u16 spare[TC_CAKE_MAX_TINS]; /* v4 - split last_len */
        __u32 max_skblen[TC_CAKE_MAX_TINS];
        __u32 capacity_estimate;  /* version 2 */
        __u32 memory_limit;       /* version 3 */
        __u32 memory_used;        /* version 3 */
        struct tc_cake_traffic_stats ack_drops[TC_CAKE_MAX_TINS]; /* v5 */
};



-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619

Re: [Cake] options for shrinking cake xstats

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 3637 bytes --]

This didn’t seem to generate any response so I’ll have a go :-)

> On 6 Dec 2017, at 00:06, Dave Taht <dave.taht@gmail.com> wrote:
> 
> as 1400+ bytes on the parisc stack, is a bit much. That said, I don't
> see much possibility for shrinkage overall.
How much over do you happen to know?  I see a number of options (6 or so) that are in essence binary flags passed as u32, could they be u8 instead saving 3 bytes per flag = 6 * 3 = 18 bytes.
> 
> A) if we resort to having a struct new_cake_xstats[q->tin_cnt], we
> still end up with a big stack on the diffserv8 case (that won't get
> caught by a static checker)
> 
> B) I'm on record as disliking any statistics calculation that is not
> directly needed by the algorithm. Putting my asbestos suit on, that's
> peak_delay_us, avge_delay_us, base_delay_us, way_indirect_hits,
> way_misses, way_collisions, bulk_flow_count, and a few others.
Dons flamethrower:  I have found most of these stats very useful in determining when odd behaviour is occurring.  unresponsive flows highlighted an ECN oddity ;-)  max_len hightlighted some unexpected behaviour in sqm-scripts for the me the other day (4 hours of my life I’ll never get back). I’m not a fan in losing them.  Removes flamethrower.
> 
> C) Can we return a tuple to tc saying "we have 8 queues, poll me for each”?
Passing a structure per tin seems an ideal solution if it can be done.  I’ve no idea how.  It must be possible, isn’t that the point of the netlink interface?I wonder if that nice Mr Shemminger can offer some clues?
> 
> D) Put the extended stats in sysfs, instead?
Not a fan - I like the way ‘tc’ returns all the useful/relevant info in one place - I personally don’t care for digging around the filesystem.
> 
> E) ??
> 
> struct tc_cake_traffic_stats {
>        __u32 packets;
>        __u32 link_ms; // essentially unused, but we could promote
> packets to u64.
>        __u64 bytes;
> };
> 
> #define TC_CAKE_MAX_TINS (8)
> struct tc_cake_xstats {
>        __u16 version;  /* == 5, increments when struct extended */
>        __u8  max_tins; /* == TC_CAKE_MAX_TINS */
>        __u8  tin_cnt;  /* <= TC_CAKE_MAX_TINS */
> 
>        __u32 threshold_rate[TC_CAKE_MAX_TINS];
>        __u32 target_us[TC_CAKE_MAX_TINS];
>        struct tc_cake_traffic_stats sent[TC_CAKE_MAX_TINS];
>        struct tc_cake_traffic_stats dropped[TC_CAKE_MAX_TINS];
>        struct tc_cake_traffic_stats ecn_marked[TC_CAKE_MAX_TINS];
>        struct tc_cake_traffic_stats backlog[TC_CAKE_MAX_TINS];
>        __u32 interval_us[TC_CAKE_MAX_TINS];
>        __u32 way_indirect_hits[TC_CAKE_MAX_TINS];
>        __u32 way_misses[TC_CAKE_MAX_TINS];
>        __u32 way_collisions[TC_CAKE_MAX_TINS];
>        __u32 peak_delay_us[TC_CAKE_MAX_TINS]; /* ~= bulk flow delay */
>        __u32 avge_delay_us[TC_CAKE_MAX_TINS];
>        __u32 base_delay_us[TC_CAKE_MAX_TINS]; /* ~= sparse flows delay */
>        __u16 sparse_flows[TC_CAKE_MAX_TINS];
>        __u16 bulk_flows[TC_CAKE_MAX_TINS];
>        __u16 unresponse_flows[TC_CAKE_MAX_TINS]; /* v4 - was u32 last_len */
>        __u16 spare[TC_CAKE_MAX_TINS]; /* v4 - split last_len */
>        __u32 max_skblen[TC_CAKE_MAX_TINS];
>        __u32 capacity_estimate;  /* version 2 */
>        __u32 memory_limit;       /* version 3 */
>        __u32 memory_used;        /* version 3 */
>        struct tc_cake_traffic_stats ack_drops[TC_CAKE_MAX_TINS]; /* v5 */
> };
> 

They’re my thoughts, but what do I know? :-)

Cheers,

Kevin D-B

GPG fingerprint: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Cake] options for shrinking cake xstats

[Sebastian Moeller]

Hi Kevin,


> On Dec 16, 2017, at 09:35, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> This didn’t seem to generate any response so I’ll have a go :-)
> 
>> On 6 Dec 2017, at 00:06, Dave Taht <dave.taht@gmail.com> wrote:
>> 
>> as 1400+ bytes on the parisc stack, is a bit much. That said, I don't
>> see much possibility for shrinkage overall.
> How much over do you happen to know?  I see a number of options (6 or so) that are in essence binary flags passed as u32, could they be u8 instead saving 3 bytes per flag = 6 * 3 = 18 bytes.

That got me thinkong, how about a u8 interpreted as a bitfield? Then we go from 6 * 4 = 24 to 1 byte... (or make it a u32 leaving plenty of room for the future ;) )

>> 
>> A) if we resort to having a struct new_cake_xstats[q->tin_cnt], we
>> still end up with a big stack on the diffserv8 case (that won't get
>> caught by a static checker)
>> 
>> B) I'm on record as disliking any statistics calculation that is not
>> directly needed by the algorithm. Putting my asbestos suit on, that's
>> peak_delay_us, avge_delay_us, base_delay_us, way_indirect_hits,
>> way_misses, way_collisions, bulk_flow_count, and a few others.
> Dons flamethrower:  I have found most of these stats very useful in determining when odd behaviour is occurring.  unresponsive flows highlighted an ECN oddity ;-)  max_len hightlighted some unexpected behaviour in sqm-scripts for the me the other day (4 hours of my life I’ll never get back). I’m not a fan in losing them.  Removes flamethrower.

	I wonder whether we could make gathering and reporting those conditional on a keyword like "expensive_stats" so that only users wanting the information pay the cost...

>> 
>> C) Can we return a tuple to tc saying "we have 8 queues, poll me for each”?
> Passing a structure per tin seems an ideal solution if it can be done.  I’ve no idea how.  It must be possible, isn’t that the point of the netlink interface?I wonder if that nice Mr Shemminger can offer some clues?
>> 
>> D) Put the extended stats in sysfs, instead?
> Not a fan - I like the way ‘tc’ returns all the useful/relevant info in one place - I personally don’t care for digging around the filesystem.

	Mmh, but what if tc did that digging for us, so no change from observable tc behavior, just a different implementation?


But as always, I have no real clue on these things and am mostly improvising on Kevin's theme here...

Best Regards
	Sebastian

>> 
>> E) ??
>> 
>> struct tc_cake_traffic_stats {
>>       __u32 packets;
>>       __u32 link_ms; // essentially unused, but we could promote
>> packets to u64.
>>       __u64 bytes;
>> };
>> 
>> #define TC_CAKE_MAX_TINS (8)
>> struct tc_cake_xstats {
>>       __u16 version;  /* == 5, increments when struct extended */
>>       __u8  max_tins; /* == TC_CAKE_MAX_TINS */
>>       __u8  tin_cnt;  /* <= TC_CAKE_MAX_TINS */
>> 
>>       __u32 threshold_rate[TC_CAKE_MAX_TINS];
>>       __u32 target_us[TC_CAKE_MAX_TINS];
>>       struct tc_cake_traffic_stats sent[TC_CAKE_MAX_TINS];
>>       struct tc_cake_traffic_stats dropped[TC_CAKE_MAX_TINS];
>>       struct tc_cake_traffic_stats ecn_marked[TC_CAKE_MAX_TINS];
>>       struct tc_cake_traffic_stats backlog[TC_CAKE_MAX_TINS];
>>       __u32 interval_us[TC_CAKE_MAX_TINS];
>>       __u32 way_indirect_hits[TC_CAKE_MAX_TINS];
>>       __u32 way_misses[TC_CAKE_MAX_TINS];
>>       __u32 way_collisions[TC_CAKE_MAX_TINS];
>>       __u32 peak_delay_us[TC_CAKE_MAX_TINS]; /* ~= bulk flow delay */
>>       __u32 avge_delay_us[TC_CAKE_MAX_TINS];
>>       __u32 base_delay_us[TC_CAKE_MAX_TINS]; /* ~= sparse flows delay */
>>       __u16 sparse_flows[TC_CAKE_MAX_TINS];
>>       __u16 bulk_flows[TC_CAKE_MAX_TINS];
>>       __u16 unresponse_flows[TC_CAKE_MAX_TINS]; /* v4 - was u32 last_len */
>>       __u16 spare[TC_CAKE_MAX_TINS]; /* v4 - split last_len */
>>       __u32 max_skblen[TC_CAKE_MAX_TINS];
>>       __u32 capacity_estimate;  /* version 2 */
>>       __u32 memory_limit;       /* version 3 */
>>       __u32 memory_used;        /* version 3 */
>>       struct tc_cake_traffic_stats ack_drops[TC_CAKE_MAX_TINS]; /* v5 */
>> };
>> 
> 
> They’re my thoughts, but what do I know? :-)
> 
> Cheers,
> 
> Kevin D-B
> 
> GPG fingerprint: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake