[Cerowrt-devel] Correctly calculating overheads on unknown connections

[Sebastian Moeller]

Hi Andy,


On Sep 24, 2014, at 11:17 , Andy Furniss <adf.lists at gmail.com> wrote:

> Sebastian Moeller wrote:
> 
>> Thanks for sharing your test case; I can repeat these results
>> exactly on my machines (I also tried htb instead hfsc for fun: same
>> result as to be expected see below). Looking back at
>> http://lxr.free-electrons.com/ident?i=qdisc_pkt_len_init (line
>> 2731):
>> 
>> qdisc_skb_cb(skb)->pkt_len += (gso_segs - 1) * hdr_len ;
>> 
>> I begin to realize this function is not responsible for adding
>> single wire packet’s ethernet header, but for figuring out in how
>> many on-the-wire packets to chop down a GSO packet , and add the
>> header overhead for the additional wire packets, I had completely
>> looked over the (gso-segs - 1) part, oops.
> 
> Glad it helped - I know from trying, and giving up, how hard/error prone
> reading kernel code can be :-)

	Especially when all one knows about C is basically from reading K&R with almost no hands-on coding experience ;)

> 
>> 
>> @cerowrt-devel: everyone using link layer ATM you might want to try
>> to reduce the the per packet overhead by 14… (but please test)
> 
> Maybe you mean overhead calculated by a script?

	Well in cerowrt’s SQM-scripts we expose the stab options so users can take link layer and overhead into account. If you naively determine the overhead, either with the help of the scrips I posted earlier or by looking it up on a table (if the encapsulation options are known) you will end up not handling the kernel’s auto-added overhead well. Currently SQM scripts does not expose PPP devices only ge00 (ethernet) so -14 seems currently the best recommendation in combination with “please test”. What I am curious after your message is what happens if the kernel terminates a pppoe connection but is connected to a “modem” via ethernet, what does the kernel do. And thanks to your pointers I know have an idea of how to test that ;)


> 
> Just to be clear, I expect that wrt would be shaping on ppp, so you
> don't need to take 14 if that's the case.

	Good to know.

> 
> 
>> So I stand corrected, you are right, tic’s stab automatically adds
>> the ethernet header. So I am off to repeat my netperf-wrapper tests
>> right now again with overhead of 30 instead of 44, again these tests
>> confirm your observation. Interestingly, it seems netperf-wrapper’s
>> RRUL test really is suited to figure out the overhead: while shaping
>> to 100% of line rate (on ADSL2+ where line rate rate is the net line
>> rate (after FEC)) specifying too small an overhead the ICMP latency
>> plot shows larger deviations from the expected unload RTT plus 10ms.
>> Too large an overhead however just decreases the good put bait while
>> leaving the latency well under control.
> 
> I wouldn't word it like "stab adds ..." This is nothing to do with stab
> really - just the only length stab knows is skb->len and that means
> different things on different interfaces because of how the kernel works.
> 
> (I haven't retested all this, but I doubt it's changed)
> 
> On ppp skb->len = ip len
> 
> On eth skb->len = ip len + 14
> 
> On vlan skb->len = ip len + 18

	So this is the information I actually wanted to find and then somehow thought qdisc_pkt_len_init() was the place. Do you by chance have any pointer where this assignment is handled?

> 
> If you ran my script on various interfaces without stab I expect you
> would still be able to see the difference - everyone who does any tc on
> eth gets shaping with ip+14 sized packets.
> 
> Even without tc involved I think you could see the difference looking at
> ip -s ls xxxx type stats on different interfaces.

Thanks again, & Best Regards
	Sebastian

>