[Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[Kevin Darbyshire-Bryant]

Greetings All,

In a bid to create yet another day of cake controversy, here's my latest 
pull request https://github.com/dtaht/tc-adv/pull/12

It removes all the atm/ptm/ethernet specific overhead keywords.

I find myself largely in agreement with Sebastian where he said:

"
I would prefer very much if we could just rip the keywords out and just 
leave “overhead N” in there. As is the keywords are inconsistent in that 
some act as modifiers that “add” to already specified values while 
others replace older values whole-sale. Also these keywords do not 
really simplify the challenges in choosing the correct overhead at all, 
so my vote would be to go from today’s:

[ atm | noatm* ] [ overhead N | conservative | raw* ]

to:
[ atm | noatm* ] [ overhead N | raw* ]

So, instead of documenting the keywords, remove even more (and add an 
info/error message to tc to warn users still using the old ones). If 
that is not acceptable then I would propose to simply create named 
keywords for the individual components that make up the overhead and 
apply these in an additive fashion, like:

ppp pppoe ethernet llc snap atmpad aal5sar

which would translate into:

2 + 6 + 14 + 3 + 5 + 2 + 8 = 40

So, I propose to just take the components and their names from 
https://web.archive.org/web/20150606220856/http://ace-host.stuart.id.au/russell/files/tc/tc-atm/ 
and ad vlan, mac, and fcs to the mix (and maybe the effective additional 
overhead required to shape ethernet, IFG and friends).

But honestly I believe we would be better off by directing users to 
decent information how to deduce the applicable overhead in each 
individual case. Say the link above and maybe a link to 
https://github.com/moeller0/ATM_overhead_detector might be of more help 
than trying to simply a complex situation?
While at it, I also would remove the “conservative” keyword since with 
that we make a promise to our users we might not be able to keep as we 
have no guarantee of a maximum overhead possible.
"

I agree that pointing people to decent info and/or having a small table 
in the cake man page with the same info as the now dropped overhead 
commands would be far better.  My concession is to keep the conservative 
keyword.  For people who are unwilling to go into this (and I expect 
those same people would not be interested in Sebastian's ppp pppoe 
ethernet llc....' keyword set) it is *likely* that the addition of a 
whole ATM user cell is going to cover the overhead.

I'd be sort of interested to know if anyone is actually using those 
keywords: ipoa-vcmux, ipoa-llcsnap, bridged-vcmux, bridged-llcsnap, 
ppoa-vcmux, pppoa-llc, pppoe-vcmux, pppoe-llcsnap, pppoe-ptm, 
bridged-ptm, via-ethernet, ether-phy, ether-all, ether-fcs, ether-vlan.

How many actually knew they even existed?

Yours controversially,

Kevin

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[Jonathan Morton]

> On 2 Jun, 2016, at 12:37, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> I'd be sort of interested to know if anyone is actually using those keywords: ipoa-vcmux, ipoa-llcsnap, bridged-vcmux, bridged-llcsnap, ppoa-vcmux, pppoa-llc, pppoe-vcmux, pppoe-llcsnap, pppoe-ptm, bridged-ptm, via-ethernet, ether-phy, ether-all, ether-fcs, ether-vlan.
> 
> How many actually knew they even existed?

Since they are poorly documented, probably not many.  But I’d rather improve the documentation than delete them.

In principle, the existence and naming of these keywords is a potential clue to the uninitiated user of the overhead feature’s purpose.  The concept of protocol overhead affecting shader function is not an obvious one outside of networking specialists; making users look up a number in a cryptic table will simply result in nobody doing it at all.

Having shortcut keywords for this purpose in tc also helps to avoid the trap of other UI layers doing their own (incomplete or inaccurate) research on the correct compensation to apply.  Currently LuCI is extremely clumsy at handling SQM configuration, and the general quality of vendor firmware doesn’t give me confidence.

I’ve just pushed an update which makes all the keywords incremental, rather than some of them being absolute.  Existing correct examples of how to use these keywords remain correct.

It would be nice if LuCI could infer information about the likely overheads from the rest of the configuration, and apply (or suggest & default) the correct keywords in sqm-scripts.  That would make the feature much more widely used.

 - Jonathan Morton

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[Toke Høiland-Jørgensen]

Jonathan Morton <chromatix99@gmail.com> writes:

> It would be nice if LuCI could infer information about the likely
> overheads from the rest of the configuration, and apply (or suggest &
> default) the correct keywords in sqm-scripts. That would make the
> feature much more widely used.

We can probably do this for the most common cases, but am not so sure
it's unambiguous when to pick what. If someone can supply a couple of
examples of configuration where we are fairly certain we know what to
pick, I can look into how that can be inferred in luci...

-Toke

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[Jonathan Morton]

>> It would be nice if LuCI could infer information about the likely
>> overheads from the rest of the configuration, and apply (or suggest &
>> default) the correct keywords in sqm-scripts. That would make the
>> feature much more widely used.
> 
> We can probably do this for the most common cases, but am not so sure
> it's unambiguous when to pick what. If someone can supply a couple of
> examples of configuration where we are fairly certain we know what to
> pick, I can look into how that can be inferred in luci…

The most obvious cases are where there is an ADSL or VDSL modem built into the router, as there is on my Buffalo WBMR (though I don’t currently have a working ADSL line to demonstrate a live example).  SQM on the WAN-facing interfaces which (directly or indirectly) use that modem should then infer the PPP and ATM modes from the modem’s configuration, which I know is visible to LuCI in some form.

It’s slightly less obvious what to do with a standalone router, where there is a separate modem handling the PPP endpoint, thus the WAN interface appears in all respects to be Ethernet.  For this case, an easy way of setting the “conservative”, “bridged-ptm" or “pppoe-ptm” keywords should do the trick, perhaps with “conservative” being the sane default.  The PTM keywords are better suited to VDSL installations.  I suspect cable and cell modems may work best *without* overhead compensation.

As for the “conservative” keyword, it assumes ATM encapsulation and exactly one ATM cell of inner encapsulation overhead, the latter exceeding all known combinations of inner encapsulation used in practice (except for bizarre cases involving IP-in-IP tunnelling).  It’s intended to be a reasonable default where it is known or suspected that encapsulation is in use, but no specifics are known.

Sebastian seems to have more detailed information about what encapsulation combinations are actually in use in Germany, and can doubtless advise on how these show up in LuCI on an ADSL router.

 - Jonathan Morton

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[moeller0]

Hi Jonathan,

> On Jun 2, 2016, at 16:49 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
>>> It would be nice if LuCI could infer information about the likely
>>> overheads from the rest of the configuration, and apply (or suggest &
>>> default) the correct keywords in sqm-scripts. That would make the
>>> feature much more widely used.
>> 
>> We can probably do this for the most common cases, but am not so sure
>> it's unambiguous when to pick what. If someone can supply a couple of
>> examples of configuration where we are fairly certain we know what to
>> pick, I can look into how that can be inferred in luci…
> 
> The most obvious cases are where there is an ADSL or VDSL modem built into the router, as there is on my Buffalo WBMR (though I don’t currently have a working ADSL line to demonstrate a live example).  SQM on the WAN-facing interfaces which (directly or indirectly) use that modem should then infer the PPP and ATM modes from the modem’s configuration, which I know is visible to LuCI in some form.
> 
> It’s slightly less obvious what to do with a standalone router, where there is a separate modem handling the PPP endpoint, thus the WAN interface appears in all respects to be Ethernet.  For this case, an easy way of setting the “conservative”, “bridged-ptm" or “pppoe-ptm” keywords should do the trick, perhaps with “conservative” being the sane default.  The PTM keywords are better suited to VDSL installations.  I suspect cable and cell modems may work best *without* overhead compensation.

	I do not agree that these are too helpful unless the user already has lots of knowledge, but then they are hardly an improvement on “overhead N”.

> 
> As for the “conservative” keyword, it assumes ATM encapsulation and exactly one ATM cell of inner encapsulation overhead, the latter exceeding all known combinations of inner encapsulation used in practice (except for bizarre cases involving IP-in-IP tunnelling).  It’s intended to be a reasonable default where it is known or suspected that encapsulation is in use, but no specifics are known.

	So far I have not encountered overhead >= 48, that is true. But I have also never seen a blue whale, which I do not want to be interpreted as me doubting their existence. In reality 48 bytes still seems like a likely upper bound for per-packet overhead on an ATM link, so of all the keywords, conservative might survive, if renamed “conservative_adsl” or “conservative_atm”. “pppoe-ptm” however is a good example for an under-defined keyword… what exactly is this supposed to entail?

> 
> Sebastian seems to have more detailed information about what encapsulation combinations are actually in use in Germany, and can doubtless advise on how these show up in LuCI on an ADSL router.

	Not at all, some details of the encapsulation are not easily extractable from the typical modem’s dsl-information tab. My attempts at measuring the per-packet-overhead culminated in what I documented under https://github.com/moeller0/ATM_overhead_detector and https://github.com/moeller0/ATM_overhead_detector/wiki I stopped believing this is “simple” a long time ago; I also failed to come up with a method to measure the actual overhead on VDSL links…

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[Jonathan Morton]

> On 2 Jun, 2016, at 18:42, moeller0 <moeller0@gmx.de> wrote:
> 
> I stopped believing this is “simple” a long time ago; I also failed to come up with a method to measure the actual overhead on VDSL links…

If it’s impossible to measure, then by definition it doesn’t have a significant performance impact, and thus it doesn’t matter if it is sloppily estimated.  In that context an overestimate of overhead is safe.

This is really a user-interface problem.  I think it would be very helpful for LuCI to get a basic, easy “conservative overhead compensation” button that simply turns on the “conservative” keyword for both ingress and egress.  That’s the bare minimum.

We can discuss details of improving granularity of that setting at greater leisure, preferably in the context of a proper redesign of the SQN configuration.

And let’s be absolutely clear: I will NOT drop the shortcut keywords from tc.

 - Jonathan Morton

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[moeller0]

Dear Jonathan,


> On Jun 2, 2016, at 19:40 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 2 Jun, 2016, at 18:42, moeller0 <moeller0@gmx.de> wrote:
>> 
>> I stopped believing this is “simple” a long time ago; I also failed to come up with a method to measure the actual overhead on VDSL links…
> 
> If it’s impossible to measure, then by definition it doesn’t have a significant performance impact,

	That is unfortunately not really true… it really is just that, hard to measure. And that might be more a statement about my methodological/creative gaps than anything else.

> and thus it doesn’t matter if it is sloppily estimated.  In that context an overestimate of overhead is safe.

	Safe, only if over-estimated; I would prefer if the user would specify the bandwidth “sacrifice” once when he sets the shapers brutto rate ;)

> 
> This is really a user-interface problem.  I think it would be very helpful for LuCI to get a basic, easy “conservative overhead compensation” button that simply turns on the “conservative” keyword for both ingress and egress.  That’s the bare minimum.

	That will, as currently implemented, drag in the ATM 48/53 cell compensation for all users, eating roughly 9% bandwidth, not sure we would do our users a favor with such a default policy. 


> 
> We can discuss details of improving granularity of that setting at greater leisure, preferably in the context of a proper redesign of the SQN configuration.
> 
> And let’s be absolutely clear: I will NOT drop the shortcut keywords from tc.

	So far I have tried to raise a number of in-detail issues (in other mails and postings) that seem to indicate that our assumptions about per-packet-overheads are not as robust and complete as they should be if we want to use them to justify changing the defaults.
	So, if you do not want to drop the keywords then please make sure they are well-documented and consistent. Currently it is not self-evident which numerical overhead corresponds to which keyword and that mapping should be crystal clear. And I would like to add that “conservative”-keyword needs special care in documentation as it is the only keyword that compounds per-packet-overhead and specific framing (ATM in this case, we simply ignore PTM’s 64/65 encoding (and rightly so)), this should be obvious from the documentation (man page and the output of “tc qdisc add cake help”) and in my opinion the keyword itself. I would propose as first half-measure “conservative_adsl” or “conservative_atm” to signify the compound nature.
	

Best Regards
 	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[Jonathan Morton]

> On 2 Jun, 2016, at 21:53, moeller0 <moeller0@gmx.de> wrote:
> 
> “conservative”-keyword needs special care in documentation as it is the only keyword that compounds per-packet-overhead and specific framing

Not true.  All of the ATM-specific encapsulation keywords - of which there are ten others - also force ATM compensation on.  This is obvious in the code.

 - Jonathan Morton

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[moeller0]

> On Jun 2, 2016, at 20:55 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 2 Jun, 2016, at 21:53, moeller0 <moeller0@gmx.de> wrote:
>> 
>> “conservative”-keyword needs special care in documentation as it is the only keyword that compounds per-packet-overhead and specific framing
> 
> Not true.  All of the ATM-specific encapsulation keywords - of which there are ten others - also force ATM compensation on.  This is obvious in the code.
> 
> - Jonathan Morton
> 

I might not have picked the best example, but the current keywords make it simple for me ;) 
Let me rephrase then , it is not self-evident which keywords are ATM-specific then… But humor me:
tc qdisc add cake help
Usage: ... cake [ bandwidth RATE | unlimited* | autorate_ingress ]
                [ rtt TIME | datacentre | lan | metro | regional | internet* | oceanic | satellite | interplanetary ]
                [ besteffort | precedence | diffserv8 | diffserv4* ]
                [ flowblind | srchost | dsthost | hosts | flows* | dual-srchost | dual-dsthost | triple-isolate ]
                [ atm | noatm* ] [ overhead N | conservative | raw* ]
                [ wash | nowash* ]
                [ memlimit LIMIT ]
    (* marks defaults)

Where is it evident that “conservative” includes atm encapsulation? And what should a user expect that specifies “noatm conservative”?

	So Jonathan, please, instead of trying to argue obvious inconsistencies in the currently assigned encapsulation keywords away, just go and make sure they are consistent and well documented. 
	Your postings in the overhead-matter make me question whether you do fully understand the issue at hand in its full complexity; so by all means go and collect input from users (usability and self-evidence of the keywords) and experts (on the actual likelihood of encapsulations).

Best Regards
	Sebastian

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[moeller0]

Hi Toke,

> On Jun 2, 2016, at 16:27 , Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> Jonathan Morton <chromatix99@gmail.com> writes:
> 
>> It would be nice if LuCI could infer information about the likely
>> overheads from the rest of the configuration, and apply (or suggest &
>> default) the correct keywords in sqm-scripts. That would make the
>> feature much more widely used.
> 
> We can probably do this for the most common cases, but am not so sure
> it's unambiguous when to pick what. If someone can supply a couple of
> examples of configuration where we are fairly certain we know what to
> pick, I can look into how that can be inferred in luci…

	As I tried to convey before the matter is far from simple. For example my ISP, DTAG, has at least 4 different sets of per packet overhead (ATM versus PTM, BRAS versus BNG) so even for this one ISP there is not one solution to the issue. And with BRAS/BNG shaping as used by say DTAG the actual VDLS2 related overhead becomes irrelevant compared to the overhead setting of that applied policer. I believe trying to simplify this complexity will lead to false overhead recommendations. I would rather direct people to better documentation how to deduce the overhead by measurements and research…

Good

> 
> -Toke
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[Jonathan Morton]

> On 2 Jun, 2016, at 17:59, moeller0 <moeller0@gmx.de> wrote:
> 
> As I tried to convey before the matter is far from simple. For example my ISP, DTAG, has at least 4 different sets of per packet overhead (ATM versus PTM, BRAS versus BNG) so even for this one ISP there is not one solution to the issue. And with BRAS/BNG shaping as used by say DTAG the actual VDLS2 related overhead becomes irrelevant compared to the overhead setting of that applied policer. I believe trying to simplify this complexity will lead to false overhead recommendations. I would rather direct people to better documentation how to deduce the overhead by measurements and research…

I think we can make a couple of usefully simplifying assumptions:

1: The encapsulation overhead on the wire is the same in both directions.

2: The BRAS is irrelevant, because we need to set an ingress qdisc below the line rate anyway in order to exert control, and the BRAS doesn’t apply on upload.

Does that help at all?

 - Jonathan Morton

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[moeller0]

Hi Jonathan,

> On Jun 2, 2016, at 17:10 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 2 Jun, 2016, at 17:59, moeller0 <moeller0@gmx.de> wrote:
>> 
>> As I tried to convey before the matter is far from simple. For example my ISP, DTAG, has at least 4 different sets of per packet overhead (ATM versus PTM, BRAS versus BNG) so even for this one ISP there is not one solution to the issue. And with BRAS/BNG shaping as used by say DTAG the actual VDLS2 related overhead becomes irrelevant compared to the overhead setting of that applied policer. I believe trying to simplify this complexity will lead to false overhead recommendations. I would rather direct people to better documentation how to deduce the overhead by measurements and research…
> 
> I think we can make a couple of usefully simplifying assumptions:
> 
> 1: The encapsulation overhead on the wire is the same in both directions.

	That seems reasonable.

> 
> 2: The BRAS is irrelevant, because we need to set an ingress qdisc below the line rate anyway in order to exert control, and the BRAS doesn’t apply on upload.

	Not at all, either the BRAS shaper is set above the XDSL link capacity (in which case it does not matter) or it is set below link capacity and then ONLY the BRAS shaper matters. As an example DTAG changed their DSLAMs/MSANs to alway sync as high as possible, so my Modem syncs at 109/38Mbps, but I have 50/10Mbps plan, all the shaping is happening at the BNG/BRAS, the parameters of the VDSL2 links do not matter at all anymore. It seems that the BNG shaper effectively limits the brutto rate and then they use dual-VLAN tags and the pppoe overhead, so that the shapers overhead in practice is equal to the PTM link’s. But even if the would not account for any overhead the difference between 50 and 100 is large enough to make the real VDSL2 overhead irrelevant. At least for DTAG’s BNG rollout the shaping/policing happens in the BNG both for upstream and downstream.
	I believe this illustrates why we should not try to sugarcoat this complexity. Instead of telling the user our best guess of his/her specific overhead, we would do them a greater service by explaining how to actually measure their overhead…

Best Regards
	Sebastian


> 
> Does that help at all?
> 
> - Jonathan Morton
> 

Re: [Cake] cake/tc - removal of atm/ptm/ethernet specific overhead keywords

[moeller0]

Hi Jonathan,

> On Jun 2, 2016, at 16:22 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> 
>> On 2 Jun, 2016, at 12:37, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>> 
>> I'd be sort of interested to know if anyone is actually using those keywords: ipoa-vcmux, ipoa-llcsnap, bridged-vcmux, bridged-llcsnap, ppoa-vcmux, pppoa-llc, pppoe-vcmux, pppoe-llcsnap, pppoe-ptm, bridged-ptm, via-ethernet, ether-phy, ether-all, ether-fcs, ether-vlan.
>> 
>> How many actually knew they even existed?
> 
> Since they are poorly documented, probably not many.  But I’d rather improve the documentation than delete them.

My take is that if you write a comprehensive description of per packet overhead that remains readable and interesting you would do a great thing, But it would be too much for just being a section in the cake man page. 

> 
> In principle, the existence and naming of these keywords is a potential clue to the uninitiated user of the overhead feature’s purpose.  

	In practice actual overheads are a) more complex than the keywords implied and b) less invariant as one would hope (in that ISPs change their composition without noticing their users).

> The concept of protocol overhead affecting shader function is not an obvious one outside of networking specialists; making users look up a number in a cryptic table will simply result in nobody doing it at all.

	Making people pick potentially wrong values from a set of (under-documented) keywords will have similar results.

> 
> Having shortcut keywords for this purpose in tc also helps to avoid the trap of other UI layers doing their own (incomplete or inaccurate) research on the correct compensation to apply.  

	Yeah, but this only is an improvement if the keywords are complete and accurate, the last batch was not…

> Currently LuCI is extremely clumsy at handling SQM configuration, and the general quality of vendor firmware doesn’t give me confidence.

	Thanks for the kind words ;).

> 
> I’ve just pushed an update which makes all the keywords incremental, rather than some of them being absolute.  Existing correct examples of how to use these keywords remain correct.
"
	This still does not fix the fact that it is quite unclear what exactly “pppoe-llcsnap" should expand to, and whether “pppoe-llcsnap vlan vlan” really is that much user friendlier than “overhead NN”.

> 
> It would be nice if LuCI could infer information about the likely overheads from the rest of the configuration, and apply (or suggest & default) the correct keywords in sqm-scripts.  That would make the feature much more widely used.

	Yes, if reality was less complex life would be easier, I agree. In reality ISPs are beginning to implement their own policers/shapers at the BRAS/BNG level making the whole thing even more difficult, as at that level the shapers sits on top of ethernet with an unknown per-packet overhead that is difficult to measure remotely…

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake