[Cake] #17

[Cake] #17

[Sebastian Moeller]

Hi list, hi Dave

On Sat Apr 11 19:33:59 PDT 2015, Dave That <dave.taht at gmail.com> wrote:
17)  the atm compensation in cake is entirely untested. And it is
unclear as to how best handle pppoe.

Regarding ATM, it seems from http://www.bufferbloat.net/projects/codel/wiki/Cake that there is only one command line argument “atm” to activate accounting for atm encapsulation. I venture a guess that this will not be sufficient as it seems  necessary to add the per packet overhead before “expanding” the packet size to the 43 in 53 atm cells. This might be really just a documentation issue (I have not looked agh cake’s code, not that I a) read C well and b) know where to find the cake code repository). In my experience the relevant per packet overhead on the ATM link needs to be measured on each link individually (and repeatedly to catch the ISPs doing funny things like adding an otherwise invisible vlan tag on the atm link).

PPPoE, is easy, either drill into the packets to get the values used for hashing, or ask people to activate cake on the pppoe interfaces in the router (these typically do not show the pppoe headers so that classifier will find the right values). Or you could argue that a PPPoE link really is just one flow, and hence does not deserve special treatment ;) (which other “multiplexors” would deserve the same special treatment SPDY/HTTP2?).

Best Regards
	Sebastian

Re: [Cake] #17

[Dave Taht]

It is not clear if atm framing and things like pppoe are used at
higher rates on newer technologies like GPON and fiber. They are not
used in cablemodems, for example. Given the headache it has been to
get that straightened out on DSL AND detect AND configure properly, I
would not mind if we abandoned the idea of supporting alternate
framings in cake, and stuck with what already works in the sqm-scripts
with htb + fq_codel.



On Sun, Apr 12, 2015 at 12:18 PM, Sebastian Moeller <moeller0@gmx.de> wrote:
> Hi list, hi Dave
>
> On Sat Apr 11 19:33:59 PDT 2015, Dave That <dave.taht at gmail.com> wrote:
> 17)  the atm compensation in cake is entirely untested. And it is
> unclear as to how best handle pppoe.
>
> Regarding ATM, it seems from http://www.bufferbloat.net/projects/codel/wiki/Cake that there is only one command line argument “atm” to activate accounting for atm encapsulation. I venture a guess that this will not be sufficient as it seems  necessary to add the per packet overhead before “expanding” the packet size to the 43 in 53 atm cells. This might be really just a documentation issue (I have not looked agh cake’s code, not that I a) read C well and b) know where to find the cake code repository). In my experience the relevant per packet overhead on the ATM link needs to be measured on each link individually (and repeatedly to catch the ISPs doing funny things like adding an otherwise invisible vlan tag on the atm link).
>
> PPPoE, is easy, either drill into the packets to get the values used for hashing, or ask people to activate cake on the pppoe interfaces in the router (these typically do not show the pppoe headers so that classifier will find the right values). Or you could argue that a PPPoE link really is just one flow, and hence does not deserve special treatment ;) (which other “multiplexors” would deserve the same special treatment SPDY/HTTP2?).
>
> Best Regards
>         Sebastian
>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Cake] #17

[Sebastian Moeller]

Hi Dave,


On Apr 12, 2015, at 23:59 , Dave Taht <dave.taht@gmail.com> wrote:

> It is not clear if atm framing and things like pppoe are used at
> higher rates on newer technologies like GPON and fiber.

	To be clear ATM is currently still a legacy tech, used where deployed, but not actively rolled out anymore. But unfortunately, ATM is still used between DSLAM and Modems if the modems run either ADSL1, ADSL2, or ADSL2+, since even in modern FTC dslams far away customers fall back to ADSL2+, so dropping ATM and per packet overhead support might not be the best idea. (But note as long as cake can be combined successfully with tc’s stab option all users should be fine, especially, IIRC as ADSL2+ tops out at around 20Mbps/3Mbps).
	PPPoE on the other hand is still used on DTAG’s (current) FTTH roll-out.

> They are not
> used in cablemodems, for example.

	But as far as I can tell almost everywhere else (ADSL/VDSL/Fiber) at least by some of the ISPs supporting such technologies.

> Given the headache it has been to
> get that straightened out on DSL AND detect AND configure properly, I
> would not mind if we abandoned the idea of supporting alternate
> framings in cake, and stuck with what already works in the sqm-scripts
> with htb + fq_codel.

	As far as I can tell most there techs just need a per-packet-overhead option (the kernel already tries to account for some overhead so the machinery is in place, we just need to expose it); ATM is mainly weird due to its insistence on padding out the last “cell” for each packet individually. As mentioned above, as long as cake can work with tc’s stab option all framing and overhead handling should be available to whoever needs it. (I note that stab will need some love for GRO packets).
	I would vote for exposing ATM framing and per packet overhead in cake to keep with the one-stop-shopping idea of its design (since enough people will be forced to live with ATM for the foreseeable future).

Best Regards
	Sebastian


> 
> 
> 
> On Sun, Apr 12, 2015 at 12:18 PM, Sebastian Moeller <moeller0@gmx.de> wrote:
>> Hi list, hi Dave
>> 
>> On Sat Apr 11 19:33:59 PDT 2015, Dave That <dave.taht at gmail.com> wrote:
>> 17)  the atm compensation in cake is entirely untested. And it is
>> unclear as to how best handle pppoe.
>> 
>> Regarding ATM, it seems from http://www.bufferbloat.net/projects/codel/wiki/Cake that there is only one command line argument “atm” to activate accounting for atm encapsulation. I venture a guess that this will not be sufficient as it seems  necessary to add the per packet overhead before “expanding” the packet size to the 43 in 53 atm cells. This might be really just a documentation issue (I have not looked agh cake’s code, not that I a) read C well and b) know where to find the cake code repository). In my experience the relevant per packet overhead on the ATM link needs to be measured on each link individually (and repeatedly to catch the ISPs doing funny things like adding an otherwise invisible vlan tag on the atm link).
>> 
>> PPPoE, is easy, either drill into the packets to get the values used for hashing, or ask people to activate cake on the pppoe interfaces in the router (these typically do not show the pppoe headers so that classifier will find the right values). Or you could argue that a PPPoE link really is just one flow, and hence does not deserve special treatment ;) (which other “multiplexors” would deserve the same special treatment SPDY/HTTP2?).
>> 
>> Best Regards
>>        Sebastian
>> 
>> 
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake
> 
> 
> 
> -- 
> Dave Täht
> Open Networking needs **Open Source Hardware**
> 
> https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Cake] #17

[Jonathan Morton]

> On 13 Apr, 2015, at 02:50, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> As mentioned above, as long as cake can work with tc’s stab option all framing and overhead handling should be available to whoever needs it.

Remind me to look into how to use that, then.  If there’s a better way than manually calculating ceil(len/48)*53, so much the better.

 - Jonathan Morton

Re: [Cake] #17

[Sebastian Moeller]

Hi Jonathan,


On Apr 13, 2015, at 01:55 , Jonathan Morton <chromatix99@gmail.com> wrote:

> 
>> On 13 Apr, 2015, at 02:50, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>> As mentioned above, as long as cake can work with tc’s stab option all framing and overhead handling should be available to whoever needs it.
> 
> Remind me to look into how to use that, then.  

basically you add:
stab mtu 2047 tsize 512 mpu 0 overhead 40 linklayer atm
to the top level qdisc invocation, it fudges the size variable in the SKB? just as the kernels internal accounting for the ethernet header does, and all qdisc will see the fudged value, this is unlike HTB private implementation of basically the same (TB’s version has some advantages with GRO though).
e.g.:
$TC qdisc add dev $DEV root handle 1: stab mtu 2047 tsize 512 mpu 0 overhead 40 linklayer atm htb default 12

Note: mtu in the stab context only defines the upper bound for which the size table gets created (this is the limit up to which stab works reliable), tsize gives the number of entries in the table, for ATM all increases are at multiples of 16, so 2048/512 = 4bits is enough, and mpu allows to specify a minimum packet size. All in all stab gives more freedom than strictly needed for atm ,link layer accounting...


> If there’s a better way than manually calculating ceil(len/48)*53, so much the better.

	Well, “stab” is so much better than statically accounting for the ~9% ATM cell overhead tax that it is just not funny anymore. Due to the AAL5’s? weird idea that each payload package has to fill an integer number of ATM cells, small packets can get an obscene amount of additional overhead: worst case is something like 1 ATM cell with of data + 1 byte, which will drag in another ATM cell with 47 bytes padding… increasing the overhead by 100% on top of the 9% cell tax and the normal per packet overhead gunk ISPs force unto the unlucky ATM users…

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] #17

[Sebastian Moeller]

Hi Jonathan,

on a added note below; Note to self "reading actually helps"

On Apr 13, 2015, at 02:15 , Sebastian Moeller <moeller0@gmx.de> wrote:

> Hi Jonathan,
> 
> 
> On Apr 13, 2015, at 01:55 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
>> 
>>> On 13 Apr, 2015, at 02:50, Sebastian Moeller <moeller0@gmx.de> wrote:
>>> 
>>> As mentioned above, as long as cake can work with tc’s stab option all framing and overhead handling should be available to whoever needs it.
>> 
>> Remind me to look into how to use that, then.  
> 
> basically you add:
> stab mtu 2047 tsize 512 mpu 0 overhead 40 linklayer atm
> to the top level qdisc invocation, it fudges the size variable in the SKB? just as the kernels internal accounting for the ethernet header does, and all qdisc will see the fudged value, this is unlike HTB private implementation of basically the same (TB’s version has some advantages with GRO though).
> e.g.:
> $TC qdisc add dev $DEV root handle 1: stab mtu 2047 tsize 512 mpu 0 overhead 40 linklayer atm htb default 12
> 
> Note: mtu in the stab context only defines the upper bound for which the size table gets created (this is the limit up to which stab works reliable), tsize gives the number of entries in the table, for ATM all increases are at multiples of 16, so 2048/512 = 4bits is enough, and mpu allows to specify a minimum packet size. All in all stab gives more freedom than strictly needed for atm ,link layer accounting...
> 
> 
>> If there’s a better way than manually calculating ceil(len/48)*53, so much the better.

	Ignore my gibberish below, that is almost all required: ceil(len+additional_per_packet_overhead/48)*53 seems the winner ;) Effectively this is what stab should offer, but its implementation with a table has issues if packets are larger than the table estimated…

Best Regards
	Sebastian

> 
> 	Well, “stab” is so much better than statically accounting for the ~9% ATM cell overhead tax that it is just not funny anymore. Due to the AAL5’s? weird idea that each payload package has to fill an integer number of ATM cells, small packets can get an obscene amount of additional overhead: worst case is something like 1 ATM cell with of data + 1 byte, which will drag in another ATM cell with 47 bytes padding… increasing the overhead by 100% on top of the 9% cell tax and the normal per packet overhead gunk ISPs force unto the unlucky ATM users…
> 
> Best Regards
> 	Sebastian
> 
>> 
>> - Jonathan Morton
>> 
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] #17

[Jonathan Morton]

> On 13 Apr, 2015, at 03:23, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> Ignore my gibberish below, that is almost all required: ceil(len+additional_per_packet_overhead/48)*53 seems the winner ;) Effectively this is what stab should offer, but its implementation with a table has issues if packets are larger than the table estimated…

The other problem with stab is that the configuration is awfully verbose - pretty much what I was trying to avoid with cake.  I’ll admit that a table lookup can be faster than a calculation (under favourable circumstances), but exposing those sorts of implementation details to userspace is just going to give people headaches while they try to figure out what to put and why.

That’s a recipe for people (and, more seriously, CPE vendors) getting it wrong.

So cake just has that simple “atm” flag, which adds roughly the right amount of overhead for cell-framing already.  Any optimisations to that calculation (which are certainly possible) are an internal matter and not for public consumption.

It could be refined by adding an additional, orthogonal setting for non-cell framing overhead, which would be added before the cell-framing calculation; this would allow adding the 8 bytes of PPPoE framing and/or the 2 bytes of Ethernet VLAN tag, for people who want to get it exactly right.  Whether these are simple, self-descriptive flags which can be combined for the desired effect, or a numeric parameter which must be set up, is open for discussion.

All of this will help with setting cake’s shaper closer to, and eventually perhaps *at* the physical link rate.  The relative lack of bursting from cake’s shaper already allows some of that, since it’s no longer necessary to allow the initial burst from the token bucket to drain from the dumb FIFO downstream.

 - Jonathan Morton

Re: [Cake] #17

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 3793 bytes --]

Hi Jonathan,

On April 13, 2015 2:45:35 AM GMT+02:00, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>> On 13 Apr, 2015, at 03:23, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>> Ignore my gibberish below, that is almost all required:
>ceil(len+additional_per_packet_overhead/48)*53 seems the winner ;)
>Effectively this is what stab should offer, but its implementation with
>a table has issues if packets are larger than the table estimated…
>
>The other problem with stab is that the configuration is awfully
>verbose - pretty much what I was trying to avoid with cake.  

I fully agree that simple and automatic are real requirements if we want this to be actually used. I also agree that tc stab has more options than absolutely necessary.

I’ll admit
>that a table lookup can be faster than a calculation (under favourable
>circumstances), but exposing those sorts of implementation details to
>userspace is just going to give people headaches while they try to
>figure out what to put and why.

        I am not arguing for the table approach, calculating the size expansion directly might be a bit more costly, but it will scale to GRO packets without requiring special configuration, so certainly your current approach seems better suited for cake than a table.


>
>That’s a recipe for people (and, more seriously, CPE vendors) getting
>it wrong.

        Well, getting it wrong is bad, but not being able to configure it at all is also going to be wrong for ATM users 100% of the time ;)

>
>So cake just has that simple “atm” flag, which adds roughly the right
>amount of overhead for cell-framing already.  

        Let me be frank here, roughly the right amount is most likely a chiffre for wrong most of the time. As long as you overestimate the overhead you are sacrificing some bandwidth, but underestimating the overhead will make the shaper misbehave...

Any optimisations to that
>calculation (which are certainly possible) are an internal matter and
>not for public consumption.

        I fear that this is not going to work to well, some things have complex solutions because the problem is complex.

>
>It could be refined by adding an additional, orthogonal setting for
>non-cell framing overhead, which would be added before the cell-framing
>calculation; this would allow adding the 8 bytes of PPPoE framing
>and/or the 2 bytes of Ethernet VLAN tag, for people who want to get it
>exactly right.  

        I fully endorse this plan!

Whether these are simple, self-descriptive flags which
>can be combined for the desired effect, or a numeric parameter which
>must be set up, is open for discussion.

        If you look at the available tables for valid ATM per packet overheads you will notice them to be quite large, even though none of them includes vlan tags. I want to argue that enumerating them to turn these into symbolic flags is going to be the opposite of simple; hence I would argue for a simple numeric parameter 'overhead'. Maybe default to your automatic estimate unless 'overhead' is specified?


>
>All of this will help with setting cake’s shaper closer to, and
>eventually perhaps *at* the physical link rate.  The relative lack of
>bursting from cake’s shaper already allows some of that, since it’s no
>longer necessary to allow the initial burst from the token bucket to
>drain from the dumb FIFO downstream.

        With the correct link layer encapsulation values I ran my ATM link with sqm-scripts shaped at 100% egress without induced latency (above the expected), so if sqm scripts can do it so sure can cake :)

Best Regards
        Sebastian

>
> - Jonathan Morton

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 95 bytes --]

Re: [Cake] #17

[Sebastian Moeller]

Hi Jonathan,


On Apr 13, 2015, at 02:45 , Jonathan Morton <chromatix99@gmail.com> wrote:

> 
>> On 13 Apr, 2015, at 03:23, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>> Ignore my gibberish below, that is almost all required: ceil(len+additional_per_packet_overhead/48)*53 seems the winner ;) Effectively this is what stab should offer, but its implementation with a table has issues if packets are larger than the table estimated…
> 
> The other problem with stab is that the configuration is awfully verbose - pretty much what I was trying to avoid with cake.  

	I agree, “overhead" and "link layer" should be sufficient and link layer can be turned into a simple “atm” flag as you already implemented.

> I’ll admit that a table lookup can be faster than a calculation (under favourable circumstances), but exposing those sorts of implementation details to userspace is just going to give people headaches while they try to figure out what to put and why.

	I think your method is superior to the table as it will work for any packet size, and will not require the user to either specify the table sizing parameters or default to a crazy 64K table to cover GRO.

> 
> That’s a recipe for people (and, more seriously, CPE vendors) getting it wrong.

	Yes, simple wins here, but it needs to be complex enough to actually cover the relevant use cases...

> 
> So cake just has that simple “atm” flag, which adds roughly the right amount of overhead for cell-framing already.  

	The cell framing with the last partly padded cell absolutely requires the correct overhead (well too large an overhead will only cost bandwidth) to work well, other wise the occasional packet will drag in up to 47 bytes of unaccounted bandwidth use.

> Any optimisations to that calculation (which are certainly possible) are an internal matter and not for public consumption.

	There is no reliably way, that I n=know to quickly auto detect the encapsulation overhead; I do have a rather crufty and slow method that will take hours.

> 
> It could be refined by adding an additional, orthogonal setting for non-cell framing overhead, which would be added before the cell-framing calculation; this would allow adding the 8 bytes of PPPoE framing and/or the 2 bytes of Ethernet VLAN tag, for people who want to get it exactly right.  

	I fully endorse this, actually this “overhead” parameter should work independent of the “atm” flag (on my VDSL link I have 16 bytes overhead that needs accounting on top of the ethernet header). For non-arm carriers getting the per-packet overhead wrong is not as terrible, but it will make the shaper suck with small packets...

> Whether these are simple, self-descriptive flags which can be combined for the desired effect, or a numeric parameter which must be set up, is open for discussion.

	Have a look at http://ace-host.stuart.id.au/russell/files/tc/tc-atm/ and http://www.faqs.org/rfcs/rfc2684.html (neither of which deals with potential VLAN tags between BRAS and Modem, that are “invisible” to packet captures as they never reach the endusers network, but still cost bottleneck bandwidth); the whole encapsulation issue is a mess that will most likely not lend it self to a small enumeration of encapsulations, so I vote for a) defaulting to 44-14 bytes overhead (this is pretty much the largest I have seen in the wild, but I have not seen much) and b) allowing the user to override this with something along “overhead 26” that will completely replace the 44-14 bytes from above. So have a reasonable default, but allow fine-tuning

> 
> All of this will help with setting cake’s shaper closer to, and eventually perhaps *at* the physical link rate.  

	Just a data point, when my link was still running ADSL2+ I manged to get the egress shaping to work well at 100% of line rate (around 2500 Kbps), so even sqm-scripts could reach that goal. (After the switch to vdsl2 I learned the hard way that my ISP had implemented a BRAS throttle that sort of mooted my attempts to reach 100% of the vdsl line rate).

> The relative lack of bursting from cake’s shaper already allows some of that, since it’s no longer necessary to allow the initial burst from the token bucket to drain from the dumb FIFO downstream.

	Interesting.

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] #17

[Jonathan Morton]

> On 13 Apr, 2015, at 17:45, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
>> It could be refined by adding an additional, orthogonal setting for non-cell framing overhead, which would be added before the cell-framing calculation; this would allow adding the 8 bytes of PPPoE framing and/or the 2 bytes of Ethernet VLAN tag, for people who want to get it exactly right.  
> 
> I fully endorse this, actually this “overhead” parameter should work independent of the “atm” flag (on my VDSL link I have 16 bytes overhead that needs accounting on top of the ethernet header). For non-atm carriers getting the per-packet overhead wrong is not as terrible, but it will make the shaper suck with small packets…

Well yes, that’s what I meant by “orthogonal”.

>> Whether these are simple, self-descriptive flags which can be combined for the desired effect, or a numeric parameter which must be set up, is open for discussion.
> 
> 	Have a look at http://ace-host.stuart.id.au/russell/files/tc/tc-atm/ and http://www.faqs.org/rfcs/rfc2684.html (neither of which deals with potential VLAN tags between BRAS and Modem, that are “invisible” to packet captures as they never reach the endusers network, but still cost bottleneck bandwidth); the whole encapsulation issue is a mess that will most likely not lend it self to a small enumeration of encapsulations

What I propose is that I add a numeric overhead parameter to cake’s configuration API, and then in iproute2 I can provide keywords as a more user-friendly way of specifying the common cases, as well as offering direct access to the numeric.  So specifying “pppoa vcmux” would be shorthand for “atm overhead 10”, “vdsl” would be shorthand for “noatm overhead 16”, and “raw” would allow reverting to an unadjusted configuration, ie. “noatm overhead 0".  And it’s easy to retrospectively refine the behaviour of those keywords, because they’re in userspace.

 - Jonathan Morton

Re: [Cake] #17

[Sebastian Moeller]

Hi Jonathan,


On Apr 14, 2015, at 12:13 , Jonathan Morton <chromatix99@gmail.com> wrote:

> 
>> On 13 Apr, 2015, at 17:45, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>>> It could be refined by adding an additional, orthogonal setting for non-cell framing overhead, which would be added before the cell-framing calculation; this would allow adding the 8 bytes of PPPoE framing and/or the 2 bytes of Ethernet VLAN tag, for people who want to get it exactly right.  
>> 
>> I fully endorse this, actually this “overhead” parameter should work independent of the “atm” flag (on my VDSL link I have 16 bytes overhead that needs accounting on top of the ethernet header). For non-atm carriers getting the per-packet overhead wrong is not as terrible, but it will make the shaper suck with small packets…
> 
> Well yes, that’s what I meant by “orthogonal”.

	Ah, good.

> 
>>> Whether these are simple, self-descriptive flags which can be combined for the desired effect, or a numeric parameter which must be set up, is open for discussion.
>> 
>> 	Have a look at http://ace-host.stuart.id.au/russell/files/tc/tc-atm/ and http://www.faqs.org/rfcs/rfc2684.html (neither of which deals with potential VLAN tags between BRAS and Modem, that are “invisible” to packet captures as they never reach the endusers network, but still cost bottleneck bandwidth); the whole encapsulation issue is a mess that will most likely not lend it self to a small enumeration of encapsulations
> 
> What I propose is that I add a numeric overhead parameter to cake’s configuration API, and then in iproute2 I can provide keywords as a more user-friendly way of specifying the common cases, as well as offering direct access to the numeric.  So specifying “pppoa vcmux” would be shorthand for “atm overhead 10”,

	This is going to be challenging to keep it simple, as we still need to think about what the kernel does on ethernet interfaces (it accounts for the ethernet header) so for PPPoE, LLC/SNAP, RFC-2684 you already need 3 flags plus a flag for ethernet header (which h the kernel does account for on ethernet devices but not on true ATM devices even if an ethernet header is used on the wire). Also most users do not really know or have a reliable way to figure out the encapsulation used (short of the slow empiric way I have been pushing for some time). But as long as there is a way to specify the overhead numerically, having a nicer symbolic way is not going to hurt.

> “vdsl” would be shorthand for “noatm overhead 16”,

	But, that is only true for my link including PPPoE and VLAN overhead, vdsl2 only drags in an additional 5 bytes:
VDSL (IEEE 802.3-2012 61.3 relevant for VDSL2): 2 Byte PPP + 6 Byte PPPoE + 4 Byte VLAN + 1 Byte Start of Frame (S), 1 Byte End of Frame (Ck), 2 Byte TC-CRC (PTM-FCS), = 16 Byte
	Also I am uncertain whether vdsl2 transmits the ethernet frame check sequence or not… (and unlike with the ATM carrier I have no idea how to measure the actual per packet overhead…)
In my case this really does not matter though, as the BRAS throttle accounts for 16 bytes (dual VLAN tags) so that is the value I need to shape for…



> and “raw” would allow reverting to an unadjusted configuration, ie. “noatm overhead 0”.

	I like raw as simple way to clear all the individual symbolic constants ;)

>  And it’s easy to retrospectively refine the behaviour of those keywords, because they’re in userspace.

	Is it simlper to get a patch into iproute2 than the kernel? and are the iproute2 maintainers open to redefinition of symbolic values?

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] #17

[Sebastian Moeller]

Hi Dave, hi Jonathan,


>> On Sat Apr 11 19:33:59 PDT 2015, Dave That <dave.taht at gmail.com> wrote:
>> 17)  the atm compensation in cake is entirely untested. And it is
>> unclear as to how best handle pppoe.

	I wonder could cake use http://lxr.free-electrons.com/ident?i=skb_flow_dissect to get the inputs for the hash function? This function seems to dive into pppoe as well as GRE so should allow proper flow separation for a number of relevant encapsulations without the need to code this up again? (For all I know cake might already be using this ;) ) Thgis would be nice since then cake could easily run on a physical interface like ge00 in cerowrt and would not need to work on pppoe-ge00 (which is a pain as it comes and goes ;) )

Best Regards
	Sebastian

Re: [Cake] #17

[Jonathan Morton]

> On 13 Apr, 2015, at 17:59, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> I wonder could cake use http://lxr.free-electrons.com/ident?i=skb_flow_dissect to get the inputs for the hash function?

It already does, just like fq_codel.

 - Jonathan Morton

Re: [Cake] #17

[Sebastian Moeller]

Hi Jonathan,

On Apr 13, 2015, at 19:06 , Jonathan Morton <chromatix99@gmail.com> wrote:

> 
>> On 13 Apr, 2015, at 17:59, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>> I wonder could cake use http://lxr.free-electrons.com/ident?i=skb_flow_dissect to get the inputs for the hash function?
> 
> It already does, just like fq_codel.

	This is great! Does skb_fow_dissect also allow access to the TOS bits? Currently in sqm-scripts we “manually” extract the TOS bits wit a different set of tc filters for each encapsulation and protocol, given the cost of the fi;ted operation this sucks; it would be sweet if this could be made cheap enough to not care anymore. Again this might already be implemented ;)

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] #17

[Jonathan Morton]

> On 13 Apr, 2015, at 22:45, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> Does skb_flow_dissect also allow access to the TOS bits?

No, but I do extract the DSCP (which is in the TOS byte) a different way.

 - Jonathan Morton

Re: [Cake] #17

[Sebastian Moeller]

Hi Jonathan,

On Apr 13, 2015, at 22:08 , Jonathan Morton <chromatix99@gmail.com> wrote:

> 
>> On 13 Apr, 2015, at 22:45, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>> Does skb_flow_dissect also allow access to the TOS bits?
> 
> No, but I do extract the DSCP (which is in the TOS byte) a different way.

	A pity, since skb_flow_dissect already delivers so much from the right vicinity ;) Does your method also work with the same encapsulations as skb_flow_dissect?

Best Regards
	Sebastian

> 
> - Jonathan Morton
> 

Re: [Cake] #17

[Jonathan Morton]

> On 13 Apr, 2015, at 23:10, Sebastian Moeller <moeller0@gmx.de> wrote:
> 
> Does your method also work with the same encapsulations as skb_flow_dissect?

I’m not sure.  Given the complexity, it’s probably neither a strict superset or subset; it’s also fairly likely that the outermost DSCP is returned from a tunnelled IP scenario.  ISTR that tunnels are supposed to copy the TOS byte on entry and the ECN field on exit, though there are almost certainly tunnel implementations which don’t do that.

 - Jonathan Morton

Re: [Cake] #17

[Dave Taht]

I think better walking down the packet than flow_dissect currently
does, and doing more stuff would be a computational and functional
win.

(boy do we need to start profiling!!)

While we are doing that, perhaps... well... see my next message.

On Mon, Apr 13, 2015 at 5:58 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>
>> On 13 Apr, 2015, at 23:10, Sebastian Moeller <moeller0@gmx.de> wrote:
>>
>> Does your method also work with the same encapsulations as skb_flow_dissect?
>
> I’m not sure.  Given the complexity, it’s probably neither a strict superset or subset; it’s also fairly likely that the outermost DSCP is returned from a tunnelled IP scenario.  ISTR that tunnels are supposed to copy the TOS byte on entry and the ECN field on exit, though there are almost certainly tunnel implementations which don’t do that.
>
>  - Jonathan Morton
>



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Cake] #17

[Sebastian Moeller]

Hi Jonathan,

On Apr 14, 2015, at 02:58 , Jonathan Morton <chromatix99@gmail.com> wrote:

> 
>> On 13 Apr, 2015, at 23:10, Sebastian Moeller <moeller0@gmx.de> wrote:
>> 
>> Does your method also work with the same encapsulations as skb_flow_dissect?
> 
> I’m not sure.  Given the complexity, it’s probably neither a strict superset or subset; it’s also fairly likely that the outermost DSCP is returned from a tunnelled IP scenario.  ISTR that tunnels are supposed to copy the TOS byte on entry and the ECN field on exit, though there are almost certainly tunnel implementations which don’t do that.

	Fair enough, I was actually just thinking about PPPoE and VLAN/double VLAN tags, both of which change the offset of the TOS bits (as well as the offset difference for IPv4 and IPv6), so it seems a bit of a hassle handling down the encapsulation twice (once for the TOS bits and once for the flow hash). Most likely I am biased due to the computational cost of doing this with a naive set of tc filters ;) I guess.

Best Regards
	Sebastian

> 
> - Jonathan Morton
>