[Cake] New to cake. Some questions

[Cake] New to cake. Some questions

[Dennis Fedtke]

Hi

Currently im running lede + cake + sqm_scripts and i have some questions:

1. What is considered the "optimal" setup atm for cake?
e.g. which cake script should i use piece or layer cake?
2. Recently squash and wash was removed.
But the sqm scripts were not updated. In the advanced options should i 
set that the dcsp marks are kept?
3. Should i use advanced options in sqm scripts and set triple-isolate + 
diffserv8 ?
4. Is it recommend to enable diffserv on ingress?
5. Is there still the udp packet dropping problem? e.g. games that are 
using udp.
If yes does it make sense to apply diffserv classes manually? How to do 
this?
6. is the autorate_ingress still under development?
This very interesting feature. especially for docsis networks. Will it 
be possible to set target ping time?
6. What difference does it make to set a different rtt?
Setting lower rtt will reduce download speed i guess but will it allow 
better ping times (because of lower downloadrate uh)?
What happens if rtt is set way higher?

Thank you!

Re: [Cake] New to cake. Some questions

[moeller0]

Hi Dennis,

let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…

> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
> 
> Hi
> 
> Currently im running lede + cake + sqm_scripts and i have some questions:
> 
> 1. What is considered the “optimal" setup atm for cake?

	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.

> e.g. which cake script should i use piece or layer cake?

	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..

> 2. Recently squash and wash was removed.
> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?

	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).

> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?

	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)

> 4. Is it recommend to enable diffserv on ingress?

	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.

> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
> If yes does it make sense to apply diffserv classes manually? How to do this?

	I am not sure what you mean, but if you test this and have some findings please report here…

> 6. is the autorate_ingress still under development?
> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?

	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.

> 6. What difference does it make to set a different rtt?
> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
> What happens if rtt is set way higher?

	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)


> 
> Thank you!

	I am sure the real experts will also chime in…

Best Regards
	Sebastian

> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] New to cake. Some questions

[Kevin Darbyshire-Bryant]

On 09/06/16 21:58, Dennis Fedtke wrote:
> Hi
>
> Currently im running lede + cake + sqm_scripts and i have some questions:
>
> 1. What is considered the "optimal" setup atm for cake?
> e.g. which cake script should i use piece or layer cake?
Piece of cake doesn't use diffserv hence doesn't 'soft shape' traffic 
into bandwidth categories.  Layer cake uses DSCP to divide packets into 
relevant flow types and 'soft shape/limit' them.  Also do you mean 'at 
the moment' or do you mean 'ATM - Async Transfer Mode'?
>
> 2. Recently squash and wash was removed.
> But the sqm scripts were not updated. In the advanced options should i 
> set that the dcsp marks are kept?
On point 2: sqm-scripts was never actually updated to take advantage of 
the 'squash' (synonym for besteffort wash) or 'wash' option anyway.  It 
used and continues to use iptables rules to set DSCP marks to '0', in 
combination with 'besteffort' to get cake to ignore the DSCP codes.
> 3. Should i use advanced options in sqm scripts and set triple-isolate 
> + diffserv8 ?
There's almost certainly no point in going to 'diffserv8' - it is 
reliant on suitable DSCP markings to put traffic in each tin and I'd be 
gobsmacked if you had that many different DSCP markings in use. 
Triple-isolate is more interesting and needs a lot more testing...if we 
can ever work out how to test it :-)  Another potential problem is 
'where are you putting the cake qdisc?'  If you're putting this on a WAN 
interface of your standard NAT router, then it won't be seeing you're 
internal IP addresses anyway...all the traffic is coming from/going to 
your router's external Internet facing IP..it doesn't see the 
de-masqueraded addresses of your internal LAN.
> 4. Is it recommend to enable diffserv on ingress?
I guess it depends if you see different DSCP markings on your ingress 
traffic or if your ISP mangles them somehow anyway.  'tc -s qdisc show' 
would show if any packets have been placed in different tins.  I happen 
to use diffserv4 on ingress even though my ISP seems to mangle them 
anyway :-)
> 5. Is there still the udp packet dropping problem? e.g. games that are 
> using udp.
> If yes does it make sense to apply diffserv classes manually? How to 
> do this?
What udp packet problem?
> 6. is the autorate_ingress still under development?
> This very interesting feature. especially for docsis networks. Will it 
> be possible to set target ping time?
no idea
> 6. What difference does it make to set a different rtt?
> Setting lower rtt will reduce download speed i guess but will it allow 
> better ping times (because of lower downloadrate uh)?
> What happens if rtt is set way higher?
Don't mess with the rtt parameter unless you're on either extremely fast 
links 10gbit+ OR you have a high latency link (satellite)  If you really 
must know more lookup info on 'codel interval' and look at 
http://queue.acm.org/detail.cfm?id=2209336  CAKE uses codel for its flow 
queueing management.
>
> Thank you!
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] New to cake. Some questions

[Jonathan Morton]

> On 10 Jun, 2016, at 00:36, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>> If yes does it make sense to apply diffserv classes manually? How to do this?

> What udp packet problem?

He’s probably referring to the tendency of non-flow-isolating AQMs to drop packets indiscriminately when under load.

Cake is flow-isolating and thus applies a separate AQM algorithm to each flow.  As such, UDP gaming/VoIP traffic won’t get dropped unless it exceeds its fair share of the link, which is unlikely for a well-designed, lightweight protocol.

We really should make an effort to put a more intuitive GUI interface on this.  These questions indicate a user overwhelmed by many options without guidance.

 - Jonathan Morton

Re: [Cake] New to cake. Some questions

[Dennis Fedtke]

Hi Sebastian,

thank you for your answers :)

The ATM overhead detector script is currently running.
I read the wiki about it but im not quite sure how to interpret the plot.
I mean what info should i read from it? maximum packet size?
If yes do i set the overhead in cake? Or do i set iptables to clamp to 
new mtu/mss?

Regarding UDP paket dropping problem:
I just read some forums and users stated that under heavy load cake 
starts to drop udp packets which causes lag ingame.
My idea was to set ingress/egress to diffserv4 and apply the EF dscp 
mark on those packets.
Will this even work? if yes how to do this? iptables?

ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j 
DSCP --set-dscp-class EF

Like thia? Is prerouting correct here? (Taken from layer cake script)


For the squash and wash feature.
Im asking because if i choose to squash in the advanced options of sqm 
scripts.
The dscp fields/marks will be overwritten by iptables to 0 (besteffort). 
(layer cake script)
So then it makes no sense to manually set dscp fields/marks or? (Or even 
setting diffserv)

Did i understand this correctly. Per rfc isps should not provide dscp 
fields/marks?


Thank you.



Am 09.06.2016 um 23:30 schrieb moeller0:
> Hi Dennis,
>
> let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…
>
>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>
>> Hi
>>
>> Currently im running lede + cake + sqm_scripts and i have some questions:
>>
>> 1. What is considered the “optimal" setup atm for cake?
> 	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.
>
>> e.g. which cake script should i use piece or layer cake?
> 	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..
>
>> 2. Recently squash and wash was removed.
>> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?
> 	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).
>
>> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?
> 	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)
>
>> 4. Is it recommend to enable diffserv on ingress?
> 	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.
>
>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>> If yes does it make sense to apply diffserv classes manually? How to do this?
> 	I am not sure what you mean, but if you test this and have some findings please report here…
>
>> 6. is the autorate_ingress still under development?
>> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?
> 	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.
>
>> 6. What difference does it make to set a different rtt?
>> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
>> What happens if rtt is set way higher?
> 	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)
>
>
>> Thank you!
> 	I am sure the real experts will also chime in…
>
> Best Regards
> 	Sebastian
>
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] New to cake. Some questions

[moeller0]

Hi Jonathan,


> On Jun 10, 2016, at 00:17 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
>> On 10 Jun, 2016, at 00:36, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>> 
>>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>>> If yes does it make sense to apply diffserv classes manually? How to do this?
> 
>> What udp packet problem?
> 
> He’s probably referring to the tendency of non-flow-isolating AQMs to drop packets indiscriminately when under load.
> 
> Cake is flow-isolating and thus applies a separate AQM algorithm to each flow.  As such, UDP gaming/VoIP traffic won’t get dropped unless it exceeds its fair share of the link, which is unlikely for a well-designed, lightweight protocol.
> 
> We really should make an effort to put a more intuitive GUI interface on this.  These questions indicate a v.

	Let me be frank to the level of impoliteness, let’s tests things first and once they work as described in the mostly missing documentation then let’s work on a GUI (a GUI is the wrong place to fix up a bad CLI interface for instance). I am sorry that I have to be so blunt, but I fear due to me not being a native english speaker, we will not be able to unambiguously communicate.

So here is my top five list of cake related topics that I would appreciate if you could address them:

1) the isolation options:
	Does triple-isolate really work, and what is the theoretical prediction how it should exactly work? 
		The last test indicates we might have problems there
	Do the dual isolation options work as intended, and what exactly is intended? 
		Recent tests indicate they might work, but they are nasty to set up for users, due to the NAT issue.
	How much additional CPU load do these options cause, or what is the computational cost expressed in lost bandwidth or additionally gained latency under load?

	We are currently trying to distribute a few test scripts to openwrt volunters to help answer these question, but before they are answered I vote to not further exposing anything isolation related in the sqm-scripts GUI. We (well at least I do) plan to use the results of these tests to decide whether to change the defaults for cake under sqm-scripts. But even if the tests work out we need the man page to describe reasonably well what tho expect, so users can decide about the applicability of the isolation option based on there needs.

2) auto-rate:
	The same recent test indicated that auto-rate might not work well on a fixed rate link. This is somewhat unexpected as a fixed rate can be easily seen as a corner-case for a variable rate. I would vote for researching why this is, and anyway documenting the suitability for variable or fixed rate links more prominently so our users can make informed policy choices for their home networks.

3) the dreaded encapsulation keywords
	Please address the issues I raised in too many mails this months… which boil down to questions of scope “why are the vdsl options suffixed with _ptm, but the atm options are not?” and of minimality “is the currently selected set of keywords minimal and complete?” and transparene “will a user of a compund keyword be easily able to predict all side effects?” and confusion “why name something conservative that will for all peop;e not using an ATM link cost between 9 to 40% of goodput?”. Since all of this is a question for tc and not so much core cake, things should be relatively quick to sort out, but they should be sorted out before distributing cake wider/ or exposing the currently marginally consistent state to more users. As if we expose to much it becomes API and immutable…

4) documantation:
	I fully agree with your conclusion that a “user overwhelmed by many options without guidance”. So please write/improve the “missing” man page and description of algorithms already. Given that you designed all the core of cake you are the best qualified person to do this.

5) development focus/sequence:
	My understanding is that cake currently contains a fair number of interesting and relevant functions that our users certainly want and will appreciate. Unfortunately, some of those do not seem to be adequately documented (see 4) and some might actually be not ready for primetime (that requires further coformatory testing that I am willing and in the process of helping with). I would be really happy if you could give a tentative development priority list, so that we can harmonize testing features with your development, so that the testing actually helps you in your process.

But please do not drag in GUI before cake and tc are fully “baked”.


Best Regards
	Sebastian


> 
> - Jonathan Morton
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] New to cake. Some questions

[moeller0]

Hi Dennis,

> On Jun 10, 2016, at 00:45 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
> 
> Hi Sebastian,
> 
> thank you for your answers :)
> 
> The ATM overhead detector script is currently running.
> I read the wiki about it but im not quite sure how to interpret the plot.
> I mean what info should i read from it? maximum packet size?

	The relevant number is reported as “Estimated overhead preceding the IP header” in the top part of the second figure created by the script. But that is only relevant.useful if you see a nice step like plot in figure 2 as well ( the second figure in https://github.com/moeller0/ATM_overhead_detector/wiki as positive and the fourth figure as negative example.

> If yes do i set the overhead in cake? Or do i set iptables to clamp to new mtu/mss?

	If you use plain cake and you know the numerical overhead (NN) the easiest is to add the following to your cake invocation: “atm overhead NN”

Please note that if you use cake on an ethernet interface the kernel will already account for 14 byte of ethernet overhead, so if the script told you 44 as actual overhead, you use ”overhead 30” to address that. If you use a pppoe interface the kernel will most likely not add the 14 bytes for you, so then you would use “overhead 44” (I excluded the atm option in the last examples for clarity…)

> 
> Regarding UDP paket dropping problem:
> I just read some forums and users stated that under heavy load cake starts to drop udp packets which causes lag ingame.
> My idea was to set ingress/egress to diffserv4 and apply the EF dscp mark on those packets.

	Ell, not a bad idea, but often the problem are in the incoming traffic, and unfortunately with the ifb we use we can not use iptables, but only tc, and remarking with tc is unpleasant.

> Will this even work? if yes how to do this? iptables?

	No, you wuld need tp use tc.

> 
> ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j DSCP --set-dscp-class EF
> 
> Like thia? Is prerouting correct here? (Taken from layer cake script)

	This will affect outgoing packets and might be a good idea in your specific case.

BUT why don’t you try the default behaviour with specific rules and tricks and report success or failure back to us, after all the fastest/easiest classification is one one does not need to perform at all.

> 
> 
> For the squash and wash feature.
> Im asking because if i choose to squash in the advanced options of sqm scripts.
> The dscp fields/marks will be overwritten by iptables to 0 (besteffort). (layer cake script)
> So then it makes no sense to manually set dscp fields/marks or? (Or even setting diffserv)

	No unfortunately on ingress cake sees the packets before iptables, so the effective behavioral emulation of wash/squash by cake is to set ingress cake to besteffort (basically cake ignores the dscp field which functionally is identical to all packets having the same value). The squashing by iptables just clears the dscp marls so that internal networking elements like potentially wifi liknks are not confuzed by the dscp information.

> 
> Did i understand this correctly. Per rfc isps should not provide dscp fields/marks?

	Not exactly, per RFC DSCPs are only ever valid/defined inside a DSCP domain and your ISPs domain ends before it reaches your CPE. Since you have no control over your ISPs markings, they can be very much not like you want them to be (Dave That reported that his ISP re-mapped almost 1/3 or so of packets into the CS1 background class). So it is recomended that each DSCP domain re-mapps the code points at its entry point, which in your case is your router…

Best Regards
	Sebastian

> 
> 
> Thank you.
> 
> 
> 
> Am 09.06.2016 um 23:30 schrieb moeller0:
>> Hi Dennis,
>> 
>> let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…
>> 
>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>> 
>>> Hi
>>> 
>>> Currently im running lede + cake + sqm_scripts and i have some questions:
>>> 
>>> 1. What is considered the “optimal" setup atm for cake?
>> 	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.
>> 
>>> e.g. which cake script should i use piece or layer cake?
>> 	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..
>> 
>>> 2. Recently squash and wash was removed.
>>> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?
>> 	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).
>> 
>>> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?
>> 	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)
>> 
>>> 4. Is it recommend to enable diffserv on ingress?
>> 	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.
>> 
>>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>>> If yes does it make sense to apply diffserv classes manually? How to do this?
>> 	I am not sure what you mean, but if you test this and have some findings please report here…
>> 
>>> 6. is the autorate_ingress still under development?
>>> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?
>> 	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.
>> 
>>> 6. What difference does it make to set a different rtt?
>>> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
>>> What happens if rtt is set way higher?
>> 	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)
>> 
>> 
>>> Thank you!
>> 	I am sure the real experts will also chime in…
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
> 
> 
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] New to cake. Some questions

[Jonathan Morton]

> why are the vdsl options suffixed with _ptm, but the atm options are not?

Because the “vcmux” and “llc” suffixes are sufficient to imply ATM cell framing.

> is the currently selected set of keywords minimal and complete?

I did some careful research back when I added that feature, including taking some suggestions from you, and according to that: yes, it is correct and complete, and every keyword is related to a real protocol.  Though some are not widely used in practice, they *are* widely supported in ubiquitous consumer-grade equipment.

I haven’t seen any evidence to the contrary; if you have any, please show it, if not, PLEASE SHUT UP about it.

That, by the way, is *me* being blunt to the point of rudeness.

> why name something conservative that will for all peop;e not using an ATM link cost between 9 to 40% of goodput?

The use-case for the “conservative” keyword is essentially: “I know what the raw bitrate of the link is, but I have no sodding clue what overhead it has”.  The goal is to prevent the dumb buffers elsewhere from filling up and undoing our good work.

Yes, it will overcompensate, leading to reduced throughput.  That’s recognised and accepted as far as I’m concerned; the worst corner cases are with very small packets, which frankly matter less.  If you don’t want overcompensation, figure out what the real overhead is and set that.

 - Jonathan Morton

Re: [Cake] New to cake. Some questions

[Dennis Fedtke]

Hi Sebastian,

thanks again :)

the first 2 pictures arent loading for me in the browser i had to save 
to hard disk.
here is my results: http://i67.tinypic.com/5cklcg.png

I think it is a negativ one?
The script gave me following log:

Found 45400 ping packets in ping_sweep__20160610_001950.txt
Elapsed time is 767.967 seconds.
Minimum size of ping payload used: 16 bytes.
warning: division by zero
warning: legend: ignoring extra labels
Unknown or ambiguous terminal name 'wxt'
Unknown or ambiguous terminal name 'wxt'
Saved figure (5) to: ping_sweep__20160610_001950_data.png
lower bound estimate for one ATM cell RTT based of specified up and 
downlink is 0.064431 ms.
estimate for one ATM cell RTT based on linear fit of the ping sweep data 
is 0.064431 ms.
Starting brute-force search for optimal stair fit, might take a while...
Unknown or ambiguous terminal name 'wxt'
Unknown or ambiguous terminal name 'wxt'
Best staircase fit cumulative difference is: 25.28
Best linear fit cumulative difference is: 25.787
Quantized ATM carrier LIKELY (cummulative residual: stair fit 25.28 
linear fit 25.787
remaining ATM cell length after ICMP header is 11 bytes.
ICMP RTT of a single ATM cell is 0.059921 ms.

Estimated overhead preceding the IP header: 42 bytes

Can the errors be ignored ?

Best Regards
Dennis


Am 10.06.2016 um 01:11 schrieb moeller0:
> Hi Dennis,
>
>> On Jun 10, 2016, at 00:45 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>
>> Hi Sebastian,
>>
>> thank you for your answers :)
>>
>> The ATM overhead detector script is currently running.
>> I read the wiki about it but im not quite sure how to interpret the plot.
>> I mean what info should i read from it? maximum packet size?
> 	The relevant number is reported as “Estimated overhead preceding the IP header” in the top part of the second figure created by the script. But that is only relevant.useful if you see a nice step like plot in figure 2 as well ( the second figure in https://github.com/moeller0/ATM_overhead_detector/wiki as positive and the fourth figure as negative example.
>
>> If yes do i set the overhead in cake? Or do i set iptables to clamp to new mtu/mss?
> 	If you use plain cake and you know the numerical overhead (NN) the easiest is to add the following to your cake invocation: “atm overhead NN”
>
> Please note that if you use cake on an ethernet interface the kernel will already account for 14 byte of ethernet overhead, so if the script told you 44 as actual overhead, you use ”overhead 30” to address that. If you use a pppoe interface the kernel will most likely not add the 14 bytes for you, so then you would use “overhead 44” (I excluded the atm option in the last examples for clarity…)
>
>> Regarding UDP paket dropping problem:
>> I just read some forums and users stated that under heavy load cake starts to drop udp packets which causes lag ingame.
>> My idea was to set ingress/egress to diffserv4 and apply the EF dscp mark on those packets.
> 	Ell, not a bad idea, but often the problem are in the incoming traffic, and unfortunately with the ifb we use we can not use iptables, but only tc, and remarking with tc is unpleasant.
>
>> Will this even work? if yes how to do this? iptables?
> 	No, you wuld need tp use tc.
>
>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j DSCP --set-dscp-class EF
>>
>> Like thia? Is prerouting correct here? (Taken from layer cake script)
> 	This will affect outgoing packets and might be a good idea in your specific case.
>
> BUT why don’t you try the default behaviour with specific rules and tricks and report success or failure back to us, after all the fastest/easiest classification is one one does not need to perform at all.
>
>>
>> For the squash and wash feature.
>> Im asking because if i choose to squash in the advanced options of sqm scripts.
>> The dscp fields/marks will be overwritten by iptables to 0 (besteffort). (layer cake script)
>> So then it makes no sense to manually set dscp fields/marks or? (Or even setting diffserv)
> 	No unfortunately on ingress cake sees the packets before iptables, so the effective behavioral emulation of wash/squash by cake is to set ingress cake to besteffort (basically cake ignores the dscp field which functionally is identical to all packets having the same value). The squashing by iptables just clears the dscp marls so that internal networking elements like potentially wifi liknks are not confuzed by the dscp information.
>
>> Did i understand this correctly. Per rfc isps should not provide dscp fields/marks?
> 	Not exactly, per RFC DSCPs are only ever valid/defined inside a DSCP domain and your ISPs domain ends before it reaches your CPE. Since you have no control over your ISPs markings, they can be very much not like you want them to be (Dave That reported that his ISP re-mapped almost 1/3 or so of packets into the CS1 background class). So it is recomended that each DSCP domain re-mapps the code points at its entry point, which in your case is your router…
>
> Best Regards
> 	Sebastian
>
>>
>> Thank you.
>>
>>
>>
>> Am 09.06.2016 um 23:30 schrieb moeller0:
>>> Hi Dennis,
>>>
>>> let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…
>>>
>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>
>>>> Hi
>>>>
>>>> Currently im running lede + cake + sqm_scripts and i have some questions:
>>>>
>>>> 1. What is considered the “optimal" setup atm for cake?
>>> 	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.
>>>
>>>> e.g. which cake script should i use piece or layer cake?
>>> 	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..
>>>
>>>> 2. Recently squash and wash was removed.
>>>> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?
>>> 	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).
>>>
>>>> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?
>>> 	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)
>>>
>>>> 4. Is it recommend to enable diffserv on ingress?
>>> 	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.
>>>
>>>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>>>> If yes does it make sense to apply diffserv classes manually? How to do this?
>>> 	I am not sure what you mean, but if you test this and have some findings please report here…
>>>
>>>> 6. is the autorate_ingress still under development?
>>>> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?
>>> 	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.
>>>
>>>> 6. What difference does it make to set a different rtt?
>>>> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
>>>> What happens if rtt is set way higher?
>>> 	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)
>>>
>>>
>>>> Thank you!
>>> 	I am sure the real experts will also chime in…
>>>
>>> Best Regards
>>> 	Sebastian
>>>
>>>> _______________________________________________
>>>> Cake mailing list
>>>> Cake@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/cake
>>
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] New to cake. Some questions

[Dennis Fedtke]

Hi Sebastian,

Sorry this is positive or?
But i need more samples ?

Thanks :)


Am 10.06.2016 um 01:11 schrieb moeller0:
> Hi Dennis,
>
>> On Jun 10, 2016, at 00:45 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>
>> Hi Sebastian,
>>
>> thank you for your answers :)
>>
>> The ATM overhead detector script is currently running.
>> I read the wiki about it but im not quite sure how to interpret the plot.
>> I mean what info should i read from it? maximum packet size?
> 	The relevant number is reported as “Estimated overhead preceding the IP header” in the top part of the second figure created by the script. But that is only relevant.useful if you see a nice step like plot in figure 2 as well ( the second figure in https://github.com/moeller0/ATM_overhead_detector/wiki as positive and the fourth figure as negative example.
>
>> If yes do i set the overhead in cake? Or do i set iptables to clamp to new mtu/mss?
> 	If you use plain cake and you know the numerical overhead (NN) the easiest is to add the following to your cake invocation: “atm overhead NN”
>
> Please note that if you use cake on an ethernet interface the kernel will already account for 14 byte of ethernet overhead, so if the script told you 44 as actual overhead, you use ”overhead 30” to address that. If you use a pppoe interface the kernel will most likely not add the 14 bytes for you, so then you would use “overhead 44” (I excluded the atm option in the last examples for clarity…)
>
>> Regarding UDP paket dropping problem:
>> I just read some forums and users stated that under heavy load cake starts to drop udp packets which causes lag ingame.
>> My idea was to set ingress/egress to diffserv4 and apply the EF dscp mark on those packets.
> 	Ell, not a bad idea, but often the problem are in the incoming traffic, and unfortunately with the ifb we use we can not use iptables, but only tc, and remarking with tc is unpleasant.
>
>> Will this even work? if yes how to do this? iptables?
> 	No, you wuld need tp use tc.
>
>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j DSCP --set-dscp-class EF
>>
>> Like thia? Is prerouting correct here? (Taken from layer cake script)
> 	This will affect outgoing packets and might be a good idea in your specific case.
>
> BUT why don’t you try the default behaviour with specific rules and tricks and report success or failure back to us, after all the fastest/easiest classification is one one does not need to perform at all.
>
>>
>> For the squash and wash feature.
>> Im asking because if i choose to squash in the advanced options of sqm scripts.
>> The dscp fields/marks will be overwritten by iptables to 0 (besteffort). (layer cake script)
>> So then it makes no sense to manually set dscp fields/marks or? (Or even setting diffserv)
> 	No unfortunately on ingress cake sees the packets before iptables, so the effective behavioral emulation of wash/squash by cake is to set ingress cake to besteffort (basically cake ignores the dscp field which functionally is identical to all packets having the same value). The squashing by iptables just clears the dscp marls so that internal networking elements like potentially wifi liknks are not confuzed by the dscp information.
>
>> Did i understand this correctly. Per rfc isps should not provide dscp fields/marks?
> 	Not exactly, per RFC DSCPs are only ever valid/defined inside a DSCP domain and your ISPs domain ends before it reaches your CPE. Since you have no control over your ISPs markings, they can be very much not like you want them to be (Dave That reported that his ISP re-mapped almost 1/3 or so of packets into the CS1 background class). So it is recomended that each DSCP domain re-mapps the code points at its entry point, which in your case is your router…
>
> Best Regards
> 	Sebastian
>
>>
>> Thank you.
>>
>>
>>
>> Am 09.06.2016 um 23:30 schrieb moeller0:
>>> Hi Dennis,
>>>
>>> let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…
>>>
>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>
>>>> Hi
>>>>
>>>> Currently im running lede + cake + sqm_scripts and i have some questions:
>>>>
>>>> 1. What is considered the “optimal" setup atm for cake?
>>> 	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.
>>>
>>>> e.g. which cake script should i use piece or layer cake?
>>> 	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..
>>>
>>>> 2. Recently squash and wash was removed.
>>>> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?
>>> 	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).
>>>
>>>> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?
>>> 	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)
>>>
>>>> 4. Is it recommend to enable diffserv on ingress?
>>> 	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.
>>>
>>>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>>>> If yes does it make sense to apply diffserv classes manually? How to do this?
>>> 	I am not sure what you mean, but if you test this and have some findings please report here…
>>>
>>>> 6. is the autorate_ingress still under development?
>>>> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?
>>> 	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.
>>>
>>>> 6. What difference does it make to set a different rtt?
>>>> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
>>>> What happens if rtt is set way higher?
>>> 	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)
>>>
>>>
>>>> Thank you!
>>> 	I am sure the real experts will also chime in…
>>>
>>> Best Regards
>>> 	Sebastian
>>>
>>>> _______________________________________________
>>>> Cake mailing list
>>>> Cake@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/cake
>>
>> _______________________________________________
>> Cake mailing list
>> Cake@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] New to cake. Some questions

[moeller0]

Hi Dennis,

> On Jun 10, 2016, at 02:41 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
> 
> Hi Sebastian,
> 
> thanks again :)
> 
> the first 2 pictures arent loading for me in the browser i had to save to hard disk.
> here is my results: http://i67.tinypic.com/5cklcg.png
> 
> I think it is a negativ one?

	Mmmh, his is quite noisy, more noisy than it should be; I would recommend to redo the test with more sampling points. How many did you use? And which target IP did you end up targeting? Also could you also post the first picture, which shows more of the data distribution?

> The script gave me following log:
> 
> Found 45400 ping packets in ping_sweep__20160610_001950.txt
> Elapsed time is 767.967 seconds.
> Minimum size of ping payload used: 16 bytes.
> warning: division by zero
> warning: legend: ignoring extra labels
> Unknown or ambiguous terminal name 'wxt'
> Unknown or ambiguous terminal name 'wxt'
> Saved figure (5) to: ping_sweep__20160610_001950_data.png
> lower bound estimate for one ATM cell RTT based of specified up and downlink is 0.064431 ms.
> estimate for one ATM cell RTT based on linear fit of the ping sweep data is 0.064431 ms.
> Starting brute-force search for optimal stair fit, might take a while...
> Unknown or ambiguous terminal name 'wxt'
> Unknown or ambiguous terminal name 'wxt'
> Best staircase fit cumulative difference is: 25.28
> Best linear fit cumulative difference is: 25.787
> Quantized ATM carrier LIKELY (cummulative residual: stair fit 25.28 linear fit 25.787
> remaining ATM cell length after ICMP header is 11 bytes.
> ICMP RTT of a single ATM cell is 0.059921 ms.
> 
> Estimated overhead preceding the IP header: 42 bytes
> 
> Can the errors be ignored ?

	I have never seen these before, so I need to see whether I can recreate them, which octave version are you using?

Best Regards
	Sebastian

> 
> Best Regards
> Dennis
> 
> 
> Am 10.06.2016 um 01:11 schrieb moeller0:
>> Hi Dennis,
>> 
>>> On Jun 10, 2016, at 00:45 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>> 
>>> Hi Sebastian,
>>> 
>>> thank you for your answers :)
>>> 
>>> The ATM overhead detector script is currently running.
>>> I read the wiki about it but im not quite sure how to interpret the plot.
>>> I mean what info should i read from it? maximum packet size?
>> 	The relevant number is reported as “Estimated overhead preceding the IP header” in the top part of the second figure created by the script. But that is only relevant.useful if you see a nice step like plot in figure 2 as well ( the second figure in https://github.com/moeller0/ATM_overhead_detector/wiki as positive and the fourth figure as negative example.
>> 
>>> If yes do i set the overhead in cake? Or do i set iptables to clamp to new mtu/mss?
>> 	If you use plain cake and you know the numerical overhead (NN) the easiest is to add the following to your cake invocation: “atm overhead NN”
>> 
>> Please note that if you use cake on an ethernet interface the kernel will already account for 14 byte of ethernet overhead, so if the script told you 44 as actual overhead, you use ”overhead 30” to address that. If you use a pppoe interface the kernel will most likely not add the 14 bytes for you, so then you would use “overhead 44” (I excluded the atm option in the last examples for clarity…)
>> 
>>> Regarding UDP paket dropping problem:
>>> I just read some forums and users stated that under heavy load cake starts to drop udp packets which causes lag ingame.
>>> My idea was to set ingress/egress to diffserv4 and apply the EF dscp mark on those packets.
>> 	Ell, not a bad idea, but often the problem are in the incoming traffic, and unfortunately with the ifb we use we can not use iptables, but only tc, and remarking with tc is unpleasant.
>> 
>>> Will this even work? if yes how to do this? iptables?
>> 	No, you wuld need tp use tc.
>> 
>>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j DSCP --set-dscp-class EF
>>> 
>>> Like thia? Is prerouting correct here? (Taken from layer cake script)
>> 	This will affect outgoing packets and might be a good idea in your specific case.
>> 
>> BUT why don’t you try the default behaviour with specific rules and tricks and report success or failure back to us, after all the fastest/easiest classification is one one does not need to perform at all.
>> 
>>> 
>>> For the squash and wash feature.
>>> Im asking because if i choose to squash in the advanced options of sqm scripts.
>>> The dscp fields/marks will be overwritten by iptables to 0 (besteffort). (layer cake script)
>>> So then it makes no sense to manually set dscp fields/marks or? (Or even setting diffserv)
>> 	No unfortunately on ingress cake sees the packets before iptables, so the effective behavioral emulation of wash/squash by cake is to set ingress cake to besteffort (basically cake ignores the dscp field which functionally is identical to all packets having the same value). The squashing by iptables just clears the dscp marls so that internal networking elements like potentially wifi liknks are not confuzed by the dscp information.
>> 
>>> Did i understand this correctly. Per rfc isps should not provide dscp fields/marks?
>> 	Not exactly, per RFC DSCPs are only ever valid/defined inside a DSCP domain and your ISPs domain ends before it reaches your CPE. Since you have no control over your ISPs markings, they can be very much not like you want them to be (Dave That reported that his ISP re-mapped almost 1/3 or so of packets into the CS1 background class). So it is recomended that each DSCP domain re-mapps the code points at its entry point, which in your case is your router…
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> 
>>> Thank you.
>>> 
>>> 
>>> 
>>> Am 09.06.2016 um 23:30 schrieb moeller0:
>>>> Hi Dennis,
>>>> 
>>>> let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…
>>>> 
>>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>> 
>>>>> Hi
>>>>> 
>>>>> Currently im running lede + cake + sqm_scripts and i have some questions:
>>>>> 
>>>>> 1. What is considered the “optimal" setup atm for cake?
>>>> 	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.
>>>> 
>>>>> e.g. which cake script should i use piece or layer cake?
>>>> 	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..
>>>> 
>>>>> 2. Recently squash and wash was removed.
>>>>> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?
>>>> 	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).
>>>> 
>>>>> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?
>>>> 	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)
>>>> 
>>>>> 4. Is it recommend to enable diffserv on ingress?
>>>> 	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.
>>>> 
>>>>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>>>>> If yes does it make sense to apply diffserv classes manually? How to do this?
>>>> 	I am not sure what you mean, but if you test this and have some findings please report here…
>>>> 
>>>>> 6. is the autorate_ingress still under development?
>>>>> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?
>>>> 	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.
>>>> 
>>>>> 6. What difference does it make to set a different rtt?
>>>>> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
>>>>> What happens if rtt is set way higher?
>>>> 	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)
>>>> 
>>>> 
>>>>> Thank you!
>>>> 	I am sure the real experts will also chime in…
>>>> 
>>>> Best Regards
>>>> 	Sebastian
>>>> 
>>>>> _______________________________________________
>>>>> Cake mailing list
>>>>> Cake@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/cake
>>> 
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
> 
> 

Re: [Cake] New to cake. Some questions

[moeller0]

Hi Dennis,


> On Jun 10, 2016, at 02:49 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
> 
> Hi Sebastian,
> 
> Sorry this is positive or?

	I would say that is unclear…

> But i need more samples ?

	I would try with more samples, after checking that the ping times in the recorded data file actually are larger for larger probes than for smaller, some hosts will reply with a fixed maximum ICMP packet instead of returning the received packet, thereby reducing the signal range (as only the upload leg of the link is meaning fully contributing useful differential signal.
	BTW I am in central europe so at times of the day my responses can be very sporadic, as I either am at work or sleeping ;)

Best Regards
	Sebastian

> 
> Thanks :)
> 
> 
> Am 10.06.2016 um 01:11 schrieb moeller0:
>> Hi Dennis,
>> 
>>> On Jun 10, 2016, at 00:45 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>> 
>>> Hi Sebastian,
>>> 
>>> thank you for your answers :)
>>> 
>>> The ATM overhead detector script is currently running.
>>> I read the wiki about it but im not quite sure how to interpret the plot.
>>> I mean what info should i read from it? maximum packet size?
>> 	The relevant number is reported as “Estimated overhead preceding the IP header” in the top part of the second figure created by the script. But that is only relevant.useful if you see a nice step like plot in figure 2 as well ( the second figure in https://github.com/moeller0/ATM_overhead_detector/wiki as positive and the fourth figure as negative example.
>> 
>>> If yes do i set the overhead in cake? Or do i set iptables to clamp to new mtu/mss?
>> 	If you use plain cake and you know the numerical overhead (NN) the easiest is to add the following to your cake invocation: “atm overhead NN”
>> 
>> Please note that if you use cake on an ethernet interface the kernel will already account for 14 byte of ethernet overhead, so if the script told you 44 as actual overhead, you use ”overhead 30” to address that. If you use a pppoe interface the kernel will most likely not add the 14 bytes for you, so then you would use “overhead 44” (I excluded the atm option in the last examples for clarity…)
>> 
>>> Regarding UDP paket dropping problem:
>>> I just read some forums and users stated that under heavy load cake starts to drop udp packets which causes lag ingame.
>>> My idea was to set ingress/egress to diffserv4 and apply the EF dscp mark on those packets.
>> 	Ell, not a bad idea, but often the problem are in the incoming traffic, and unfortunately with the ifb we use we can not use iptables, but only tc, and remarking with tc is unpleasant.
>> 
>>> Will this even work? if yes how to do this? iptables?
>> 	No, you wuld need tp use tc.
>> 
>>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j DSCP --set-dscp-class EF
>>> 
>>> Like thia? Is prerouting correct here? (Taken from layer cake script)
>> 	This will affect outgoing packets and might be a good idea in your specific case.
>> 
>> BUT why don’t you try the default behaviour with specific rules and tricks and report success or failure back to us, after all the fastest/easiest classification is one one does not need to perform at all.
>> 
>>> 
>>> For the squash and wash feature.
>>> Im asking because if i choose to squash in the advanced options of sqm scripts.
>>> The dscp fields/marks will be overwritten by iptables to 0 (besteffort). (layer cake script)
>>> So then it makes no sense to manually set dscp fields/marks or? (Or even setting diffserv)
>> 	No unfortunately on ingress cake sees the packets before iptables, so the effective behavioral emulation of wash/squash by cake is to set ingress cake to besteffort (basically cake ignores the dscp field which functionally is identical to all packets having the same value). The squashing by iptables just clears the dscp marls so that internal networking elements like potentially wifi liknks are not confuzed by the dscp information.
>> 
>>> Did i understand this correctly. Per rfc isps should not provide dscp fields/marks?
>> 	Not exactly, per RFC DSCPs are only ever valid/defined inside a DSCP domain and your ISPs domain ends before it reaches your CPE. Since you have no control over your ISPs markings, they can be very much not like you want them to be (Dave That reported that his ISP re-mapped almost 1/3 or so of packets into the CS1 background class). So it is recomended that each DSCP domain re-mapps the code points at its entry point, which in your case is your router…
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> 
>>> Thank you.
>>> 
>>> 
>>> 
>>> Am 09.06.2016 um 23:30 schrieb moeller0:
>>>> Hi Dennis,
>>>> 
>>>> let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…
>>>> 
>>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>> 
>>>>> Hi
>>>>> 
>>>>> Currently im running lede + cake + sqm_scripts and i have some questions:
>>>>> 
>>>>> 1. What is considered the “optimal" setup atm for cake?
>>>> 	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.
>>>> 
>>>>> e.g. which cake script should i use piece or layer cake?
>>>> 	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..
>>>> 
>>>>> 2. Recently squash and wash was removed.
>>>>> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?
>>>> 	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).
>>>> 
>>>>> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?
>>>> 	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)
>>>> 
>>>>> 4. Is it recommend to enable diffserv on ingress?
>>>> 	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.
>>>> 
>>>>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>>>>> If yes does it make sense to apply diffserv classes manually? How to do this?
>>>> 	I am not sure what you mean, but if you test this and have some findings please report here…
>>>> 
>>>>> 6. is the autorate_ingress still under development?
>>>>> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?
>>>> 	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.
>>>> 
>>>>> 6. What difference does it make to set a different rtt?
>>>>> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
>>>>> What happens if rtt is set way higher?
>>>> 	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)
>>>> 
>>>> 
>>>>> Thank you!
>>>> 	I am sure the real experts will also chime in…
>>>> 
>>>> Best Regards
>>>> 	Sebastian
>>>> 
>>>>> _______________________________________________
>>>>> Cake mailing list
>>>>> Cake@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/cake
>>> 
>>> _______________________________________________
>>> Cake mailing list
>>> Cake@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/cake
> 
> 

Re: [Cake] New to cake. Some questions

[moeller0]

Hi Jonathan,

I assume you will the other points I proposed for discussion also in dedicated emails, correct?

> On Jun 10, 2016, at 01:27 , Jonathan Morton <chromatix99@gmail.com> wrote:
> 
>> why are the vdsl options suffixed with _ptm, but the atm options are not?
> 
> Because the “vcmux” and “llc” suffixes are sufficient to imply ATM cell framing.

	The operational word here is “imply”, in a user interface unlike poetry explicit beats implicit any day in my opinion. And whether we want or not tc’s command line arguments are cake’s user interface. We are talking about exposing this to people who have done probably next to zero research on the matter. I would propose to you that you poll actual novice users whether they easily recognize that vcmux and llc should imply ATM-based carriers instead of repesating this like a mantra to me. If you follow the wikipedia entry for llc (something an interested novice might actually do) you end up with https://en.wikipedia.org/wiki/Logical_link_control which states a number of different systems that can use LLC, same for SNAP with no mentioning of ATM exclusively, only the wikipedia page for vcmux has strong indicators for ATM. But honestly in user interface design  user’s should not be forced to guess or rely on intuition, at least as far as I can see it.
	But the missing “_ATM" suffix is especially annoying as these compound keywords do not only set a specific overhead value, but also set the atm variable to one. So I repeat something most users will find puzzling:
“pppoe-vcmuc noatm” results in a different cake state than “noatm pppoe-vcmux” that at least requires an explanation in on-line help and man-page. My beef here is that your compounds comingle two different things, the atm cell accounting and the pure overheads without reflecting this in the keyword names. Speaking of inconsistencies, pppoe-vcmux will silently account for ATM’s 48/53 encoding, but pppoe-ptm will NOT account for PTM’s 64/65 encoding… I am not sure what the best solution is for that, but at least I want to point out that this is too subtle for naive users… (heck I believe this is too subtle for you ;) )

> 
>> is the currently selected set of keywords minimal and complete?
> 
> I did some careful research back when I added that feature, including taking some suggestions from you,

	I want reject responsibility for the current inconsistency, but I am thankful that you added the numeric overhead option/keyword.

> and according to that: yes, it is correct and complete, and every keyword is related to a real protocol.  Though some are not widely used in practice, they *are* widely supported in ubiquitous consumer-grade equipment.
> 
> I haven’t seen any evidence to the contrary; if you have any,

	That just shows that your research can be improved upon ;) . At least in my eyes, annoy Sebastian enough to report more real-world encapsulation examples does NOT count as original research…
	But here you go: Deutsche Telekom, DTAG for short, uses PPPoE encapsulation on its FTTH-GPON links, so at least pure pppoe seems to be missing from the set of keywords.
	And that in a nut shell is a problem with only giving compound keywords, they do not easily allow users to assemble their own known encapsulation from atomic base elements, so you should make sure your set covers all bases. Saying there is always “overhead NN” is not a solution, because with that argument you could get rid of the whole bunch (with the possible exemption of “conservative_atm” as that really is a nice gesture to offer novice users, that at least know they have an adsl link…).

> please show it, if not, PLEASE SHUT UP about it.

	To me this does not sound like you are honestly discussion the design of the keywords with me, but that you are deep in the trenches and are not willing to change anything at all. But as I happen to know a thing or two about this specific topic I will not let you end the discussion by essentially “bullying". Eviscerate my arguments dissect my logic, fine, but expect me to shut up just because you repeat your same incomplete/inconsistent arguments over and over again? That will just result in my repeating my questions/challenges over and over again… After all, as it looks I will be babysitting people in e.g. the openwrt forum that actually try to use cake.



> 
> That, by the way, is *me* being blunt to the point of rudeness.

	Fine, the “gloves are off”, no issue from my side, even though I prefer strong arguments to strong words; to each his own, I guess.

> 
>> why name something conservative that will for all peop;e not using an ATM link cost between 9 to 40% of goodput?
> 
> The use-case for the “conservative” keyword is essentially: “I know what the raw bitrate of the link is, but I have no sodding clue what overhead it has”.  The goal is to prevent the dumb buffers elsewhere from filling up and undoing our good work.
> 
> Yes, it will overcompensate, leading to reduced throughput.  That’s recognised and accepted as far as I’m concerned; the worst corner cases are with very small packets, which frankly matter less.

	Now this is a good example why I think you stopped discussing with me long ago and simply automatically defend past decisions, there is a typical use case that creates a meaningful number of small packets, the ACK traffic of large downloads. Depending on the acking strategy we might see one ACK per downloaded packet, at a 16/1 VDSL link with conservative we get 16Mbps: 16000000bps * 0.9(atm 48/53 encapsulation factor) / (1500*8) = 1200 packets per second On the uplink we see the need for 1200packets per second * 3*48*8 (assuming one ACK requires 3 cells say due to TCP timestamps) = 1382400  bps add the framing 1382400/0.9 = 1536000 so we would need more than the 1000000 we have and hence the download will suffer due to ACK clocking. Now you say nobody still uses one ACK per packet, but the fact remains that the upload link will not be used efficiently as there will be lot of dead time when the shaper thinks overhead is transported. Really, tell me why I am wrong and you are right.


>  If you don’t want overcompensation, figure out what the real overhead is and set that.

	It does not work that way, words have meaning you know. If you name a thing conservative and recommend to use it as a default, as you have done in the past, the onus is on you to make sure it does not do (too much) harm. I really wonder why you believe even the even simple act of renaming conservative to conservative_atm or conservative_adsl will “spoil" your beautiful system while adding no useful information to our users, some of which (on openwrt/lede) will only ever see the output of “tc qdisc add cake help” and no man-page (not that there is a man page yet describing all of your rationale and reasoning).
	The recent discussion of the overhead topic leave me believing that the Dunning Kruger paper has some merits, it is up to the peanut gallery to decide which of us is deeper in that trap… ;)

Best Regards
	Sebastian


P.S.: Oh and by the way the comment in q_cake.c:    
                /* Typical VDSL2 framing schemes */
                /* NB: PTM includes HDLC’s 0x7D/7E expansion, adds extra 1/128 */
still is wrong for VDSL2’s data bearers, they do not use HDLC, and even VDSL1 uses byte mode so 1/128 is just an approximation… I believe I have pointed that out some months ago already.


> 
> - Jonathan Morton
> 

Re: [Cake] New to cake. Some questions

[Dennis Fedtke]

Hi Sebastian,

i used the default setting of 1000.
But it seems that my isp is dropping icmp packets if there are exceeding 
some sending threshold.
So there is a lot of none usable ping data. I increased the send delay 
to 50ms. 25 ms already shows dropped requests.
This is the third run now. Waiting for completion.
The ping target is my first hop.

Actually my ping always varies around +-5ms even at idle and 
independently of ping target.
When i look through the ping file the increase in ping times are 
actually appear to be random to me.
So how to test if my isp responses with fixed icmp packet size?

Im in central europe too :D

Thanks :)


Am 10.06.2016 um 07:20 schrieb moeller0:
> Hi Dennis,
>
>
>> On Jun 10, 2016, at 02:49 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>
>> Hi Sebastian,
>>
>> Sorry this is positive or?
> 	I would say that is unclear…
>
>> But i need more samples ?
> 	I would try with more samples, after checking that the ping times in the recorded data file actually are larger for larger probes than for smaller, some hosts will reply with a fixed maximum ICMP packet instead of returning the received packet, thereby reducing the signal range (as only the upload leg of the link is meaning fully contributing useful differential signal.
> 	BTW I am in central europe so at times of the day my responses can be very sporadic, as I either am at work or sleeping ;)
>
> Best Regards
> 	Sebastian
>
>> Thanks :)
>>
>>
>> Am 10.06.2016 um 01:11 schrieb moeller0:
>>> Hi Dennis,
>>>
>>>> On Jun 10, 2016, at 00:45 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>
>>>> Hi Sebastian,
>>>>
>>>> thank you for your answers :)
>>>>
>>>> The ATM overhead detector script is currently running.
>>>> I read the wiki about it but im not quite sure how to interpret the plot.
>>>> I mean what info should i read from it? maximum packet size?
>>> 	The relevant number is reported as “Estimated overhead preceding the IP header” in the top part of the second figure created by the script. But that is only relevant.useful if you see a nice step like plot in figure 2 as well ( the second figure in https://github.com/moeller0/ATM_overhead_detector/wiki as positive and the fourth figure as negative example.
>>>
>>>> If yes do i set the overhead in cake? Or do i set iptables to clamp to new mtu/mss?
>>> 	If you use plain cake and you know the numerical overhead (NN) the easiest is to add the following to your cake invocation: “atm overhead NN”
>>>
>>> Please note that if you use cake on an ethernet interface the kernel will already account for 14 byte of ethernet overhead, so if the script told you 44 as actual overhead, you use ”overhead 30” to address that. If you use a pppoe interface the kernel will most likely not add the 14 bytes for you, so then you would use “overhead 44” (I excluded the atm option in the last examples for clarity…)
>>>
>>>> Regarding UDP paket dropping problem:
>>>> I just read some forums and users stated that under heavy load cake starts to drop udp packets which causes lag ingame.
>>>> My idea was to set ingress/egress to diffserv4 and apply the EF dscp mark on those packets.
>>> 	Ell, not a bad idea, but often the problem are in the incoming traffic, and unfortunately with the ifb we use we can not use iptables, but only tc, and remarking with tc is unpleasant.
>>>
>>>> Will this even work? if yes how to do this? iptables?
>>> 	No, you wuld need tp use tc.
>>>
>>>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j DSCP --set-dscp-class EF
>>>>
>>>> Like thia? Is prerouting correct here? (Taken from layer cake script)
>>> 	This will affect outgoing packets and might be a good idea in your specific case.
>>>
>>> BUT why don’t you try the default behaviour with specific rules and tricks and report success or failure back to us, after all the fastest/easiest classification is one one does not need to perform at all.
>>>
>>>> For the squash and wash feature.
>>>> Im asking because if i choose to squash in the advanced options of sqm scripts.
>>>> The dscp fields/marks will be overwritten by iptables to 0 (besteffort). (layer cake script)
>>>> So then it makes no sense to manually set dscp fields/marks or? (Or even setting diffserv)
>>> 	No unfortunately on ingress cake sees the packets before iptables, so the effective behavioral emulation of wash/squash by cake is to set ingress cake to besteffort (basically cake ignores the dscp field which functionally is identical to all packets having the same value). The squashing by iptables just clears the dscp marls so that internal networking elements like potentially wifi liknks are not confuzed by the dscp information.
>>>
>>>> Did i understand this correctly. Per rfc isps should not provide dscp fields/marks?
>>> 	Not exactly, per RFC DSCPs are only ever valid/defined inside a DSCP domain and your ISPs domain ends before it reaches your CPE. Since you have no control over your ISPs markings, they can be very much not like you want them to be (Dave That reported that his ISP re-mapped almost 1/3 or so of packets into the CS1 background class). So it is recomended that each DSCP domain re-mapps the code points at its entry point, which in your case is your router…
>>>
>>> Best Regards
>>> 	Sebastian
>>>
>>>> Thank you.
>>>>
>>>>
>>>>
>>>> Am 09.06.2016 um 23:30 schrieb moeller0:
>>>>> Hi Dennis,
>>>>>
>>>>> let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…
>>>>>
>>>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Currently im running lede + cake + sqm_scripts and i have some questions:
>>>>>>
>>>>>> 1. What is considered the “optimal" setup atm for cake?
>>>>> 	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.
>>>>>
>>>>>> e.g. which cake script should i use piece or layer cake?
>>>>> 	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..
>>>>>
>>>>>> 2. Recently squash and wash was removed.
>>>>>> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?
>>>>> 	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).
>>>>>
>>>>>> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?
>>>>> 	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)
>>>>>
>>>>>> 4. Is it recommend to enable diffserv on ingress?
>>>>> 	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.
>>>>>
>>>>>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>>>>>> If yes does it make sense to apply diffserv classes manually? How to do this?
>>>>> 	I am not sure what you mean, but if you test this and have some findings please report here…
>>>>>
>>>>>> 6. is the autorate_ingress still under development?
>>>>>> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?
>>>>> 	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.
>>>>>
>>>>>> 6. What difference does it make to set a different rtt?
>>>>>> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
>>>>>> What happens if rtt is set way higher?
>>>>> 	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)
>>>>>
>>>>>
>>>>>> Thank you!
>>>>> 	I am sure the real experts will also chime in…
>>>>>
>>>>> Best Regards
>>>>> 	Sebastian
>>>>>
>>>>>> _______________________________________________
>>>>>> Cake mailing list
>>>>>> Cake@lists.bufferbloat.net
>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>> _______________________________________________
>>>> Cake mailing list
>>>> Cake@lists.bufferbloat.net
>>>> https://lists.bufferbloat.net/listinfo/cake
>>

Re: [Cake] New to cake. Some questions

[moeller0]

Hi Dennis,

> On Jun 10, 2016, at 14:43 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
> 
> Hi Sebastian,
> 
> i used the default setting of 1000.

	Okay, that should work i assume unless you have a very fast link… What link at what ISP do you actually have?

> But it seems that my isp is dropping icmp packets if there are exceeding some sending threshold.

	I would be amazed if they did, a sympotom of that would be rsate reduction to all ICMP probe flows independent of target host. If however you only see this with specific hosts it is very likely that that host rate limits its ICMP responses. In either case try another host further upstream. II think I has reasonable decent results with targeting 8.8.8.8, googles dns servers.

> So there is a lot of none usable ping data.

	Again, try another host…

> I increased the send delay to 50ms. 25 ms already shows dropped requests.

	That might also help, as long as you stay below their throttling rate the chosen host might still work okay.

> This is the third run now. Waiting for completion.

	Well, sorry that the method is not as slick and streamlined, but there are no guarded good ICMP reflectors available on the net.

> The ping target is my first hop.

	Try the next hop then ;)

> 
> Actually my ping always varies around +-5ms even at idle and independently of ping target.

	This is via wifi/wlan? If so try from a wired connection instead.

> When i look through the ping file the increase in ping times are actually appear to be random to me.

	Well, we expect variability of the individual “trials” to exist, that is why we collect so many and try to select the best measure in the matlab code to remove the unwanted variance. Could you post a link to both of the generated plots please, the first one showing te different aggregation measures might be helpful in diagnosing the issues deeper.

> So how to test if my isp responses with fixed icmp packet size?

	You could try manually. In the folloewing example I pinged gstatic.com (which belongs to googles CDN as far as I know):

bash-3.2$ ping -s 1 -c 1 gstatic.com
PING gstatic.com (216.58.213.195): 1 data bytes
9 bytes from 216.58.213.195: icmp_seq=0 ttl=55

--- gstatic.com ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss


bash-3.2$ ping -s 64 -c 1 gstatic.com
PING gstatic.com (216.58.213.195): 64 data bytes
72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=19.446 ms

--- gstatic.com ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 19.446/19.446/19.446/0.000 ms


bash-3.2$ ping -s 65 -c 1 gstatic.com
PING gstatic.com (216.58.213.195): 65 data bytes
72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=21.138 ms
wrong total length 92 instead of 93

--- gstatic.com ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 21.138/21.138/21.138/0.000 ms
bash-3.2$ 


bash-3.2$ ping -s 1400 -c 1 gstatic.com
PING gstatic.com (216.58.213.195): 1400 data bytes
72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=6.878 ms
wrong total length 92 instead of 1428

--- gstatic.com ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 6.878/6.878/6.878/0.000 ms

Once I try to send 65 Bytes of ICMP payload the response is cut short to 92 bytes, the same might happen with your isp. But also if all your ISP does is rate limiting the ICMP packests that still can lead to to much variance in the RTTs…


> 
> Im in central europe too :D

	Ah, then you just have a different work/sleep cycle than I do ;). Where in central Europe, if I might as Ii am, as you might have guessed based in Germany…

Best Regards
	Sebastian

> 
> Thanks :)
> 
> 
> Am 10.06.2016 um 07:20 schrieb moeller0:
>> Hi Dennis,
>> 
>> 
>>> On Jun 10, 2016, at 02:49 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>> 
>>> Hi Sebastian,
>>> 
>>> Sorry this is positive or?
>> 	I would say that is unclear…
>> 
>>> But i need more samples ?
>> 	I would try with more samples, after checking that the ping times in the recorded data file actually are larger for larger probes than for smaller, some hosts will reply with a fixed maximum ICMP packet instead of returning the received packet, thereby reducing the signal range (as only the upload leg of the link is meaning fully contributing useful differential signal.
>> 	BTW I am in central europe so at times of the day my responses can be very sporadic, as I either am at work or sleeping ;)
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> Thanks :)
>>> 
>>> 
>>> Am 10.06.2016 um 01:11 schrieb moeller0:
>>>> Hi Dennis,
>>>> 
>>>>> On Jun 10, 2016, at 00:45 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>> 
>>>>> Hi Sebastian,
>>>>> 
>>>>> thank you for your answers :)
>>>>> 
>>>>> The ATM overhead detector script is currently running.
>>>>> I read the wiki about it but im not quite sure how to interpret the plot.
>>>>> I mean what info should i read from it? maximum packet size?
>>>> 	The relevant number is reported as “Estimated overhead preceding the IP header” in the top part of the second figure created by the script. But that is only relevant.useful if you see a nice step like plot in figure 2 as well ( the second figure in https://github.com/moeller0/ATM_overhead_detector/wiki as positive and the fourth figure as negative example.
>>>> 
>>>>> If yes do i set the overhead in cake? Or do i set iptables to clamp to new mtu/mss?
>>>> 	If you use plain cake and you know the numerical overhead (NN) the easiest is to add the following to your cake invocation: “atm overhead NN”
>>>> 
>>>> Please note that if you use cake on an ethernet interface the kernel will already account for 14 byte of ethernet overhead, so if the script told you 44 as actual overhead, you use ”overhead 30” to address that. If you use a pppoe interface the kernel will most likely not add the 14 bytes for you, so then you would use “overhead 44” (I excluded the atm option in the last examples for clarity…)
>>>> 
>>>>> Regarding UDP paket dropping problem:
>>>>> I just read some forums and users stated that under heavy load cake starts to drop udp packets which causes lag ingame.
>>>>> My idea was to set ingress/egress to diffserv4 and apply the EF dscp mark on those packets.
>>>> 	Ell, not a bad idea, but often the problem are in the incoming traffic, and unfortunately with the ifb we use we can not use iptables, but only tc, and remarking with tc is unpleasant.
>>>> 
>>>>> Will this even work? if yes how to do this? iptables?
>>>> 	No, you wuld need tp use tc.
>>>> 
>>>>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j DSCP --set-dscp-class EF
>>>>> 
>>>>> Like thia? Is prerouting correct here? (Taken from layer cake script)
>>>> 	This will affect outgoing packets and might be a good idea in your specific case.
>>>> 
>>>> BUT why don’t you try the default behaviour with specific rules and tricks and report success or failure back to us, after all the fastest/easiest classification is one one does not need to perform at all.
>>>> 
>>>>> For the squash and wash feature.
>>>>> Im asking because if i choose to squash in the advanced options of sqm scripts.
>>>>> The dscp fields/marks will be overwritten by iptables to 0 (besteffort). (layer cake script)
>>>>> So then it makes no sense to manually set dscp fields/marks or? (Or even setting diffserv)
>>>> 	No unfortunately on ingress cake sees the packets before iptables, so the effective behavioral emulation of wash/squash by cake is to set ingress cake to besteffort (basically cake ignores the dscp field which functionally is identical to all packets having the same value). The squashing by iptables just clears the dscp marls so that internal networking elements like potentially wifi liknks are not confuzed by the dscp information.
>>>> 
>>>>> Did i understand this correctly. Per rfc isps should not provide dscp fields/marks?
>>>> 	Not exactly, per RFC DSCPs are only ever valid/defined inside a DSCP domain and your ISPs domain ends before it reaches your CPE. Since you have no control over your ISPs markings, they can be very much not like you want them to be (Dave That reported that his ISP re-mapped almost 1/3 or so of packets into the CS1 background class). So it is recomended that each DSCP domain re-mapps the code points at its entry point, which in your case is your router…
>>>> 
>>>> Best Regards
>>>> 	Sebastian
>>>> 
>>>>> Thank you.
>>>>> 
>>>>> 
>>>>> 
>>>>> Am 09.06.2016 um 23:30 schrieb moeller0:
>>>>>> Hi Dennis,
>>>>>> 
>>>>>> let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…
>>>>>> 
>>>>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>>>> 
>>>>>>> Hi
>>>>>>> 
>>>>>>> Currently im running lede + cake + sqm_scripts and i have some questions:
>>>>>>> 
>>>>>>> 1. What is considered the “optimal" setup atm for cake?
>>>>>> 	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.
>>>>>> 
>>>>>>> e.g. which cake script should i use piece or layer cake?
>>>>>> 	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..
>>>>>> 
>>>>>>> 2. Recently squash and wash was removed.
>>>>>>> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?
>>>>>> 	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).
>>>>>> 
>>>>>>> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?
>>>>>> 	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)
>>>>>> 
>>>>>>> 4. Is it recommend to enable diffserv on ingress?
>>>>>> 	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.
>>>>>> 
>>>>>>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>>>>>>> If yes does it make sense to apply diffserv classes manually? How to do this?
>>>>>> 	I am not sure what you mean, but if you test this and have some findings please report here…
>>>>>> 
>>>>>>> 6. is the autorate_ingress still under development?
>>>>>>> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?
>>>>>> 	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.
>>>>>> 
>>>>>>> 6. What difference does it make to set a different rtt?
>>>>>>> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
>>>>>>> What happens if rtt is set way higher?
>>>>>> 	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)
>>>>>> 
>>>>>> 
>>>>>>> Thank you!
>>>>>> 	I am sure the real experts will also chime in…
>>>>>> 
>>>>>> Best Regards
>>>>>> 	Sebastian
>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> Cake mailing list
>>>>>>> Cake@lists.bufferbloat.net
>>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>> _______________________________________________
>>>>> Cake mailing list
>>>>> Cake@lists.bufferbloat.net
>>>>> https://lists.bufferbloat.net/listinfo/cake
>>> 
> 
> 

Re: [Cake] New to cake. Some questions

[Dennis Fedtke]

Hi Sebastian,

yes this is wired connection. As i stated my ping times always vary 
independently of target.
My ISP is overloaded in certain regions. So i assume they do some 
shaping/limiting on certain protocols (icmp for example)
Connection speed is 200/20 Mbit.
ISP is unitymedia which doesn't allow you to use your own hardware.
So actually i have to run my router behind theirs with exposed host 
enabled :<

Ping response:

ping -s 1400 -c 1 109.90.x.x
PING 109.90.28.1 (109.90.28.1) 1400(1428) bytes of data.
1408 bytes from 109.90.28.1: icmp_seq=1 ttl=253 time=11.6 ms

--- 109.90.28.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 11.677/11.677/11.677/0.000 ms

This looks good or?

Yes i am from germany. So you are from germany too?

Thanks for your time and help :)


Best regards
Dennis

Am 10.06.2016 um 15:02 schrieb moeller0:
> Hi Dennis,
>
>> On Jun 10, 2016, at 14:43 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>
>> Hi Sebastian,
>>
>> i used the default setting of 1000.
> 	Okay, that should work i assume unless you have a very fast link… What link at what ISP do you actually have?
>
>> But it seems that my isp is dropping icmp packets if there are exceeding some sending threshold.
> 	I would be amazed if they did, a sympotom of that would be rsate reduction to all ICMP probe flows independent of target host. If however you only see this with specific hosts it is very likely that that host rate limits its ICMP responses. In either case try another host further upstream. II think I has reasonable decent results with targeting 8.8.8.8, googles dns servers.
>
>> So there is a lot of none usable ping data.
> 	Again, try another host…
>
>> I increased the send delay to 50ms. 25 ms already shows dropped requests.
> 	That might also help, as long as you stay below their throttling rate the chosen host might still work okay.
>
>> This is the third run now. Waiting for completion.
> 	Well, sorry that the method is not as slick and streamlined, but there are no guarded good ICMP reflectors available on the net.
>
>> The ping target is my first hop.
> 	Try the next hop then ;)
>
>> Actually my ping always varies around +-5ms even at idle and independently of ping target.
> 	This is via wifi/wlan? If so try from a wired connection instead.
>
>> When i look through the ping file the increase in ping times are actually appear to be random to me.
> 	Well, we expect variability of the individual “trials” to exist, that is why we collect so many and try to select the best measure in the matlab code to remove the unwanted variance. Could you post a link to both of the generated plots please, the first one showing te different aggregation measures might be helpful in diagnosing the issues deeper.
>
>> So how to test if my isp responses with fixed icmp packet size?
> 	You could try manually. In the folloewing example I pinged gstatic.com (which belongs to googles CDN as far as I know):
>
> bash-3.2$ ping -s 1 -c 1 gstatic.com
> PING gstatic.com (216.58.213.195): 1 data bytes
> 9 bytes from 216.58.213.195: icmp_seq=0 ttl=55
>
> --- gstatic.com ping statistics ---
> 1 packets transmitted, 1 packets received, 0.0% packet loss
>
>
> bash-3.2$ ping -s 64 -c 1 gstatic.com
> PING gstatic.com (216.58.213.195): 64 data bytes
> 72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=19.446 ms
>
> --- gstatic.com ping statistics ---
> 1 packets transmitted, 1 packets received, 0.0% packet loss
> round-trip min/avg/max/stddev = 19.446/19.446/19.446/0.000 ms
>
>
> bash-3.2$ ping -s 65 -c 1 gstatic.com
> PING gstatic.com (216.58.213.195): 65 data bytes
> 72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=21.138 ms
> wrong total length 92 instead of 93
>
> --- gstatic.com ping statistics ---
> 1 packets transmitted, 1 packets received, 0.0% packet loss
> round-trip min/avg/max/stddev = 21.138/21.138/21.138/0.000 ms
> bash-3.2$
>
>
> bash-3.2$ ping -s 1400 -c 1 gstatic.com
> PING gstatic.com (216.58.213.195): 1400 data bytes
> 72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=6.878 ms
> wrong total length 92 instead of 1428
>
> --- gstatic.com ping statistics ---
> 1 packets transmitted, 1 packets received, 0.0% packet loss
> round-trip min/avg/max/stddev = 6.878/6.878/6.878/0.000 ms
>
> Once I try to send 65 Bytes of ICMP payload the response is cut short to 92 bytes, the same might happen with your isp. But also if all your ISP does is rate limiting the ICMP packests that still can lead to to much variance in the RTTs…
>
>
>> Im in central europe too :D
> 	Ah, then you just have a different work/sleep cycle than I do ;). Where in central Europe, if I might as Ii am, as you might have guessed based in Germany…
>
> Best Regards
> 	Sebastian
>
>> Thanks :)
>>
>>
>> Am 10.06.2016 um 07:20 schrieb moeller0:
>>> Hi Dennis,
>>>
>>>
>>>> On Jun 10, 2016, at 02:49 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>
>>>> Hi Sebastian,
>>>>
>>>> Sorry this is positive or?
>>> 	I would say that is unclear…
>>>
>>>> But i need more samples ?
>>> 	I would try with more samples, after checking that the ping times in the recorded data file actually are larger for larger probes than for smaller, some hosts will reply with a fixed maximum ICMP packet instead of returning the received packet, thereby reducing the signal range (as only the upload leg of the link is meaning fully contributing useful differential signal.
>>> 	BTW I am in central europe so at times of the day my responses can be very sporadic, as I either am at work or sleeping ;)
>>>
>>> Best Regards
>>> 	Sebastian
>>>
>>>> Thanks :)
>>>>
>>>>
>>>> Am 10.06.2016 um 01:11 schrieb moeller0:
>>>>> Hi Dennis,
>>>>>
>>>>>> On Jun 10, 2016, at 00:45 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>>>
>>>>>> Hi Sebastian,
>>>>>>
>>>>>> thank you for your answers :)
>>>>>>
>>>>>> The ATM overhead detector script is currently running.
>>>>>> I read the wiki about it but im not quite sure how to interpret the plot.
>>>>>> I mean what info should i read from it? maximum packet size?
>>>>> 	The relevant number is reported as “Estimated overhead preceding the IP header” in the top part of the second figure created by the script. But that is only relevant.useful if you see a nice step like plot in figure 2 as well ( the second figure in https://github.com/moeller0/ATM_overhead_detector/wiki as positive and the fourth figure as negative example.
>>>>>
>>>>>> If yes do i set the overhead in cake? Or do i set iptables to clamp to new mtu/mss?
>>>>> 	If you use plain cake and you know the numerical overhead (NN) the easiest is to add the following to your cake invocation: “atm overhead NN”
>>>>>
>>>>> Please note that if you use cake on an ethernet interface the kernel will already account for 14 byte of ethernet overhead, so if the script told you 44 as actual overhead, you use ”overhead 30” to address that. If you use a pppoe interface the kernel will most likely not add the 14 bytes for you, so then you would use “overhead 44” (I excluded the atm option in the last examples for clarity…)
>>>>>
>>>>>> Regarding UDP paket dropping problem:
>>>>>> I just read some forums and users stated that under heavy load cake starts to drop udp packets which causes lag ingame.
>>>>>> My idea was to set ingress/egress to diffserv4 and apply the EF dscp mark on those packets.
>>>>> 	Ell, not a bad idea, but often the problem are in the incoming traffic, and unfortunately with the ifb we use we can not use iptables, but only tc, and remarking with tc is unpleasant.
>>>>>
>>>>>> Will this even work? if yes how to do this? iptables?
>>>>> 	No, you wuld need tp use tc.
>>>>>
>>>>>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j DSCP --set-dscp-class EF
>>>>>>
>>>>>> Like thia? Is prerouting correct here? (Taken from layer cake script)
>>>>> 	This will affect outgoing packets and might be a good idea in your specific case.
>>>>>
>>>>> BUT why don’t you try the default behaviour with specific rules and tricks and report success or failure back to us, after all the fastest/easiest classification is one one does not need to perform at all.
>>>>>
>>>>>> For the squash and wash feature.
>>>>>> Im asking because if i choose to squash in the advanced options of sqm scripts.
>>>>>> The dscp fields/marks will be overwritten by iptables to 0 (besteffort). (layer cake script)
>>>>>> So then it makes no sense to manually set dscp fields/marks or? (Or even setting diffserv)
>>>>> 	No unfortunately on ingress cake sees the packets before iptables, so the effective behavioral emulation of wash/squash by cake is to set ingress cake to besteffort (basically cake ignores the dscp field which functionally is identical to all packets having the same value). The squashing by iptables just clears the dscp marls so that internal networking elements like potentially wifi liknks are not confuzed by the dscp information.
>>>>>
>>>>>> Did i understand this correctly. Per rfc isps should not provide dscp fields/marks?
>>>>> 	Not exactly, per RFC DSCPs are only ever valid/defined inside a DSCP domain and your ISPs domain ends before it reaches your CPE. Since you have no control over your ISPs markings, they can be very much not like you want them to be (Dave That reported that his ISP re-mapped almost 1/3 or so of packets into the CS1 background class). So it is recomended that each DSCP domain re-mapps the code points at its entry point, which in your case is your router…
>>>>>
>>>>> Best Regards
>>>>> 	Sebastian
>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Am 09.06.2016 um 23:30 schrieb moeller0:
>>>>>>> Hi Dennis,
>>>>>>>
>>>>>>> let me start with a disclaimer, I am not the best information source for cake on this mailing list, but I assume the others will chime in if I say something questionable…
>>>>>>>
>>>>>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Currently im running lede + cake + sqm_scripts and i have some questions:
>>>>>>>>
>>>>>>>> 1. What is considered the “optimal" setup atm for cake?
>>>>>>> 	The same as without cake; really, proper per-packet-overhead accounting is important for bandwidth shaping, especially for ATM -based links. I would recommend to follow the method on https://github.com/moeller0/ATM_overhead_detector to m\empirically measure whether your link uses ATM encapsulation and what exact overhead is in use.
>>>>>>>
>>>>>>>> e.g. which cake script should i use piece or layer cake?
>>>>>>> 	piece_of_cake has only one tier of priority, while layer_cake currently offers 4. Packets are put into the different priority bands based on the content of their TOS/DSCP filed in the IP header; if this is greek to you, I guess piece_of_cake most likely is what you are looking for..
>>>>>>>
>>>>>>>> 2. Recently squash and wash was removed.
>>>>>>>> But the sqm scripts were not updated. In the advanced options should i set that the dcsp marks are kept?
>>>>>>> 	This really is an implementation detail that has no immediate effect if you choose piece_of_cake as typically only the bottleneck is sensitive to DSCP based priority banding. (Typically in that if you are unlucky your WLAN will use the DSCP marks to move packets into 4 different priority classes, which is fine if you want that, but bad for not sanity checked packets coming in from the wider internet (one is not supposed to assume incoming packets have sensible dscp markings as per RFC) that is why the wash/squash option is missed by some of us, independent of the fact that it was a layering violation).
>>>>>>>
>>>>>>>> 3. Should i use advanced options in sqm scripts and set triple-isolate + diffserv8 ?
>>>>>>> 	If you understand what these options do and believe that this is the best for your network go ahead, otherwise… The triple-isolate option will try to be fair to host_IP addresses first and then for each hostIP fair to each flow, but for that to do something you will most likely want this requires that cake sees internal IP addresses of your end-hosts. In the typical configuration with SQM on the WAN interface of a NAT router all internal addresses are replaced with the external IP address of the router it self and triple-isolates per host fairness will pretty much be equal to per flow fairness (not exactly, but in essence). So if you want to try tiple-isolate or its better defined brothers dual-srchost and dual-dsthost you would need to instantiate SQM on an internal interface like LAN. But then the direction of ingress and egress from the routers perspective changes with regards to the internet download and upload direction and you will need to put the internet upload bandwidth into the download field of the sqm GUI and vice versa. Also SQM on an internal interface will also shape internal traffic over the same interface, and that often affects traffic to and from the wifi/wlan radios to the lan switch… (I guess you would have preferred a shorter less vague response, but such are the constraints…)
>>>>>>>
>>>>>>>> 4. Is it recommend to enable diffserv on ingress?
>>>>>>> 	If you trust/konw/have confirmed that your upstream (ISP?) sends you sensible and reasonable DSCP markings by all means enable diffserv on ingress. But the default assumption should be that your upstream used a dscp mapping that only makes sense for them and not for you.
>>>>>>>
>>>>>>>> 5. Is there still the udp packet dropping problem? e.g. games that are using udp.
>>>>>>>> If yes does it make sense to apply diffserv classes manually? How to do this?
>>>>>>> 	I am not sure what you mean, but if you test this and have some findings please report here…
>>>>>>>
>>>>>>>> 6. is the autorate_ingress still under development?
>>>>>>>> This very interesting feature. especially for docsis networks. Will it be possible to set target ping time?
>>>>>>> 	The last tests did indicate that this feature is not ready for primetime at least not on typically fixed bandwidth links and I assume docsis links are fixed enough.
>>>>>>>
>>>>>>>> 6. What difference does it make to set a different rtt?
>>>>>>>> Setting lower rtt will reduce download speed i guess but will it allow better ping times (because of lower downloadrate uh)?
>>>>>>>> What happens if rtt is set way higher?
>>>>>>> 	With the RTT parameter you in essence specify how much time you give the endpoints of a flow to respond to a congestion signal (ECN marking or packet drop) if you select this way to small you will sacrifice bandwidth, if you set this too high you will accumulate more latency under load. The good thing seems to be that this does not need to be terribly precise, order of magnitude correctness seems to be sufficient (at least in base2)
>>>>>>>
>>>>>>>
>>>>>>>> Thank you!
>>>>>>> 	I am sure the real experts will also chime in…
>>>>>>>
>>>>>>> Best Regards
>>>>>>> 	Sebastian
>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Cake mailing list
>>>>>>>> Cake@lists.bufferbloat.net
>>>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>>> _______________________________________________
>>>>>> Cake mailing list
>>>>>> Cake@lists.bufferbloat.net
>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>

Re: [Cake] New to cake. Some questions

[Sebastian Moeller]

Hi Dennis,?

On June 10, 2016 4:05:11 PM GMT+02:00, Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>Hi Sebastian,
>
>yes this is wired connection. As i stated my ping times always vary 
>independently of target.
>My ISP is overloaded in certain regions. So i assume they do some 
>shaping/limiting on certain protocols (icmp for example)
>Connection speed is 200/20 Mbit.

Okay that is a Docsis cable Link, so no atm encapsulation at all. There a several lines of reasoning to one to this conclusion, but mainly ATM links top out at 22Mbps, and in Germany typically at 16-17Mbps, and your ISP is a pure cable. Company. So you can stop running the overhead detector as that only works on ATM links, sorry. I have not yet found a way to measure the overhead without ATM cells.

>ISP is unitymedia which doesn't allow you to use your own hardware.

The law changed an soon, August I believe they will have to give you the access information, but you will still need a cable modem or Docsis router...


>So actually i have to run my router behind theirs with exposed host 
>enabled :<
>
>Ping response:
>
>ping -s 1400 -c 1 109.90.x.x
>PING 109.90.28.1 (109.90.28.1) 1400(1428) bytes of data.
>1408 bytes from 109.90.28.1: icmp_seq=1 ttl=253 time=11.6 ms
>
>--- 109.90.28.1 ping statistics ---
>1 packets transmitted, 1 received, 0% packet loss, time 0ms
>rtt min/avg/max/mdev = 11.677/11.677/11.677/0.000 ms
>
>This looks good or?

      Yes the ping is fine, I assume that your node is quite overbooked and you the 4ms variance from the Docsis grant request system or so. But I am no Docsis expert, so that could be wrong...


>
>Yes i am from germany. So you are from germany too?

       Yes.

>
>Thanks for your time and help :)

        Happy to be able to help....
>
>
>Best regards
>Dennis

Freundlichem Gruessen
        Sebastian
>
>Am 10.06.2016 um 15:02 schrieb moeller0:
>> Hi Dennis,
>>
>>> On Jun 10, 2016, at 14:43 , Dennis Fedtke <dennisfedtke@gmail.com>
>wrote:
>>>
>>> Hi Sebastian,
>>>
>>> i used the default setting of 1000.
>> 	Okay, that should work i assume unless you have a very fast link…
>What link at what ISP do you actually have?
>>
>>> But it seems that my isp is dropping icmp packets if there are
>exceeding some sending threshold.
>> 	I would be amazed if they did, a sympotom of that would be rsate
>reduction to all ICMP probe flows independent of target host. If
>however you only see this with specific hosts it is very likely that
>that host rate limits its ICMP responses. In either case try another
>host further upstream. II think I has reasonable decent results with
>targeting 8.8.8.8, googles dns servers.
>>
>>> So there is a lot of none usable ping data.
>> 	Again, try another host…
>>
>>> I increased the send delay to 50ms. 25 ms already shows dropped
>requests.
>> 	That might also help, as long as you stay below their throttling
>rate the chosen host might still work okay.
>>
>>> This is the third run now. Waiting for completion.
>> 	Well, sorry that the method is not as slick and streamlined, but
>there are no guarded good ICMP reflectors available on the net.
>>
>>> The ping target is my first hop.
>> 	Try the next hop then ;)
>>
>>> Actually my ping always varies around +-5ms even at idle and
>independently of ping target.
>> 	This is via wifi/wlan? If so try from a wired connection instead.
>>
>>> When i look through the ping file the increase in ping times are
>actually appear to be random to me.
>> 	Well, we expect variability of the individual “trials” to exist,
>that is why we collect so many and try to select the best measure in
>the matlab code to remove the unwanted variance. Could you post a link
>to both of the generated plots please, the first one showing te
>different aggregation measures might be helpful in diagnosing the
>issues deeper.
>>
>>> So how to test if my isp responses with fixed icmp packet size?
>> 	You could try manually. In the folloewing example I pinged
>gstatic.com (which belongs to googles CDN as far as I know):
>>
>> bash-3.2$ ping -s 1 -c 1 gstatic.com
>> PING gstatic.com (216.58.213.195): 1 data bytes
>> 9 bytes from 216.58.213.195: icmp_seq=0 ttl=55
>>
>> --- gstatic.com ping statistics ---
>> 1 packets transmitted, 1 packets received, 0.0% packet loss
>>
>>
>> bash-3.2$ ping -s 64 -c 1 gstatic.com
>> PING gstatic.com (216.58.213.195): 64 data bytes
>> 72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=19.446 ms
>>
>> --- gstatic.com ping statistics ---
>> 1 packets transmitted, 1 packets received, 0.0% packet loss
>> round-trip min/avg/max/stddev = 19.446/19.446/19.446/0.000 ms
>>
>>
>> bash-3.2$ ping -s 65 -c 1 gstatic.com
>> PING gstatic.com (216.58.213.195): 65 data bytes
>> 72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=21.138 ms
>> wrong total length 92 instead of 93
>>
>> --- gstatic.com ping statistics ---
>> 1 packets transmitted, 1 packets received, 0.0% packet loss
>> round-trip min/avg/max/stddev = 21.138/21.138/21.138/0.000 ms
>> bash-3.2$
>>
>>
>> bash-3.2$ ping -s 1400 -c 1 gstatic.com
>> PING gstatic.com (216.58.213.195): 1400 data bytes
>> 72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=6.878 ms
>> wrong total length 92 instead of 1428
>>
>> --- gstatic.com ping statistics ---
>> 1 packets transmitted, 1 packets received, 0.0% packet loss
>> round-trip min/avg/max/stddev = 6.878/6.878/6.878/0.000 ms
>>
>> Once I try to send 65 Bytes of ICMP payload the response is cut short
>to 92 bytes, the same might happen with your isp. But also if all your
>ISP does is rate limiting the ICMP packests that still can lead to to
>much variance in the RTTs…
>>
>>
>>> Im in central europe too :D
>> 	Ah, then you just have a different work/sleep cycle than I do ;).
>Where in central Europe, if I might as Ii am, as you might have guessed
>based in Germany…
>>
>> Best Regards
>> 	Sebastian
>>
>>> Thanks :)
>>>
>>>
>>> Am 10.06.2016 um 07:20 schrieb moeller0:
>>>> Hi Dennis,
>>>>
>>>>
>>>>> On Jun 10, 2016, at 02:49 , Dennis Fedtke <dennisfedtke@gmail.com>
>wrote:
>>>>>
>>>>> Hi Sebastian,
>>>>>
>>>>> Sorry this is positive or?
>>>> 	I would say that is unclear…
>>>>
>>>>> But i need more samples ?
>>>> 	I would try with more samples, after checking that the ping times
>in the recorded data file actually are larger for larger probes than
>for smaller, some hosts will reply with a fixed maximum ICMP packet
>instead of returning the received packet, thereby reducing the signal
>range (as only the upload leg of the link is meaning fully contributing
>useful differential signal.
>>>> 	BTW I am in central europe so at times of the day my responses can
>be very sporadic, as I either am at work or sleeping ;)
>>>>
>>>> Best Regards
>>>> 	Sebastian
>>>>
>>>>> Thanks :)
>>>>>
>>>>>
>>>>> Am 10.06.2016 um 01:11 schrieb moeller0:
>>>>>> Hi Dennis,
>>>>>>
>>>>>>> On Jun 10, 2016, at 00:45 , Dennis Fedtke
><dennisfedtke@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi Sebastian,
>>>>>>>
>>>>>>> thank you for your answers :)
>>>>>>>
>>>>>>> The ATM overhead detector script is currently running.
>>>>>>> I read the wiki about it but im not quite sure how to interpret
>the plot.
>>>>>>> I mean what info should i read from it? maximum packet size?
>>>>>> 	The relevant number is reported as “Estimated overhead preceding
>the IP header” in the top part of the second figure created by the
>script. But that is only relevant.useful if you see a nice step like
>plot in figure 2 as well ( the second figure in
>https://github.com/moeller0/ATM_overhead_detector/wiki as positive and
>the fourth figure as negative example.
>>>>>>
>>>>>>> If yes do i set the overhead in cake? Or do i set iptables to
>clamp to new mtu/mss?
>>>>>> 	If you use plain cake and you know the numerical overhead (NN)
>the easiest is to add the following to your cake invocation: “atm
>overhead NN”
>>>>>>
>>>>>> Please note that if you use cake on an ethernet interface the
>kernel will already account for 14 byte of ethernet overhead, so if the
>script told you 44 as actual overhead, you use ”overhead 30” to address
>that. If you use a pppoe interface the kernel will most likely not add
>the 14 bytes for you, so then you would use “overhead 44” (I excluded
>the atm option in the last examples for clarity…)
>>>>>>
>>>>>>> Regarding UDP paket dropping problem:
>>>>>>> I just read some forums and users stated that under heavy load
>cake starts to drop udp packets which causes lag ingame.
>>>>>>> My idea was to set ingress/egress to diffserv4 and apply the EF
>dscp mark on those packets.
>>>>>> 	Ell, not a bad idea, but often the problem are in the incoming
>traffic, and unfortunately with the ifb we use we can not use iptables,
>but only tc, and remarking with tc is unpleasant.
>>>>>>
>>>>>>> Will this even work? if yes how to do this? iptables?
>>>>>> 	No, you wuld need tp use tc.
>>>>>>
>>>>>>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports
>5000:5500 -j DSCP --set-dscp-class EF
>>>>>>>
>>>>>>> Like thia? Is prerouting correct here? (Taken from layer cake
>script)
>>>>>> 	This will affect outgoing packets and might be a good idea in
>your specific case.
>>>>>>
>>>>>> BUT why don’t you try the default behaviour with specific rules
>and tricks and report success or failure back to us, after all the
>fastest/easiest classification is one one does not need to perform at
>all.
>>>>>>
>>>>>>> For the squash and wash feature.
>>>>>>> Im asking because if i choose to squash in the advanced options
>of sqm scripts.
>>>>>>> The dscp fields/marks will be overwritten by iptables to 0
>(besteffort). (layer cake script)
>>>>>>> So then it makes no sense to manually set dscp fields/marks or?
>(Or even setting diffserv)
>>>>>> 	No unfortunately on ingress cake sees the packets before
>iptables, so the effective behavioral emulation of wash/squash by cake
>is to set ingress cake to besteffort (basically cake ignores the dscp
>field which functionally is identical to all packets having the same
>value). The squashing by iptables just clears the dscp marls so that
>internal networking elements like potentially wifi liknks are not
>confuzed by the dscp information.
>>>>>>
>>>>>>> Did i understand this correctly. Per rfc isps should not provide
>dscp fields/marks?
>>>>>> 	Not exactly, per RFC DSCPs are only ever valid/defined inside a
>DSCP domain and your ISPs domain ends before it reaches your CPE. Since
>you have no control over your ISPs markings, they can be very much not
>like you want them to be (Dave That reported that his ISP re-mapped
>almost 1/3 or so of packets into the CS1 background class). So it is
>recomended that each DSCP domain re-mapps the code points at its entry
>point, which in your case is your router…
>>>>>>
>>>>>> Best Regards
>>>>>> 	Sebastian
>>>>>>
>>>>>>> Thank you.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Am 09.06.2016 um 23:30 schrieb moeller0:
>>>>>>>> Hi Dennis,
>>>>>>>>
>>>>>>>> let me start with a disclaimer, I am not the best information
>source for cake on this mailing list, but I assume the others will
>chime in if I say something questionable…
>>>>>>>>
>>>>>>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke
><dennisfedtke@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Currently im running lede + cake + sqm_scripts and i have some
>questions:
>>>>>>>>>
>>>>>>>>> 1. What is considered the “optimal" setup atm for cake?
>>>>>>>> 	The same as without cake; really, proper per-packet-overhead
>accounting is important for bandwidth shaping, especially for ATM
>-based links. I would recommend to follow the method on
>https://github.com/moeller0/ATM_overhead_detector to m\empirically
>measure whether your link uses ATM encapsulation and what exact
>overhead is in use.
>>>>>>>>
>>>>>>>>> e.g. which cake script should i use piece or layer cake?
>>>>>>>> 	piece_of_cake has only one tier of priority, while layer_cake
>currently offers 4. Packets are put into the different priority bands
>based on the content of their TOS/DSCP filed in the IP header; if this
>is greek to you, I guess piece_of_cake most likely is what you are
>looking for..
>>>>>>>>
>>>>>>>>> 2. Recently squash and wash was removed.
>>>>>>>>> But the sqm scripts were not updated. In the advanced options
>should i set that the dcsp marks are kept?
>>>>>>>> 	This really is an implementation detail that has no immediate
>effect if you choose piece_of_cake as typically only the bottleneck is
>sensitive to DSCP based priority banding. (Typically in that if you are
>unlucky your WLAN will use the DSCP marks to move packets into 4
>different priority classes, which is fine if you want that, but bad for
>not sanity checked packets coming in from the wider internet (one is
>not supposed to assume incoming packets have sensible dscp markings as
>per RFC) that is why the wash/squash option is missed by some of us,
>independent of the fact that it was a layering violation).
>>>>>>>>
>>>>>>>>> 3. Should i use advanced options in sqm scripts and set
>triple-isolate + diffserv8 ?
>>>>>>>> 	If you understand what these options do and believe that this
>is the best for your network go ahead, otherwise… The triple-isolate
>option will try to be fair to host_IP addresses first and then for each
>hostIP fair to each flow, but for that to do something you will most
>likely want this requires that cake sees internal IP addresses of your
>end-hosts. In the typical configuration with SQM on the WAN interface
>of a NAT router all internal addresses are replaced with the external
>IP address of the router it self and triple-isolates per host fairness
>will pretty much be equal to per flow fairness (not exactly, but in
>essence). So if you want to try tiple-isolate or its better defined
>brothers dual-srchost and dual-dsthost you would need to instantiate
>SQM on an internal interface like LAN. But then the direction of
>ingress and egress from the routers perspective changes with regards to
>the internet download and upload direction and you will need to put the
>internet upload bandwidth into the download field of the sqm GUI and
>vice versa. Also SQM on an internal interface will also shape internal
>traffic over the same interface, and that often affects traffic to and
>from the wifi/wlan radios to the lan switch… (I guess you would have
>preferred a shorter less vague response, but such are the constraints…)
>>>>>>>>
>>>>>>>>> 4. Is it recommend to enable diffserv on ingress?
>>>>>>>> 	If you trust/konw/have confirmed that your upstream (ISP?)
>sends you sensible and reasonable DSCP markings by all means enable
>diffserv on ingress. But the default assumption should be that your
>upstream used a dscp mapping that only makes sense for them and not for
>you.
>>>>>>>>
>>>>>>>>> 5. Is there still the udp packet dropping problem? e.g. games
>that are using udp.
>>>>>>>>> If yes does it make sense to apply diffserv classes manually?
>How to do this?
>>>>>>>> 	I am not sure what you mean, but if you test this and have
>some findings please report here…
>>>>>>>>
>>>>>>>>> 6. is the autorate_ingress still under development?
>>>>>>>>> This very interesting feature. especially for docsis networks.
>Will it be possible to set target ping time?
>>>>>>>> 	The last tests did indicate that this feature is not ready for
>primetime at least not on typically fixed bandwidth links and I assume
>docsis links are fixed enough.
>>>>>>>>
>>>>>>>>> 6. What difference does it make to set a different rtt?
>>>>>>>>> Setting lower rtt will reduce download speed i guess but will
>it allow better ping times (because of lower downloadrate uh)?
>>>>>>>>> What happens if rtt is set way higher?
>>>>>>>> 	With the RTT parameter you in essence specify how much time
>you give the endpoints of a flow to respond to a congestion signal (ECN
>marking or packet drop) if you select this way to small you will
>sacrifice bandwidth, if you set this too high you will accumulate more
>latency under load. The good thing seems to be that this does not need
>to be terribly precise, order of magnitude correctness seems to be
>sufficient (at least in base2)
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thank you!
>>>>>>>> 	I am sure the real experts will also chime in…
>>>>>>>>
>>>>>>>> Best Regards
>>>>>>>> 	Sebastian
>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Cake mailing list
>>>>>>>>> Cake@lists.bufferbloat.net
>>>>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>>>> _______________________________________________
>>>>>>> Cake mailing list
>>>>>>> Cake@lists.bufferbloat.net
>>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [Cake] New to cake. Some questions

[Benjamin Cronce]

[-- Attachment #1: Type: text/plain, Size: 17789 bytes --]

At least you ISP's trunk seems decent

ping -t 109.90.28.1

Packets: sent=150, rcvd=150, error=0, lost=0 (0.0% loss) in 74.660177 sec
RTTs in ms: min/avg/max/dev: 158.255 / 159.140 / 161.922 / 0.528
Bandwidth in kbytes/sec: sent=0.120, rcvd=0.120

On Fri, Jun 10, 2016 at 9:05 AM, Dennis Fedtke <dennisfedtke@gmail.com>
wrote:

> Hi Sebastian,
>
> yes this is wired connection. As i stated my ping times always vary
> independently of target.
> My ISP is overloaded in certain regions. So i assume they do some
> shaping/limiting on certain protocols (icmp for example)
> Connection speed is 200/20 Mbit.
> ISP is unitymedia which doesn't allow you to use your own hardware.
> So actually i have to run my router behind theirs with exposed host
> enabled :<
>
> Ping response:
>
> ping -s 1400 -c 1 109.90.x.x
> PING 109.90.28.1 (109.90.28.1) 1400(1428) bytes of data.
> 1408 bytes from 109.90.28.1: icmp_seq=1 ttl=253 time=11.6 ms
>
> --- 109.90.28.1 ping statistics ---
> 1 packets transmitted, 1 received, 0% packet loss, time 0ms
> rtt min/avg/max/mdev = 11.677/11.677/11.677/0.000 ms
>
> This looks good or?
>
> Yes i am from germany. So you are from germany too?
>
> Thanks for your time and help :)
>
>
> Best regards
> Dennis
>
>
> Am 10.06.2016 um 15:02 schrieb moeller0:
>
>> Hi Dennis,
>>
>> On Jun 10, 2016, at 14:43 , Dennis Fedtke <dennisfedtke@gmail.com> wrote:
>>>
>>> Hi Sebastian,
>>>
>>> i used the default setting of 1000.
>>>
>>         Okay, that should work i assume unless you have a very fast link…
>> What link at what ISP do you actually have?
>>
>> But it seems that my isp is dropping icmp packets if there are exceeding
>>> some sending threshold.
>>>
>>         I would be amazed if they did, a sympotom of that would be rsate
>> reduction to all ICMP probe flows independent of target host. If however
>> you only see this with specific hosts it is very likely that that host rate
>> limits its ICMP responses. In either case try another host further
>> upstream. II think I has reasonable decent results with targeting 8.8.8.8,
>> googles dns servers.
>>
>> So there is a lot of none usable ping data.
>>>
>>         Again, try another host…
>>
>> I increased the send delay to 50ms. 25 ms already shows dropped requests.
>>>
>>         That might also help, as long as you stay below their throttling
>> rate the chosen host might still work okay.
>>
>> This is the third run now. Waiting for completion.
>>>
>>         Well, sorry that the method is not as slick and streamlined, but
>> there are no guarded good ICMP reflectors available on the net.
>>
>> The ping target is my first hop.
>>>
>>         Try the next hop then ;)
>>
>> Actually my ping always varies around +-5ms even at idle and
>>> independently of ping target.
>>>
>>         This is via wifi/wlan? If so try from a wired connection instead.
>>
>> When i look through the ping file the increase in ping times are actually
>>> appear to be random to me.
>>>
>>         Well, we expect variability of the individual “trials” to exist,
>> that is why we collect so many and try to select the best measure in the
>> matlab code to remove the unwanted variance. Could you post a link to both
>> of the generated plots please, the first one showing te different
>> aggregation measures might be helpful in diagnosing the issues deeper.
>>
>> So how to test if my isp responses with fixed icmp packet size?
>>>
>>         You could try manually. In the folloewing example I pinged
>> gstatic.com (which belongs to googles CDN as far as I know):
>>
>> bash-3.2$ ping -s 1 -c 1 gstatic.com
>> PING gstatic.com (216.58.213.195): 1 data bytes
>> 9 bytes from 216.58.213.195: icmp_seq=0 ttl=55
>>
>> --- gstatic.com ping statistics ---
>> 1 packets transmitted, 1 packets received, 0.0% packet loss
>>
>>
>> bash-3.2$ ping -s 64 -c 1 gstatic.com
>> PING gstatic.com (216.58.213.195): 64 data bytes
>> 72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=19.446 ms
>>
>> --- gstatic.com ping statistics ---
>> 1 packets transmitted, 1 packets received, 0.0% packet loss
>> round-trip min/avg/max/stddev = 19.446/19.446/19.446/0.000 ms
>>
>>
>> bash-3.2$ ping -s 65 -c 1 gstatic.com
>> PING gstatic.com (216.58.213.195): 65 data bytes
>> 72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=21.138 ms
>> wrong total length 92 instead of 93
>>
>> --- gstatic.com ping statistics ---
>> 1 packets transmitted, 1 packets received, 0.0% packet loss
>> round-trip min/avg/max/stddev = 21.138/21.138/21.138/0.000 ms
>> bash-3.2$
>>
>>
>> bash-3.2$ ping -s 1400 -c 1 gstatic.com
>> PING gstatic.com (216.58.213.195): 1400 data bytes
>> 72 bytes from 216.58.213.195: icmp_seq=0 ttl=55 time=6.878 ms
>> wrong total length 92 instead of 1428
>>
>> --- gstatic.com ping statistics ---
>> 1 packets transmitted, 1 packets received, 0.0% packet loss
>> round-trip min/avg/max/stddev = 6.878/6.878/6.878/0.000 ms
>>
>> Once I try to send 65 Bytes of ICMP payload the response is cut short to
>> 92 bytes, the same might happen with your isp. But also if all your ISP
>> does is rate limiting the ICMP packests that still can lead to to much
>> variance in the RTTs…
>>
>>
>> Im in central europe too :D
>>>
>>         Ah, then you just have a different work/sleep cycle than I do ;).
>> Where in central Europe, if I might as Ii am, as you might have guessed
>> based in Germany…
>>
>> Best Regards
>>         Sebastian
>>
>> Thanks :)
>>>
>>>
>>> Am 10.06.2016 um 07:20 schrieb moeller0:
>>>
>>>> Hi Dennis,
>>>>
>>>>
>>>> On Jun 10, 2016, at 02:49 , Dennis Fedtke <dennisfedtke@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi Sebastian,
>>>>>
>>>>> Sorry this is positive or?
>>>>>
>>>>         I would say that is unclear…
>>>>
>>>> But i need more samples ?
>>>>>
>>>>         I would try with more samples, after checking that the ping
>>>> times in the recorded data file actually are larger for larger probes than
>>>> for smaller, some hosts will reply with a fixed maximum ICMP packet instead
>>>> of returning the received packet, thereby reducing the signal range (as
>>>> only the upload leg of the link is meaning fully contributing useful
>>>> differential signal.
>>>>         BTW I am in central europe so at times of the day my responses
>>>> can be very sporadic, as I either am at work or sleeping ;)
>>>>
>>>> Best Regards
>>>>         Sebastian
>>>>
>>>> Thanks :)
>>>>>
>>>>>
>>>>> Am 10.06.2016 um 01:11 schrieb moeller0:
>>>>>
>>>>>> Hi Dennis,
>>>>>>
>>>>>> On Jun 10, 2016, at 00:45 , Dennis Fedtke <dennisfedtke@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Sebastian,
>>>>>>>
>>>>>>> thank you for your answers :)
>>>>>>>
>>>>>>> The ATM overhead detector script is currently running.
>>>>>>> I read the wiki about it but im not quite sure how to interpret the
>>>>>>> plot.
>>>>>>> I mean what info should i read from it? maximum packet size?
>>>>>>>
>>>>>>         The relevant number is reported as “Estimated overhead
>>>>>> preceding the IP header” in the top part of the second figure created by
>>>>>> the script. But that is only relevant.useful if you see a nice step like
>>>>>> plot in figure 2 as well ( the second figure in
>>>>>> https://github.com/moeller0/ATM_overhead_detector/wiki as positive
>>>>>> and the fourth figure as negative example.
>>>>>>
>>>>>> If yes do i set the overhead in cake? Or do i set iptables to clamp
>>>>>>> to new mtu/mss?
>>>>>>>
>>>>>>         If you use plain cake and you know the numerical overhead
>>>>>> (NN) the easiest is to add the following to your cake invocation: “atm
>>>>>> overhead NN”
>>>>>>
>>>>>> Please note that if you use cake on an ethernet interface the kernel
>>>>>> will already account for 14 byte of ethernet overhead, so if the script
>>>>>> told you 44 as actual overhead, you use ”overhead 30” to address that. If
>>>>>> you use a pppoe interface the kernel will most likely not add the 14 bytes
>>>>>> for you, so then you would use “overhead 44” (I excluded the atm option in
>>>>>> the last examples for clarity…)
>>>>>>
>>>>>> Regarding UDP paket dropping problem:
>>>>>>> I just read some forums and users stated that under heavy load cake
>>>>>>> starts to drop udp packets which causes lag ingame.
>>>>>>> My idea was to set ingress/egress to diffserv4 and apply the EF dscp
>>>>>>> mark on those packets.
>>>>>>>
>>>>>>         Ell, not a bad idea, but often the problem are in the
>>>>>> incoming traffic, and unfortunately with the ifb we use we can not use
>>>>>> iptables, but only tc, and remarking with tc is unpleasant.
>>>>>>
>>>>>> Will this even work? if yes how to do this? iptables?
>>>>>>>
>>>>>>         No, you wuld need tp use tc.
>>>>>>
>>>>>> ipt -t mangle -A PREROUTING -p udp -m multiport --ports 5000:5500 -j
>>>>>>> DSCP --set-dscp-class EF
>>>>>>>
>>>>>>> Like thia? Is prerouting correct here? (Taken from layer cake script)
>>>>>>>
>>>>>>         This will affect outgoing packets and might be a good idea in
>>>>>> your specific case.
>>>>>>
>>>>>> BUT why don’t you try the default behaviour with specific rules and
>>>>>> tricks and report success or failure back to us, after all the
>>>>>> fastest/easiest classification is one one does not need to perform at all.
>>>>>>
>>>>>> For the squash and wash feature.
>>>>>>> Im asking because if i choose to squash in the advanced options of
>>>>>>> sqm scripts.
>>>>>>> The dscp fields/marks will be overwritten by iptables to 0
>>>>>>> (besteffort). (layer cake script)
>>>>>>> So then it makes no sense to manually set dscp fields/marks or? (Or
>>>>>>> even setting diffserv)
>>>>>>>
>>>>>>         No unfortunately on ingress cake sees the packets before
>>>>>> iptables, so the effective behavioral emulation of wash/squash by cake is
>>>>>> to set ingress cake to besteffort (basically cake ignores the dscp field
>>>>>> which functionally is identical to all packets having the same value). The
>>>>>> squashing by iptables just clears the dscp marls so that internal
>>>>>> networking elements like potentially wifi liknks are not confuzed by the
>>>>>> dscp information.
>>>>>>
>>>>>> Did i understand this correctly. Per rfc isps should not provide dscp
>>>>>>> fields/marks?
>>>>>>>
>>>>>>         Not exactly, per RFC DSCPs are only ever valid/defined inside
>>>>>> a DSCP domain and your ISPs domain ends before it reaches your CPE. Since
>>>>>> you have no control over your ISPs markings, they can be very much not like
>>>>>> you want them to be (Dave That reported that his ISP re-mapped almost 1/3
>>>>>> or so of packets into the CS1 background class). So it is recomended that
>>>>>> each DSCP domain re-mapps the code points at its entry point, which in your
>>>>>> case is your router…
>>>>>>
>>>>>> Best Regards
>>>>>>         Sebastian
>>>>>>
>>>>>> Thank you.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Am 09.06.2016 um 23:30 schrieb moeller0:
>>>>>>>
>>>>>>>> Hi Dennis,
>>>>>>>>
>>>>>>>> let me start with a disclaimer, I am not the best information
>>>>>>>> source for cake on this mailing list, but I assume the others will chime in
>>>>>>>> if I say something questionable…
>>>>>>>>
>>>>>>>> On Jun 9, 2016, at 22:58 , Dennis Fedtke <dennisfedtke@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Currently im running lede + cake + sqm_scripts and i have some
>>>>>>>>> questions:
>>>>>>>>>
>>>>>>>>> 1. What is considered the “optimal" setup atm for cake?
>>>>>>>>>
>>>>>>>>         The same as without cake; really, proper
>>>>>>>> per-packet-overhead accounting is important for bandwidth shaping,
>>>>>>>> especially for ATM -based links. I would recommend to follow the method on
>>>>>>>> https://github.com/moeller0/ATM_overhead_detector to m\empirically
>>>>>>>> measure whether your link uses ATM encapsulation and what exact overhead is
>>>>>>>> in use.
>>>>>>>>
>>>>>>>> e.g. which cake script should i use piece or layer cake?
>>>>>>>>>
>>>>>>>>         piece_of_cake has only one tier of priority, while
>>>>>>>> layer_cake currently offers 4. Packets are put into the different priority
>>>>>>>> bands based on the content of their TOS/DSCP filed in the IP header; if
>>>>>>>> this is greek to you, I guess piece_of_cake most likely is what you are
>>>>>>>> looking for..
>>>>>>>>
>>>>>>>> 2. Recently squash and wash was removed.
>>>>>>>>> But the sqm scripts were not updated. In the advanced options
>>>>>>>>> should i set that the dcsp marks are kept?
>>>>>>>>>
>>>>>>>>         This really is an implementation detail that has no
>>>>>>>> immediate effect if you choose piece_of_cake as typically only the
>>>>>>>> bottleneck is sensitive to DSCP based priority banding. (Typically in that
>>>>>>>> if you are unlucky your WLAN will use the DSCP marks to move packets into 4
>>>>>>>> different priority classes, which is fine if you want that, but bad for not
>>>>>>>> sanity checked packets coming in from the wider internet (one is not
>>>>>>>> supposed to assume incoming packets have sensible dscp markings as per RFC)
>>>>>>>> that is why the wash/squash option is missed by some of us, independent of
>>>>>>>> the fact that it was a layering violation).
>>>>>>>>
>>>>>>>> 3. Should i use advanced options in sqm scripts and set
>>>>>>>>> triple-isolate + diffserv8 ?
>>>>>>>>>
>>>>>>>>         If you understand what these options do and believe that
>>>>>>>> this is the best for your network go ahead, otherwise… The triple-isolate
>>>>>>>> option will try to be fair to host_IP addresses first and then for each
>>>>>>>> hostIP fair to each flow, but for that to do something you will most likely
>>>>>>>> want this requires that cake sees internal IP addresses of your end-hosts.
>>>>>>>> In the typical configuration with SQM on the WAN interface of a NAT router
>>>>>>>> all internal addresses are replaced with the external IP address of the
>>>>>>>> router it self and triple-isolates per host fairness will pretty much be
>>>>>>>> equal to per flow fairness (not exactly, but in essence). So if you want to
>>>>>>>> try tiple-isolate or its better defined brothers dual-srchost and
>>>>>>>> dual-dsthost you would need to instantiate SQM on an internal interface
>>>>>>>> like LAN. But then the direction of ingress and egress from the routers
>>>>>>>> perspective changes with regards to the internet download and upload
>>>>>>>> direction and you will need to put the internet upload bandwidth into the
>>>>>>>> download field of the sqm GUI and vice versa. Also SQM on an internal
>>>>>>>> interface will also shape internal traffic over the same interface, and
>>>>>>>> that often affects traffic to and from the wifi/wlan radios to the lan
>>>>>>>> switch… (I guess you would have preferred a shorter less vague response,
>>>>>>>> but such are the constraints…)
>>>>>>>>
>>>>>>>> 4. Is it recommend to enable diffserv on ingress?
>>>>>>>>>
>>>>>>>>         If you trust/konw/have confirmed that your upstream (ISP?)
>>>>>>>> sends you sensible and reasonable DSCP markings by all means enable
>>>>>>>> diffserv on ingress. But the default assumption should be that your
>>>>>>>> upstream used a dscp mapping that only makes sense for them and not for you.
>>>>>>>>
>>>>>>>> 5. Is there still the udp packet dropping problem? e.g. games that
>>>>>>>>> are using udp.
>>>>>>>>> If yes does it make sense to apply diffserv classes manually? How
>>>>>>>>> to do this?
>>>>>>>>>
>>>>>>>>         I am not sure what you mean, but if you test this and have
>>>>>>>> some findings please report here…
>>>>>>>>
>>>>>>>> 6. is the autorate_ingress still under development?
>>>>>>>>> This very interesting feature. especially for docsis networks.
>>>>>>>>> Will it be possible to set target ping time?
>>>>>>>>>
>>>>>>>>         The last tests did indicate that this feature is not ready
>>>>>>>> for primetime at least not on typically fixed bandwidth links and I assume
>>>>>>>> docsis links are fixed enough.
>>>>>>>>
>>>>>>>> 6. What difference does it make to set a different rtt?
>>>>>>>>> Setting lower rtt will reduce download speed i guess but will it
>>>>>>>>> allow better ping times (because of lower downloadrate uh)?
>>>>>>>>> What happens if rtt is set way higher?
>>>>>>>>>
>>>>>>>>         With the RTT parameter you in essence specify how much time
>>>>>>>> you give the endpoints of a flow to respond to a congestion signal (ECN
>>>>>>>> marking or packet drop) if you select this way to small you will sacrifice
>>>>>>>> bandwidth, if you set this too high you will accumulate more latency under
>>>>>>>> load. The good thing seems to be that this does not need to be terribly
>>>>>>>> precise, order of magnitude correctness seems to be sufficient (at least in
>>>>>>>> base2)
>>>>>>>>
>>>>>>>>
>>>>>>>> Thank you!
>>>>>>>>>
>>>>>>>>         I am sure the real experts will also chime in…
>>>>>>>>
>>>>>>>> Best Regards
>>>>>>>>         Sebastian
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>>> Cake mailing list
>>>>>>>>> Cake@lists.bufferbloat.net
>>>>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>> Cake mailing list
>>>>>>> Cake@lists.bufferbloat.net
>>>>>>> https://lists.bufferbloat.net/listinfo/cake
>>>>>>>
>>>>>>
>>>
>
> _______________________________________________
> Cake mailing list
> Cake@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cake
>

[-- Attachment #2: Type: text/html, Size: 23246 bytes --]