Re: [Cerowrt-devel] VDSL

Re: [Cerowrt-devel] VDSL

[Rich Brown]

The VDSL option is burning up way too many of my brain cells. Given that VDSL may well use ATM, while VDSL2 is unlikely to use ATM, I think that it isn’t useful to include it as a top-level choice .

In the new pages, I propose removing the VDSL choice, leaving:

-  “ATM (any kind of DSL link)” and 
- “None”

and letting people tune the settings after reading the “Everything you wanted to know…” page if they want to optimize. 

http://www.bufferbloat.net/projects/cerowrt/wiki/Setting_up_AQM_for_CeroWrt_310

-and-

http://www.bufferbloat.net/projects/cerowrt/wiki/Everything_you_wanted_to_know_about_Link_Layer_Adaptation

Rich

Re: [Cerowrt-devel] VDSL

[Sebastian Moeller]

Hi Rich,

On Jan 11, 2014, at 17:29 , Rich Brown <richb.hanover@gmail.com> wrote:

> The VDSL option is burning up way too many of my brain cells. Given that VDSL may well use ATM, while VDSL2 is unlikely to use ATM, I think that it isn’t useful to include it as a top-level choice .
> 
> In the new pages, I propose removing the VDSL choice, leaving:
> 
> -  “ATM (any kind of DSL link)” and 
> - “None”

	Well, that looks like a decent recommendation for the wiki. The SQM configuration page still needs to expose all three values, atm, ethernet, and  none so that people can actually change things...

> 
> and letting people tune the settings after reading the “Everything you wanted to know…” page if they want to optimize. 

	Exactly, we still need options exposed for making this possible.

> 
> http://www.bufferbloat.net/projects/cerowrt/wiki/Setting_up_AQM_for_CeroWrt_310
> 
> -and-
> 
> http://www.bufferbloat.net/projects/cerowrt/wiki/Everything_you_wanted_to_know_about_Link_Layer_Adaptation
> 
> Rich

Best Regards
	Sebastian
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] VDSL

[Rich Brown]

Hi Sebastian,

>> In the new pages, I propose removing the VDSL choice, leaving:
>> 
>> -  “ATM (any kind of DSL link)” and 
>> - “None”
> 
> 	Well, that looks like a decent recommendation for the wiki. The SQM configuration page still needs to expose all three values, atm, ethernet, and  none so that people can actually change things…

Ahah. What’s the difference between “Ethernet” and “None”? I had thought there were two link layer adaptations (ATM, and None, with the possibility of some number of overhead bytes.

Rich

Re: [Cerowrt-devel] VDSL

[Rich Brown]

HI Sebastian,

> 	Well, that looks like a decent recommendation for the wiki. The SQM configuration page still needs to expose all three values, atm, ethernet, and  none so that people can actually change things...

So two questions really:

1) (From my previous note) What’s the difference between the current “Ethernet” (for VDSL) and “None” link layer adaptations?

2) When we distinguish the Ethernet/VDSL case, I would really like to use a different name from “Ethernet” because it seems confusingly similar to  having a real Ethernet path/link (e.g., direct connection to internet backbone without any ADSL, cable modem, etc.) 

Thanks.

Rich

Re: [Cerowrt-devel] VDSL

[Sebastian Moeller]

Hi Rich,


On Jan 11, 2014, at 18:55 , Rich Brown <richb.hanover@gmail.com> wrote:

> HI Sebastian,
> 
>> 	Well, that looks like a decent recommendation for the wiki. The SQM configuration page still needs to expose all three values, atm, ethernet, and  none so that people can actually change things...
> 
> So two questions really:
> 
> 1) (From my previous note) What’s the difference between the current “Ethernet” (for VDSL) and “None” link layer adaptations?

	Currently, "none" completely disables the link layer adjustments, "ethernet" enables them, but will only use the overhead (unless you specify an tcMPU, but that is truly exotic).

> 
> 2) When we distinguish the Ethernet/VDSL case, I would really like to use a different name from “Ethernet” because it seems confusingly similar to  having a real Ethernet path/link (e.g., direct connection to internet backbone without any ADSL, cable modem, etc.) 

	On the one hand I agree, but the two options are called "ATM" (well for tc "adsl" is a valid alias for ATM) and "ethernet" if you pass them to tc (what we do), and I would really hate it to hide this under fancy names. I see no chance of renaming those options in tc, so we are sort of stuck with them and adding another layer of indirection seems too opaque to me. This is why I put some explanation behind the option names in the list box…

Best
	Sebastian

> 
> Thanks.
> 
> Rich

Re: [Cerowrt-devel] VDSL

[Rich Brown]

Hi Sebastian,

>>> 	Well, that looks like a decent recommendation for the wiki. The SQM configuration page still needs to expose all three values, atm, ethernet, and  none so that people can actually change things...
>> 
>> So two questions really:
>> 
>> 1) (From my previous note) What’s the difference between the current “Ethernet” (for VDSL) and “None” link layer adaptations?
> 
> 	Currently, "none" completely disables the link layer adjustments, "ethernet" enables them, but will only use the overhead (unless you specify an tcMPU, but that is truly exotic).
> 
>> 
>> 2) When we distinguish the Ethernet/VDSL case, I would really like to use a different name from “Ethernet” because it seems confusingly similar to  having a real Ethernet path/link (e.g., direct connection to internet backbone without any ADSL, cable modem, etc.) 
> 
> 	On the one hand I agree, but the two options are called "ATM" (well for tc "adsl" is a valid alias for ATM) and "ethernet" if you pass them to tc (what we do), and I would really hate it to hide this under fancy names. I see no chance of renaming those options in tc, so we are sort of stuck with them and adding another layer of indirection seems too opaque to me. This is why I put some explanation behind the option names in the list box…

Now I see how it works. (I didn’t understand that “None” really meant NONE.) The following choices in the Link Layer Adaptation would have eased my confusion:

- ATM (almost every type of ADSL or DSL)
- Ethernet with overhead
- None (default)

Then the text can say:

—
You must set the Link Layer Adaptation options so that CeroWrt can perform its best with video and audio chat, gaming, and other protocols that rely on short packets. The general rule for selecting the Link Layer Adaption is: 

* If you use any kind of DSL/ADSL connection to the Internet (that is, if you get your internet service through the telephone line), you should choose the “ATM (almost every type of ADSL or DSL)" item. Set the Per-packet Overhead to 44. 

* If you know you have a VDSL connection, you should choose “Ethernet with overhead" and set the Per-packet Overhead to 8. 

* If you use Cable modem, Fiber, or another kind of connection to the Internet, you should choose “None (default)”. All other parameters will be ignored.

If you cannot tell what kind of link you have, first try using "None", then run the [[Quick Test for Bufferbloat]]. If the results are good, you’re done. Next,  try the ADSL/ATM choice, then the VDSL choice to see which performs best. Read the **Details** (below) to learn more about tuning the parameters for your link. 
—

Would that be better? Thanks.

Rich

Re: [Cerowrt-devel] VDSL

[Sebastian Moeller]

Hi Rich,


On Jan 11, 2014, at 20:06 , Rich Brown <richb.hanover@gmail.com> wrote:

> Hi Sebastian,
> 
>>>> 	Well, that looks like a decent recommendation for the wiki. The SQM configuration page still needs to expose all three values, atm, ethernet, and  none so that people can actually change things...
>>> 
>>> So two questions really:
>>> 
>>> 1) (From my previous note) What’s the difference between the current “Ethernet” (for VDSL) and “None” link layer adaptations?
>> 
>> 	Currently, "none" completely disables the link layer adjustments, "ethernet" enables them, but will only use the overhead (unless you specify an tcMPU, but that is truly exotic).
>> 
>>> 
>>> 2) When we distinguish the Ethernet/VDSL case, I would really like to use a different name from “Ethernet” because it seems confusingly similar to  having a real Ethernet path/link (e.g., direct connection to internet backbone without any ADSL, cable modem, etc.) 
>> 
>> 	On the one hand I agree, but the two options are called "ATM" (well for tc "adsl" is a valid alias for ATM) and "ethernet" if you pass them to tc (what we do), and I would really hate it to hide this under fancy names. I see no chance of renaming those options in tc, so we are sort of stuck with them and adding another layer of indirection seems too opaque to me. This is why I put some explanation behind the option names in the list box…
> 
> Now I see how it works. (I didn’t understand that “None” really meant NONE.)

	Ah, I see, the reason most likely is that initially I had it set up differently, and only later I realized that there are only three relevant states...

> The following choices in the Link Layer Adaptation would have eased my confusion:
> 
> - ATM (almost every type of ADSL or DSL)

	Question, for me DSL is short hand for SDSL, ADSL, and VDSL, but i take it too mean whatever the ISP put in the product name so most likely ADSL, is that correct?

> - Ethernet with overhead
> - None (default)
> 
> Then the text can say:
> 
> —
> You must set the Link Layer Adaptation options so that CeroWrt can perform its best with video and audio chat, gaming, and other protocols that rely on short packets. The general rule for selecting the Link Layer Adaption is: 
> 
> * If you use any kind of DSL/ADSL connection to the Internet (that is, if you get your internet service through the telephone line), you should choose the “ATM (almost every type of ADSL or DSL)" item. Set the Per-packet Overhead to 44. 

	Sounds like a reasonable default, we only sacrifice a bit of bandwidth but keep the latency spiffy.

> 
> * If you know you have a VDSL connection, you should choose “Ethernet with overhead" and set the Per-packet Overhead to 8. 

	This depends on whether there is PPPoE in use (then 8 seems correct), I do not know about potential ethernet heads or VLAN headers, so * might be too small. Also I have not seen much data whatsoever from a VDSL link so I really have no good handle on this (except Germany's lardiest ISP uses PPPoE). Note I might switch to VDSL later this year, then I can see (if I manage to get anything useful out of non ATM links, my overhead detector critically relies on ATM cell quantization)

> 
> * If you use Cable modem, Fiber, or another kind of connection to the Internet, you should choose “None (default)”. All other parameters will be ignored.

	I think this is the status quo; I have no data to argue any change here (I have an intuition though that most likely all of these will also suffer from additional per packet overhead by virtue of being shared media).

> 
> If you cannot tell what kind of link you have, first try using "None", then run the [[Quick Test for Bufferbloat]]. If the results are good, you’re done.

	I will ned to test this with link layer disabled, to better describe what to expect. I fear a typical speediest.net test will not show ATMs small packet volatility; so there might be need for a dedicated test...

> Next,  try the ADSL/ATM choice, then the VDSL choice to see which performs best. Read the **Details** (below) to learn more about tuning the parameters for your link. 
> —
> 
> Would that be better? Thanks.

	Oh, I think that is quite reasonable and helpful. Now I need to put more information into the **Details**

Best
	Sebastian

> 
> Rich

Re: [Cerowrt-devel] VDSL

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 5449 bytes --]

Rich, Sebastian, (and others),

First, a hello.  I've been lurking on the bufferbloat mailing list for a
bit, and just joined here as well to better follow what's going on, and see
if there's any way I can help.

Next, I have an ADSL+ link in Paris (Free.fr), and am willing to run a
number of tests using the various LLA options and overhead estimations.
 But 3 hours on a dead-quiet link could be hard to deal with.  I'm happy to
run an hour's worth of netperf tests to a nearby server, slowly working
through the parameter space, and then comparing the results.

My ad-hoc comparisons last week with various modes showed that too high of
settings for the overhead (coupled with the already reduced bw limit in the
shaper) killed the bulk upload/download performance (which I care about on
a meager 18Mbps/1Mbps link).  But I found that setting the bw shaper limit
to the reported line speed (from the modem), and then adjusting the
overhead parameters got me the same bulk performances, with the same
latencies (or what appeared to be the same, I need to do more data
crunching on the results, and run again in a quieter setting).

It would also help if my target server was closer than it is.  The only
server I know of is in Germany, 55ms away (unloaded).

OTOH, I can say that any changes here over the defaults are still gilding
the lily (to an end user).  But Free.fr's router/modem already uses codel,
so it wasn't that bad to begin with (vs. Numericable on a docsis 3 Netgear
router/modem).

===

I can also say that I found the current verbiage on that particular page a
bit "clear as mud".  Even knowing what my network is (to some degree, since
the Free.fr modem can tell me), it was difficult to follow, and I quickly
found myself at about 3/4 my previous speeds, with no visible improvement
in latency (although that could have been a measurement tool issue, as I
was doing 60-second runs using the rrul netperf loads).

The main question I have is:

- Should we both limit the bandwidth well below the reported line rate (or
measured IP rate) AND use the link layer adaption settings? (85-90% of
bandwidth)
 or
- rely on the LLA settings to do the overhead, and shape to just a tiny bit
under the reported line rate? (95-99% of bandwidth)

Thanks,

- Aaron Wood


On Sat, Jan 11, 2014 at 8:06 PM, Rich Brown <richb.hanover@gmail.com> wrote:

> Hi Sebastian,
>
> >>>     Well, that looks like a decent recommendation for the wiki. The
> SQM configuration page still needs to expose all three values, atm,
> ethernet, and  none so that people can actually change things...
> >>
> >> So two questions really:
> >>
> >> 1) (From my previous note) What’s the difference between the current
> “Ethernet” (for VDSL) and “None” link layer adaptations?
> >
> >       Currently, "none" completely disables the link layer adjustments,
> "ethernet" enables them, but will only use the overhead (unless you specify
> an tcMPU, but that is truly exotic).
> >
> >>
> >> 2) When we distinguish the Ethernet/VDSL case, I would really like to
> use a different name from “Ethernet” because it seems confusingly similar
> to  having a real Ethernet path/link (e.g., direct connection to internet
> backbone without any ADSL, cable modem, etc.)
> >
> >       On the one hand I agree, but the two options are called "ATM"
> (well for tc "adsl" is a valid alias for ATM) and "ethernet" if you pass
> them to tc (what we do), and I would really hate it to hide this under
> fancy names. I see no chance of renaming those options in tc, so we are
> sort of stuck with them and adding another layer of indirection seems too
> opaque to me. This is why I put some explanation behind the option names in
> the list box…
>
> Now I see how it works. (I didn’t understand that “None” really meant
> NONE.) The following choices in the Link Layer Adaptation would have eased
> my confusion:
>
> - ATM (almost every type of ADSL or DSL)
> - Ethernet with overhead
> - None (default)
>
> Then the text can say:
>
> —
> You must set the Link Layer Adaptation options so that CeroWrt can perform
> its best with video and audio chat, gaming, and other protocols that rely
> on short packets. The general rule for selecting the Link Layer Adaption is:
>
> * If you use any kind of DSL/ADSL connection to the Internet (that is, if
> you get your internet service through the telephone line), you should
> choose the “ATM (almost every type of ADSL or DSL)" item. Set the
> Per-packet Overhead to 44.
>
> * If you know you have a VDSL connection, you should choose “Ethernet with
> overhead" and set the Per-packet Overhead to 8.
>
> * If you use Cable modem, Fiber, or another kind of connection to the
> Internet, you should choose “None (default)”. All other parameters will be
> ignored.
>
> If you cannot tell what kind of link you have, first try using "None",
> then run the [[Quick Test for Bufferbloat]]. If the results are good,
> you’re done. Next,  try the ADSL/ATM choice, then the VDSL choice to see
> which performs best. Read the **Details** (below) to learn more about
> tuning the parameters for your link.
> —
>
> Would that be better? Thanks.
>
> Rich
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

[-- Attachment #2: Type: text/html, Size: 6402 bytes --]

Re: [Cerowrt-devel] VDSL

[Sebastian Moeller]

Hi Aaron,


On Jan 11, 2014, at 21:23 , Aaron Wood <woody77@gmail.com> wrote:

> Rich, Sebastian, (and others),
> 
> First, a hello.  I've been lurking on the bufferbloat mailing list for a bit, and just joined here as well to better follow what's going on, and see if there's any way I can help.
> 
> Next, I have an ADSL+ link in Paris (Free.fr), and am willing to run a number of tests using the various LLA options and overhead estimations.  But 3 hours on a dead-quiet link could be hard to deal with.  

	Well, it takes 3 hours to collect 10000 samples for 100 different packet sizes (creating a maximum transient traffic of < 150kbit/s), you could collect less (your RTT will be dominated by your uplink, so 2000 to 3000 samples per size will be plenty, reducing the time to something more manageable). And as long as your link has 150 kbit/sec reserved on the uplink (since it is more critical there) you should be fine. Unless you saturate your link, additional traffic to the probe will increase the variance of the ping times, but you might be able to get away with, just try to measure for as long as you are comfortable with. (I typically just run this over night… and there is not much going on in our network at that time, but it is not dead-quiet either).

> I'm happy to run an hour's worth of netperf tests to a nearby server, slowly working through the parameter space, and then comparing the results.\

	Again, I recommend to perform the ATM quantization probe against the nearest host (already in the ISPs network, ideally the DSLAM/MSAN) to reduce the variance in the data...

> 
> My ad-hoc comparisons last week with various modes showed that too high of settings for the overhead (coupled with the already reduced bw limit in the shaper) killed the bulk upload/download performance (which I care about on a meager 18Mbps/1Mbps link).  

	Yes, as expected you pay a bandwidth price for getting the overhead too large. The price you pay for specifying the overhead to low is that for some packet sizes there is no difference while some other packet sizes will drag in an additional ATM cell of 43 bytes. A larger packet sizes typical for bulk upload this additional 43 bytes will not really kill your through put, for small packets this can be catastrophic. Think your specify verhead 0 no link layer adjustments and send a 50 byte packet, the shaper thinks this takes 64 bytes, but on an ATM link with PPPoE LLC/SNAP you actually have 40 bytes overhead + 64 byte packet = 104 bytes which require 3 ATM cells of 48 byte totaling 3*48 = 144 bytes on the wire, your packet more than doubled (factor 2.25). And if you specify an overhead that is too small, you still get some packet sizes that drag in an additional cell with the same consequences.
	But note that these effects depend on the size of the packets, which people typically do not vary during bandwidth tests...

> But I found that setting the bw shaper limit to the reported line speed (from the modem), and then adjusting the overhead parameters got me the same bulk performances, with the same latencies (or what appeared to be the same, I need to do more data crunching on the results, and run again in a quieter setting).

	As I tried to indicate above for bulk traffic the effect of 47 bytes in 1500 might be lost in the noise (and 47 bytes per packet is the worst case), think roughly 3 percent worst case link capacity overestimation (and if you shape to 95% this already fits into the shaped margin).

> 
> It would also help if my target server was closer than it is.  The only server I know of is in Germany, 55ms away (unloaded).

	I think you should measure the per racket overhead where it actually matters, on the link to the DSLAM/ISP, any congestion on shared network segments further away from the router cannot really be controlled well by shaping in the router, so let's not try to fix that part of the route :) .


> 
> OTOH, I can say that any changes here over the defaults are still gilding the lily (to an end user).  

	I think I need to come up with a worst case ATM carrier effects test, so people can feel the pain quicker. The dang dependence on packet size really is unfortunate as no bulk speed tester I know tries a decent set of packet sizes...

> But Free.fr's router/modem already uses codel, so it wasn't that bad to begin with (vs. Numericable on a docsis 3 Netgear router/modem).

	If I understand correctly, at least the newer free boxes should be very hard to beat, can you disable the modems AQM for testing?

> 
> ===
> 
> I can also say that I found the current verbiage on that particular page a bit "clear as mud".  

	Which page?

> Even knowing what my network is (to some degree, since the Free.fr modem can tell me), it was difficult to follow, and I quickly found myself at about 3/4 my previous speeds, with no visible improvement in latency (although that could have been a measurement tool issue, as I was doing 60-second runs using the rrul netperf loads).

	So codel/fq_codel typically try to keep the latency increase bound to 5ms (per leg) so the ping times should stay nicely somewhere between 55ms and 65ms, if the rrul ping trace stays in that band you are already golden; instead of reducing the shaped bandwidth I would start increasing it again, until you notice the latencies increase again. Note for RRUL I typically aim for 300 seconds (I think Dave once recommended that) to get over all transient effects.


> 
> The main question I have is:
> 
> - Should we both limit the bandwidth well below the reported line rate (or measured IP rate) AND use the link layer adaption settings? (85-90% of bandwidth)
>  or
> - rely on the LLA settings to do the overhead, and shape to just a tiny bit under the reported line rate? (95-99% of bandwidth)

	I wou;d go for the second option. But note while you have full control over your uplink, on the downlink there might be not expected packets arriving (think you connect to your network via VPN) so the current recommendation is to shape the downlink a bit more aggressively than the uplink. Currently I use 95% on uplink and 90% on downlink (I have not yet found the time to see how far up I can push these, but latency is generally quite pleasent).

Best Regards
	Sebastian

> 
> Thanks,
> 
> - Aaron Wood
> 
> 
> On Sat, Jan 11, 2014 at 8:06 PM, Rich Brown <richb.hanover@gmail.com> wrote:
> Hi Sebastian,
> 
> >>>     Well, that looks like a decent recommendation for the wiki. The SQM configuration page still needs to expose all three values, atm, ethernet, and  none so that people can actually change things...
> >>
> >> So two questions really:
> >>
> >> 1) (From my previous note) What’s the difference between the current “Ethernet” (for VDSL) and “None” link layer adaptations?
> >
> >       Currently, "none" completely disables the link layer adjustments, "ethernet" enables them, but will only use the overhead (unless you specify an tcMPU, but that is truly exotic).
> >
> >>
> >> 2) When we distinguish the Ethernet/VDSL case, I would really like to use a different name from “Ethernet” because it seems confusingly similar to  having a real Ethernet path/link (e.g., direct connection to internet backbone without any ADSL, cable modem, etc.)
> >
> >       On the one hand I agree, but the two options are called "ATM" (well for tc "adsl" is a valid alias for ATM) and "ethernet" if you pass them to tc (what we do), and I would really hate it to hide this under fancy names. I see no chance of renaming those options in tc, so we are sort of stuck with them and adding another layer of indirection seems too opaque to me. This is why I put some explanation behind the option names in the list box…
> 
> Now I see how it works. (I didn’t understand that “None” really meant NONE.) The following choices in the Link Layer Adaptation would have eased my confusion:
> 
> - ATM (almost every type of ADSL or DSL)
> - Ethernet with overhead
> - None (default)
> 
> Then the text can say:
> 
> —
> You must set the Link Layer Adaptation options so that CeroWrt can perform its best with video and audio chat, gaming, and other protocols that rely on short packets. The general rule for selecting the Link Layer Adaption is:
> 
> * If you use any kind of DSL/ADSL connection to the Internet (that is, if you get your internet service through the telephone line), you should choose the “ATM (almost every type of ADSL or DSL)" item. Set the Per-packet Overhead to 44.
> 
> * If you know you have a VDSL connection, you should choose “Ethernet with overhead" and set the Per-packet Overhead to 8.
> 
> * If you use Cable modem, Fiber, or another kind of connection to the Internet, you should choose “None (default)”. All other parameters will be ignored.
> 
> If you cannot tell what kind of link you have, first try using "None", then run the [[Quick Test for Bufferbloat]]. If the results are good, you’re done. Next,  try the ADSL/ATM choice, then the VDSL choice to see which performs best. Read the **Details** (below) to learn more about tuning the parameters for your link.
> —
> 
> Would that be better? Thanks.
> 
> Rich
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>