[Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Dave Taht]

one of the things that makes me happy with all-up testing is that
occasionally after completely blowing up my own work, I get to
critique fresh work that isn't mine, in an area with which I have no
expertise, with gratitude that I don't have to figure out the answer.
:)

So I spent some time clicking wildly all over the AQM gui webpage to
see what I could break.

1) the aqm gui code doesn't work due to a bug at line 66.
sc:depends("advanced", "1").
sc has to be initialized first, which happens later in the file. Extra
line removed in ceropackages, committed, pushed, you will need to do a
pull. Merge failure?

2) it's not clear to me we have to support both the stab and
htb_private methods of fixing htb's linklayer. It was important that
these be fixed for everyone else that uses htb,
but is one of these is faster than the other? I seem to recall one was
a calculated value in the kernel, the other some sort of table. Does
this choice need to be made by the
user? The two variants benchmarked? Jesper?

3) Clicking "advanced configuration" on and off toggles display of the
qdisc and qdisc script, and twiddling with the linklayer value brings
up all the extra DSL detail. Yea!

... and I think I was wrong in mentally visualizing the thing

If these were made tabs [Basic, Queueing Discipline, Linklayer,
Priorities], there would be more room for explanatory text in
particular and better alignment with the
"look and feel" of the rest of the gui. Note that "priorities" is a
placeholder for somehow
bringing out something remotely similar to what openwrt's qos system
already does
and what AQM (ceroshaper? some other name is needed) does implicitly
with optimizing for dns and ntp.

ECN enablement should be brought out in "Queueing discipline" via the
ALLECN variable. It seems likely ALLECN needs to have 4 states rather
than 3, which needs to also be fixed in the scripts.

While I'm at it, perhaps having tabs for each physical interface is
not a horrible idea,
but I shudder to think of people rate-limiting their wifi in the hope
that that would help.

?

5) Adding a second interface shows @ge01 as an option, which isn't a
real interface, and se00 as an option and not the gw* or sw*
interfaces. Adding se00 with the default option
gives me an error

One or more required fields have no value!
One or more required fields have no value!
One or more required fields have no value!
One or more required fields have no value!

(and I'm pretty sure the aqm-scripts break even if this is correctly
written to the config file)

6) feel free to add your copyright to the code.  :)

I return now to figuring out why bringing up the wifi is so hosed. I
will probably be reverting the kernel, netifd, and other things, way,
way, way back to when they used to work.

-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Sebastian Moeller]

Hi Dave,


On Dec 14, 2013, at 07:26 , Dave Taht <dave.taht@gmail.com> wrote:

> one of the things that makes me happy with all-up testing is that
> occasionally after completely blowing up my own work, I get to
> critique fresh work that isn't mine, in an area with which I have no
> expertise, with gratitude that I don't have to figure out the answer.
> :)
> 
> So I spent some time clicking wildly all over the AQM gui webpage to
> see what I could break.
> 
> 1) the aqm gui code doesn't work due to a bug at line 66.
> sc:depends("advanced", "1").
> sc has to be initialized first, which happens later in the file. Extra
> line removed in ceropackages, committed, pushed, you will need to do a
> pull. Merge failure?

	Worse, process failure, instead of actually committing tested code I manually edited some changes ("that could not break") into the repository and pushed those without actually testing them. Will try to be more careful in the future
I put in an additional "s" so in stead of "sc:depends("advanced", "1")" it should have read "c:depends("advanced", "1")" as the current identifier in that section is c. I pulled your changes, made my edit and puhed it again.

> 
> 2) it's not clear to me we have to support both the stab and
> htb_private methods of fixing htb's linklayer. It was important that
> these be fixed for everyone else that uses htb,
> but is one of these is faster than the other?

	So, tc_stab is generic in that it should work with HFSC, HTB and TBF, while htb_private will only work with HTB (it seems TBF also has a private method but I have no clue whether that actually works, I just noticed that someone from huawei is posting changes to TBF on linux net-dev). My take is that we should just stick to stab so we can keep the same configuration fiels for most scripts people might come up with (thing free.qos without HTB). I have no idea which is faster though.

> I seem to recall one was
> a calculated value in the kernel, the other some sort of table.

	If I recall correctly HTB lost its internal tables to allow higher rates and/or precision; Jesper than fixed the htb_private method to account for the link layer on the fly. So currently HTB is more advanced than tc_stab, since HTB will allow arbitrarily large packets and still do the right thing, while tc_stab will either need humungous tables or will not work with jumbo packets and GRO. I think for shaping on a home router we could not care less. People who can afford such large packets and GRO on the router probably have bandwidths available that cerowrt does not handle well anyway (picture me heavily handwaving here).

> Does
> this choice need to be made by the
> user?

	Well, no the user should not care. We should make that decision for the user; then again unless we are able to constantly check both methods against each other one will bitrott, so maybe we should default to tc_stab and make htb_private an advanced configuration option? Also we should try to drape tc_stab into the future and teach it the same trick htb_private does (and then also fix the fact that tc_stab ignores the information we have about overhead). If no one beats me to it I will try to prepare my first patch to the kernel to fix this sometime next year; but I reallr really have no time for that in the near future, as I have to papers to write and grants to write as well as apply for a new position. (Also I am not too keen of getting a patch into the kernel, I just want this issue fixed; but since it looks this itches me most...)

> The two variants benchmarked? Jesper?
> 
> 3) Clicking "advanced configuration" on and off toggles display of the
> qdisc and qdisc script,

	Did it really? I think with your change only the qdisc
  script should have toggled (but I am no Luci expert).

> and twiddling with the linklayer value brings
> up all the extra DSL detail. Yea!
> 
> ... and I think I was wrong in mentally visualizing the thing
> 
> If these were made tabs [Basic, Queueing Discipline, Linklayer,
> Priorities], there would be more room for explanatory text in
> particular and better alignment with the
> "look and feel" of the rest of the gui.

	I agree, then we would not need to hide anything, and could bring on more options like "interval" and "target" (and/or some of the pie details)

> Note that "priorities" is a
> placeholder for somehow
> bringing out something remotely similar to what openwrt's qos system
> already does
> and what AQM (ceroshaper? some other name is needed) does implicitly
> with optimizing for dns and ntp.

	So a way to place different packets into the different priority bands (in simple.qos)?

> 
> ECN enablement should be brought out in "Queueing discipline" via the
> ALLECN variable. It seems likely ALLECN needs to have 4 states rather
> than 3, which needs to also be fixed in the scripts.

	What about two fields then "ECN inbound" and "ECN outbound", same for states but easier to understand...

> 
> While I'm at it, perhaps having tabs for each physical interface is
> not a horrible idea,
> but I shudder to think of people rate-limiting their wifi in the hope
> that that would help.
> 
> ?

	I would rather propose, I have a look at disabling the ability to add and delete "Queues" sections, one should be enough.

> 
> 5) Adding a second interface shows @ge01 as an option, which isn't a
> real interface, and se00 as an option and not the gw* or sw*
> interfaces.

	If I press the add option I can see all interfaces?

> Adding se00 with the default option
> gives me an error
> 
> One or more required fields have no value!
> One or more required fields have no value!
> One or more required fields have no value!
> One or more required fields have no value!

	I assume you did not fill the bandwidth fields?

> 
> (and I'm pretty sure the aqm-scripts break even if this is correctly
> written to the config file)

	Yeah, I would say, let's try to disable this on more than one interface. This requirement seems advanced enough that requesters can be bothered to script it themselves :)

> 
> 6) feel free to add your copyright to the code.  :)

	Thanks, but basically I am still just only rearranging your code and Toke's so I am not sure there is anything to copy yet :)

> 
> I return now to figuring out why bringing up the wifi is so hosed. I
> will probably be reverting the kernel, netifd, and other things, way,
> way, way back to when they used to work.

	So for me things worked well with 3.10.21-1. I have no idea how much work it is for you to roll a new -2 version, but with a 3.10.21-2 including Sujith's atheros patch I would be happy to see whether the TX DMA failures are gone (assuming that those are connected to the disappearing radios). But just an offer, I would not be unhappy if abler hands than mine would fix this :).


Best Regards
	Sebastian
 

> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Sebastian Moeller]

Hi Dave,

On Dec 14, 2013, at 07:26 , Dave Taht <dave.taht@gmail.com> wrote:

> one of the things that makes me happy with all-up testing is that
> occasionally after completely blowing up my own work, I get to
> critique fresh work that isn't mine, in an area with which I have no
> expertise, with gratitude that I don't have to figure out the answer.
> :)
> 
> So I spent some time clicking wildly all over the AQM gui webpage to
> see what I could break.
> 
> 1) the aqm gui code doesn't work due to a bug at line 66.
> sc:depends("advanced", "1").
> sc has to be initialized first, which happens later in the file. Extra
> line removed in ceropackages, committed, pushed, you will need to do a
> pull. Merge failure?
> 
> 2) it's not clear to me we have to support both the stab and
> htb_private methods of fixing htb's linklayer. It was important that
> these be fixed for everyone else that uses htb,
> but is one of these is faster than the other? I seem to recall one was
> a calculated value in the kernel, the other some sort of table. Does
> this choice need to be made by the
> user? The two variants benchmarked? Jesper?
> 
> 3) Clicking "advanced configuration" on and off toggles display of the
> qdisc and qdisc script, and twiddling with the linklayer value brings
> up all the extra DSL detail. Yea!
> 
> ... and I think I was wrong in mentally visualizing the thing
> 
> If these were made tabs [Basic, Queueing Discipline, Linklayer,
> Priorities], there would be more room for explanatory text in
> particular and better alignment with the
> "look and feel" of the rest of the gui. Note that "priorities" is a
> placeholder for somehow
> bringing out something remotely similar to what openwrt's qos system
> already does
> and what AQM (ceroshaper? some other name is needed) does implicitly
> with optimizing for dns and ntp.
> 
> ECN enablement should be brought out in "Queueing discipline" via the
> ALLECN variable. It seems likely ALLECN needs to have 4 states rather
> than 3, which needs to also be fixed in the scripts.
> 
> While I'm at it, perhaps having tabs for each physical interface is
> not a horrible idea,
> but I shudder to think of people rate-limiting their wifi in the hope
> that that would help.
> 
> ?
> 
> 5) Adding a second interface shows @ge01 as an option, which isn't a
> real interface, and se00 as an option and not the gw* or sw*
> interfaces. Adding se00 with the default option
> gives me an error
> 
> One or more required fields have no value!
> One or more required fields have no value!
> One or more required fields have no value!
> One or more required fields have no value!
> 
> (and I'm pretty sure the aqm-scripts break even if this is correctly
> written to the config file)

	Fixed. I disabled the GUI option to show the add and delete buttons, so ceroshaper now is just configurable for one interface...


> 6) feel free to add your copyright to the code.  :)
> 
> I return now to figuring out why bringing up the wifi is so hosed. I
> will probably be reverting the kernel, netifd, and other things, way,
> way, way back to when they used to work.
> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Sebastian Moeller]

hI Dave hi list,


On Dec 14, 2013, at 07:26 , Dave Taht <dave.taht@gmail.com> wrote:

> one of the things that makes me happy with all-up testing is that
> occasionally after completely blowing up my own work, I get to
> critique fresh work that isn't mine, in an area with which I have no
> expertise, with gratitude that I don't have to figure out the answer.
> :)
> 
> So I spent some time clicking wildly all over the AQM gui webpage to
> see what I could break.
> 
> 1) the aqm gui code doesn't work due to a bug at line 66.
> sc:depends("advanced", "1").
> sc has to be initialized first, which happens later in the file. Extra
> line removed in ceropackages, committed, pushed, you will need to do a
> pull. Merge failure?
> 
> 2) it's not clear to me we have to support both the stab and
> htb_private methods of fixing htb's linklayer. It was important that
> these be fixed for everyone else that uses htb,
> but is one of these is faster than the other? I seem to recall one was
> a calculated value in the kernel, the other some sort of table. Does
> this choice need to be made by the
> user? The two variants benchmarked? Jesper?

	So I just went ahead and hid htb_private for the time being (by commenting out the definition in aqm.lua, can 
only be reenabled by editing, this is not ideal, but at lease confusing than the situation before)

> 
> 3) Clicking "advanced configuration" on and off toggles display of the
> qdisc and qdisc script, and twiddling with the linklayer value brings
> up all the extra DSL detail. Yea!
> 
> ... and I think I was wrong in mentally visualizing the thing
> 
> If these were made tabs [Basic, Queueing Discipline, Linklayer,
> Priorities], there would be more room for explanatory text in
> particular and better alignment with the
> "look and feel" of the rest of the gui. Note that "priorities" is a
> placeholder for somehow
> bringing out something remotely similar to what openwrt's qos system
> already does
> and what AQM (ceroshaper? some other name is needed) does implicitly
> with optimizing for dns and ntp.

	The tabs are in. Since "Priorities" would be empty it does not exist yet. Let's see how you like the rest...


> 
> ECN enablement should be brought out in "Queueing discipline" via the
> ALLECN variable. It seems likely ALLECN needs to have 4 states rather
> than 3, which needs to also be fixed in the scripts.
> 
> While I'm at it, perhaps having tabs for each physical interface is
> not a horrible idea,
> but I shudder to think of people rate-limiting their wifi in the hope
> that that would help.
> 
> ?
> 
> 5) Adding a second interface shows @ge01 as an option, which isn't a
> real interface, and se00 as an option and not the gw* or sw*
> interfaces. Adding se00 with the default option
> gives me an error
> 
> One or more required fields have no value!
> One or more required fields have no value!
> One or more required fields have no value!
> One or more required fields have no value!
> 
> (and I'm pretty sure the aqm-scripts break even if this is correctly
> written to the config file)
> 
> 6) feel free to add your copyright to the code.  :)
> 
> I return now to figuring out why bringing up the wifi is so hosed. I
> will probably be reverting the kernel, netifd, and other things, way,
> way, way back to when they used to work.
> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Sebastian Moeller]

Hi Dave,

On Dec 14, 2013, at 07:26 , Dave Taht <dave.taht@gmail.com> wrote:

> one of the things that makes me happy with all-up testing is that
> occasionally after completely blowing up my own work, I get to
> critique fresh work that isn't mine, in an area with which I have no
> expertise, with gratitude that I don't have to figure out the answer.
> :)
> 
> So I spent some time clicking wildly all over the AQM gui webpage to
> see what I could break.
> 
> 1) the aqm gui code doesn't work due to a bug at line 66.
> sc:depends("advanced", "1").
> sc has to be initialized first, which happens later in the file. Extra
> line removed in ceropackages, committed, pushed, you will need to do a
> pull. Merge failure?
> 
> 2) it's not clear to me we have to support both the stab and
> htb_private methods of fixing htb's linklayer. It was important that
> these be fixed for everyone else that uses htb,
> but is one of these is faster than the other? I seem to recall one was
> a calculated value in the kernel, the other some sort of table. Does
> this choice need to be made by the
> user? The two variants benchmarked? Jesper?
> 
> 3) Clicking "advanced configuration" on and off toggles display of the
> qdisc and qdisc script, and twiddling with the linklayer value brings
> up all the extra DSL detail. Yea!
> 
> ... and I think I was wrong in mentally visualizing the thing
> 
> If these were made tabs [Basic, Queueing Discipline, Linklayer,
> Priorities], there would be more room for explanatory text in
> particular and better alignment with the
> "look and feel" of the rest of the gui. Note that "priorities" is a
> placeholder for somehow
> bringing out something remotely similar to what openwrt's qos system
> already does
> and what AQM (ceroshaper? some other name is needed) does implicitly
> with optimizing for dns and ntp.
> 
> ECN enablement should be brought out in "Queueing discipline" via the
> ALLECN variable. It seems likely ALLECN needs to have 4 states rather
> than 3, which needs to also be fixed in the scripts.

	Done, that is all 4 states for inbound and outbound ECN can be configured via the GUI now.

> 
> While I'm at it, perhaps having tabs for each physical interface is
> not a horrible idea,
> but I shudder to think of people rate-limiting their wifi in the hope
> that that would help.
> 
> ?
> 
> 5) Adding a second interface shows @ge01 as an option, which isn't a
> real interface, and se00 as an option and not the gw* or sw*
> interfaces. Adding se00 with the default option
> gives me an error
> 
> One or more required fields have no value!
> One or more required fields have no value!
> One or more required fields have no value!
> One or more required fields have no value!
> 
> (and I'm pretty sure the aqm-scripts break even if this is correctly
> written to the config file)
> 
> 6) feel free to add your copyright to the code.  :)
> 
> I return now to figuring out why bringing up the wifi is so hosed. I
> will probably be reverting the kernel, netifd, and other things, way,
> way, way back to when they used to work.
> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Dave Taht]

On Sun, Dec 15, 2013 at 6:28 PM, Sebastian Moeller <moeller0@gmx.de> wrote:
> Hi Dave,
>
> On Dec 14, 2013, at 07:26 , Dave Taht <dave.taht@gmail.com> wrote:
>
>> one of the things that makes me happy with all-up testing is that
>> occasionally after completely blowing up my own work, I get to
>> critique fresh work that isn't mine, in an area with which I have no
>> expertise, with gratitude that I don't have to figure out the answer.
>> :)
>>
>> So I spent some time clicking wildly all over the AQM gui webpage to
>> see what I could break.
>>
>> 1) the aqm gui code doesn't work due to a bug at line 66.
>> sc:depends("advanced", "1").
>> sc has to be initialized first, which happens later in the file. Extra
>> line removed in ceropackages, committed, pushed, you will need to do a
>> pull. Merge failure?
>>
>> 2) it's not clear to me we have to support both the stab and
>> htb_private methods of fixing htb's linklayer. It was important that
>> these be fixed for everyone else that uses htb,
>> but is one of these is faster than the other? I seem to recall one was
>> a calculated value in the kernel, the other some sort of table. Does
>> this choice need to be made by the
>> user? The two variants benchmarked? Jesper?
>>
>> 3) Clicking "advanced configuration" on and off toggles display of the
>> qdisc and qdisc script, and twiddling with the linklayer value brings
>> up all the extra DSL detail. Yea!
>>
>> ... and I think I was wrong in mentally visualizing the thing
>>
>> If these were made tabs [Basic, Queueing Discipline, Linklayer,
>> Priorities], there would be more room for explanatory text in
>> particular and better alignment with the
>> "look and feel" of the rest of the gui. Note that "priorities" is a
>> placeholder for somehow
>> bringing out something remotely similar to what openwrt's qos system
>> already does
>> and what AQM (ceroshaper? some other name is needed) does implicitly
>> with optimizing for dns and ntp.
>>
>> ECN enablement should be brought out in "Queueing discipline" via the
>> ALLECN variable. It seems likely ALLECN needs to have 4 states rather
>> than 3, which needs to also be fixed in the scripts.
>
>         Done, that is all 4 states for inbound and outbound ECN can be configured via the GUI now.
>
>>
>> While I'm at it, perhaps having tabs for each physical interface is
>> not a horrible idea,
>> but I shudder to think of people rate-limiting their wifi in the hope
>> that that would help.
>>
>> ?
>>
>> 5) Adding a second interface shows @ge01 as an option, which isn't a
>> real interface, and se00 as an option and not the gw* or sw*
>> interfaces. Adding se00 with the default option
>> gives me an error
>>
>> One or more required fields have no value!
>> One or more required fields have no value!
>> One or more required fields have no value!
>> One or more required fields have no value!
>>
>> (and I'm pretty sure the aqm-scripts break even if this is correctly
>> written to the config file)
>>
>> 6) feel free to add your copyright to the code.  :)
>>
>> I return now to figuring out why bringing up the wifi is so hosed. I
>> will probably be reverting the kernel, netifd, and other things, way,
>> way, way back to when they used to work.
>>
>> --
>> Dave Täht
>>
>> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>

This patch didn't quite make 3.10.24-4. But nice work all the same.
I'll get it next time.

-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Jesper Dangaard Brouer]

On Sat, 14 Dec 2013 13:24:06 +0100 Sebastian Moeller <moeller0@gmx.de> wrote:

> On Dec 14, 2013, at 07:26 , Dave Taht <dave.taht@gmail.com> wrote:
> 
> > 
> > 2) it's not clear to me we have to support both the stab and
> > htb_private methods of fixing htb's linklayer. It was important that
> > these be fixed for everyone else that uses htb,
> > but is one of these is faster than the other?
> 
> 	So, tc_stab is generic in that it should work with HFSC, HTB
> and TBF, while htb_private will only work with HTB (it seems TBF also
> has a private method but I have no clue whether that actually works, I
> just noticed that someone from huawei is posting changes to TBF on
> linux net-dev). My take is that we should just stick to stab so we can
> keep the same configuration fiels for most scripts people might come up
> with (thing free.qos without HTB). I have no idea which is faster
> though.
> 
> > I seem to recall one was
> > a calculated value in the kernel, the other some sort of table.
> 
> 	If I recall correctly HTB lost its internal tables to allow
> higher rates and/or precision; Jesper than fixed the htb_private method
> to account for the link layer on the fly. So currently HTB is more
> advanced than tc_stab, since HTB will allow arbitrarily large packets
> and still do the right thing, while tc_stab will either need humungous
> tables or will not work with jumbo packets and GRO. I think for shaping
> on a home router we could not care less. People who can afford such
> large packets and GRO on the router probably have bandwidths available
> that cerowrt does not handle well anyway (picture me heavily handwaving
> here).
> 

Yes, with my recent fix, the HTB linklayer should be more precise than
stab (as HTB linklayer is no longer table based).

BUT as stab is more generic, e.g. works on all schedulers, we should
move towards that.  We should fix stab, in the kernel, to account for
stuff like GSO, and if needed we could easily do "on-the-fly" ATM cell
alignment (like the HTB linklayer patch).


> > Does
> > this choice need to be made by the
> > user?
> 
> 	Well, no the user should not care. We should make that
> decision for the user; then again unless we are able to constantly
> check both methods against each other one will bitrott, so maybe we
> should default to tc_stab and make htb_private an advanced
> configuration option? Also we should try to drape tc_stab into the
> future and teach it the same trick htb_private does (and then also fix
> the fact that tc_stab ignores the information we have about overhead).

What! - Does stab ignore the overhead?!?  The overhead for small (e.g.
ACK) packet is *very* important in the ATM/ADSL case, as the small
encap overhead cause the packet to use two ATM frames, which is
important to account for, because this represent a very big percentage
overhead (62%).  Over-more ADSL is especially prone to have many ACK
packets travel their upload link (due to the larger download link
capacity).


> If no one beats me to it I will try to prepare my first patch to the
> kernel to fix this sometime next year; but I reallr really have no time
> for that in the near future, as I have to papers to write and grants to
> write as well as apply for a new position. (Also I am not too keen of
> getting a patch into the kernel, I just want this issue fixed; but
> since it looks this itches me most...)
> 
> > The two variants benchmarked? Jesper?

I have actually not played with stab.

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Re: [Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Sebastian Moeller]

Hi Jesper,


On Dec 17, 2013, at 09:03 , Jesper Dangaard Brouer <brouer@redhat.com> wrote:

> 
> On Sat, 14 Dec 2013 13:24:06 +0100 Sebastian Moeller <moeller0@gmx.de> wrote:
> 
>> On Dec 14, 2013, at 07:26 , Dave Taht <dave.taht@gmail.com> wrote:
>> 
>>> 
>>> 2) it's not clear to me we have to support both the stab and
>>> htb_private methods of fixing htb's linklayer. It was important that
>>> these be fixed for everyone else that uses htb,
>>> but is one of these is faster than the other?
>> 
>> 	So, tc_stab is generic in that it should work with HFSC, HTB
>> and TBF, while htb_private will only work with HTB (it seems TBF also
>> has a private method but I have no clue whether that actually works, I
>> just noticed that someone from huawei is posting changes to TBF on
>> linux net-dev). My take is that we should just stick to stab so we can
>> keep the same configuration fiels for most scripts people might come up
>> with (thing free.qos without HTB). I have no idea which is faster
>> though.
>> 
>>> I seem to recall one was
>>> a calculated value in the kernel, the other some sort of table.
>> 
>> 	If I recall correctly HTB lost its internal tables to allow
>> higher rates and/or precision; Jesper than fixed the htb_private method
>> to account for the link layer on the fly. So currently HTB is more
>> advanced than tc_stab, since HTB will allow arbitrarily large packets
>> and still do the right thing, while tc_stab will either need humungous
>> tables or will not work with jumbo packets and GRO. I think for shaping
>> on a home router we could not care less. People who can afford such
>> large packets and GRO on the router probably have bandwidths available
>> that cerowrt does not handle well anyway (picture me heavily handwaving
>> here).
>> 
> 
> Yes, with my recent fix, the HTB linklayer should be more precise than
> stab (as HTB linklayer is no longer table based).

	But for DSL stab is precise unless MTU is larger than table size + overhead. stab modifies the apparent size of packets and that has no precision issue :) But I think stab should do the same you did with HTB, namely calculate the link layer adjustment on the fly.

> 
> BUT as stab is more generic, e.g. works on all schedulers, we should
> move towards that.  We should fix stab, in the kernel, to account for
> stuff like GSO, and if needed we could easily do "on-the-fly" ATM cell
> alignment (like the HTB linklayer patch).

	I agree, the account for GSO is i single line change, so should be easy, then the fly calculation is a tiny bit more involved. But in difference to HTB for stab the kernel knows the requested link layer so no heuristic is needed!

> 
> 
>>> Does
>>> this choice need to be made by the
>>> user?
>> 
>> 	Well, no the user should not care. We should make that
>> decision for the user; then again unless we are able to constantly
>> check both methods against each other one will bitrott, so maybe we
>> should default to tc_stab and make htb_private an advanced
>> configuration option? Also we should try to drape tc_stab into the
>> future and teach it the same trick htb_private does (and then also fix
>> the fact that tc_stab ignores the information we have about overhead).
> 
> What! - Does stab ignore the overhead?!?

	Oh, sorry for being imprecise here. Stab does take the overhead into account you put in the stab invocation just like HTB. It does not currently use the kernels information about GSO, so if handed a GSO packet it will not account for any ethernet header. For the non offload situation not a big deal, you just include the 14? bytes ethernet header in overhead, but hopelessly wrong in the GSO situation. Currently cerowrt does not use GSO so that is theoretical for now.

>  The overhead for small (e.g.
> ACK) packet is *very* important in the ATM/ADSL case, as the small
> encap overhead cause the packet to use two ATM frames, which is
> important to account for, because this represent a very big percentage
> overhead (62%).  Over-more ADSL is especially prone to have many ACK
> packets travel their upload link (due to the larger download link
> capacity).
> 
> 
>> If no one beats me to it I will try to prepare my first patch to the
>> kernel to fix this sometime next year; but I reallr really have no time
>> for that in the near future, as I have to papers to write and grants to
>> write as well as apply for a new position. (Also I am not too keen of
>> getting a patch into the kernel, I just want this issue fixed; but
>> since it looks this itches me most...)
>> 
>>> The two variants benchmarked? Jesper?
> 
> I have actually not played with stab.


Best Regards
	Sebastian
> 
> -- 
> Best regards,
>  Jesper Dangaard Brouer
>  MSc.CS, Sr. Network Kernel Developer at Red Hat
>  Author of http://www.iptv-analyzer.org
>  LinkedIn: http://www.linkedin.com/in/brouer

Re: [Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Fred Stratton]

In the new interface, htb_private is not an explicit option.

IF aqm  is enabled, and linklayer protocol is 'none', is htb_private 
implicitly chosen?


On 17/12/13 08:22, Sebastian Moeller wrote:
> Hi Jesper,
>
>
> On Dec 17, 2013, at 09:03 , Jesper Dangaard Brouer <brouer@redhat.com> wrote:
>
>> On Sat, 14 Dec 2013 13:24:06 +0100 Sebastian Moeller <moeller0@gmx.de> wrote:
>>
>>> On Dec 14, 2013, at 07:26 , Dave Taht <dave.taht@gmail.com> wrote:
>>>
>>>> 2) it's not clear to me we have to support both the stab and
>>>> htb_private methods of fixing htb's linklayer. It was important that
>>>> these be fixed for everyone else that uses htb,
>>>> but is one of these is faster than the other?
>>> 	So, tc_stab is generic in that it should work with HFSC, HTB
>>> and TBF, while htb_private will only work with HTB (it seems TBF also
>>> has a private method but I have no clue whether that actually works, I
>>> just noticed that someone from huawei is posting changes to TBF on
>>> linux net-dev). My take is that we should just stick to stab so we can
>>> keep the same configuration fiels for most scripts people might come up
>>> with (thing free.qos without HTB). I have no idea which is faster
>>> though.
>>>
>>>> I seem to recall one was
>>>> a calculated value in the kernel, the other some sort of table.
>>> 	If I recall correctly HTB lost its internal tables to allow
>>> higher rates and/or precision; Jesper than fixed the htb_private method
>>> to account for the link layer on the fly. So currently HTB is more
>>> advanced than tc_stab, since HTB will allow arbitrarily large packets
>>> and still do the right thing, while tc_stab will either need humungous
>>> tables or will not work with jumbo packets and GRO. I think for shaping
>>> on a home router we could not care less. People who can afford such
>>> large packets and GRO on the router probably have bandwidths available
>>> that cerowrt does not handle well anyway (picture me heavily handwaving
>>> here).
>>>
>> Yes, with my recent fix, the HTB linklayer should be more precise than
>> stab (as HTB linklayer is no longer table based).
> 	But for DSL stab is precise unless MTU is larger than table size + overhead. stab modifies the apparent size of packets and that has no precision issue :) But I think stab should do the same you did with HTB, namely calculate the link layer adjustment on the fly.
>
>> BUT as stab is more generic, e.g. works on all schedulers, we should
>> move towards that.  We should fix stab, in the kernel, to account for
>> stuff like GSO, and if needed we could easily do "on-the-fly" ATM cell
>> alignment (like the HTB linklayer patch).
> 	I agree, the account for GSO is i single line change, so should be easy, then the fly calculation is a tiny bit more involved. But in difference to HTB for stab the kernel knows the requested link layer so no heuristic is needed!
>
>>
>>>> Does
>>>> this choice need to be made by the
>>>> user?
>>> 	Well, no the user should not care. We should make that
>>> decision for the user; then again unless we are able to constantly
>>> check both methods against each other one will bitrott, so maybe we
>>> should default to tc_stab and make htb_private an advanced
>>> configuration option? Also we should try to drape tc_stab into the
>>> future and teach it the same trick htb_private does (and then also fix
>>> the fact that tc_stab ignores the information we have about overhead).
>> What! - Does stab ignore the overhead?!?
> 	Oh, sorry for being imprecise here. Stab does take the overhead into account you put in the stab invocation just like HTB. It does not currently use the kernels information about GSO, so if handed a GSO packet it will not account for any ethernet header. For the non offload situation not a big deal, you just include the 14? bytes ethernet header in overhead, but hopelessly wrong in the GSO situation. Currently cerowrt does not use GSO so that is theoretical for now.
>
>>   The overhead for small (e.g.
>> ACK) packet is *very* important in the ATM/ADSL case, as the small
>> encap overhead cause the packet to use two ATM frames, which is
>> important to account for, because this represent a very big percentage
>> overhead (62%).  Over-more ADSL is especially prone to have many ACK
>> packets travel their upload link (due to the larger download link
>> capacity).
>>
>>
>>> If no one beats me to it I will try to prepare my first patch to the
>>> kernel to fix this sometime next year; but I reallr really have no time
>>> for that in the near future, as I have to papers to write and grants to
>>> write as well as apply for a new position. (Also I am not too keen of
>>> getting a patch into the kernel, I just want this issue fixed; but
>>> since it looks this itches me most...)
>>>
>>>> The two variants benchmarked? Jesper?
>> I have actually not played with stab.
>
> Best Regards
> 	Sebastian
>> -- 
>> Best regards,
>>   Jesper Dangaard Brouer
>>   MSc.CS, Sr. Network Kernel Developer at Red Hat
>>   Author of http://www.iptv-analyzer.org
>>   LinkedIn: http://www.linkedin.com/in/brouer
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel

Re: [Cerowrt-devel] aqm gui feedback on cerowrt-3.10.24-1 for linklayer adaption

[Sebastian Moeller]

Hi Fred,


On Dec 17, 2013, at 12:39 , Fred Stratton <fredstratton@imap.cc> wrote:

> In the new interface, htb_private is not an explicit option.
> 
> IF aqm  is enabled, and linklayer protocol is 'none', is htb_private implicitly chosen?

	No, currently, you have no access to htb_private. I am still thinking about how to expose this option or if at all. The best way forward would be to teach the kernel's stab implementation the two tricks htb_private knows better and then forget about htb_private at all…
	BUt if you need htb_private, all you need to foo is to uncomment one line in aqm.lua.

best
	Sebastian

> 
> 
> On 17/12/13 08:22, Sebastian Moeller wrote:
>> Hi Jesper,
>> 
>> 
>> On Dec 17, 2013, at 09:03 , Jesper Dangaard Brouer <brouer@redhat.com> wrote:
>> 
>>> On Sat, 14 Dec 2013 13:24:06 +0100 Sebastian Moeller <moeller0@gmx.de> wrote:
>>> 
>>>> On Dec 14, 2013, at 07:26 , Dave Taht <dave.taht@gmail.com> wrote:
>>>> 
>>>>> 2) it's not clear to me we have to support both the stab and
>>>>> htb_private methods of fixing htb's linklayer. It was important that
>>>>> these be fixed for everyone else that uses htb,
>>>>> but is one of these is faster than the other?
>>>> 	So, tc_stab is generic in that it should work with HFSC, HTB
>>>> and TBF, while htb_private will only work with HTB (it seems TBF also
>>>> has a private method but I have no clue whether that actually works, I
>>>> just noticed that someone from huawei is posting changes to TBF on
>>>> linux net-dev). My take is that we should just stick to stab so we can
>>>> keep the same configuration fiels for most scripts people might come up
>>>> with (thing free.qos without HTB). I have no idea which is faster
>>>> though.
>>>> 
>>>>> I seem to recall one was
>>>>> a calculated value in the kernel, the other some sort of table.
>>>> 	If I recall correctly HTB lost its internal tables to allow
>>>> higher rates and/or precision; Jesper than fixed the htb_private method
>>>> to account for the link layer on the fly. So currently HTB is more
>>>> advanced than tc_stab, since HTB will allow arbitrarily large packets
>>>> and still do the right thing, while tc_stab will either need humungous
>>>> tables or will not work with jumbo packets and GRO. I think for shaping
>>>> on a home router we could not care less. People who can afford such
>>>> large packets and GRO on the router probably have bandwidths available
>>>> that cerowrt does not handle well anyway (picture me heavily handwaving
>>>> here).
>>>> 
>>> Yes, with my recent fix, the HTB linklayer should be more precise than
>>> stab (as HTB linklayer is no longer table based).
>> 	But for DSL stab is precise unless MTU is larger than table size + overhead. stab modifies the apparent size of packets and that has no precision issue :) But I think stab should do the same you did with HTB, namely calculate the link layer adjustment on the fly.
>> 
>>> BUT as stab is more generic, e.g. works on all schedulers, we should
>>> move towards that.  We should fix stab, in the kernel, to account for
>>> stuff like GSO, and if needed we could easily do "on-the-fly" ATM cell
>>> alignment (like the HTB linklayer patch).
>> 	I agree, the account for GSO is i single line change, so should be easy, then the fly calculation is a tiny bit more involved. But in difference to HTB for stab the kernel knows the requested link layer so no heuristic is needed!
>> 
>>> 
>>>>> Does
>>>>> this choice need to be made by the
>>>>> user?
>>>> 	Well, no the user should not care. We should make that
>>>> decision for the user; then again unless we are able to constantly
>>>> check both methods against each other one will bitrott, so maybe we
>>>> should default to tc_stab and make htb_private an advanced
>>>> configuration option? Also we should try to drape tc_stab into the
>>>> future and teach it the same trick htb_private does (and then also fix
>>>> the fact that tc_stab ignores the information we have about overhead).
>>> What! - Does stab ignore the overhead?!?
>> 	Oh, sorry for being imprecise here. Stab does take the overhead into account you put in the stab invocation just like HTB. It does not currently use the kernels information about GSO, so if handed a GSO packet it will not account for any ethernet header. For the non offload situation not a big deal, you just include the 14? bytes ethernet header in overhead, but hopelessly wrong in the GSO situation. Currently cerowrt does not use GSO so that is theoretical for now.
>> 
>>>  The overhead for small (e.g.
>>> ACK) packet is *very* important in the ATM/ADSL case, as the small
>>> encap overhead cause the packet to use two ATM frames, which is
>>> important to account for, because this represent a very big percentage
>>> overhead (62%).  Over-more ADSL is especially prone to have many ACK
>>> packets travel their upload link (due to the larger download link
>>> capacity).
>>> 
>>> 
>>>> If no one beats me to it I will try to prepare my first patch to the
>>>> kernel to fix this sometime next year; but I reallr really have no time
>>>> for that in the near future, as I have to papers to write and grants to
>>>> write as well as apply for a new position. (Also I am not too keen of
>>>> getting a patch into the kernel, I just want this issue fixed; but
>>>> since it looks this itches me most...)
>>>> 
>>>>> The two variants benchmarked? Jesper?
>>> I have actually not played with stab.
>> 
>> Best Regards
>> 	Sebastian
>>> -- 
>>> Best regards,
>>>  Jesper Dangaard Brouer
>>>  MSc.CS, Sr. Network Kernel Developer at Red Hat
>>>  Author of http://www.iptv-analyzer.org
>>>  LinkedIn: http://www.linkedin.com/in/brouer
>> _______________________________________________
>> Cerowrt-devel mailing list
>> Cerowrt-devel@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>