[Bloat] curious.....

[Bloat] curious.....

[Outback Dingo]

[-- Attachment #1: Type: text/plain, Size: 114 bytes --]

found this blog about bufferbloat..... and their answer.

http://blog.lxgr.net/posts/2013/01/28/my-openwrt-setup/

[-- Attachment #2: Type: text/html, Size: 234 bytes --]

Re: [Bloat] curious.....

[Kenyon Ralph]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

On 2013-12-03T13:40:00-0500, Outback Dingo <outbackdingo@gmail.com> wrote:
> found this blog about bufferbloat..... and their answer.
> 
> http://blog.lxgr.net/posts/2013/01/28/my-openwrt-setup/

OpenWrt's qos-scripts package uses fq_codel, so his manual approach is
not really necessary. Just install the qos-scripts package and
configure /etc/config/qos. Documentation is here, but site seems to be
down at the moment: http://wiki.openwrt.org/doc/uci/qos

-- 
Kenyon Ralph

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [Bloat] curious.....

[Outback Dingo]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

On Tue, Dec 3, 2013 at 5:25 PM, Kenyon Ralph <kenyon@kenyonralph.com> wrote:

> On 2013-12-03T13:40:00-0500, Outback Dingo <outbackdingo@gmail.com> wrote:
> > found this blog about bufferbloat..... and their answer.
> >
> > http://blog.lxgr.net/posts/2013/01/28/my-openwrt-setup/
>
> OpenWrt's qos-scripts package uses fq_codel, so his manual approach is
> not really necessary. Just install the qos-scripts package and
> configure /etc/config/qos. Documentation is here, but site seems to be
> down at the moment: http://wiki.openwrt.org/doc/uci/qos


Yeah i know how to achieve it, i was just thinking some of us might get a
chuckle from it. :) but thanks


>
>
> --
> Kenyon Ralph
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>
>

[-- Attachment #2: Type: text/html, Size: 1767 bytes --]

Re: [Bloat] curious.....

[Dave Taht]

On Tue, Dec 3, 2013 at 4:25 PM, Outback Dingo <outbackdingo@gmail.com> wrote:
>
>
>
> On Tue, Dec 3, 2013 at 5:25 PM, Kenyon Ralph <kenyon@kenyonralph.com> wrote:
>>
>> On 2013-12-03T13:40:00-0500, Outback Dingo <outbackdingo@gmail.com> wrote:
>> > found this blog about bufferbloat..... and their answer.
>> >
>> > http://blog.lxgr.net/posts/2013/01/28/my-openwrt-setup/
>>
>> OpenWrt's qos-scripts package uses fq_codel, so his manual approach is
>> not really necessary. Just install the qos-scripts package and
>> configure /etc/config/qos. Documentation is here, but site seems to be
>> down at the moment: http://wiki.openwrt.org/doc/uci/qos
>
>
> Yeah i know how to achieve it, i was just thinking some of us might get a
> chuckle from it. :) but thanks

It's simple and illustrates the idea, so I applaud the blog post. I
wish that the
constants were better explained.

I note that openwrt's qos-scripts still do not do ipv6 properly, and I think
cerowrt's aqm system is better in most respects, and it's easy to
include on an openwrt system.


>>
>>
>>
>> --
>> Kenyon Ralph
>>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
>
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Bloat] curious.....

[Juliusz Chroboczek]

> I note that openwrt's qos-scripts still do not do ipv6 properly, and I think
> cerowrt's aqm system is better in most respects, and it's easy to
> include on an openwrt system.

Perhaps you should push your system to OpenWRT?

-- Juliusz

Re: [Bloat] curious.....

[Dave Taht]

On Fri, Dec 6, 2013 at 9:19 AM, Juliusz Chroboczek
<jch@pps.univ-paris-diderot.fr> wrote:
>> I note that openwrt's qos-scripts still do not do ipv6 properly, and I think
>> cerowrt's aqm system is better in most respects, and it's easy to
>> include on an openwrt system.
>
> Perhaps you should push your system to OpenWRT?

There is still some work going on to streamline the gui. there are
less features in the aqm-scripts for
prioritizing packet types than qos-scripts. I just had to come up with
a way to disable it at high (> 80 mbit) rates on incoming traffic (not
enough cpu in cerowrt), so I'd like it to run faster, maybe using drr
in that case, or something like what free.fr uses...

And there are actually two aqm/packet scheduling shapers in there (a
simple 1 tier and a 3 tier one), and it supports a variety of
aqm/scheduling  algorithms, not just fq_codel, which is there for the
research but will confuse the end users... and I am not happy with
"aqm" at a word, because the packet scheduling part counts for a LOT,
so i'd rename it to sqm or someting like that.

So I'd argue it needs some love. And a solid set of requirements that
it meets before it is as standardized as, say, wondershaper is. And if
it got that standardized, I think it would run a heck of a lot faster
if it got poured into C.

However it is pretty stable and could definitely use more eyeballs,
and it works right with ipv6 and seems better than qos-scripts on most
benchmarks... so I'll ask the openwrt folk if they want it upstream.

In the interim, existing openwrt users can add ceropackages-3.3 into
their feeds.

And incorporate it into their build with

./scripts feeds install aqm-scripts luci-app-aqm



> -- Juliusz



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Bloat] curious.....

[Sebastian Moeller]

Hi Dave,


On Dec 6, 2013, at 19:15 , Dave Taht <dave.taht@gmail.com> wrote:

> On Fri, Dec 6, 2013 at 9:19 AM, Juliusz Chroboczek
> <jch@pps.univ-paris-diderot.fr> wrote:
>>> I note that openwrt's qos-scripts still do not do ipv6 properly, and I think
>>> cerowrt's aqm system is better in most respects, and it's easy to
>>> include on an openwrt system.
>> 
>> Perhaps you should push your system to OpenWRT?
> 
> There is still some work going on to streamline the gui. there are
> less features in the aqm-scripts for
> prioritizing packet types than qos-scripts.

	Do you think these additional features are essential? I assume that openwork would not immediately drop their QOS packet so "AQM/SQM" and QOS will live together for some time, so having slightly different features might still be okay.

> I just had to come up with
> a way to disable it at high (> 80 mbit) rates on incoming traffic (not
> enough cpu in cerowrt), so I'd like it to run faster, maybe using drr
> in that case, or something like what free.fr uses...
> 
> And there are actually two aqm/packet scheduling shapers in there (a
> simple 1 tier and a 3 tier one)

	What is your opinion, should we default to the single tier version or the 3 tier version?

> , and it supports a variety of
> aqm/scheduling  algorithms, not just fq_codel, which is there for the
> research but will confuse the end users…

	I guess it should be relatively easy to hide this under the control of a "I know what I am doing give me all the knobs to play with" checkbox with a scary name, say "research"… :)
	Question, I tried to send you a version of aqm.lua that hides the detailed link layer fields in the GUI if none is selected as link layer adaptation mechanism did that ever reach you (I have some issues with my mailer so it might not have made it out of my system for good)? But in the same vain, hiding "Queueing discipline" and "Queue setup script" should be simple...

> and I am not happy with
> "aqm" at a word, because the packet scheduling part counts for a LOT,
> so i'd rename it to sqm or someting like that.

	What about the long and verbose "Latency and Bandwidth Control" or something else that talks about the "benefit" to the user?

> 
> So I'd argue it needs some love. And a solid set of requirements that
> it meets before it is as standardized as, say, wondershaper is. And if
> it got that standardized, I think it would run a heck of a lot faster
> if it got poured into C.

	Just curious, isn't the kernel doing all the work for us? I ,foolishly?, assumed that the script is just configuring the kernel and since this is done rarely why would we bother about the run time...

> 
> However it is pretty stable and could definitely use more eyeballs,
> and it works right with ipv6 and seems better than qos-scripts on most
> benchmarks... so I'll ask the openwrt folk if they want it upstream.
> 
> In the interim, existing openwrt users can add ceropackages-3.3 into
> their feeds.
> 
> And incorporate it into their build with
> 
> ./scripts feeds install aqm-scripts luci-app-aqm

	Note: people should at least disable openwork's QOS before activating "AQM"; or should we teach AQM to either disable qos on activation or at least warn the user about a potential conflict. I assume people will want to compare the performance of both…

best
	Sebastian

> 
> 
> 
>> -- Juliusz
> 
> 
> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] curious.....

[Dave Taht]

I am here:

http://conferences.sigcomm.org/co-next/2013/program.html

this week, so expect emails from me to be sparse to non-existent.

On Sat, Dec 7, 2013 at 3:24 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
> Hi Dave,
>
>
> On Dec 6, 2013, at 19:15 , Dave Taht <dave.taht@gmail.com> wrote:
>
>> On Fri, Dec 6, 2013 at 9:19 AM, Juliusz Chroboczek
>> <jch@pps.univ-paris-diderot.fr> wrote:
>>>> I note that openwrt's qos-scripts still do not do ipv6 properly, and I think
>>>> cerowrt's aqm system is better in most respects, and it's easy to
>>>> include on an openwrt system.
>>>
>>> Perhaps you should push your system to OpenWRT?
>>
>> There is still some work going on to streamline the gui. there are
>> less features in the aqm-scripts for
>> prioritizing packet types than qos-scripts.
>
>         Do you think these additional features are essential? I assume that openwork would not immediately drop their QOS packet so "AQM/SQM" and QOS will live together for some time, so having slightly different features might still be okay.

Most people like knobs. I don't - most people also have a tendency to
turn up everything to 11. So it is perhaps too idealistic of me to try
and come up with something that handles the bulk of shaping needs with
no knobs whatsoever.

I would certainly like openwrt folk to fix qos-scripts to work right
with ipv6 in particular, but the connection tracking element is
required in the current qos-scripts, and substantial changes to how it
works would need to be made.

So offering sqm-scripts as an alternative in the main distro would be good,
especially for ipv6 users, but also because I do think it's generally better.


>> I just had to come up with
>> a way to disable it at high (> 80 mbit) rates on incoming traffic (not
>> enough cpu in cerowrt), so I'd like it to run faster, maybe using drr
>> in that case, or something like what free.fr uses...
>>
>> And there are actually two aqm/packet scheduling shapers in there (a
>> simple 1 tier and a 3 tier one)
>
>         What is your opinion, should we default to the single tier version or the 3 tier version?

I actually get such great results from simplest.qos on normal traffic,
*on systems running at 5Mbit+* even with bit torrent, that I'm tempted
to recommend that rather than the three tier system. But that breaks
with tradition too much, and ietfers
are off specifying 1024 different behaviors based on the 8 bits of the
tos field, so I fear that only 3 bins is controversial, and 1 bin,
anathema.

Below that prioritization is truly needed and helps, and there
certainly is a vague zone between 5 and 30MB on the traffic  types I
see in the home. So 3 bins
seems to continue to make sense in the general case.

Recently I had some correspondence with the person at free that
deployed fq_codel, who has deployed *at scale* a three tier system
similar to cero's but it runs at line rate w/adsl, and combines prio,
drr, and fq_codel. It's running on the "revolution V6" box. There was
an interesting short algo to increase target at bandwidths below
5mbit, which seems to make sense.

Does anyone here have free.fr access and can do a rrul measurement?

And in terms of research still ongoing I'd like (someone) to take a
hard look at gargoyles' "acc".

>> , and it supports a variety of
>> aqm/scheduling  algorithms, not just fq_codel, which is there for the
>> research but will confuse the end users…
>
>         I guess it should be relatively easy to hide this under the control of a "I know what I am doing give me all the knobs to play with" checkbox with a scary name, say "research"… :)

Sure. "Advanced".

>         Question, I tried to send you a version of aqm.lua that hides the detailed link layer fields in the GUI if none is selected as link layer adaptation mechanism did that ever reach you (I have some issues with my mailer so it might not have made it out of my system for good)? But in the same vain, hiding "Queueing discipline" and "Queue setup script" should be simple…

I saw it whiz by as I was doing a zillion other things that week.
Sorry! There were two other patches I really wanted in the that
release - the fix to TSQ and pie v4. The first was put into 3.10.23,
and the second hit netdev a few days ago.

I am limited to working on cero on sundays right now so I can
incorporate your latest patch if it arrives by sunday morning.

>
>> and I am not happy with
>> "aqm" at a word, because the packet scheduling part counts for a LOT,
>> so i'd rename it to sqm or someting like that.
>
>         What about the long and verbose "Latency and Bandwidth Control" or something else that talks about the "benefit" to the user?

That's better. I agree that makes sense to accentuate the positives.

"Latency and Bandwidth Optimization"? LBO?

>>
>> So I'd argue it needs some love. And a solid set of requirements that
>> it meets before it is as standardized as, say, wondershaper is. And if
>> it got that standardized, I think it would run a heck of a lot faster
>> if it got poured into C.
>
>         Just curious, isn't the kernel doing all the work for us? I ,foolishly?, assumed that the script is just configuring the kernel and since this is done rarely why would we bother about the run time…

HTB's code is highly flexible and highly inefficient .

tc is an incredibly flexible construction set for discs. This also
comes at the expense of inefficiency as well.

One of the points of fq_codel is it combines a drr-like algorithm with
the codel algorithm in the same code so, for example, we get buffer
sizes much more under control than if we were to use
tc to combine, say DRR + codel or DRR + pie (or QFQ + pie). And there
is a great deal of synergy between several characteristics of the two
basic algorithms.

So I keep hoping with a successor to aqm-scripts, call it "cake", that
we can make something even better - that scales to 200+mbit on current
hardware, uses less memory, groks diffserv and packet prioritization
sanely, makes your networks scale farther, and will make your teeth
whiter and improve your sex life.

>> However it is pretty stable and could definitely use more eyeballs,
>> and it works right with ipv6 and seems better than qos-scripts on most
>> benchmarks... so I'll ask the openwrt folk if they want it upstream.
>>
>> In the interim, existing openwrt users can add ceropackages-3.3 into
>> their feeds.
>>
>> And incorporate it into their build with
>>
>> ./scripts feeds install aqm-scripts luci-app-aqm
>
>         Note: people should at least disable openwork's QOS before activating "AQM"; or should we teach AQM to either disable qos on activation or at least warn the user about a potential conflict. I assume people will want to compare the performance of both…

Yes, the two packages should either be mutually exclusive or aware of
each other.

> best
>         Sebastian
>
>>
>>
>>
>>> -- Juliusz
>>
>>
>>
>> --
>> Dave Täht
>>
>> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Bloat] curious.....

[Sebastian Moeller]

Hi Dave,


On Dec 10, 2013, at 20:05 , Dave Taht <dave.taht@gmail.com> wrote:

> I am here:
> 
> http://conferences.sigcomm.org/co-next/2013/program.html
> 
> this week, so expect emails from me to be sparse to non-existent.

	Santa Barbara, how nice. Since we left California this spring I started to miss the Pacific; funny since we rarely went to the beach. Anyway I envy you :)

> 
> On Sat, Dec 7, 2013 at 3:24 AM, Sebastian Moeller <moeller0@gmx.de> wrote:
>> Hi Dave,
>> 
>> 
>> On Dec 6, 2013, at 19:15 , Dave Taht <dave.taht@gmail.com> wrote:
>> 
>>> On Fri, Dec 6, 2013 at 9:19 AM, Juliusz Chroboczek
>>> <jch@pps.univ-paris-diderot.fr> wrote:
>>>>> I note that openwrt's qos-scripts still do not do ipv6 properly, and I think
>>>>> cerowrt's aqm system is better in most respects, and it's easy to
>>>>> include on an openwrt system.
>>>> 
>>>> Perhaps you should push your system to OpenWRT?
>>> 
>>> There is still some work going on to streamline the gui. there are
>>> less features in the aqm-scripts for
>>> prioritizing packet types than qos-scripts.
>> 
>>        Do you think these additional features are essential? I assume that openwork would not immediately drop their QOS packet so "AQM/SQM" and QOS will live together for some time, so having slightly different features might still be okay.
> 
> Most people like knobs. I don't - most people also have a tendency to
> turn up everything to 11. So it is perhaps too idealistic of me to try
> and come up with something that handles the bulk of shaping needs with
> no knobs whatsoever.

	I guess I like knobs, they give me a warm fuzzy feeling; I do prefer not having to touch those knobs though :)

> 
> I would certainly like openwrt folk to fix qos-scripts to work right
> with ipv6 in particular, but the connection tracking element is
> required in the current qos-scripts, and substantial changes to how it
> works would need to be made.
> 
> So offering sqm-scripts as an alternative in the main distro would be good,
> especially for ipv6 users, but also because I do think it's generally better.

	At least in the past there were alternatives to QOS in OpenWRT, I am not sure whether these survived to recent restructuring though.

> 
> 
>>> I just had to come up with
>>> a way to disable it at high (> 80 mbit) rates on incoming traffic (not
>>> enough cpu in cerowrt), so I'd like it to run faster, maybe using drr
>>> in that case, or something like what free.fr uses...
>>> 
>>> And there are actually two aqm/packet scheduling shapers in there (a
>>> simple 1 tier and a 3 tier one)
>> 
>>        What is your opinion, should we default to the single tier version or the 3 tier version?
> 
> I actually get such great results from simplest.qos on normal traffic,
> *on systems running at 5Mbit+* even with bit torrent, that I'm tempted
> to recommend that rather than the three tier system.

	Ah, I guess I should actually try to test how this works with my VoIP. What I like about the 3 tier system is that if the need arises it will be relatively simple to control the latency of specific traffic by marking it appropriately. I guess it is a belts and suspender thing. I have a question though; with simplest.qos we still use HTB svn though there is no hierarchy, so maybe we could switch to TBF (assuming TBF is actually less computationally demanding than HTB)?

> But that breaks
> with tradition too much, and ietfers
> are off specifying 1024 different behaviors based on the 8 bits of the
> tos field,

	This sounds quite complicated, and I am unsure whether that is really required for a home router :)

> so I fear that only 3 bins is controversial, and 1 bin,
> anathema.
> 
> Below that prioritization is truly needed and helps, and there
> certainly is a vague zone between 5 and 30MB on the traffic  types I
> see in the home. So 3 bins
> seems to continue to make sense in the general case.

	Ah, okay, so would you agree that simple.qos then should be the default?

> 
> Recently I had some correspondence with the person at free that
> deployed fq_codel, who has deployed *at scale* a three tier system
> similar to cero's but it runs at line rate w/adsl, and combines prio,
> drr, and fq_codel. It's running on the "revolution V6" box. There was
> an interesting short algo to increase target at bandwidths below
> 5mbit, which seems to make sense.

	To allow at least a few packets worth of queueing?

> 
> Does anyone here have free.fr access and can do a rrul measurement?

	Sorry wrong country, I have to pass.

> 
> And in terms of research still ongoing I'd like (someone) to take a
> hard look at gargoyles' "acc".
> 
>>> , and it supports a variety of
>>> aqm/scheduling  algorithms, not just fq_codel, which is there for the
>>> research but will confuse the end users…
>> 
>>        I guess it should be relatively easy to hide this under the control of a "I know what I am doing give me all the knobs to play with" checkbox with a scary name, say "research"… :)
> 
> Sure. "Advanced".

	Done, okay, this was easy :) Since I am still on 3.10.18-1 I will first upgrade to the most recent one to see the changes you made to the AQM code… Also this will allow me to test my hypothesis about sysupgrade being affected by mount-utils. 

> 
>>        Question, I tried to send you a version of aqm.lua that hides the detailed link layer fields in the GUI if none is selected as link layer adaptation mechanism did that ever reach you (I have some issues with my mailer so it might not have made it out of my system for good)? But in the same vain, hiding "Queueing discipline" and "Queue setup script" should be simple…
> 
> I saw it whiz by as I was doing a zillion other things that week.
> Sorry! There were two other patches I really wanted in the that
> release - the fix to TSQ and pie v4. The first was put into 3.10.23,
> and the second hit netdev a few days ago.

	If I understand it correctly the TSQ patch should only affect traffic that terminates at the router; so it might affect netsurf sessions to the router but should not affect normal routing, or am I wrong here?


> 
> I am limited to working on cero on sundays right now so I can
> incorporate your latest patch if it arrives by sunday morning.

	I should be able to pull this off, even though I am under a deadline right now.

> 
>> 
>>> and I am not happy with
>>> "aqm" at a word, because the packet scheduling part counts for a LOT,
>>> so i'd rename it to sqm or someting like that.
>> 
>>        What about the long and verbose "Latency and Bandwidth Control" or something else that talks about the "benefit" to the user?
> 
> That's better. I agree that makes sense to accentuate the positives.
> 
> "Latency and Bandwidth Optimization"? LBO?

	Do you want to rename every occurrence of AQM or would you be happy with just changing the labels on the webpage? Since that is easy I will fold this into my modification...

> 
>>> 
>>> So I'd argue it needs some love. And a solid set of requirements that
>>> it meets before it is as standardized as, say, wondershaper is. And if
>>> it got that standardized, I think it would run a heck of a lot faster
>>> if it got poured into C.
>> 
>>        Just curious, isn't the kernel doing all the work for us? I ,foolishly?, assumed that the script is just configuring the kernel and since this is done rarely why would we bother about the run time…
> 
> HTB's code is highly flexible and highly inefficient .
> 
> tc is an incredibly flexible construction set for discs. This also
> comes at the expense of inefficiency as well.

	Yeah, but we only run tc while setting things up and even if it would take a minute we would not care?

> 
> One of the points of fq_codel is it combines a drr-like algorithm with
> the codel algorithm in the same code so, for example, we get buffer
> sizes much more under control than if we were to use
> tc to combine, say DRR + codel or DRR + pie (or QFQ + pie). And there
> is a great deal of synergy between several characteristics of the two
> basic algorithms.
> 
> So I keep hoping with a successor to aqm-scripts, call it "cake", that
> we can make something even better - that scales to 200+mbit on current
> hardware, uses less memory, groks diffserv and packet prioritization
> sanely, makes your networks scale farther, and will make your teeth
> whiter and improve your sex life.

	Ah, I see, also if included in the kernel this actually would have a fighting chance to make it into the firmware of commercial routers and help people who have no time/fun to tinker with things.

> 
>>> However it is pretty stable and could definitely use more eyeballs,
>>> and it works right with ipv6 and seems better than qos-scripts on most
>>> benchmarks... so I'll ask the openwrt folk if they want it upstream.
>>> 
>>> In the interim, existing openwrt users can add ceropackages-3.3 into
>>> their feeds.
>>> 
>>> And incorporate it into their build with
>>> 
>>> ./scripts feeds install aqm-scripts luci-app-aqm
>> 
>>        Note: people should at least disable openwork's QOS before activating "AQM"; or should we teach AQM to either disable qos on activation or at least warn the user about a potential conflict. I assume people will want to compare the performance of both…
> 
> Yes, the two packages should either be mutually exclusive or aware of
> each other.

	Mmmh, I see qos-scripts is still part of cero's packages, I guess I will try figuring out whether/how that would work.

> 
>> best
>>        Sebastian
>> 
>>> 
>>> 
>>> 
>>>> -- Juliusz
>>> 
>>> 
>>> 
>>> --
>>> Dave Täht
>>> 
>>> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html
>>> _______________________________________________
>>> Bloat mailing list
>>> Bloat@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/bloat
>> 
> 
> 
> 
> -- 
> Dave Täht
> 
> Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Bloat] curious.....

[Juliusz Chroboczek]

> > Perhaps you should push your system to OpenWRT?

> There is still some work going on to streamline the gui.

Fair enough.  That's important.

> there are less features in the aqm-scripts for prioritizing packet
> types than qos-scripts.

I wouldn't bother much with that.  The promise of fq_codel is that we
can get rid of our prioritising hacks -- if we need that kind of
features, then fq_codel has failed.

> I just had to come up with a way to disable it at high (> 80 mbit)
> rates on incoming traffic (not enough cpu in cerowrt),

I wouldn't bother with that either.  120 Mbit/s is the highest rate you
can get in Europe as far as I can tell, so being able to push 80 Mbit/s
on a four year old router is fine (as long as you're careful to avoid
shaping traffic between LAN and WLAN -- I certainly wouldn't want
backing up my laptop to be capped at 80 Mbit/s).

> so I'd like it to run faster, maybe using drr in that case, or
> something like what free.fr uses...

What are they using?

> And there are actually two aqm/packet scheduling shapers in there (a
> simple 1 tier and a 3 tier one),

Remove the non-default features.  Be a man, Dave, dump it all.

> In the interim, existing openwrt users can add ceropackages-3.3 into
> their feeds.

They won't.  If you want to have an effect on the world, you need to
push it into the default OpenWRT scripts.

-- Juliusz

Re: [Bloat] curious.....

[Steinar H. Gunderson]

On Sat, Dec 07, 2013 at 01:59:04PM +0100, Juliusz Chroboczek wrote:
> I wouldn't bother with that either.  120 Mbit/s is the highest rate you
> can get in Europe as far as I can tell, so being able to push 80 Mbit/s
> on a four year old router is fine (as long as you're careful to avoid
> shaping traffic between LAN and WLAN -- I certainly wouldn't want
> backing up my laptop to be capped at 80 Mbit/s).

FYI: Norway has at least two entirely distinct ISPs that offer 200 Mbit/sec
or more. Switzerland has at least two ISPs that offer 150 Mbit/sec or more.
(All four examples are to private end users; of course, availability varies
with where you live.)

/* Steinar */
-- 
Homepage: http://www.sesse.net/

Re: [Bloat] curious.....

[Mikael Abrahamsson]

On Sun, 8 Dec 2013, Steinar H. Gunderson wrote:

> FYI: Norway has at least two entirely distinct ISPs that offer 200 
> Mbit/sec or more. Switzerland has at least two ISPs that offer 150 
> Mbit/sec or more. (All four examples are to private end users; of 
> course, availability varies with where you live.)

Yeah, Sweden has a bunch of them as well, 1000/100 is quite common, 
1000/1000 can be had, there is even a small ISP that offers 10000/1000, 
but that's mostly a marketing thing, they don't have any customers willing 
to pay 400 USD per month for that service.

The comment about the WNDR3800 not being able to push this is of course 
relevant, so I guess we need a better platform if we want to do testing 
for these higher speeds.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: [Bloat] curious.....

[Mark Constable]

On 12/08/2013 03:24 PM, Mikael Abrahamsson wrote:
> The comment about the WNDR3800 not being able to push this is of course
>  relevant, so I guess we need a better platform if we want to do testing
>  for these higher speeds.

Not cheap at $307 delivered (to AU) but ready to use straight away...

http://utilite-computer.com/web/utilite-pro-specifications

tl;dr

Freescale i.MX6 quad-core Cortex-A9 @ 1.2GHz
2GB DDR3-1066 (soldered on-board)
SATA SSD 32GB + Micro-SD socket
Two 1000 BaseT Ethernet ports
802.11b/g/n Wi-Fi, single antenna
Bluetooth 3.0

Re: [Bloat] curious.....

[Outback Dingo]

[-- Attachment #1: Type: text/plain, Size: 1002 bytes --]

On Sun, Dec 8, 2013 at 6:00 AM, Mark Constable <markc@renta.net> wrote:

> On 12/08/2013 03:24 PM, Mikael Abrahamsson wrote:
>
>> The comment about the WNDR3800 not being able to push this is of course
>>  relevant, so I guess we need a better platform if we want to do testing
>>  for these higher speeds.
>>
>
> Not cheap at $307 delivered (to AU) but ready to use straight away...
>
> http://utilite-computer.com/web/utilite-pro-specifications
>
> tl;dr
>
> Freescale i.MX6 quad-core Cortex-A9 @ 1.2GHz
> 2GB DDR3-1066 (soldered on-board)
> SATA SSD 32GB + Micro-SD socket
> Two 1000 BaseT Ethernet ports
> 802.11b/g/n Wi-Fi, single antenna
> Bluetooth 3.0
>
>
>
Mark...... http://cubieboard.org/
Id look at the Cubieboard3: Cubietruck

you can source these in AU from littlebird also......
http://littlebirdelectronics.com/products/cubieboard





> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 2361 bytes --]

Re: [Bloat] curious.....

[Outback Dingo]

[-- Attachment #1: Type: text/plain, Size: 1190 bytes --]

On Sun, Dec 8, 2013 at 9:01 AM, Outback Dingo <outbackdingo@gmail.com>wrote:

>
>
>
> On Sun, Dec 8, 2013 at 6:00 AM, Mark Constable <markc@renta.net> wrote:
>
>> On 12/08/2013 03:24 PM, Mikael Abrahamsson wrote:
>>
>>> The comment about the WNDR3800 not being able to push this is of course
>>>  relevant, so I guess we need a better platform if we want to do testing
>>>  for these higher speeds.
>>>
>>
>> Not cheap at $307 delivered (to AU) but ready to use straight away...
>>
>> http://utilite-computer.com/web/utilite-pro-specifications
>>
>> tl;dr
>>
>> Freescale i.MX6 quad-core Cortex-A9 @ 1.2GHz
>> 2GB DDR3-1066 (soldered on-board)
>> SATA SSD 32GB + Micro-SD socket
>> Two 1000 BaseT Ethernet ports
>> 802.11b/g/n Wi-Fi, single antenna
>> Bluetooth 3.0
>>
>>
>>
> Mark...... http://cubieboard.org/
> Id look at the Cubieboard3: Cubietruck
>
> you can source these in AU from littlebird also......
> http://littlebirdelectronics.com/products/cubieboard
>
> and of course....  http://stores.ebay.com.au/Cubietruck
>
>
>
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
>>
>
>

[-- Attachment #2: Type: text/html, Size: 2948 bytes --]

Re: [Bloat] curious.....

[Mark Constable]

On 12/09/2013 12:01 AM, Outback Dingo wrote:
>> Not cheap at $307 delivered (to AU) but ready to use straight away...
>> http://utilite-computer.com/web/utilite-pro-specifications
>
> Mark...... http://cubieboard.org/
> Id look at the Cubieboard3: Cubietruck

The SATA connector is good but it's not quite "ready to use straight away".

Networking：10/100 ethernet, optional wifi

vs

Two 1000 BaseT Ethernet ports + 802.11b/g/n Wi-Fi

Re: [Bloat] curious.....

[Sebastian Moeller]

Hi Mark,

On Dec 8, 2013, at 17:44 , Mark Constable <markc@renta.net> wrote:

> On 12/09/2013 12:01 AM, Outback Dingo wrote:
>>> Not cheap at $307 delivered (to AU) but ready to use straight away...
>>> http://utilite-computer.com/web/utilite-pro-specifications
>> 
>> Mark...... http://cubieboard.org/
>> Id look at the Cubieboard3: Cubietruck
> 
> The SATA connector is good but it's not quite "ready to use straight away".
> 
> Networking：10/100 ethernet, optional wifi
> 
> vs
> 
> Two 1000 BaseT Ethernet ports + 802.11b/g/n Wi-Fi

	Well, that is a just one antenna in the 2.4GHz band. The WNDR sports concurrent 2.4 and 5GHz radios with two antennas each allowing MIMO and higher wireless bandwidth. At least for the wireless part both ARM boxes will not cut it as sole home router. They might be useful to use as a beefier internet router used with a wndr3[7|8]00 as the AP, but that is neither cheap nor power efficient. Now maybe there are decent debated USB multi antenna radios we could plug into such a device, but I am not hopeful...


Best Regards
	Sebastian

> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] curious.....

[Juliusz Chroboczek]

>> FYI: Norway has at least two entirely distinct ISPs that offer 200
>> Mbit/sec or more.

> Sweden has a bunch of them as well, 1000/100 is quite common,
> 1000/1000 can be had

I stand corrected.

> The comment about the WNDR3800 not being able to push this is of
> course relevant,

I think that the WNDR3800 should be able to push a gigabit, even with
NAT.  Dave's point, as far as I understand, is that the WNDR is not
able to push a gigabit *through fq_codel*, so he's looking for ways to
automatically replace fq_codel with something simpler at higher rates.

My point is that, Scandinavians excepted, most people don't have
multigigabit links at home, and so Dave's scripts should still be
useful even if they top at 80 Mbit.  Of course, (1) optimising fq_codel,
(2) making fq_codel do something fast at high data rates, and
(3) finding a successor to the WNDR3800 are all laudable goals.

-- Juliusz

Re: [Bloat] curious.....

[Jonathan Morton]

On 8 Dec, 2013, at 3:12 pm, Juliusz Chroboczek wrote:

> (2) making fq_codel do something fast at high data rates

At those sorts of data rates, the "usual" condition will be that the queue is empty *and* the output interface can immediately accept a packet.  With both of those conditions satisfied, I would assume that fq_codel can be bypassed with extremely little overhead.

 - Jonathan Morton

Re: [Bloat] curious.....

[Dave Taht]

On Sun, Dec 8, 2013 at 5:12 AM, Juliusz Chroboczek
<jch@pps.univ-paris-diderot.fr> wrote:
>>> FYI: Norway has at least two entirely distinct ISPs that offer 200
>>> Mbit/sec or more.
>
>> Sweden has a bunch of them as well, 1000/100 is quite common,
>> 1000/1000 can be had
>
> I stand corrected.
>
>> The comment about the WNDR3800 not being able to push this is of
>> course relevant,
>
> I think that the WNDR3800 should be able to push a gigabit, even with
> NAT.  Dave's point, as far as I understand, is that the WNDR is not

No, it tops out at about 330Mbit/sec currently with no firewall rules,
no nat.

> able to push a gigabit *through fq_codel*, so he's looking for ways to
> automatically replace fq_codel with something simpler at higher rates.

No. aqm-scripts can't push 110KB/sec through *HTB*. The rate limiter is the
biggest cpu pig in the system.

fq_codel itself barely shows up on benchmarks vs red OR pfifo_fast.
It's overhead is trivial. It's on by default on the ethernet and wifi
interfaces, so at bursty line rates you see it come into play. So htb
is the problem.

What I was thinking about was taking the 3 tier structure of the
current aqm-scripts system, turning it into something more drr-like
and adding support for rate limiting directly to it, which I hope
would give a larger outer limit for rate shaping than htb on this
hardware.

There are some tweakable knobs for htb that might help, notably
setting a larger quantum at higher rates.

I've had various variants of this idea running off and on for the last
year, (been calling it "cake"). I'm aware that google has something
similar already so I keep hoping they will release theirs...

I have some feedback on some of the lines of thought expressed on this
thread earlier but I have to jump on a train now. Notably it seems few
have internalized what the "sparse stream" optimization does for
things like voip.


> My point is that, Scandinavians excepted, most people don't have
> multigigabit links at home, and so Dave's scripts should still be
> useful even if they top at 80 Mbit.  Of course, (1) optimising fq_codel,
> (2) making fq_codel do something fast at high data rates, and
> (3) finding a successor to the WNDR3800 are all laudable goals.
>
> -- Juliusz
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Bloat] curious.....

[Juliusz Chroboczek]

> > I think that the WNDR3800 should be able to push a gigabit, even with NAT.
> 
> No, it tops out at about 330Mbit/sec currently with no firewall rules,
> no nat.

Full-size frames?  That sucks.  (And thanks for the hard data.)

> No. aqm-scripts can't push 110KB/sec through *HTB*.

Typo?

> The rate limiter is the biggest cpu pig in the system.

Have you spoken to Dumazet about that?

> What I was thinking about was taking the 3 tier structure of the
> current aqm-scripts system, turning it into something more drr-like
> and adding support for rate limiting directly to it

(Nods.)

> Notably it seems few have internalized what the "sparse stream"
> optimization does for things like voip.

We're waiting with held breath until you care to enlighten us ;-)

-- Juliusz

Re: [Bloat] curious.....

[Jonathan Morton]

On 8 Dec, 2013, at 7:56 pm, Juliusz Chroboczek wrote:

>> Notably it seems few have internalized what the "sparse stream"
>> optimization does for things like voip.
> 
> We're waiting with held breath until you care to enlighten us ;-)

I think I know what he's talking about.  This is where a packet arriving for an empty queue will be prioritised over packets waiting at the head of queues - but a flag will be set on that empty queue to ensure that it only happens once per cycle.  That reduces latency for stuff that isn't a bulk data flow.

The trouble is that if there's a sufficiently large number of congested flows (eg. BitTorrent) that VoIP's "fair share" is less than what it needs, that optimisation is disabled and VoIP stops working - unless it can detect and adapt to a lower bandwidth, or unless it is classified and managed as distinct from bulk data.

 - Jonathan Morton

Re: [Bloat] curious.....

[Aaron Wood]

[-- Attachment #1: Type: text/plain, Size: 682 bytes --]

> The comment about the WNDR3800 not being able to push this is of course
relevant, so I guess we need a better platform if we want to do testing for
these higher speeds.

One thing that I've noticed a number of newer chipsets doing is moving
"network acceleration" into hardware, as a way to get to gigabit speeds
with a <1GHz processor.

I've been leery of them for fear of how much the linux network stack has to
get hacked up to make that hardware work.  If it's just DMA, that's one
thing, but if it's implementing packet inspection in hardware...  Not so
much.  At least for a research/testing platform, a pure software solution
is definitely going to be more useful.

-Aaron

[-- Attachment #2: Type: text/html, Size: 925 bytes --]

Re: [Bloat] curious.....

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 1212 bytes --]

Yeah, Comcast upped my 50/10 service to 100/20 without asking early in the
year.

We definitely need a better platform; but we've known that for quite a
while....


On Sun, Dec 8, 2013 at 12:24 AM, Mikael Abrahamsson <swmike@swm.pp.se>wrote:

> On Sun, 8 Dec 2013, Steinar H. Gunderson wrote:
>
>  FYI: Norway has at least two entirely distinct ISPs that offer 200
>> Mbit/sec or more. Switzerland has at least two ISPs that offer 150 Mbit/sec
>> or more. (All four examples are to private end users; of course,
>> availability varies with where you live.)
>>
>
> Yeah, Sweden has a bunch of them as well, 1000/100 is quite common,
> 1000/1000 can be had, there is even a small ISP that offers 10000/1000, but
> that's mostly a marketing thing, they don't have any customers willing to
> pay 400 USD per month for that service.
>
> The comment about the WNDR3800 not being able to push this is of course
> relevant, so I guess we need a better platform if we want to do testing for
> these higher speeds.
>
> --
> Mikael Abrahamsson    email: swmike@swm.pp.se
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 2174 bytes --]

Re: [Bloat] curious.....

[Sebastian Moeller]

Hi Juliusz,


On Dec 7, 2013, at 13:59 , Juliusz Chroboczek <jch@pps.univ-paris-diderot.fr> wrote:

>>> Perhaps you should push your system to OpenWRT?
> 
>> There is still some work going on to streamline the gui.
> 
> Fair enough.  That's important.
> 
>> there are less features in the aqm-scripts for prioritizing packet
>> types than qos-scripts.
> 
> I wouldn't bother much with that.  The promise of fq_codel is that we
> can get rid of our prioritising hacks -- if we need that kind of
> features, then fq_codel has failed.

	Is that really true? given enough concurrent flows, critical flows might be delayed purely be the round robin scheduling of equally "worthy" packets in fq_codel, so some residual priory system might still make sense...

> 
>> I just had to come up with a way to disable it at high (> 80 mbit)
>> rates on incoming traffic (not enough cpu in cerowrt),
> 
> I wouldn't bother with that either.  120 Mbit/s is the highest rate you
> can get in Europe as far as I can tell, so being able to push 80 Mbit/s
> on a four year old router is fine (as long as you're careful to avoid
> shaping traffic between LAN and WLAN -- I certainly wouldn't want
> backing up my laptop to be capped at 80 Mbit/s).

	mmmh, currently in Germany residential fiber tops out at 200Mbit/s, cable at 150, and VDSL tops out at 50Mbit/s. Next year cable companies promised 200Mbit/s and the biggest VDSL provider promised100Mbit/sec (G.993.5 Vectoring). So, the wndr3[7,8]00 are getting a bit long in the tooth.
	What would be a reasonable replacement, anybody any good ideas?
	@Dave: for the one tier shaper, maybe using TBF instead of HTB will allow higher shaping rates? (I happily admit , I have no clue which part of HTB is the expensive one, the token bucket filter or the hierarchy.)

> 
>> so I'd like it to run faster, maybe using drr in that case, or
>> something like what free.fr uses...
> 
> What are they using?
> 
>> And there are actually two aqm/packet scheduling shapers in there (a
>> simple 1 tier and a 3 tier one),
> 
> Remove the non-default features.  Be a man, Dave, dump it all.

	Being more cautious, not to say cowardly, I would opt for hiding the non-default features :) Kidding aside, what should be the default?

> 
>> In the interim, existing openwrt users can add ceropackages-3.3 into
>> their feeds.
> 
> They won't.  If you want to have an effect on the world, you need to
> push it into the default OpenWRT scripts.

Best Regards
	Sebastian


> 
> -- Juliusz
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] curious.....

[Juliusz Chroboczek]

>> The promise of fq_codel is that we can get rid of our prioritising
>> hacks -- if we need that kind of features, then fq_codel has
>> failed.

> Is that really true? given enough concurrent flows, critical flows
> might be delayed purely be the round robin scheduling of equally
> "worthy" packets in fq_codel

At 100 Mbit, one full-size Ethernet frame is 120us.  This means that
if you want your VoIP traffic to have less than 30ms delay, you should
in principle reach your deadline as long as you have fewer than 250
congestion-limited flows at a given time.

> so some residual priory system might still make sense...

For throughput-sharing reasons, perhaps.  For latency reasons, hopefully not.

-- Juliusz

Re: [Bloat] curious.....

[Sebastian Moeller]

Hi Juliusz,

On Dec 8, 2013, at 14:25 , Juliusz Chroboczek <jch@pps.univ-paris-diderot.fr> wrote:

>>> The promise of fq_codel is that we can get rid of our prioritising
>>> hacks -- if we need that kind of features, then fq_codel has
>>> failed.
> 
>> Is that really true? given enough concurrent flows, critical flows
>> might be delayed purely be the round robin scheduling of equally
>> "worthy" packets in fq_codel
> 
> At 100 Mbit, one full-size Ethernet frame is 120us.  This means that
> if you want your VoIP traffic to have less than 30ms delay, you should
> in principle reach your deadline as long as you have fewer than 250
> congestion-limited flows at a given time.

	Well, is 250 enough and are the 250 really realistic in a residential setting? Currently not doing much of anything my router has 142 active connections (according to conntrack) so 250 might be on the low size for a device that routes multiple devices. Plus, unfortunately, most residential internet connections are asymmetric, so on the upload will allow fewer congestion-limited concurrent flows before the round robin delay will impede the VOIP session. (In Germany residential VDSL with 100Mbit/s downlink will run at 40Mbit/s uplink, so hopefully not a big issue, but most cable providers keep the upload below 10Mbit/s, typically 5Mbit/s for 100Mbit/s download).  So we talk about an order of magnitude fewer flows required to make phone calls "interesting".
	So I still think that for VoIP prioritizing might still be required until supplied minimum bandwidth gets higher.

> 
>> so some residual priory system might still make sense...
> 
> For throughput-sharing reasons, perhaps.  For latency reasons, hopefully not.

	Even at 1000 symmetric I still think it would be a good idea to isolate really latency critical traffic from the rest, even if under normal circumstances there should be no problem, I guess a "better safe than sorry" approach. But, hey I do not do this for a living so I might be on the wrong track here.

best
	Sebastian

> 
> -- Juliusz

Re: [Bloat] curious.....

[Juliusz Chroboczek]

> > as long as you have fewer than 250 congestion-limited flows at
> > a given time.

> Currently not doing much of anything my router has 142 active
> connections (according to conntrack)

142 active conntrack entries, as opposed to congestion-limited flows.
Roughly speaking, an active conntrack entry is a flow that has sent at
least a packet within the last minute or so, while a congestion-limited
flow is one that sends data as fast as it reasonably can.  I certainly
hope you don't have 142 congestion-limited flows on an idle router.

> I still think that for VoIP prioritizing might still be required
> until supplied minimum bandwidth gets higher.

I agree, in the short term.  In the medium term, I'm optimistic we can
get rid of these nasty hacks, which is why I'd rather see Dave spend
his copious free time on something else.

> Even at 1000 symmetric I still think it would be a good idea to
> isolate really latency critical traffic from the rest,

It would certainly be good to have hard data rather than confronting
your intuitions with my back-of-the-envelope computations.  (Hint, hint.)

-- Juliusz

Re: [Bloat] curious.....

[Sebastian Moeller]

Hi Juliusz,

On Dec 8, 2013, at 18:47 , Juliusz Chroboczek <jch@pps.univ-paris-diderot.fr> wrote:

>>> as long as you have fewer than 250 congestion-limited flows at
>>> a given time.
> 
>> Currently not doing much of anything my router has 142 active
>> connections (according to conntrack)
> 
> 142 active conntrack entries, as opposed to congestion-limited flows.
> Roughly speaking, an active conntrack entry is a flow that has sent at
> least a packet within the last minute or so, while a congestion-limited
> flow is one that sends data as fast as it reasonably can.  I certainly
> hope you don't have 142 congestion-limited flows on an idle router.

	Well, I guess the crux of the problem is that even at 100Mbit/s  with 250 packets (according to your estimate) we are not in the clear as we can only round robin 250 equally weighted queues, and much less at say 10Mbit/s uplinks. So opening around 90 tabs in a browser increased the number of connections to ~2000, all of them new, so all of them will be expedited by fq_codel. So I am still not convinced that real time critical applications will work well if the bottleneck does not do priority management even at 100Mbit/s.

> 
>> I still think that for VoIP prioritizing might still be required
>> until supplied minimum bandwidth gets higher.
> 
> I agree, in the short term.  In the medium term, I'm optimistic we can
> get rid of these nasty hacks, which is why I'd rather see Dave spend
> his copious free time on something else.

	I fully agree about Dave's time :).
About prioritizing, I am less optimistic as I can not see how the system can behave well under extreme load; I for one would be quite unhappy if an device on my internal network could effectively DOS my ability to make phone calls (I switched to IP telephony so this theoretical issue has become practical for me).
	Or put it that way, I hope that the core internet is over-provided and will not cause congestion, at the same time I want to be able to max out my internet connection and still be able to phone.

> 
>> Even at 1000 symmetric I still think it would be a good idea to
>> isolate really latency critical traffic from the rest,
> 
> It would certainly be good to have hard data rather than confronting
> your intuitions with my back-of-the-envelope computations.  (Hint, hint.)

	I guess I will chime in again once I have symmetric 1000Mbit/s link then :) I guess until I have an internet connection I cannot easily saturate I will stubbornly stick to my conviction that some sort of hard polity system is needed, until convinced by data :)

best regards
	Sebastian

> 
> -- Juliusz

Re: [Bloat] curious.....

[Dan Siemon]

On Sun, 2013-12-08 at 20:02 +0100, Sebastian Moeller wrote:
> About prioritizing, I am less optimistic as I can not see how the system can behave well under extreme load; I for one would be quite unhappy if an device on my internal network could effectively DOS my ability to make phone calls (I switched to IP telephony so this theoretical issue has become practical for me).
> 	Or put it that way, I hope that the core internet is over-provided and will not cause congestion, at the same time I want to be able to max out my internet connection and still be able to phone.

Making it impossible for an internal host to DOS VoIP quality is one of
the reasons I've been playing with per host fairness in addition to
three priority levels (per host). The tc script is below with a link to
performance results.

http://git.coverfire.com/?p=linux-qos-scripts.git;a=blob;f=src-3tos.sh;hb=HEAD

Re: [Bloat] curious.....

[Stephen Hemminger]

On Sat, 21 Dec 2013 20:38:41 -0500
Dan Siemon <dan@coverfire.com> wrote:

> On Sun, 2013-12-08 at 20:02 +0100, Sebastian Moeller wrote:
> > About prioritizing, I am less optimistic as I can not see how the system can behave well under extreme load; I for one would be quite unhappy if an device on my internal network could effectively DOS my ability to make phone calls (I switched to IP telephony so this theoretical issue has become practical for me).
> > 	Or put it that way, I hope that the core internet is over-provided and will not cause congestion, at the same time I want to be able to max out my internet connection and still be able to phone.
> 
> Making it impossible for an internal host to DOS VoIP quality is one of
> the reasons I've been playing with per host fairness in addition to
> three priority levels (per host). The tc script is below with a link to
> performance results.
> 
> http://git.coverfire.com/?p=linux-qos-scripts.git;a=blob;f=src-3tos.sh;hb=HEAD
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Looks like wonder shaper. It is a fine idea if you know the number of hosts and flows
and traffic and upstream bandwidth. But if you don't then it gets to be a mess.

Re: [Bloat] curious.....

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 2792 bytes --]

Data point: Annex M ADSL2 can be approximated as 10M down, 2M up in
practice. Throw BitTorrent at that, and round-robin delay absolutely is
relevant. ADSL1 connections will be even more so. Not everyone lives in a
city in Scandinavia.

So a simple tiered scheme which can distinguish VoIP from BitTorrent and
both from general traffic, and applies fq_codel to each tier, is a good
idea.

- Jonathan Morton
 On 8 Dec 2013 20:49, "Sebastian Moeller" <moeller0@gmx.de> wrote:

> Hi Juliusz,
>
> On Dec 8, 2013, at 14:25 , Juliusz Chroboczek <
> jch@pps.univ-paris-diderot.fr> wrote:
>
> >>> The promise of fq_codel is that we can get rid of our prioritising
> >>> hacks -- if we need that kind of features, then fq_codel has
> >>> failed.
> >
> >> Is that really true? given enough concurrent flows, critical flows
> >> might be delayed purely be the round robin scheduling of equally
> >> "worthy" packets in fq_codel
> >
> > At 100 Mbit, one full-size Ethernet frame is 120us.  This means that
> > if you want your VoIP traffic to have less than 30ms delay, you should
> > in principle reach your deadline as long as you have fewer than 250
> > congestion-limited flows at a given time.
>
>         Well, is 250 enough and are the 250 really realistic in a
> residential setting? Currently not doing much of anything my router has 142
> active connections (according to conntrack) so 250 might be on the low size
> for a device that routes multiple devices. Plus, unfortunately, most
> residential internet connections are asymmetric, so on the upload will
> allow fewer congestion-limited concurrent flows before the round robin
> delay will impede the VOIP session. (In Germany residential VDSL with
> 100Mbit/s downlink will run at 40Mbit/s uplink, so hopefully not a big
> issue, but most cable providers keep the upload below 10Mbit/s, typically
> 5Mbit/s for 100Mbit/s download).  So we talk about an order of magnitude
> fewer flows required to make phone calls "interesting".
>         So I still think that for VoIP prioritizing might still be
> required until supplied minimum bandwidth gets higher.
>
> >
> >> so some residual priory system might still make sense...
> >
> > For throughput-sharing reasons, perhaps.  For latency reasons, hopefully
> not.
>
>         Even at 1000 symmetric I still think it would be a good idea to
> isolate really latency critical traffic from the rest, even if under normal
> circumstances there should be no problem, I guess a "better safe than
> sorry" approach. But, hey I do not do this for a living so I might be on
> the wrong track here.
>
> best
>         Sebastian
>
> >
> > -- Juliusz
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat
>

[-- Attachment #2: Type: text/html, Size: 3458 bytes --]

Re: [Bloat] curious.....

[Sebastian Moeller]

Hi Jonathan,


On Dec 8, 2013, at 20:01 , Jonathan Morton <chromatix99@gmail.com> wrote:

> Data point: Annex M ADSL2 can be approximated as 10M down, 2M up in practice. Throw BitTorrent at that, and round-robin delay absolutely is relevant. ADSL1 connections will be even more so. Not everyone lives in a city in Scandinavia.

	Sure, currently at adsl2+ 16M down 2.8M up, I feel the need for a prio system to keep the telephone separated from the rest.

> 
> So a simple tiered scheme which can distinguish VoIP from BitTorrent and both from general traffic, and applies fq_codel to each tier, is a good idea.

	So have a look at /usr/lib/aqm/simple.qos, I think Dave has that already prepared for you, all you need to do is make sure that VoIP and BitTorrent packets are properly DiffServ labeled. Now for VoIP that should work well, but for BitTorrent it would be nice if the router could preemptively label those packets. (I guess most torrent clients allow to control the number of flows and the consumed bandwidth, so properly configured torrents would not need any special care, but who can guarantee the "properly configured" part.) But I digress…

Best
	Sebastian

> 
> - Jonathan Morton
> On 8 Dec 2013 20:49, "Sebastian Moeller" <moeller0@gmx.de> wrote:
> Hi Juliusz,
> 
> On Dec 8, 2013, at 14:25 , Juliusz Chroboczek <jch@pps.univ-paris-diderot.fr> wrote:
> 
> >>> The promise of fq_codel is that we can get rid of our prioritising
> >>> hacks -- if we need that kind of features, then fq_codel has
> >>> failed.
> >
> >> Is that really true? given enough concurrent flows, critical flows
> >> might be delayed purely be the round robin scheduling of equally
> >> "worthy" packets in fq_codel
> >
> > At 100 Mbit, one full-size Ethernet frame is 120us.  This means that
> > if you want your VoIP traffic to have less than 30ms delay, you should
> > in principle reach your deadline as long as you have fewer than 250
> > congestion-limited flows at a given time.
> 
>         Well, is 250 enough and are the 250 really realistic in a residential setting? Currently not doing much of anything my router has 142 active connections (according to conntrack) so 250 might be on the low size for a device that routes multiple devices. Plus, unfortunately, most residential internet connections are asymmetric, so on the upload will allow fewer congestion-limited concurrent flows before the round robin delay will impede the VOIP session. (In Germany residential VDSL with 100Mbit/s downlink will run at 40Mbit/s uplink, so hopefully not a big issue, but most cable providers keep the upload below 10Mbit/s, typically 5Mbit/s for 100Mbit/s download).  So we talk about an order of magnitude fewer flows required to make phone calls "interesting".
>         So I still think that for VoIP prioritizing might still be required until supplied minimum bandwidth gets higher.
> 
> >
> >> so some residual priory system might still make sense...
> >
> > For throughput-sharing reasons, perhaps.  For latency reasons, hopefully not.
> 
>         Even at 1000 symmetric I still think it would be a good idea to isolate really latency critical traffic from the rest, even if under normal circumstances there should be no problem, I guess a "better safe than sorry" approach. But, hey I do not do this for a living so I might be on the wrong track here.
> 
> best
>         Sebastian
> 
> >
> > -- Juliusz
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] curious.....

[Neil Davies]

[-- Attachment #1: Type: text/plain, Size: 1020 bytes --]


On 8 Dec 2013, at 10:40, Sebastian Moeller <moeller0@gmx.de> wrote:
> ...
> 	Is that really true? given enough concurrent flows, critical flows might be delayed purely be the round robin scheduling of equally "worthy" packets in fq_codel, so some residual priory system might still make sense…
\x10

Sebastian - you are describing a property that we've measured in large scale (countrywide) deployed networks, it is also the underlying property that is causing mobile network operators issues with packet based networks as backhaul, as well as several other mission critical environments we've had dealings with.

When I pointed out that such service approaches will create extreme non-stationarity (in the particular case 2+ seconds) - the designers said - not their problem, not in their brief from management, they were told to bandwidth share.

It is not just that the 2+ seconds can exist, but that the overall system (under the right offered traffic pattern) WILL display that dynamic property.

Neil


[-- Attachment #2: Type: text/html, Size: 1515 bytes --]

Re: [Bloat] curious.....

[Hal Murray]

> 	Even at 1000 symmetric I still think it would be a good idea to isolate
> really latency critical traffic from the rest, even if under normal
> circumstances there should be no problem, I guess a "better safe than sorry"
> approach. But, hey I do not do this for a living so I might be on the wrong
> track here. 

The problem is who gets to decide what is latency critical and/or how to do 
it such that the bad-guys can't game the system.

Has somebody made a list of various interesting cases, and where they break?




-- 
These are my opinions.  I hate spam.

Re: [Bloat] curious.....

[Stephen Hemminger]

On Sun, 08 Dec 2013 12:41:52 -0800
Hal Murray <hmurray@megapathdsl.net> wrote:

> 
> > 	Even at 1000 symmetric I still think it would be a good idea to isolate
> > really latency critical traffic from the rest, even if under normal
> > circumstances there should be no problem, I guess a "better safe than sorry"
> > approach. But, hey I do not do this for a living so I might be on the wrong
> > track here. 
> 
> The problem is who gets to decide what is latency critical and/or how to do 
> it such that the bad-guys can't game the system.
> 
> Has somebody made a list of various interesting cases, and where they break?
> 
> 
> 
> 

That is why Diffserv only makes sense within a trusted domain. Most ISP's just ignore
DSCP.

Re: [Bloat] curious.....

[Sebastian Moeller]

Hi Hal,


On Dec 8, 2013, at 21:41 , Hal Murray <hmurray@megapathdsl.net> wrote:

> 
>> 	Even at 1000 symmetric I still think it would be a good idea to isolate
>> really latency critical traffic from the rest, even if under normal
>> circumstances there should be no problem, I guess a "better safe than sorry"
>> approach. But, hey I do not do this for a living so I might be on the wrong
>> track here. 
> 
> The problem is who gets to decide what is latency critical and/or how to do 
> it such that the bad-guys can't game the system.

	Ah, that is easy, in your home-router that is you :); no one else knows how much you value uninterrupted VoIP sessions. Now as always this is not perfect as you can not fully control the marking on the downlink, but in that case I still think it makes sense to mark on your router and shape the downlink, while not as good as the uplink shaping and marking it still improves things noticeably.

best
	Sebastian

> 
> Has somebody made a list of various interesting cases, and where they break?
> 
> 
> 
> 
> -- 
> These are my opinions.  I hate spam.
> 
> 
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat