[Cerowrt-users] QOS settings vs speedboost and random bandwidth

[Cerowrt-users] QOS settings vs speedboost and random bandwidth

[Marc MERLIN]

I'm setting up QOS as explained in the FAQ
http://www.bufferbloat.net/projects/cerowrt/wiki/FAQ

Not surprisingly
speednest.net gives me 33Mbit/7.65Mbit while dslreports.com/speedtest gives
me a meager 16.2Mbit/2.5Mbit (for a line that's supposed to be 22Mbit/5Mbit).

I understand the queuing and that if I put too high a number, I'll fill
queues and end up with bad latency.

At the same time, am I correct that if I put numbers that are lower than
what I can really get when the line doesn't suck, I'll cap myself to those
numbers (here 16.2Mbit instead of 22Mbit or so)?

Is it also possible for linux/cerowrt to dynmically detect the queuing/delay
in my cable modem and dynamically adjust the QOS values to limit buffering
on the comcast side without throttling me down to lower than what the line
can do at any given time?

I'm also a bit concerned of what happens when comcasts ups my bandwidth
without telling me (they do that sometimes) and me never finding out if I
have a static QOS value in cerowrt.

Can others advise?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: [Cerowrt-users] QOS settings vs speedboost and random bandwidth

[Dave Taht]

there are not presently a lot of cerowrt-users members, so I'm cc-ing
cerowrt-devel.

On Mon, Nov 26, 2012 at 12:20 AM, Marc MERLIN <marc@merlins.org> wrote:
> I'm setting up QOS as explained in the FAQ
> http://www.bufferbloat.net/projects/cerowrt/wiki/FAQ
>
> Not surprisingly
> speednest.net gives me 33Mbit/7.65Mbit while dslreports.com/speedtest gives
> me a meager 16.2Mbit/2.5Mbit (for a line that's supposed to be 22Mbit/5Mbit).

Yes, you get a boost for about 10 seconds.

> I understand the queuing and that if I put too high a number, I'll fill
> queues and end up with bad latency.

yep.

>
> At the same time, am I correct that if I put numbers that are lower than
> what I can really get when the line doesn't suck, I'll cap myself to those
> numbers (here 16.2Mbit instead of 22Mbit or so)?


yep.

>
> Is it also possible for linux/cerowrt to dynmically detect the queuing/delay
> in my cable modem and dynamically adjust the QOS values to limit buffering
> on the comcast side without throttling me down to lower than what the line
> can do at any given time?

Nope. The right answer is to not have to rate shape in the router, but
to move the fq_codel algorithm into the cable modem, OR to have the
cable modem exert flow control like ADSL modems do. Then have the
smarter device (the router) then do smarter queuing.

There are other dynamic bandwidth effects on a cable network that
result in more widely varying backend bandwidth.

A little birdie tells me that the cable modem guys are on this...
speedboost is a good idea but the only way to not lose it is to move
the smarts around.

That said, it does seem possible to develop a speedboost emulator on
the router that would give you most of the benefit.


>
> I'm also a bit concerned of what happens when comcasts ups my bandwidth
> without telling me (they do that sometimes) and me never finding out if I
> have a static QOS value in cerowrt.
>
> Can others advise?
>
> Thanks,
> Marc
> --
> "A mouse is a device used to point at the xterm you want to type in" - A.S.R.
> Microsoft is to operating systems ....
>                                       .... what McDonalds is to gourmet cooking
> Home page: http://marc.merlins.org/
> _______________________________________________
> Cerowrt-users mailing list
> Cerowrt-users@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-users



-- 
Dave Täht

Fixing bufferbloat with cerowrt: http://www.teklibre.com/cerowrt/subscribe.html

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Michael Richardson]

>>>>> "Dave" == Dave Taht <dave.taht@gmail.com> writes:
    Dave> to move the fq_codel algorithm into the cable modem, OR to have the
    Dave> cable modem exert flow control like ADSL modems do. Then have the
    Dave> smarter device (the router) then do smarter queuing.

I was unaware that DSL models were doing this... is this with GbE
ethernet level XON/XOFF?   Should I assume that any DSL model with less
than a GbE sync doesn't/can't have it?  Any brands you'd like to
acknowledge?

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
   Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
	               then sign the petition. 

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 2483 bytes --]


You can use a small fraction of the capacity of the cable uplink path to measure its queueing delay dynamically, and when it gets longer than latency*"expected bitrate", reduce "expected bitrate".
 
You want to do this *as quickly as possible*, so what you do is insert a "link monitor" task in the driver that sends tiny probe packets addressed to the nearest "loopback point" you can find/create on the other side, and measure the RTT.  You can use, for example, the technique used by traceroute, which is to set the hop count to the smallest number that causes a return ICMP packet to be sent, and send one of those periodically.
 
I used this specific technique to cause my uplink queue to move back into my router, where I could manage it.  You can also use it for the downlink queue measurement, but it doesn't move the queue into the router smoothly, instead you have to drop/ECN-mark the IP frames coming in.
 
This can all be done between the IP layer and layer 2.  Since it exploits speedboost better, it might be worth adding as an option to cerowrt, so you don't have to set a speed limit explicitly when you have a single connection to the public Internet.
 
-----Original Message-----
From: "Michael Richardson" <mcr@sandelman.ca>
Sent: Monday, November 26, 2012 9:22am
To: cerowrt-users@lists.bufferbloat.net, cerowrt-devel@lists.bufferbloat.net
Subject: Re: [Cerowrt-devel] [Cerowrt-users] QOS settings vs speedboost and random bandwidth



>>>>> "Dave" == Dave Taht <dave.taht@gmail.com> writes:
 Dave> to move the fq_codel algorithm into the cable modem, OR to have the
 Dave> cable modem exert flow control like ADSL modems do. Then have the
 Dave> smarter device (the router) then do smarter queuing.

I was unaware that DSL models were doing this... is this with GbE
ethernet level XON/XOFF?   Should I assume that any DSL model with less
than a GbE sync doesn't/can't have it?  Any brands you'd like to
acknowledge?

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
 Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
 then sign the petition. 
_______________________________________________
Cerowrt-devel mailing list
Cerowrt-devel@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cerowrt-devel

[-- Attachment #2: Type: text/html, Size: 3059 bytes --]

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Michael Richardson]

[-- Attachment #1: Type: text/plain, Size: 2147 bytes --]


>>>>> "dpreed" == dpreed  <dpreed@reed.com> writes:
    dpreed> You can use a small fraction of the capacity of the cable
    dpreed> uplink path to measure its queueing delay dynamically, and
    dpreed> when it gets longer than latency*"expected bitrate", reduce
    dpreed> "expected bitrate". 
 
    dpreed> You want to do this *as quickly as possible*, so what you do
    dpreed> is insert a "link monitor" task in the driver that sends
    dpreed> tiny probe packets addressed to the nearest "loopback point"
    dpreed> you can find/create on the other side, and measure the RTT.
    dpreed> You can use, for example, the technique used by traceroute,
    dpreed> which is to set the hop count to the smallest number that
    dpreed> causes a return ICMP packet to be sent, and send one of
    dpreed> those periodically. 

As I understand it, you can do this with 802.1ag
  http://en.wikipedia.org/wiki/IEEE_802.1ag, 
with the Loop-back frames as well.

Whether or not any of this is enabled on typical broadband networks, I
have no idea.

    dpreed> I used this specific technique to cause my uplink queue to
    dpreed> move back into my router, where I could manage it.  You can
    dpreed> also use it for the downlink queue measurement, but it
    dpreed> doesn't move the queue into the router smoothly, instead you
    dpreed> have to drop/ECN-mark the IP frames coming in. 

    dpreed> This can all be done between the IP layer and layer 2.
    dpreed> Since it exploits speedboost better, it might be worth
    dpreed> adding as an option to cerowrt, so you don't have to set a
    dpreed> speed limit explicitly when you have a single connection to
    dpreed> the public Internet. 

wow, this would be awesome... code??
-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
   Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
	               then sign the petition. 


[-- Attachment #2: Type: application/pgp-signature, Size: 307 bytes --]

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 3307 bytes --]


Michael - My kludge code predated all of the "bloat" activity (I wrote it in 2002, when I had a Linux box as my home router, and I stopped using it because as a practical matter it was easier to use off-the-shelf home routers to support my family when I travel).  It was a complete kludge using a modified kernel, etc.  Not the right way to do it, and probably impossible to understand.
 
But I've thought about coding it again for cerowrt.  Where to modularly slot it in seems to be worth thinking about.  Perhaps in two key pieces: an iptables/xfilter module and a routing/traffic control module - with some direct interaction between the two using some appropriate intermodule bus/link/coordination link.
 
I'd be happy to think about defining the pieces, but I really don't have time to code it, given all the other stuff I've done.  I wonder if by putting it in these modules, one can use existing kernel APIs.
 
 
 
-----Original Message-----
From: "Michael Richardson" <mcr@sandelman.ca>
Sent: Monday, November 26, 2012 10:38am
To: dpreed@reed.com
Cc: cerowrt-users@lists.bufferbloat.net, cerowrt-devel@lists.bufferbloat.net
Subject: Re: [Cerowrt-devel] [Cerowrt-users] QOS settings vs speedboost and random bandwidth



>>>>> "dpreed" == dpreed  <dpreed@reed.com> writes:
 dpreed> You can use a small fraction of the capacity of the cable
 dpreed> uplink path to measure its queueing delay dynamically, and
 dpreed> when it gets longer than latency*"expected bitrate", reduce
 dpreed> "expected bitrate". 

 dpreed> You want to do this *as quickly as possible*, so what you do
 dpreed> is insert a "link monitor" task in the driver that sends
 dpreed> tiny probe packets addressed to the nearest "loopback point"
 dpreed> you can find/create on the other side, and measure the RTT.
 dpreed> You can use, for example, the technique used by traceroute,
 dpreed> which is to set the hop count to the smallest number that
 dpreed> causes a return ICMP packet to be sent, and send one of
 dpreed> those periodically. 

As I understand it, you can do this with 802.1ag
 http://en.wikipedia.org/wiki/IEEE_802.1ag, 
with the Loop-back frames as well.

Whether or not any of this is enabled on typical broadband networks, I
have no idea.

 dpreed> I used this specific technique to cause my uplink queue to
 dpreed> move back into my router, where I could manage it.  You can
 dpreed> also use it for the downlink queue measurement, but it
 dpreed> doesn't move the queue into the router smoothly, instead you
 dpreed> have to drop/ECN-mark the IP frames coming in. 

 dpreed> This can all be done between the IP layer and layer 2.
 dpreed> Since it exploits speedboost better, it might be worth
 dpreed> adding as an option to cerowrt, so you don't have to set a
 dpreed> speed limit explicitly when you have a single connection to
 dpreed> the public Internet. 

wow, this would be awesome... code??
-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
 Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
 then sign the petition. 


[-- Attachment #2: Type: text/html, Size: 4051 bytes --]

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Michael Richardson]

>>>>> "dpreed" == dpreed  <dpreed@reed.com> writes:
    dpreed> But I've thought about coding it again for cerowrt.  Where
    dpreed> to modularly slot it in seems to be worth thinking about.
    dpreed> Perhaps in two key pieces: an iptables/xfilter module and a
    dpreed> routing/traffic control module - with some direct
    dpreed> interaction between the two using some appropriate
    dpreed> intermodule bus/link/coordination link. 

So an uplink bitrate value with an easy to reach sysctl that
userspace can toggle?  It would be an enhancement to existing tc/qos code.
 
    dpreed> I'd be happy to think about defining the pieces, but I
    dpreed> really don't have time to code it, given all the other stuff
    dpreed> I've done.  I wonder if by putting it in these modules, one
    dpreed> can use existing kernel APIs. 

How precise timing do you think we need?

As I understand what you are saying, by periodically sending a few ICMP
messages (does it help if they are back to back?) and looking when they 
are returned, one can calculate the uplink bandwidth?

Or are you saying that we are measuring the point in uplink usage where
the latency begins to peak?

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
   Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
	               then sign the petition. 

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Marc MERLIN]

On Mon, Nov 26, 2012 at 01:11:47PM -0500, Michael Richardson wrote:
>     dpreed> I'd be happy to think about defining the pieces, but I
>     dpreed> really don't have time to code it, given all the other stuff
>     dpreed> I've done.  I wonder if by putting it in these modules, one
>     dpreed> can use existing kernel APIs. 
> 
> How precise timing do you think we need?
> 
> As I understand what you are saying, by periodically sending a few ICMP
> messages (does it help if they are back to back?) and looking when they 
> are returned, one can calculate the uplink bandwidth?

By the way, that was kind of what I had in mind.
Capping bandwidth on the router works great for DSL where line bandwidth is
known, but can't really be done right on a cable modem with effectively
variable bandwidth.

I kind of envisionned something where I'd enter the min and max bandwidth
expected, and just like an analog modem, the router would try higher speeds
first, look at how the latency is doing (vs line noise for a modem) and if
it's too high, tone it down.
Then, just like a modem, it could have some algorithm to try ramping back up
from time to time and see what happens (I see this over periods of minutes,
or even hours, not seconds). It wouldn't adapt super quickly, but at the
same time it would also do better than me changing the numbers in the web
interface every so often :)

"How the latency is doing" could indeed be done by sending a traceroute
like packet (TCP or UDP, whatever works) and an ICMP packet to the next hop
up that will reply. All 3 could be done maybe once a minute?

Now I'm not too sure how well that would work for seeing congestion on
uplink vs downlink, but if we know the data that is being shoved in each
direction and it's clearly asymetrical, it ought to be able to make a
reasonable guess that's correct most of the time, no?

Note, I'm not a TCP/IP congestion expert, sorry if what I just wrote isn't
possible for some reason that isn't obvious to me :)

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 4436 bytes --]


Hi Michael -
 
In specific, what my code did was this:
 
It observed the IPv4 headers of *large* TCP/IP datagrams going upstream, so that it could construct "no-op" "content-free" datagrams that would certainly pass muster through all the filters and be routed exactly the same as the TCP/IP datagrams  that were carrying large flows.  It would remember only the most recent one.
 
Every K bytes of upstream traffic (K chosen so that the overhead [= minimal TCP/IP datagram divided by K] is a tiny percentage) it would construct a NO-OP TCP/IP datagram that appears to be part of that flow (same source/dest addr/port info, and just for grins, a duplicate sequence number and no content bytes at all), and set its TTL to make it time out very close to the "other side" of the CMTS, and queue it normally.
 
The TTL expiration causes an ICMP packet to be sent back.   My code intercepts that packet based on its contents, and removes it as "handled" before it gets processed by the TCP/IP state machines.
 
The time between the queueing of the TCP/IP NO-OP and the return of the ICMP packet is a direct measure of the queueing delays through the cable modem and CMTS.  When this grows by around "1 full datagram" from its minimum, the upload queue is becoming congested, and it's time to stop sending content for a bit.  Immediately when content is held on the egress link into the cable modem from the router, we send another NO-OP with the short TTL, and as soon as its ICMP comes back, you  know the queue in the CMTS is drained, so you can resume sending into an empty CMTS, at a lower rate (you've just gotten a good estimate of the rate that you should reduce to, if you've been keeping track of how many bytes are flowing over the egress link.)
 
Symmetrically, you can periodically (less frequently) experiment with a possible rate *increase* by sending a small NO-OP packet immediately followed by a large/maximal sized NO-OP datagram, and using the "packet pair" concept to determine the bottleneck rate by measuring the time between ICMP responses.  The time between the ICMP responses is an estimator of the achievable peak rate through the upstream path.
 
This assumes that the downstream (incoming) path is uncongested.   But you can elaborate this scheme further.
 
The goal of the "tcptraceroute" method is to get a "loopback" that follows the same path as an existing TCP connection, in order to get timing right.
 
If options exist to get intermediate timestamps on a route, you can also use similar techniques under TCP with the "NO-OP" datagram technique.
 
-----Original Message-----
From: "Michael Richardson" <mcr@sandelman.ca>
Sent: Monday, November 26, 2012 1:11pm
To: dpreed@reed.com
Cc: cerowrt-users@lists.bufferbloat.net, cerowrt-devel@lists.bufferbloat.net
Subject: Re: [Cerowrt-devel] [Cerowrt-users] QOS settings vs speedboost and random bandwidth



>>>>> "dpreed" == dpreed  <dpreed@reed.com> writes:
 dpreed> But I've thought about coding it again for cerowrt.  Where
 dpreed> to modularly slot it in seems to be worth thinking about.
 dpreed> Perhaps in two key pieces: an iptables/xfilter module and a
 dpreed> routing/traffic control module - with some direct
 dpreed> interaction between the two using some appropriate
 dpreed> intermodule bus/link/coordination link. 

So an uplink bitrate value with an easy to reach sysctl that
userspace can toggle?  It would be an enhancement to existing tc/qos code.

 dpreed> I'd be happy to think about defining the pieces, but I
 dpreed> really don't have time to code it, given all the other stuff
 dpreed> I've done.  I wonder if by putting it in these modules, one
 dpreed> can use existing kernel APIs. 

How precise timing do you think we need?

As I understand what you are saying, by periodically sending a few ICMP
messages (does it help if they are back to back?) and looking when they 
are returned, one can calculate the uplink bandwidth?

Or are you saying that we are measuring the point in uplink usage where
the latency begins to peak?

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
 Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
 then sign the petition.

[-- Attachment #2: Type: text/html, Size: 5566 bytes --]

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Michael Richardson]

>>>>> "dpreed" == dpreed  <dpreed@reed.com> writes:
    dpreed> It observed the IPv4 headers of *large* TCP/IP datagrams
    dpreed> going upstream, so that it could construct "no-op"
    dpreed> "content-free" datagrams that would certainly pass muster
    dpreed> through all the filters and be routed exactly the same as
    dpreed> the TCP/IP datagrams  that were carrying large flows.  It
    dpreed> would remember only the most recent one. 

I don't know that you need to be so precise in creating the packet, but
I guess the point is not just the ACLs, but also any traffic shapers?

    dpreed> The TTL expiration causes an ICMP packet to be sent back.
    dpreed> My code intercepts that packet based on its contents, and
    dpreed> removes it as "handled" before it gets processed by the
    dpreed> TCP/IP state machines. 

This is perhaps the biggest problem with this method... having to remove
the magic ICMP so that it does no harm.  Without this requirement, it
could be done entirely in userspace I think.

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
   Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
	               then sign the petition. 

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[dpreed]

[-- Attachment #1: Type: text/plain, Size: 2624 bytes --]


I think you can intercept and drop the ICMP packet from userspace as well, if you have the right modules installed in iptables/Xtables.  But I haven't looked closely lately  (I just patched the kernel code in a kernel that probably predated iptables itself).  Probably need "root", but on the router itself, you have root.
 
This CMTS-queue-management is a router function anyway, for the router adjacent to the cable modem/CMTS.  Using it from ordinary clients and servers probably just generates randomness.
 
The only difference from tcptraceroute (note the tcp in front) is that you sneak into an active TCP connection selected for active full size packet transfer.
 
I'll have to trace the logic in the current Internet stack in the latest kernels, but I'm pretty sure that iptables processes packets very low in the stack.  It ought to - one of the things you might want to do is reject forged ICMP packets, or not forward them.
 
-----Original Message-----
From: "Michael Richardson" <mcr@sandelman.ca>
Sent: Monday, November 26, 2012 4:27pm
To: dpreed@reed.com
Cc: cerowrt-users@lists.bufferbloat.net, cerowrt-devel@lists.bufferbloat.net
Subject: Re: [Cerowrt-devel] [Cerowrt-users] QOS settings vs speedboost and random bandwidth



>>>>> "dpreed" == dpreed  <dpreed@reed.com> writes:
 dpreed> It observed the IPv4 headers of *large* TCP/IP datagrams
 dpreed> going upstream, so that it could construct "no-op"
 dpreed> "content-free" datagrams that would certainly pass muster
 dpreed> through all the filters and be routed exactly the same as
 dpreed> the TCP/IP datagrams  that were carrying large flows.  It
 dpreed> would remember only the most recent one. 

I don't know that you need to be so precise in creating the packet, but
I guess the point is not just the ACLs, but also any traffic shapers?

 dpreed> The TTL expiration causes an ICMP packet to be sent back.
 dpreed> My code intercepts that packet based on its contents, and
 dpreed> removes it as "handled" before it gets processed by the
 dpreed> TCP/IP state machines. 

This is perhaps the biggest problem with this method... having to remove
the magic ICMP so that it does no harm.  Without this requirement, it
could be done entirely in userspace I think.

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
 Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
 then sign the petition.

[-- Attachment #2: Type: text/html, Size: 3259 bytes --]

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 2342 bytes --]

On Mon, Nov 26, 2012 at 9:22 AM, Michael Richardson <mcr@sandelman.ca>wrote:

>
> >>>>> "Dave" == Dave Taht <dave.taht@gmail.com> writes:
>     Dave> to move the fq_codel algorithm into the cable modem, OR to have
> the
>     Dave> cable modem exert flow control like ADSL modems do. Then have the
>     Dave> smarter device (the router) then do smarter queuing.
>
> I was unaware that DSL models were doing this... is this with GbE
> ethernet level XON/XOFF?   Should I assume that any DSL model with less
> than a GbE sync doesn't/can't have it?  Any brands you'd like to
> acknowledge?
>

We don't know that all DSL modems do this.

However, in corresponding with Michael Tremer about his results (documented
here
http://planet.ipfire.org/post/ipfire-2-13-tech-preview-fighting-bufferbloat),
it appears that a DSL modem commonly used in Germany does implement
Ethernet flow control (or he could not have gotten the results he reports
there at all; all he did was turn on fq_codel on the x86 geode router he is
using for IPfire).  Whether this is "standard" in DSL modems, I have no
clue (anyone want to plow through DSL standards documents?).


I've been unable to find a supplier in the U.S. for this particular DSL
modem.  Short of bidding on ebay in Europe, I was defeated (despite there
being 10's of millions of them in Germany); this isn't too surprising, as
the modem is supplied by the carrier in this case.

Note that I'm testing a DGN3500 this week with John Crispin (blogic) which
has an integrated Lantiq DSL chip (along with Atheros radios) running
fq_codel that John maintains.  That may be a better option for DSL, as it
can be purchased easily via Amazon.  Wish us luck (though John's early data
looks really nice).
                              - Jim


> --
> ]       He who is tired of Weird Al is tired of life!           |
>  firewalls  [
> ]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net
> architect[
> ] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/
>
> |device driver[
>    Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE
>
> >
>                        then sign the petition.
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>
>

[-- Attachment #2: Type: text/html, Size: 3935 bytes --]

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Michael Richardson]

>>>>> "Jim" == Jim Gettys <jg@freedesktop.org> writes:
    >> I was unaware that DSL models were doing this... is this with GbE
    >> ethernet level XON/XOFF?   Should I assume that any DSL model with less
    >> than a GbE sync doesn't/can't have it?  Any brands you'd like to
    >> acknowledge?


    Jim> We don't know that all DSL modems do this.

sure...   I figured none of them are doing this.

    Jim> Note that I'm testing a DGN3500 this week with John Crispin
    Jim> (blogic) which 
    Jim> has an integrated Lantiq DSL chip (along with Atheros radios)

so, this would be a home router with wifi and DSL?

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
   Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
	               then sign the petition. 

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 1227 bytes --]

On Mon, Nov 26, 2012 at 1:13 PM, Michael Richardson <mcr@sandelman.ca>wrote:

>
> >>>>> "Jim" == Jim Gettys <jg@freedesktop.org> writes:
>     >> I was unaware that DSL models were doing this... is this with GbE
>     >> ethernet level XON/XOFF?   Should I assume that any DSL model with
> less
>     >> than a GbE sync doesn't/can't have it?  Any brands you'd like to
>     >> acknowledge?
>
>
>     Jim> We don't know that all DSL modems do this.
>
> sure...   I figured none of them are doing this.
>
>     Jim> Note that I'm testing a DGN3500 this week with John Crispin
>     Jim> (blogic) which
>     Jim> has an integrated Lantiq DSL chip (along with Atheros radios)
>
> so, this would be a home router with wifi and DSL?
>

Yes; it's somewhat similar to the Netgear WNDR3700v2's.  Atheros radios.
                   - Jim


>
> --
> ]       He who is tired of Weird Al is tired of life!           |
>  firewalls  [
> ]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net
> architect[
> ] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/
>
> |device driver[
>    Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE
>
> >
>                        then sign the petition.
>

[-- Attachment #2: Type: text/html, Size: 2260 bytes --]

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Michael Richardson]

>>>>> "Jim" == Jim Gettys <jg@freedesktop.org> writes:
    >> >> I was unaware that DSL models were doing this... is this with GbE
    >> >> ethernet level XON/XOFF?   Should I assume that any DSL model with
    >> less
    >> >> than a GbE sync doesn't/can't have it?  Any brands you'd like to
    >> >> acknowledge?
    >> 
    >> 
    Jim> We don't know that all DSL modems do this.
    >> 
    >> sure...   I figured none of them are doing this.
    >> 
    Jim> Note that I'm testing a DGN3500 this week with John Crispin
    Jim> (blogic) which
    Jim> has an integrated Lantiq DSL chip (along with Atheros radios)
    >> 
    >> so, this would be a home router with wifi and DSL?

    Jim> Yes; it's somewhat similar to the Netgear WNDR3700v2's.  Atheros radios.

I'm parsing your Yes, ambiguously.
Yes: it's a home router with DSL model integrated.

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] mcr@sandelman.ottawa.on.ca http://www.sandelman.ottawa.on.ca/ |device driver[
   Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
	               then sign the petition. 

Re: [Cerowrt-users] [Cerowrt-devel] QOS settings vs speedboost and random bandwidth

[Jim Gettys]

[-- Attachment #1: Type: text/plain, Size: 3891 bytes --]

On Mon, Nov 26, 2012 at 3:32 AM, Dave Taht <dave.taht@gmail.com> wrote:

> there are not presently a lot of cerowrt-users members, so I'm cc-ing
> cerowrt-devel.
>
> On Mon, Nov 26, 2012 at 12:20 AM, Marc MERLIN <marc@merlins.org> wrote:
> > I'm setting up QOS as explained in the FAQ
> > http://www.bufferbloat.net/projects/cerowrt/wiki/FAQ
>
>
> >
> > Not surprisingly
> > speednest.net
>
> gives me 33Mbit/7.65Mbit while dslreports.com/speedtest
>
> gives
> > me a meager 16.2Mbit/2.5Mbit (for a line that's supposed to be
> 22Mbit/5Mbit).
>
> Yes, you get a boost for about 10 seconds.
>
> > I understand the queuing and that if I put too high a number, I'll fill
> > queues and end up with bad latency.
>
> yep.
>
> >
> > At the same time, am I correct that if I put numbers that are lower than
> > what I can really get when the line doesn't suck, I'll cap myself to
> those
> > numbers (here 16.2Mbit instead of 22Mbit or so)?
>
>
> yep.
>
> >
> > Is it also possible for linux/cerowrt to dynmically detect the
> queuing/delay
> > in my cable modem and dynamically adjust the QOS values to limit
> buffering
> > on the comcast side without throttling me down to lower than what the
> line
> > can do at any given time?
>
> Nope. The right answer is to not have to rate shape in the router, but
> to move the fq_codel algorithm into the cable modem, OR to have the
> cable modem exert flow control like ADSL modems do. Then have the
> smarter device (the router) then do smarter queuing.
>
> There are other dynamic bandwidth effects on a cable network that
> result in more widely varying backend bandwidth.
>

Yes, and under conditions of overall network congestion, if you are using
your link at a high fraction (> 80% or so, IIRC), they may also take some
action after an extended period (something like 10-15 minutes), as part of
their protocol neutral congestion management system.

There are also physical effects (temperature) that can cause some bandwidth
variation.


>
> A little birdie tells me that the cable modem guys are on this...
> speedboost is a good idea but the only way to not lose it is to move
> the smarts around.
>

Yup.  It's a good feature and it's sad to have to defeat it (unless someone
does a Powerboost emulator).

>
> That said, it does seem possible to develop a speedboost emulator on
> the router that would give you most of the benefit.
>

The details of at least Comcast's "Powerboost" may be documented in the RFC
published describing their provisioning.   I don't have the RFC handy.  I
think the algorithm they use is described there, even if the exact
parameters for a particular provisioning tier is not...

>
>
> >
> > I'm also a bit concerned of what happens when comcasts ups my bandwidth
> > without telling me (they do that sometimes) and me never finding out if I
> > have a static QOS value in cerowrt.
>

They certainly tell you in the advertising spam that we all delete ;-(.
                         - Jim


> >
> > Can others advise?
> >
> > Thanks,
> > Marc
> > --
> > "A mouse is a device used to point at the xterm you want to type in" -
> A.S.R.
> > Microsoft is to operating systems ....
> >                                       .... what McDonalds is to gourmet
> cooking
> > Home page: http://marc.merlins.org/
>
>
> > _______________________________________________
> > Cerowrt-users mailing list
> > Cerowrt-users@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/cerowrt-users
>
>
>
>
>
> --
> Dave Täht
>
> Fixing bufferbloat with cerowrt:
> http://www.teklibre.com/cerowrt/subscribe.html
>
>
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>
>
>

[-- Attachment #2: Type: text/html, Size: 6192 bytes --]