[Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Brandon Applegate]

[-- Attachment #1.1: Type: text/plain, Size: 1584 bytes --]

Disclaimer: if this is the wrong list for such a question - let me know.  This is specifically about the sqm-scripts package...

Hello,

I’ve been reading all I can on the bufferbloat website and also trying to understand the evolution of the various scripts (debloat, sqm, etc).

I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC - no *wrt etc).  Got it built with the ‘linux’ platform.  Since this is Ubuntu 12.04 - I had to cheat a bit and pull down the iproute2 source from 14.04.  I’ve tweaked the main sqm script to reflect this for the tc bindary - this is working.  I also updated my kernel to a later version that supports fq_codel.

My topology is ‘on a stick’.  I have one gig interface to a managed switch, on which are eth0.666 (outside/wan) and eth0.10 (inside).

I have 30/5 cable service, and have tried both those values as well as 90% in my /etc/sqm/*conf file.

I’ve tried both eth0 (raw/parent interface) as well as eth0.666.

No matter what I do - my bandwidth is 10% of what it should be.  I get approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat looks great though - A+.

Is there something inherent I’m doing wrong ?  Something to do with my ‘on a stick’ topology biting me ?  Kernel version (Ubuntu’s 3.13.0-74-generic btw).

Thanks in advance for any help or info (or pointer to a more appropriate list).

--
Brandon Applegate - CCIE 10273
PGP Key fingerprint:
830B 4802 1DD4 F4F9 63FE  B966 C0A7 189E 9EC0 3A74
"SH1-0151.  This is the serial number, of our orbital gun."


[-- Attachment #1.2: Type: text/html, Size: 3445 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Jonathan Morton]

> On 20 Jan, 2016, at 01:33, Brandon Applegate <brandon@burn.net> wrote:
> 
> My topology is ‘on a stick’.  I have one gig interface to a managed switch, on which are eth0.666 (outside/wan) and eth0.10 (inside).
> 
> No matter what I do - my bandwidth is 10% of what it should be.  I get approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat looks great though - A+.
> 
> Is there something inherent I’m doing wrong ?  Something to do with my ‘on a stick’ topology biting me ?

I’m not sure whether a qdisc can be attached to a virtualised interface in this way.  Can you run ‘tc -s qdisc’ and show us the output?

 - Jonathan Morton

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Brandon Applegate]

[-- Attachment #1: Type: text/plain, Size: 2937 bytes --]


> On Jan 19, 2016, at 7:09 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
> 
> I’m not sure whether a qdisc can be attached to a virtualised interface in this way.  Can you run ‘tc -s qdisc’ and show us the output?
> 

It seems to have the same effect ether I’m operating on eth0 or eth0.666.  Here’s the requested output when I’m operating on eth0.666.

PS: I’ve changed as few defaults / parameters as possible.  Bandwidth and the tweak for the tc binary is about it.

—
root@ice:/etc/sqm# /root/iproute2-3.12.0/tc/tc -s qdisc
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
 Sent 299400 bytes 1954 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
 Sent 490962005 bytes 605821 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc htb 1: dev eth0.666 root refcnt 2 r2q 10 default 12 direct_packets_stat 0 direct_qlen 2
 Sent 70076 bytes 422 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 110: dev eth0.666 parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
 Sent 1298 bytes 15 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 256 drop_overlimit 0 new_flow_count 15 ecn_mark 0
  new_flows_len 1 old_flows_len 0
qdisc fq_codel 120: dev eth0.666 parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
 Sent 68778 bytes 407 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1514 drop_overlimit 0 new_flow_count 94 ecn_mark 0
  new_flows_len 1 old_flows_len 2
qdisc fq_codel 130: dev eth0.666 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 256 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc ingress ffff: dev eth0.666 parent ffff:fff1 ----------------
 Sent 412033 bytes 516 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc htb 1: dev ifb4eth0.666 root refcnt 2 r2q 10 default 10 direct_packets_stat 0 direct_qlen 32
 Sent 426995 bytes 517 pkt (dropped 0, overlimits 119 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 110: dev ifb4eth0.666 parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms ecn
 Sent 426995 bytes 517 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 11518 drop_overlimit 0 new_flow_count 132 ecn_mark 0
  new_flows_len 1 old_flows_len 3
—


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Jonathan Morton]

> On 20 Jan, 2016, at 02:14, Brandon Applegate <brandon@burn.net> wrote:
> 
> It seems to have the same effect ether I’m operating on eth0 or eth0.666.  Here’s the requested output when I’m operating on eth0.666.

Is that after running a bandwidth test?  It looks like most of the traffic is on eth0.

 - Jonathan Morton

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Simon Barber]

Last I looked at this you can attach a qdisc to a virtual interface, but 
there is no backpressure, since the interface is virtual and the real 
interface it feeds will never 'stop' accepting packets (since you are 
really feeding into another qdisc).

So you can do things like rate pacing, logging, etc.

Simon


On 1/19/2016 4:09 PM, Jonathan Morton wrote:
>> On 20 Jan, 2016, at 01:33, Brandon Applegate <brandon@burn.net> wrote:
>>
>> My topology is ‘on a stick’.  I have one gig interface to a managed switch, on which are eth0.666 (outside/wan) and eth0.10 (inside).
>>
>> No matter what I do - my bandwidth is 10% of what it should be.  I get approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat looks great though - A+.
>>
>> Is there something inherent I’m doing wrong ?  Something to do with my ‘on a stick’ topology biting me ?
> I’m not sure whether a qdisc can be attached to a virtualised interface in this way.  Can you run ‘tc -s qdisc’ and show us the output?
>
>   - Jonathan Morton
>
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Alan Jenkins]

On 19/01/2016, Brandon Applegate <brandon@burn.net> wrote:
> Disclaimer: if this is the wrong list for such a question - let me know.
> This is specifically about the sqm-scripts package...
>
> Hello,
>
> I’ve been reading all I can on the bufferbloat website and also trying to
> understand the evolution of the various scripts (debloat, sqm, etc).
>
> I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC - no *wrt
> etc).  Got it built with the ‘linux’ platform.  Since this is Ubuntu 12.04 -
> I had to cheat a bit and pull down the iproute2 source from 14.04.  I’ve
> tweaked the main sqm script to reflect this for the tc bindary - this is
> working.  I also updated my kernel to a later version that supports
> fq_codel.
>
> My topology is ‘on a stick’.  I have one gig interface to a managed switch,
> on which are eth0.666 (outside/wan) and eth0.10 (inside).
>
> I have 30/5 cable service, and have tried both those values as well as 90%
> in my /etc/sqm/*conf file.
>
> I’ve tried both eth0 (raw/parent interface) as well as eth0.666.
>
> No matter what I do - my bandwidth is 10% of what it should be.  I get
> approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat
> looks great though - A+.
>
> Is there something inherent I’m doing wrong ?  Something to do with my ‘on a
> stick’ topology biting me ?  Kernel version (Ubuntu’s 3.13.0-74-generic
> btw).
>
> Thanks in advance for any help or info (or pointer to a more appropriate
> list).

It doesn't sound like you're doing anything wrong :(.

I would make sure to check the rates on `tc class show dev eth0.666`
(and ifb4eth0.666).  Switching to `simplest.qos` could be easier to
debug.  With your simple.qos, there'll be several tracffic classes...
the `root` should be the specified `rate`, and it looks like all
classes save 1:11 should have a `ceil` just under the specified rate.

Not sure how to debug qos-scripts itself.  However the Gentoo wiki has
a 50-line script, which was corrected by dtaht :).  Like simplest.qos
this has a single class.
https://wiki.gentoo.org/wiki/Traffic_shaping

That would let you investigate the commands finely, as well as the
resulting state shown by `tc qdisc` and `tc class`, and really narrow
it down.

`dslreports.com` will show bandwidth and latency-under-load in each
direction independently, so you could work on a single direction.  I
would look at ingress only (the IFB) since that's where your bandwidth
decimation is so visible.  E.g. just comment out the egress section,
to avoid distractions.

I think you can run the htb without the fq_codel command at the end -
that is, it will default to a massive fifo, which will replace the
fq_codel in the output of `tc qdisc`, but to a first approximation it
will affect bandwidth.

Good luck
Alan

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Alan Jenkins]

On 20/01/2016, Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:
> On 19/01/2016, Brandon Applegate <brandon@burn.net> wrote:
>> Disclaimer: if this is the wrong list for such a question - let me know.
>> This is specifically about the sqm-scripts package...
>>
>> Hello,
>>
>> I’ve been reading all I can on the bufferbloat website and also trying to
>> understand the evolution of the various scripts (debloat, sqm, etc).
>>
>> I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC - no
>> *wrt
>> etc).  Got it built with the ‘linux’ platform.  Since this is Ubuntu 12.04
>> -
>> I had to cheat a bit and pull down the iproute2 source from 14.04.  I’ve
>> tweaked the main sqm script to reflect this for the tc bindary - this is
>> working.  I also updated my kernel to a later version that supports
>> fq_codel.
>>
>> My topology is ‘on a stick’.  I have one gig interface to a managed
>> switch,
>> on which are eth0.666 (outside/wan) and eth0.10 (inside).
>>
>> I have 30/5 cable service, and have tried both those values as well as
>> 90%
>> in my /etc/sqm/*conf file.
>>
>> I’ve tried both eth0 (raw/parent interface) as well as eth0.666.
>>
>> No matter what I do - my bandwidth is 10% of what it should be.  I get
>> approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat
>> looks great though - A+.
>>
>> Is there something inherent I’m doing wrong ?  Something to do with my ‘on
>> a
>> stick’ topology biting me ?  Kernel version (Ubuntu’s 3.13.0-74-generic
>> btw).
>>
>> Thanks in advance for any help or info (or pointer to a more appropriate
>> list).
>
> It doesn't sound like you're doing anything wrong :(.
>
> I would make sure to check the rates on `tc class show dev eth0.666`
> (and ifb4eth0.666).  Switching to `simplest.qos` could be easier to
> debug.  With your simple.qos, there'll be several tracffic classes...
> the `root` should be the specified `rate`, and it looks like all
> classes save 1:11 should have a `ceil` just under the specified rate.
>
> Not sure how to debug qos-scripts itself.  However the Gentoo wiki has
> a 50-line script, which was corrected by dtaht :).  Like simplest.qos
> this has a single class.
> https://wiki.gentoo.org/wiki/Traffic_shaping
>
> That would let you investigate the commands finely, as well as the
> resulting state shown by `tc qdisc` and `tc class`, and really narrow
> it down.
>
> `dslreports.com` will show bandwidth and latency-under-load in each
> direction independently, so you could work on a single direction.  I
> would look at ingress only (the IFB) since that's where your bandwidth
> decimation is so visible.  E.g. just comment out the egress section,
> to avoid distractions.
>
> I think you can run the htb without the fq_codel command at the end -
> that is, it will default to a massive fifo, which will replace the
> fq_codel in the output of `tc qdisc`, but to a first approximation it
> will affect bandwidth.
>
> Good luck
> Alan

PS

The `ifb` part might be a bit mysterious.  If sqm-scripts is renaming
them, you could just remove the IFB kernel module & reload it to clear
the entire IFB config:

modprobe -r ifb

(the script includes "modprobe ifb" to load it as required)

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[moeller0]

Hi Alan, hi Brandon:

> On Jan 20, 2016, at 11:15 , Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:
> 
> On 20/01/2016, Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:
>> On 19/01/2016, Brandon Applegate <brandon@burn.net> wrote:
>>> Disclaimer: if this is the wrong list for such a question - let me know.
>>> This is specifically about the sqm-scripts package...
>>> 
>>> Hello,
>>> 
>>> I’ve been reading all I can on the bufferbloat website and also trying to
>>> understand the evolution of the various scripts (debloat, sqm, etc).
>>> 
>>> I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC - no
>>> *wrt
>>> etc).  Got it built with the ‘linux’ platform.  Since this is Ubuntu 12.04
>>> -
>>> I had to cheat a bit and pull down the iproute2 source from 14.04.  I’ve
>>> tweaked the main sqm script to reflect this for the tc bindary - this is
>>> working.  I also updated my kernel to a later version that supports
>>> fq_codel.
>>> 
>>> My topology is ‘on a stick’.  I have one gig interface to a managed
>>> switch,
>>> on which are eth0.666 (outside/wan) and eth0.10 (inside).
>>> 
>>> I have 30/5 cable service, and have tried both those values as well as
>>> 90%
>>> in my /etc/sqm/*conf file.
>>> 
>>> I’ve tried both eth0 (raw/parent interface) as well as eth0.666.
>>> 
>>> No matter what I do - my bandwidth is 10% of what it should be.  I get
>>> approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat
>>> looks great though - A+.
>>> 
>>> Is there something inherent I’m doing wrong ?  Something to do with my ‘on
>>> a
>>> stick’ topology biting me ?  Kernel version (Ubuntu’s 3.13.0-74-generic
>>> btw).
>>> 
>>> Thanks in advance for any help or info (or pointer to a more appropriate
>>> list).
>> 
>> It doesn't sound like you're doing anything wrong :(.
>> 
>> I would make sure to check the rates on `tc class show dev eth0.666`
>> (and ifb4eth0.666).  Switching to `simplest.qos` could be easier to
>> debug.  With your simple.qos, there'll be several tracffic classes...
>> the `root` should be the specified `rate`, and it looks like all
>> classes save 1:11 should have a `ceil` just under the specified rate.
>> 
>> Not sure how to debug qos-scripts itself.  However the Gentoo wiki has
>> a 50-line script, which was corrected by dtaht :).  Like simplest.qos
>> this has a single class.
>> https://wiki.gentoo.org/wiki/Traffic_shaping
>> 
>> That would let you investigate the commands finely, as well as the
>> resulting state shown by `tc qdisc` and `tc class`, and really narrow
>> it down.
>> 
>> `dslreports.com` will show bandwidth and latency-under-load in each
>> direction independently, so you could work on a single direction.  I
>> would look at ingress only (the IFB) since that's where your bandwidth
>> decimation is so visible.  E.g. just comment out the egress section,
>> to avoid distractions.
>> 
>> I think you can run the htb without the fq_codel command at the end -
>> that is, it will default to a massive fifo, which will replace the
>> fq_codel in the output of `tc qdisc`, but to a first approximation it
>> will affect bandwidth.
>> 
>> Good luck
>> Alan
> 
> PS
> 
> The `ifb` part might be a bit mysterious.  If sqm-scripts is renaming
> them, you could just remove the IFB kernel module & reload it to clear
> the entire IFB config:
> 
> modprobe -r ifb
> 
> (the script includes "modprobe ifb" to load it as required)

Sqm is supposed to delete its ifb interfaces if they are not used anymore, to test whether an existing interface is redirected to an ifb (and if yes which ifb) just issue the following (pppoe-ge00 being the name of the interface in question):
root@computer:~# tc -p filter show parent ffff: dev pppoe-ge00
filter protocol all pref 10 u32 
filter protocol all pref 10 u32 fh 800: ht divisor 1 
filter protocol all pref 10 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:1 
  match 00000000/00000000 at 0
	action order 1: mirred (Egress Redirect to device ifb4pppoe-ge00) stolen
 	index 145 ref 1 bind 1

Best Regards
	Sebastian


> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[moeller0]

> On Jan 20, 2016, at 11:12 , Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:
> 
> On 19/01/2016, Brandon Applegate <brandon@burn.net> wrote:
>> Disclaimer: if this is the wrong list for such a question - let me know.
>> This is specifically about the sqm-scripts package...
>> 
>> Hello,
>> 
>> I’ve been reading all I can on the bufferbloat website and also trying to
>> understand the evolution of the various scripts (debloat, sqm, etc).
>> 
>> I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC - no *wrt
>> etc).  Got it built with the ‘linux’ platform.  Since this is Ubuntu 12.04 -
>> I had to cheat a bit and pull down the iproute2 source from 14.04.  I’ve
>> tweaked the main sqm script to reflect this for the tc bindary - this is
>> working.  I also updated my kernel to a later version that supports
>> fq_codel.
>> 
>> My topology is ‘on a stick’.  I have one gig interface to a managed switch,
>> on which are eth0.666 (outside/wan) and eth0.10 (inside).
>> 
>> I have 30/5 cable service, and have tried both those values as well as 90%
>> in my /etc/sqm/*conf file.
>> 
>> I’ve tried both eth0 (raw/parent interface) as well as eth0.666.
>> 
>> No matter what I do - my bandwidth is 10% of what it should be.  I get
>> approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat
>> looks great though - A+.
>> 
>> Is there something inherent I’m doing wrong ?  Something to do with my ‘on a
>> stick’ topology biting me ?  Kernel version (Ubuntu’s 3.13.0-74-generic
>> btw).
>> 
>> Thanks in advance for any help or info (or pointer to a more appropriate
>> list).
> 
> It doesn't sound like you're doing anything wrong :(.
> 
> I would make sure to check the rates on `tc class show dev eth0.666`
> (and ifb4eth0.666).  Switching to `simplest.qos` could be easier to
> debug.  With your simple.qos, there'll be several tracffic classes...
> the `root` should be the specified `rate`, and it looks like all
> classes save 1:11 should have a `ceil` just under the specified rate.
> 
> Not sure how to debug qos-scripts itself.  However the Gentoo wiki has
> a 50-line script, which was corrected by dtaht :).  Like simplest.qos
> this has a single class.
> https://wiki.gentoo.org/wiki/Traffic_shaping
> 
> That would let you investigate the commands finely, as well as the
> resulting state shown by `tc qdisc` and `tc class`, and really narrow
> it down.
> 
> `dslreports.com` will show bandwidth and latency-under-load in each
> direction independently, so you could work on a single direction.  I
> would look at ingress only (the IFB) since that's where your bandwidth
> decimation is so visible.  E.g. just comment out the egress section,
> to avoid distractions.

	It should be sufficient to set the egress rate to 0 then, as for sqm zero denotes no shaping and not a rate of zero kbps (good luck using TCP on a purely unidirectional link...)

> 
> I think you can run the htb without the fq_codel command at the end -
> that is, it will default to a massive fifo, which will replace the
> fq_codel in the output of `tc qdisc`, but to a first approximation it
> will affect bandwidth.

	Simply put
QDISC=pfifo_fast
into the .conf file for eth0.666 to test this.

Best Regards
	Sebastian

> 
> Good luck
> Alan
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Alan Jenkins]

On 20/01/16 11:43, moeller0 wrote:
>> On Jan 20, 2016, at 11:12 , Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:
>>
>> On 19/01/2016, Brandon Applegate <brandon@burn.net> wrote:
>>> Disclaimer: if this is the wrong list for such a question - let me know.
>>> This is specifically about the sqm-scripts package...
>>>
>>> Hello,
>>>
>>> I’ve been reading all I can on the bufferbloat website and also trying to
>>> understand the evolution of the various scripts (debloat, sqm, etc).
>>>
>>> I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC - no *wrt
>>> etc).  Got it built with the ‘linux’ platform.  Since this is Ubuntu 12.04 -
>>> I had to cheat a bit and pull down the iproute2 source from 14.04.  I’ve
>>> tweaked the main sqm script to reflect this for the tc bindary - this is
>>> working.  I also updated my kernel to a later version that supports
>>> fq_codel.
>>>
>>> My topology is ‘on a stick’.  I have one gig interface to a managed switch,
>>> on which are eth0.666 (outside/wan) and eth0.10 (inside).
>>>
>>> I have 30/5 cable service, and have tried both those values as well as 90%
>>> in my /etc/sqm/*conf file.
>>>
>>> I’ve tried both eth0 (raw/parent interface) as well as eth0.666.
>>>
>>> No matter what I do - my bandwidth is 10% of what it should be.  I get
>>> approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat
>>> looks great though - A+.
>>>
>>> Is there something inherent I’m doing wrong ?  Something to do with my ‘on a
>>> stick’ topology biting me ?  Kernel version (Ubuntu’s 3.13.0-74-generic
>>> btw).
>>>
>>> Thanks in advance for any help or info (or pointer to a more appropriate
>>> list).
>> It doesn't sound like you're doing anything wrong :(.
>>
>> I would make sure to check the rates on `tc class show dev eth0.666`
>> (and ifb4eth0.666).  Switching to `simplest.qos` could be easier to
>> debug.  With your simple.qos, there'll be several tracffic classes...
>> the `root` should be the specified `rate`, and it looks like all
>> classes save 1:11 should have a `ceil` just under the specified rate.
>>
>> Not sure how to debug qos-scripts itself.  However the Gentoo wiki has
>> a 50-line script, which was corrected by dtaht :).  Like simplest.qos
>> this has a single class.
>> https://wiki.gentoo.org/wiki/Traffic_shaping
>>
>> That would let you investigate the commands finely, as well as the
>> resulting state shown by `tc qdisc` and `tc class`, and really narrow
>> it down.
>>
>> `dslreports.com` will show bandwidth and latency-under-load in each
>> direction independently, so you could work on a single direction.  I
>> would look at ingress only (the IFB) since that's where your bandwidth
>> decimation is so visible.  E.g. just comment out the egress section,
>> to avoid distractions.
> 	It should be sufficient to set the egress rate to 0 then, as for sqm zero denotes no shaping and not a rate of zero kbps (good luck using TCP on a purely unidirectional link...)
>
>> I think you can run the htb without the fq_codel command at the end -
>> that is, it will default to a massive fifo, which will replace the
>> fq_codel in the output of `tc qdisc`, but to a first approximation it
>> will affect bandwidth.
> 	Simply put
> QDISC=pfifo_fast
> into the .conf file for eth0.666 to test this.
>
> Best Regards
> 	Sebastian

No offense to your work on sqm-scripts.


1) It's just in case the problem is *outside* of sqm-scripts, it could 
be useful to try the minimum commands necessary to demonstrate the 
problem.  Maybe it's unfair but I assumed running on a PC is also a less 
tested case, as well as the "firewall on a stick" part.

Equally, since AFAICT Brandon hasn't had a working AQM setup on this box 
before.  It could be useful if the Gentoo script works, to prove that 
AQM + fq_codel can work correctly on Brandon's box.

If the Gentoo script _wasn't_ working, that's when I'd suggest tearing 
it down e.g. to eliminate fq_codel as an issue.  Rather than specify 
fifo explicitly, just don't run the fq_codel command (assuming that 
actually does something sensible, which can be checked using `tc qdisc`).


2) More specifically, you didn't mention trying "simplest.qos". This 
would i) simplify the setup we're trying to debug, and ii) it might show 
if there's a bug with the more complex bandwidth calculations / 
assignments in "simple.qos".


And again - I suggest starting by checking `tc class show dev eth0.666`, 
because it's not a 100% obvious command, and we don't want to miss if 
there are bad rates there :).


Alan

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[moeller0]

Hi Alan,

> On Jan 20, 2016, at 13:06 , Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:
> [...]
> No offense to your work on sqm-scripts.

	None taken ;) I really just want to use the opportunity to a) get Brandon up and running and b) fix any short-comings on sqm-scripts on the way.

> 
> 
> 1) It's just in case the problem is *outside* of sqm-scripts, it could be useful to try the minimum commands necessary to demonstrate the problem.

	I fully agree.

>  Maybe it's unfair but I assumed running on a PC is also a less tested case, as well as the "firewall on a stick" part.

	You are spot on, both are rather exotic test cases at least as far as reports go; that is what intrigues me even more ;)

> 
> Equally, since AFAICT Brandon hasn't had a working AQM setup on this box before.  It could be useful if the Gentoo script works, to prove that AQM + fq_codel can work correctly on Brandon's box.

	I guess since we just thought sqm-scripts to allow verbose debugging output (into a log file) I thought it is time to advertise that feature ;)

> 
> If the Gentoo script _wasn't_ working, that's when I'd suggest tearing it down e.g. to eliminate fq_codel as an issue.  Rather than specify fifo explicitly, just don't run the fq_codel command (assuming that actually does something sensible, which can be checked using `tc qdisc`).
> 
> 
> 2) More specifically, you didn't mention trying "simplest.qos". This would i) simplify the setup we're trying to debug, and ii) it might show if there's a bug with the more complex bandwidth calculations / assignments in "simple.qos”.

	Oh, you had mentioned that already and I fully agree that getting a simple test cases going is going to help a lot. So we violently agree ;)

> 
> 
> And again - I suggest starting by checking `tc class show dev eth0.666`, because it's not a 100% obvious command, and we don't want to miss if there are bad rates there :).

	Again, I fully agree, I guess I should have mentioned that I have no genuinely new advice but rather just recommendations for slight variations on the way. Sorry for muddying the water here.

Best Regards
	Sebastian

> 
> 
> Alan

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Brandon Applegate]

[-- Attachment #1: Type: text/plain, Size: 2130 bytes --]


> On Jan 20, 2016, at 7:06 AM, Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:
> 
> 
> No offense to your work on sqm-scripts.
> 
> 
> 1) It's just in case the problem is *outside* of sqm-scripts, it could be useful to try the minimum commands necessary to demonstrate the problem.  Maybe it's unfair but I assumed running on a PC is also a less tested case, as well as the "firewall on a stick" part.
> 
> Equally, since AFAICT Brandon hasn't had a working AQM setup on this box before.  It could be useful if the Gentoo script works, to prove that AQM + fq_codel can work correctly on Brandon's box.
> 
> If the Gentoo script _wasn't_ working, that's when I'd suggest tearing it down e.g. to eliminate fq_codel as an issue.  Rather than specify fifo explicitly, just don't run the fq_codel command (assuming that actually does something sensible, which can be checked using `tc qdisc`).
> 
> 
> 2) More specifically, you didn't mention trying "simplest.qos". This would i) simplify the setup we're trying to debug, and ii) it might show if there's a bug with the more complex bandwidth calculations / assignments in "simple.qos".
> 
> 
> And again - I suggest starting by checking `tc class show dev eth0.666`, because it's not a 100% obvious command, and we don't want to miss if there are bad rates there :).
> 
> 
> Alan

Wow, thanks for all the ideas and feedback.  I did indeed find the Gentoo wiki script and gave it a shot - basically the same behavior as sqm-scripts.

I’m really thinking my on-a-stick topology is tying things in knots as my traffic is on the same interface twice in inverse directions.  I have another NIC in the box used for some back end NFS stuff - I can steal him temporarily to rejigger my interfaces.  My goal is to get eth0.666 to just be eth0 (public wan, no dot1q).  I’ll post my results from this - and barring any great success there - will loop back to this thread and try what folks have mentioned.

We are snowed in today so the kids are home from school - I’m sure I’ll get plenty of feedback from them - ‘dad the internet is down’.  :)

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[moeller0]

Hi Brandon,

> On Jan 20, 2016, at 15:51 , Brandon Applegate <brandon@burn.net> wrote:
> 
>> 
>> On Jan 20, 2016, at 7:06 AM, Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:
>> 
>> 
>> No offense to your work on sqm-scripts.
>> 
>> 
>> 1) It's just in case the problem is *outside* of sqm-scripts, it could be useful to try the minimum commands necessary to demonstrate the problem.  Maybe it's unfair but I assumed running on a PC is also a less tested case, as well as the "firewall on a stick" part.
>> 
>> Equally, since AFAICT Brandon hasn't had a working AQM setup on this box before.  It could be useful if the Gentoo script works, to prove that AQM + fq_codel can work correctly on Brandon's box.
>> 
>> If the Gentoo script _wasn't_ working, that's when I'd suggest tearing it down e.g. to eliminate fq_codel as an issue.  Rather than specify fifo explicitly, just don't run the fq_codel command (assuming that actually does something sensible, which can be checked using `tc qdisc`).
>> 
>> 
>> 2) More specifically, you didn't mention trying "simplest.qos". This would i) simplify the setup we're trying to debug, and ii) it might show if there's a bug with the more complex bandwidth calculations / assignments in "simple.qos".
>> 
>> 
>> And again - I suggest starting by checking `tc class show dev eth0.666`, because it's not a 100% obvious command, and we don't want to miss if there are bad rates there :).
>> 
>> 
>> Alan
> 
> Wow, thanks for all the ideas and feedback.  I did indeed find the Gentoo wiki script and gave it a shot - basically the same behavior as sqm-scripts.
> 
> I’m really thinking my on-a-stick topology is tying things in knots as my traffic is on the same interface twice in inverse directions.  I have another NIC in the box used for some back end NFS stuff - I can steal him temporarily to rejigger my interfaces.  My goal is to get eth0.666 to just be eth0 (public wan, no dot1q).  I’ll post my results from this - and barring any great success there - will loop back to this thread and try what folks have mentioned.

	Please, before you change the set up, run the few commands Alan (and I asked for) so we can get a rough idea what was happening. This smells like a bug in sqm-scripts and I would love to either confirm and fix it or get sqm-scripts acquitted ;)


> 
> We are snowed in today so the kids are home from school - I’m sure I’ll get plenty of feedback from them - ‘dad the internet is down’.  :)

So I would fully understand if any testing will have to wait until the weather permits outdoor activity again ;)

Best Regards
	Sebastian

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Brandon Applegate]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]

Here’s the output to check the rates.

root@ice:/etc/sqm# head -2 eth0.666.iface.conf
UPLINK=5000
DOWNLINK=30000

root@ice:/etc/sqm# tc class show dev eth0.666
class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 1666Kbit burst 1600b cburst 1599b
class htb 1:1 root rate 5000Kbit ceil 5000Kbit burst 1600b cburst 1600b
class htb 1:10 parent 1:1 prio 0 rate 5000Kbit ceil 5000Kbit burst 1600b cburst 1600b
class htb 1:13 parent 1:1 leaf 130: prio 3 rate 833000bit ceil 4984Kbit burst 1599b cburst 1599b
class htb 1:12 parent 1:1 leaf 120: prio 2 rate 833000bit ceil 4984Kbit burst 1599b cburst 1599b
class fq_codel 110:160 parent 110:
class fq_codel 120:1f5 parent 120:

--
Brandon Applegate - CCIE 10273
PGP Key fingerprint:
830B 4802 1DD4 F4F9 63FE  B966 C0A7 189E 9EC0 3A74
"SH1-0151.  This is the serial number, of our orbital gun."


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Sebastian Moeller]

[-- Attachment #1: Type: text/plain, Size: 1559 bytes --]

Okay so at least the egress uplinks seems okay, maybe a bit too generous, at 5Mbps contracted speed, maybe 4500Kbps would be a more cautious starting point. But in essence this looks good, I fail to see the ingress ifb4eth0.666, so can offer any opinion on that.

On January 20, 2016 4:15:35 PM GMT+01:00, Brandon Applegate <brandon@burn.net> wrote:
>Here’s the output to check the rates.
>
>root@ice:/etc/sqm# head -2 eth0.666.iface.conf
>UPLINK=5000
>DOWNLINK=30000
>
>root@ice:/etc/sqm# tc class show dev eth0.666
>class htb 1:11 parent 1:1 leaf 110: prio 1 rate 128000bit ceil 1666Kbit
>burst 1600b cburst 1599b
>class htb 1:1 root rate 5000Kbit ceil 5000Kbit burst 1600b cburst 1600b
>class htb 1:10 parent 1:1 prio 0 rate 5000Kbit ceil 5000Kbit burst
>1600b cburst 1600b
>class htb 1:13 parent 1:1 leaf 130: prio 3 rate 833000bit ceil 4984Kbit
>burst 1599b cburst 1599b
>class htb 1:12 parent 1:1 leaf 120: prio 2 rate 833000bit ceil 4984Kbit
>burst 1599b cburst 1599b
>class fq_codel 110:160 parent 110:
>class fq_codel 120:1f5 parent 120:
>
>--
>Brandon Applegate - CCIE 10273
>PGP Key fingerprint:
>830B 4802 1DD4 F4F9 63FE  B966 C0A7 189E 9EC0 3A74
>"SH1-0151.  This is the serial number, of our orbital gun."
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Bloat mailing list
>Bloat@lists.bufferbloat.net
>https://lists.bufferbloat.net/listinfo/bloat

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 1926 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Brandon Applegate]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

I’m getting more confused as I go on :)

So I’ve rebooted and the tc classes seem to have come back.  Since this is Ubuntu 12.04 - it doesn NOT have systemd.  I mention this because I see there is a systemd config in the sqm-scripts package.

I have not added any hooks to run 'sqm start’ - neverthless - I have all the rules seemingly there on a fresh boot.  I also have a new ‘interface’ - ifb4eth0.666.  Since I’ve never messed with tc - I have no idea how/where/what is ‘saving’ these rules and making them persistent.  I’m struggling to use google and grep -ir to see where Ubuntu is saving this.

Furthermore - the scripts seem to be working now.  Slightly embarassing - it could be the result of having mucked with the gentoo script and having two many variables flying around at once.

I currently lose a small fraction of my bandwidth - but bufferbloat gets an A on the dslreports speedtest.

I’m going to concentrate on understanding how / where these rules are getting made ‘persistent’.

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Alan Jenkins]

On 20/01/16 16:10, Brandon Applegate wrote:
> I’m getting more confused as I go on :)
>
> So I’ve rebooted and the tc classes seem to have come back.  Since this is Ubuntu 12.04 - it doesn NOT have systemd.  I mention this because I see there is a systemd config in the sqm-scripts package.
>
> I have not added any hooks to run 'sqm start’ - neverthless - I have all the rules seemingly there on a fresh boot.  I also have a new ‘interface’ - ifb4eth0.666.  Since I’ve never messed with tc - I have no idea how/where/what is ‘saving’ these rules and making them persistent.  I’m struggling to use google and grep -ir to see where Ubuntu is saving this.
>
> Furthermore - the scripts seem to be working now.  Slightly embarassing - it could be the result of having mucked with the gentoo script and having two many variables flying around at once.
>
> I currently lose a small fraction of my bandwidth - but bufferbloat gets an A on the dslreports speedtest.
>
> I’m going to concentrate on understanding how / where these rules are getting made ‘persistent’.

Excellent.

There's nothing that saves `tc` rules (like iptables-save + 
iptables-restore).  The fact that you have "ifb4eth0.666" re-appearing 
shows pretty clearly you've got something running "sqm start".

On OpenWRT the scripts run automatically when the network interface 
comes up.  udev is different, although you could have something hooked 
in like `/etc/network/if-up.d`.

Alan

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Brandon Applegate]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]



> On Jan 20, 2016, at 11:10 AM, Brandon Applegate <brandon@burn.net> wrote:
> 
> I’m getting more confused as I go on :)
> 
> So I’ve rebooted and the tc classes seem to have come back.  Since this is Ubuntu 12.04 - it doesn NOT have systemd.  I mention this because I see there is a systemd config in the sqm-scripts package.
> 
> I have not added any hooks to run 'sqm start’ - neverthless - I have all the rules seemingly there on a fresh boot.  I also have a new ‘interface’ - ifb4eth0.666.  Since I’ve never messed with tc - I have no idea how/where/what is ‘saving’ these rules and making them persistent.  I’m struggling to use google and grep -ir to see where Ubuntu is saving this.
> 

I see that the install for sqm-scripts puts a script in:

/etc/network/if-up.d

This is where my ‘persistence’ is coming from.

Sorry for the machinegun postings guys - I’m just underprepared to dig into tc.  My world is usually iptables, routes, etc.  Never worked much with tc and the unfamiliarity is frustrating me.

Having said that - I can see that this is excellent work.  I’ve had it on my todo for a while to reinstall the firewall with Ubuntu 14.04.  This will bring a lot of utilities and tools forward version wise and give me a better starting point.  I’m aware that I’m coloring outside the lines a bit to make this work as it stands.

Thanks for everyone’s patience and time so far.

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Etienne Champetier]

[-- Attachment #1: Type: text/plain, Size: 2023 bytes --]

Hi Brandon,

2016-01-20 17:34 GMT+01:00 Brandon Applegate <brandon@burn.net>:

>
>
> > On Jan 20, 2016, at 11:10 AM, Brandon Applegate <brandon@burn.net>
> wrote:
> >
> > I’m getting more confused as I go on :)
> >
> > So I’ve rebooted and the tc classes seem to have come back.  Since this
> is Ubuntu 12.04 - it doesn NOT have systemd.  I mention this because I see
> there is a systemd config in the sqm-scripts package.
> >
> > I have not added any hooks to run 'sqm start’ - neverthless - I have all
> the rules seemingly there on a fresh boot.  I also have a new ‘interface’ -
> ifb4eth0.666.  Since I’ve never messed with tc - I have no idea
> how/where/what is ‘saving’ these rules and making them persistent.  I’m
> struggling to use google and grep -ir to see where Ubuntu is saving this.
> >
>
> I see that the install for sqm-scripts puts a script in:
>
> /etc/network/if-up.d
>
> This is where my ‘persistence’ is coming from.
>
> Sorry for the machinegun postings guys - I’m just underprepared to dig
> into tc.  My world is usually iptables, routes, etc.  Never worked much
> with tc and the unfamiliarity is frustrating me.
>
> Having said that - I can see that this is excellent work.  I’ve had it on
> my todo for a while to reinstall the firewall with Ubuntu 14.04.  This will
> bring a lot of utilities and tools forward version wise and give me a
> better starting point.  I’m aware that I’m coloring outside the lines a bit
> to make this work as it stands.
>
> Thanks for everyone’s patience and time so far.
>
> _______________________________________________
>
>

I've quickly read the thread (maybe too quickly),
from what i remember for virtual interface txqueuelen is 0 by default, so
nothing work as expected
can you check the txqueuelen with ip -a ?
(i've found no reference to txqueuelen in sqm-scripts)

see also
https://wiki.openwrt.org/doc/howto/packet.scheduler/packet.scheduler#tips

Regards
Etienne

[-- Attachment #2: Type: text/html, Size: 2711 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Brandon Applegate]

[-- Attachment #1.1: Type: text/plain, Size: 1978 bytes --]


> On Jan 20, 2016, at 11:47 AM, Etienne Champetier <champetier.etienne@gmail.com> wrote:
> 
> Hi Brandon,
> 
> e quickly read the thread (maybe too quickly),
> from what i remember for virtual interface txqueuelen is 0 by default, so nothing work as expected
> can you check the txqueuelen with ip -a ?
> (i've found no reference to txqueuelen in sqm-scripts)
> 
> see also
> https://wiki.openwrt.org/doc/howto/packet.scheduler/packet.scheduler#tips <https://wiki.openwrt.org/doc/howto/packet.scheduler/packet.scheduler#tips>


I haven’t done anything manually to change txqueuelen.  Whatever is here is default (1000 on eth) or I assume set by sqm-scripts / tc (32).

I believe this is the output you are looking for:

—

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
4: eth0.666@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
5: eth0.10@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
6: eth0.11@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
7: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
    link/ether 52:18:cd:a7:ad:71 brd ff:ff:ff:ff:ff:ff
8: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
    link/ether 5e:f0:86:88:8d:8e brd ff:ff:ff:ff:ff:ff
11: ifb4eth0.666: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc htb state UNKNOWN qlen 32
    link/ether 86:68:83:69:2e:b2 brd ff:ff:ff:ff:ff:ff
—


[-- Attachment #1.2: Type: text/html, Size: 3860 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Etienne Champetier]

[-- Attachment #1: Type: text/plain, Size: 2126 bytes --]

2016-01-20 17:54 GMT+01:00 Brandon Applegate <brandon@burn.net>:

>
> On Jan 20, 2016, at 11:47 AM, Etienne Champetier <
> champetier.etienne@gmail.com> wrote:
>
> Hi Brandon,
>
> e quickly read the thread (maybe too quickly),
>
> from what i remember for virtual interface txqueuelen is 0 by default, so
> nothing work as expected
> can you check the txqueuelen with ip -a ?
> (i've found no reference to txqueuelen in sqm-scripts)
>
> see also
> https://wiki.openwrt.org/doc/howto/packet.scheduler/packet.scheduler#tips
>
>
> I haven’t done anything manually to change txqueuelen.  Whatever is here
> is default (1000 on eth) or I assume set by sqm-scripts / tc (32).
>
> I believe this is the output you are looking for:
>
> —
>
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state
> UP qlen 1000
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state
> UP qlen 1000
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 4: eth0.666@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb
> state UP
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 5: eth0.10@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> state UP
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 6: eth0.11@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
> state UP
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 7: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
>     link/ether 52:18:cd:a7:ad:71 brd ff:ff:ff:ff:ff:ff
> 8: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
>     link/ether 5e:f0:86:88:8d:8e brd ff:ff:ff:ff:ff:ff
> 11: ifb4eth0.666: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc htb state
> UNKNOWN qlen 32
>     link/ether 86:68:83:69:2e:b2 brd ff:ff:ff:ff:ff:ff
> —
>
>
no qlen means 0 I think
maybe try
ip link set eth0.666 txqueuelen 1000

[-- Attachment #2: Type: text/html, Size: 3510 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[moeller0]

Hi Etienne,

I could be out to lunch, but I always assumed for virtual interfaces the txque does not matter as packets are immediate passed to the real interface.

Best Regards
	Sebastian

> On Jan 20, 2016, at 18:09 , Etienne Champetier <champetier.etienne@gmail.com> wrote:
> 
> 
> 
> 2016-01-20 17:54 GMT+01:00 Brandon Applegate <brandon@burn.net>:
> 
>> On Jan 20, 2016, at 11:47 AM, Etienne Champetier <champetier.etienne@gmail.com> wrote:
>> 
>> Hi Brandon,
>> 
>> e quickly read the thread (maybe too quickly),
>> from what i remember for virtual interface txqueuelen is 0 by default, so nothing work as expected
>> can you check the txqueuelen with ip -a ?
>> (i've found no reference to txqueuelen in sqm-scripts)
>> 
>> see also
>> https://wiki.openwrt.org/doc/howto/packet.scheduler/packet.scheduler#tips 
> 
> I haven’t done anything manually to change txqueuelen.  Whatever is here is default (1000 on eth) or I assume set by sqm-scripts / tc (32).
> 
> I believe this is the output you are looking for:
> 
> —
> 
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP qlen 1000
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP qlen 1000
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 4: eth0.666@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UP 
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 5: eth0.10@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 6: eth0.11@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
>     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> 7: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
>     link/ether 52:18:cd:a7:ad:71 brd ff:ff:ff:ff:ff:ff
> 8: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
>     link/ether 5e:f0:86:88:8d:8e brd ff:ff:ff:ff:ff:ff
> 11: ifb4eth0.666: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc htb state UNKNOWN qlen 32
>     link/ether 86:68:83:69:2e:b2 brd ff:ff:ff:ff:ff:ff
> —
> 
> 
> no qlen means 0 I think
> maybe try
> ip link set eth0.666 txqueuelen 1000
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Etienne Champetier]

[-- Attachment #1: Type: text/plain, Size: 2979 bytes --]

Hi,

2016-01-20 18:42 GMT+01:00 moeller0 <moeller0@gmx.de>:

> Hi Etienne,
>
> I could be out to lunch, but I always assumed for virtual interfaces the
> txque does not matter as packets are immediate passed to the real interface.
>

This was not the case in the past (I remember losing quite some time on
this some years ago),
but i don't know if it's still the case.

2 choices: tests or read the code :)



>
> Best Regards
>         Sebastian
>
> > On Jan 20, 2016, at 18:09 , Etienne Champetier <
> champetier.etienne@gmail.com> wrote:
> >
> >
> >
> > 2016-01-20 17:54 GMT+01:00 Brandon Applegate <brandon@burn.net>:
> >
> >> On Jan 20, 2016, at 11:47 AM, Etienne Champetier <
> champetier.etienne@gmail.com> wrote:
> >>
> >> Hi Brandon,
> >>
> >> e quickly read the thread (maybe too quickly),
> >> from what i remember for virtual interface txqueuelen is 0 by default,
> so nothing work as expected
> >> can you check the txqueuelen with ip -a ?
> >> (i've found no reference to txqueuelen in sqm-scripts)
> >>
> >> see also
> >>
> https://wiki.openwrt.org/doc/howto/packet.scheduler/packet.scheduler#tips
> >
> > I haven’t done anything manually to change txqueuelen.  Whatever is here
> is default (1000 on eth) or I assume set by sqm-scripts / tc (32).
> >
> > I believe this is the output you are looking for:
> >
> > —
> >
> > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
> >     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> > 2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state
> UP qlen 1000
> >     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> > 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state
> UP qlen 1000
> >     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> > 4: eth0.666@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb
> state UP
> >     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> > 5: eth0.10@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UP
> >     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> > 6: eth0.11@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UP
> >     link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
> > 7: ifb0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
> >     link/ether 52:18:cd:a7:ad:71 brd ff:ff:ff:ff:ff:ff
> > 8: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 32
> >     link/ether 5e:f0:86:88:8d:8e brd ff:ff:ff:ff:ff:ff
> > 11: ifb4eth0.666: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc htb state
> UNKNOWN qlen 32
> >     link/ether 86:68:83:69:2e:b2 brd ff:ff:ff:ff:ff:ff
> > —
> >
> >
> > no qlen means 0 I think
> > maybe try
> > ip link set eth0.666 txqueuelen 1000
> >
> > _______________________________________________
> > Bloat mailing list
> > Bloat@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/bloat
>
>

[-- Attachment #2: Type: text/html, Size: 4330 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[moeller0]

Hi Brandon,

> On Jan 20, 2016, at 17:10 , Brandon Applegate <brandon@burn.net> wrote:
> 
> I’m getting more confused as I go on :)
> 
> So I’ve rebooted and the tc classes seem to have come back.  Since this is Ubuntu 12.04 - it doesn NOT have systemd.  I mention this because I see there is a systemd config in the sqm-scripts package.
> 
> I have not added any hooks to run 'sqm start’ - neverthless - I have all the rules seemingly there on a fresh boot.  I also have a new ‘interface’ - ifb4eth0.666.  

	This is ours ;) sqm-scripts needs an intermediary functional device or IFB for short to allow attaching a qdisc to the ingress direction; and the IFB4${IFACE} is the convention we adopted in sqm scripts. If you set ingress shaping to 0 this should not appear at all (or only transiently).

> Since I’ve never messed with tc - I have no idea how/where/what is ‘saving’ these rules and making them persistent.  I’m struggling to use google and grep -ir to see where Ubuntu is saving this.

	I believe that has been resolved in later posts already...

> 
> Furthermore - the scripts seem to be working now.  Slightly embarassing - it could be the result of having mucked with the gentoo script and having two many variables flying around at once.

	So sqm-scripts now works in both direction as you expect it? Preserving latency under load performance for a relatively small bandwidth sacrifice?

> 
> I currently lose a small fraction of my bandwidth - but bufferbloat gets an A on the dslreports speedtest.

	Well, you can always try interative to relax the shaper settings again to recover some of the lost bandwidth. But please note that in the ingress direction the shaper works the better the larger the difference between the artificial bottleneck rate (you set for the shaper) and the true bottleneck rate is; I seem to recall that people typically are sufficiently happy with ingress shaping rates of around 85-90% of ingress line-rate.

Best Regards
	Sebastian

> 
> I’m going to concentrate on understanding how / where these rules are getting made ‘persistent’.

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[moeller0]

Hi Brandon,


> On Jan 20, 2016, at 00:33 , Brandon Applegate <brandon@burn.net> wrote:
> 
> Disclaimer: if this is the wrong list for such a question - let me know.  This is specifically about the sqm-scripts package...
> 
> Hello,
> 
> I’ve been reading all I can on the bufferbloat website and also trying to understand the evolution of the various scripts (debloat, sqm, etc).
> 
> I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC - no *wrt etc).  Got it built with the ‘linux’ platform.  Since this is Ubuntu 12.04 - I had to cheat a bit and pull down the iproute2 source from 14.04.  I’ve tweaked the main sqm script to reflect this for the tc bindary - this is working.  I also updated my kernel to a later version that supports fq_codel.

	Great, could I convince you to post a quick description of the required steps somewhere linkeable on the net (in case we actually get it to work correctly first ;) )

> 
> My topology is ‘on a stick’.  I have one gig interface to a managed switch, on which are eth0.666 (outside/wan) and eth0.10 (inside).

	Okay, that is a configuration that has not received much testing in the past…

> 
> I have 30/5 cable service, and have tried both those values as well as 90% in my /etc/sqm/*conf file.
> 
> I’ve tried both eth0 (raw/parent interface) as well as eth0.666.

	Eth0 is not going to work in that situation as you will send and receive both incoming and outgoing traffic on that interface, so we really need to configure it correctly on the VLAN interfaces. I would like to propose to start with egress/uplink first and handle ingress afterwards. If you select eth0.666 (the superstitious among us would probably try a different VLAN number ;) ) and only configure the upload bandwidth but set download to 0, effectively sqm will only try to shape the egress traffic. I would propose to try this first, if that works let’s tackle download/ingress okay?

> 
> No matter what I do - my bandwidth is 10% of what it should be.  

	Here a comprehensive list what you actually did might help to form educated hypothesis what is going on...

> I get approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat looks great though - A+.

	Mmmh, while I would love to declare success (bufferbloat quashed, let’s move on) I have a hunch you are not too impressed with that solution ;)

> 
> Is there something inherent I’m doing wrong ?  

	No, by all means you are doing the right thing, namely helping us making sqm robust under more different conditions, thanks.


> Something to do with my ‘on a stick’ topology biting me ?  

	Could be, but nobody knows.

> Kernel version (Ubuntu’s 3.13.0-74-generic btw).
> 
> Thanks in advance for any help or info (or pointer to a more appropriate list).

	We recently taught sqm a whole new level of verbosity, please have a look at Readme.md on https://github.com/tohojo/sqm-scripts , I believe under non openwrt systems you might need to set:
[ -z "$SQM_VERBOSITY" ] && SQM_VERBOSITY=$VERBOSITY_DEBUG
instead of the default:
[ -z "$SQM_VERBOSITY" ] && SQM_VERBOSITY=$VERBOSITY_INFO
and then issue:
/usr/bin/sqm/sqm-bin stop
/usr/bin/sqm/sqm-bin start
manually to get more verbose output, You could also try setting SQM_DEBUG unconditionally to 1 in defaults.sh (in addition to raising the default log level to SQM_DEBUG) to get debug logs under:
/var/run/sqm/eth0.666.debug.log
or similar. That ideally will contain all debug output as well as all calls to the tc and ip and iptables binaries and their output, which should be plenty information to figure out the root cause of your issues.

Best Regards
	Sebastian


> 
> --
> Brandon Applegate - CCIE 10273
> PGP Key fingerprint:
> 830B 4802 1DD4 F4F9 63FE  B966 C0A7 189E 9EC0 3A74
> "SH1-0151.  This is the serial number, of our orbital gun."
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Rich Brown]

Hi Brandon,

One more (very) simple test - what are your speeds *without* any SQM/shaping? I didn't see it mentioned in your report, and let's be sure that you're shaping the traffic to the actual achievable link speeds...

I recently helped a friend whose "15 mbps/1mbps" DSL just wouldn't get above ~725kbps up without inducing latency/bloat. It turns out that the uplink could only achieve 3/4 of its "rated" speed. 

Best,

Rich



> On Jan 20, 2016, at 6:30 AM, moeller0 <moeller0@gmx.de> wrote:
> 
> Hi Brandon,
> 
> 
>> On Jan 20, 2016, at 00:33 , Brandon Applegate <brandon@burn.net> wrote:
>> 
>> Disclaimer: if this is the wrong list for such a question - let me know.  This is specifically about the sqm-scripts package...
>> 
>> Hello,
>> 
>> I’ve been reading all I can on the bufferbloat website and also trying to understand the evolution of the various scripts (debloat, sqm, etc).
>> 
>> I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC - no *wrt etc).  Got it built with the ‘linux’ platform.  Since this is Ubuntu 12.04 - I had to cheat a bit and pull down the iproute2 source from 14.04.  I’ve tweaked the main sqm script to reflect this for the tc bindary - this is working.  I also updated my kernel to a later version that supports fq_codel.
> 
> 	Great, could I convince you to post a quick description of the required steps somewhere linkeable on the net (in case we actually get it to work correctly first ;) )
> 
>> 
>> My topology is ‘on a stick’.  I have one gig interface to a managed switch, on which are eth0.666 (outside/wan) and eth0.10 (inside).
> 
> 	Okay, that is a configuration that has not received much testing in the past…
> 
>> 
>> I have 30/5 cable service, and have tried both those values as well as 90% in my /etc/sqm/*conf file.
>> 
>> I’ve tried both eth0 (raw/parent interface) as well as eth0.666.
> 
> 	Eth0 is not going to work in that situation as you will send and receive both incoming and outgoing traffic on that interface, so we really need to configure it correctly on the VLAN interfaces. I would like to propose to start with egress/uplink first and handle ingress afterwards. If you select eth0.666 (the superstitious among us would probably try a different VLAN number ;) ) and only configure the upload bandwidth but set download to 0, effectively sqm will only try to shape the egress traffic. I would propose to try this first, if that works let’s tackle download/ingress okay?
> 
>> 
>> No matter what I do - my bandwidth is 10% of what it should be.  
> 
> 	Here a comprehensive list what you actually did might help to form educated hypothesis what is going on...
> 
>> I get approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat looks great though - A+.
> 
> 	Mmmh, while I would love to declare success (bufferbloat quashed, let’s move on) I have a hunch you are not too impressed with that solution ;)
> 
>> 
>> Is there something inherent I’m doing wrong ?  
> 
> 	No, by all means you are doing the right thing, namely helping us making sqm robust under more different conditions, thanks.
> 
> 
>> Something to do with my ‘on a stick’ topology biting me ?  
> 
> 	Could be, but nobody knows.
> 
>> Kernel version (Ubuntu’s 3.13.0-74-generic btw).
>> 
>> Thanks in advance for any help or info (or pointer to a more appropriate list).
> 
> 	We recently taught sqm a whole new level of verbosity, please have a look at Readme.md on https://github.com/tohojo/sqm-scripts , I believe under non openwrt systems you might need to set:
> [ -z "$SQM_VERBOSITY" ] && SQM_VERBOSITY=$VERBOSITY_DEBUG
> instead of the default:
> [ -z "$SQM_VERBOSITY" ] && SQM_VERBOSITY=$VERBOSITY_INFO
> and then issue:
> /usr/bin/sqm/sqm-bin stop
> /usr/bin/sqm/sqm-bin start
> manually to get more verbose output, You could also try setting SQM_DEBUG unconditionally to 1 in defaults.sh (in addition to raising the default log level to SQM_DEBUG) to get debug logs under:
> /var/run/sqm/eth0.666.debug.log
> or similar. That ideally will contain all debug output as well as all calls to the tc and ip and iptables binaries and their output, which should be plenty information to figure out the root cause of your issues.
> 
> Best Regards
> 	Sebastian
> 
> 
>> 
>> --
>> Brandon Applegate - CCIE 10273
>> PGP Key fingerprint:
>> 830B 4802 1DD4 F4F9 63FE  B966 C0A7 189E 9EC0 3A74
>> "SH1-0151.  This is the serial number, of our orbital gun."
>> 
>> _______________________________________________
>> Bloat mailing list
>> Bloat@lists.bufferbloat.net
>> https://lists.bufferbloat.net/listinfo/bloat
> 
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Brandon Applegate]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

> On Jan 20, 2016, at 7:37 AM, Rich Brown <richb.hanover@gmail.com> wrote:
> 
> Hi Brandon,
> 
> One more (very) simple test - what are your speeds *without* any SQM/shaping? I didn't see it mentioned in your report, and let's be sure that you're shaping the traffic to the actual achievable link speeds...
> 
> I recently helped a friend whose "15 mbps/1mbps" DSL just wouldn't get above ~725kbps up without inducing latency/bloat. It turns out that the uplink could only achieve 3/4 of its "rated" speed.
> 

I get the advertised 30mbit down / 5mbit up.  This is doing one test at a time.  Obviously because of induced latency / bufferbloat - I can tell if my wife is uploading a bunch of pictures somewhere (my ssh sessions get very choppy).  But my vanilla speeds are exactly what I would expect.

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Brandon Applegate]

[-- Attachment #1: Type: text/plain, Size: 4461 bytes --]

I’ll try to tackle what I can for now below inline.

> On Jan 20, 2016, at 6:30 AM, moeller0 <moeller0@gmx.de> wrote:
> 
> Hi Brandon,
> 
> 
>> On Jan 20, 2016, at 00:33 , Brandon Applegate <brandon@burn.net> wrote:
>> 
>> Disclaimer: if this is the wrong list for such a question - let me know.  This is specifically about the sqm-scripts package...
>> 
>> Hello,
>> 
>> I’ve been reading all I can on the bufferbloat website and also trying to understand the evolution of the various scripts (debloat, sqm, etc).
>> 
>> I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC - no *wrt etc).  Got it built with the ‘linux’ platform.  Since this is Ubuntu 12.04 - I had to cheat a bit and pull down the iproute2 source from 14.04.  I’ve tweaked the main sqm script to reflect this for the tc bindary - this is working.  I also updated my kernel to a later version that supports fq_codel.
> 
> 	Great, could I convince you to post a quick description of the required steps somewhere linkeable on the net (in case we actually get it to work correctly first ;) )
> 
>> 
>> My topology is ‘on a stick’.  I have one gig interface to a managed switch, on which are eth0.666 (outside/wan) and eth0.10 (inside).
> 
> 	Okay, that is a configuration that has not received much testing in the past…
> 
>> 
>> I have 30/5 cable service, and have tried both those values as well as 90% in my /etc/sqm/*conf file.
>> 
>> I’ve tried both eth0 (raw/parent interface) as well as eth0.666.
> 
> 	Eth0 is not going to work in that situation as you will send and receive both incoming and outgoing traffic on that interface, so we really need to configure it correctly on the VLAN interfaces. I would like to propose to start with egress/uplink first and handle ingress afterwards. If you select eth0.666 (the superstitious among us would probably try a different VLAN number ;) ) and only configure the upload bandwidth but set download to 0, effectively sqm will only try to shape the egress traffic. I would propose to try this first, if that works let’s tackle download/ingress okay

I set DOWNLINK to 0 and here are the results (again - using speedtest.dslreports.com)

Download: 36 mbit (bufferbloat through the roof)
Upload: Slowly climbed to ~ 4.5mbit (bufferbloat good/very low)

>> 
>> No matter what I do - my bandwidth is 10% of what it should be.
> 
> 	Here a comprehensive list what you actually did might help to form educated hypothesis what is going on...
> 
>> I get approx. 3/4mbit down + 2/3mbit up on dslreports speedtest.  Bufferbloat looks great though - A+.
> 
> 	Mmmh, while I would love to declare success (bufferbloat quashed, let’s move on) I have a hunch you are not too impressed with that solution ;)
> 
>> 
>> Is there something inherent I’m doing wrong ?
> 
> 	No, by all means you are doing the right thing, namely helping us making sqm robust under more different conditions, thanks.
> 
> 
>> Something to do with my ‘on a stick’ topology biting me ?
> 
> 	Could be, but nobody knows.
> 
>> Kernel version (Ubuntu’s 3.13.0-74-generic btw).
>> 
>> Thanks in advance for any help or info (or pointer to a more appropriate list).
> 
> 	We recently taught sqm a whole new level of verbosity, please have a look at Readme.md on https://github.com/tohojo/sqm-scripts , I believe under non openwrt systems you might need to set:
> [ -z "$SQM_VERBOSITY" ] && SQM_VERBOSITY=$VERBOSITY_DEBUG
> instead of the default:
> [ -z "$SQM_VERBOSITY" ] && SQM_VERBOSITY=$VERBOSITY_INFO
> and then issue:
> /usr/bin/sqm/sqm-bin stop
> /usr/bin/sqm/sqm-bin start
> manually to get more verbose output, You could also try setting SQM_DEBUG unconditionally to 1 in defaults.sh (in addition to raising the default log level to SQM_DEBUG) to get debug logs under:
> /var/run/sqm/eth0.666.debug.log
> or similar. That ideally will contain all debug output as well as all calls to the tc and ip and iptables binaries and their output, which should be plenty information to figure out the root cause of your issues.

I can’t get it to log anything for the life of me.  I’ve tried:

SQM_DEBUG=1 SQM_VERBOSITY=8 /etc/init.d/sqm stop ; SQM_DEBUG=1 SQM_VERBOSITY=8 /etc/init.d/sqm start

As well as temporarily mucking around in the defaults.sh script.  It never writes anything to the /var/run (but the state gets written, so I don’t think it’s perms).

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 842 bytes --]

Re: [Bloat] Getting started with sqm-scripts - latency good, bandwidth decimated

[Sebastian Moeller]

Hi Brandon,

First, thanks a lot.

On January 20, 2016 4:30:25 PM GMT+01:00, Brandon Applegate <brandon@burn.net> wrote:
>I’ll try to tackle what I can for now below inline.
>
>> On Jan 20, 2016, at 6:30 AM, moeller0 <moeller0@gmx.de> wrote:
>> 
>> Hi Brandon,
>> 
>> 
>>> On Jan 20, 2016, at 00:33 , Brandon Applegate <brandon@burn.net>
>wrote:
>>> 
>>> Disclaimer: if this is the wrong list for such a question - let me
>know.  This is specifically about the sqm-scripts package...
>>> 
>>> Hello,
>>> 
>>> I’ve been reading all I can on the bufferbloat website and also
>trying to understand the evolution of the various scripts (debloat,
>sqm, etc).
>>> 
>>> I managed to get sqm-scripts on my firewall (Ubuntu linux on a PC -
>no *wrt etc).  Got it built with the ‘linux’ platform.  Since this is
>Ubuntu 12.04 - I had to cheat a bit and pull down the iproute2 source
>from 14.04.  I’ve tweaked the main sqm script to reflect this for the
>tc bindary - this is working.  I also updated my kernel to a later
>version that supports fq_codel.
>> 
>> 	Great, could I convince you to post a quick description of the
>required steps somewhere linkeable on the net (in case we actually get
>it to work correctly first ;) )
>> 
>>> 
>>> My topology is ‘on a stick’.  I have one gig interface to a managed
>switch, on which are eth0.666 (outside/wan) and eth0.10 (inside).
>> 
>> 	Okay, that is a configuration that has not received much testing in
>the past…
>> 
>>> 
>>> I have 30/5 cable service, and have tried both those values as well
>as 90% in my /etc/sqm/*conf file.
>>> 
>>> I’ve tried both eth0 (raw/parent interface) as well as eth0.666.
>> 
>> 	Eth0 is not going to work in that situation as you will send and
>receive both incoming and outgoing traffic on that interface, so we
>really need to configure it correctly on the VLAN interfaces. I would
>like to propose to start with egress/uplink first and handle ingress
>afterwards. If you select eth0.666 (the superstitious among us would
>probably try a different VLAN number ;) ) and only configure the upload
>bandwidth but set download to 0, effectively sqm will only try to shape
>the egress traffic. I would propose to try this first, if that works
>let’s tackle download/ingress okay
>
>I set DOWNLINK to 0 and here are the results (again - using
>speedtest.dslreports.com)
>
>Download: 36 mbit (bufferbloat through the roof)
>Upload: Slowly climbed to ~ 4.5mbit (bufferbloat good/very low)

Okay so it seems the uplink shaping is working as intended. Good so it seems you are ready to tackle ingress/download next ;)

>
>>> 
>>> No matter what I do - my bandwidth is 10% of what it should be.
>> 
>> 	Here a comprehensive list what you actually did might help to form
>educated hypothesis what is going on...
>> 
>>> I get approx. 3/4mbit down + 2/3mbit up on dslreports speedtest. 
>Bufferbloat looks great though - A+.
>> 
>> 	Mmmh, while I would love to declare success (bufferbloat quashed,
>let’s move on) I have a hunch you are not too impressed with that
>solution ;)
>> 
>>> 
>>> Is there something inherent I’m doing wrong ?
>> 
>> 	No, by all means you are doing the right thing, namely helping us
>making sqm robust under more different conditions, thanks.
>> 
>> 
>>> Something to do with my ‘on a stick’ topology biting me ?
>> 
>> 	Could be, but nobody knows.
>> 
>>> Kernel version (Ubuntu’s 3.13.0-74-generic btw).
>>> 
>>> Thanks in advance for any help or info (or pointer to a more
>appropriate list).
>> 
>> 	We recently taught sqm a whole new level of verbosity, please have a
>look at Readme.md on https://github.com/tohojo/sqm-scripts , I believe
>under non openwrt systems you might need to set:
>> [ -z "$SQM_VERBOSITY" ] && SQM_VERBOSITY=$VERBOSITY_DEBUG
>> instead of the default:
>> [ -z "$SQM_VERBOSITY" ] && SQM_VERBOSITY=$VERBOSITY_INFO
>> and then issue:
>> /usr/bin/sqm/sqm-bin stop
>> /usr/bin/sqm/sqm-bin start
>> manually to get more verbose output, You could also try setting
>SQM_DEBUG unconditionally to 1 in defaults.sh (in addition to raising
>the default log level to SQM_DEBUG) to get debug logs under:
>> /var/run/sqm/eth0.666.debug.log
>> or similar. That ideally will contain all debug output as well as all
>calls to the tc and ip and iptables binaries and their output, which
>should be plenty information to figure out the root cause of your
>issues.
>
>I can’t get it to log anything for the life of me.  I’ve tried:
>
>SQM_DEBUG=1 SQM_VERBOSITY=8 /etc/init.d/sqm stop ; SQM_DEBUG=1
>SQM_VERBOSITY=8 /etc/init.d/sqm start

I am unsure whether this will work outside of openwrt, as we needed some extra code in run.sh to pass these on from the command line.
>
>As well as temporarily mucking around in the defaults.sh script.  It
>never writes anything to the /var/run (but the state gets written, so I
>don’t think it’s perms).

That is sad. Maybe we try to solve your real problem first and then maybe if you are still patient enough we could try to improve sqm scripts?
Best Regards
        Sebastian

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.