[Bloat] ADSL/ATM linklayer tc shaping regression fix commits for stable

[Bloat] ADSL/ATM linklayer tc shaping regression fix commits for stable

[Jesper Dangaard Brouer]

Hi Greg (and google search),

ADSL/ATM linklayer shaping were broken in kernel release from 3.8 to
3.10 by commit 56b765b79 ("htb: improved accuracy at high rates").

I can see that 3.9-stable is marked EOL, and 3.8-stable also seems
closed.  I guess it is up to Greg, how far the commits described below
can be backported.  Below is the commit needed for a 3.9 backport.

This was not the only regression introduced by the regression commit,
several fixes were required:

(Refactor/change commit for easy 3.9 backport step (in v3.10-rc1)):
 commit 6906f4ed6f8 (htb: add HTB_DIRECT_QLEN attribute)

The following commit fixes are in 3.10-rc5:
 commit 01cb71d2d47 (net_sched: restore "overhead xxx" handling)
 commit 5343a7f8be1 (net_sched: htb: do not mix 1ns and 64ns time units)

The following commit fixes are in 3.10-rc6:
 commit 40edeff6e1c (net_sched: qdisc_get_rtab() must check data[] array)
 (this fix goes further back, but needed for linklayer atm, see below[1])

Refactor improvements (v3.11-rc1):
 commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)

The linklayer ATM/ADSL fix, reached 3.11-rc6:
 commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)


[1] Eric Dumazet also found a general "linklayer atm" regression
(dating way-back), which could cause rate-tables to get wrongly shared
between the same rates with-and-without linklayer atm settings.
Addressed/fixed in (3.10-rc6:):
 - commit 40edeff6e1c (net_sched: qdisc_get_rtab() must check data[] array)
When configuring two completely equal rates, the kernel detects that
these two equal rates can share the same rate-table.  But the kernel
didn't check if the rate-table data had been modified, which is done
in the linklayer atm case.
In practice, this often isn't a problem, as overhead parameter is
usually combined with the linklayer parameter.


The regression commit 56b765b79 ("htb: improved accuracy at high
rates"), is not all bad, it did provided a significantly improved rate
shaping accuracy.  The old rate-table based system, which this commit
removed, would get inaccurate at rates above 100Mbit/s.

(ps. I'm the original author of http://www.adsl-optimizer.dk/ and
http://sourceforge.net/projects/adsl-optimizer/)
-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Re: [Bloat] ADSL/ATM linklayer tc shaping regression fix commits for stable

[Greg Kroah-Hartman]

On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:
> Hi Greg (and google search),
> 
> ADSL/ATM linklayer shaping were broken in kernel release from 3.8 to
> 3.10 by commit 56b765b79 ("htb: improved accuracy at high rates").
> 
> I can see that 3.9-stable is marked EOL, and 3.8-stable also seems
> closed.  I guess it is up to Greg, how far the commits described below
> can be backported.  Below is the commit needed for a 3.9 backport.

As you say, 3.8 and 3.9 are end-of-life, so there's nothing I can, or
even want, to do with them, sorry.

If there are issues with 3.10, that's a different story.

> Refactor improvements (v3.11-rc1):
>  commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)
> 
> The linklayer ATM/ADSL fix, reached 3.11-rc6:
>  commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

David sends me the networking patches for the stable tree, and if he
thinks these are applicable, then I'll take them.

thanks,

greg k-h

Re: [Bloat] ADSL/ATM linklayer tc shaping regression fix commits for stable

[Jesper Dangaard Brouer]

On Tue, 20 Aug 2013 08:16:50 -0700
Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:
[...]
 
> If there are issues with 3.10, that's a different story.
> 
> > Refactor improvements (v3.11-rc1):
> >  commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)

Needed because it fixes the accuracy of the rate calc.

> > The linklayer ATM/ADSL fix, reached 3.11-rc6:
> >  commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)

Needed because we broke userspace interface.

> David sends me the networking patches for the stable tree, and if he
> thinks these are applicable, then I'll take them.

DaveM, please?  What do you want me to do, submit this req/patches somehow?

(Can see that is not currently on your stable queue via:
http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Re: [Bloat] ADSL/ATM linklayer tc shaping regression fix commits for stable

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

On Wed, Aug 21, 2013 at 5:03 AM, Jesper Dangaard Brouer
<brouer@redhat.com>wrote:

>
> On Tue, 20 Aug 2013 08:16:50 -0700
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> > On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:
> [...]
>
> > If there are issues with 3.10, that's a different story.
> >
> > > Refactor improvements (v3.11-rc1):
> > >  commit 130d3d68b52 (net_sched: psched_ratecfg_precompute()
> improvements)
>
> Needed because it fixes the accuracy of the rate calc.
>
> > > The linklayer ATM/ADSL fix, reached 3.11-rc6:
> > >  commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)
>
> Needed because we broke userspace interface.
>
> > David sends me the networking patches for the stable tree, and if he
> > thinks these are applicable, then I'll take them.
>
> DaveM, please?  What do you want me to do, submit this req/patches somehow?
>
> (Can see that is not currently on your stable queue via:
> http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)
>
>
+1 Dealing with the busted DSL stuff since 3.8 has been no fun and it would
be nice if these last fixes made it into 3.10.X at least so router/fw
distros can pick them up more easily. The difference in dsl behavior since
the bustage in 3.8 to the saner now is remarkable.

Probably not meeting the criteria for 3.10-stable inclusion (as they aren't
exactly bugfixes) are the tiny improvements to the flow dissector made in
3.11 for better ipip, ipv6 encapsulation, and 802.1ad support, but I
thought I'd mention them so other fq_codel folk pick them up that need them.

Not ready for any tree and increasingly doubtful for 3.12 are some
improvements under test to codel and cisco's sch_pie, if anyone wants to
play with them, let me know off list.


> --
> Best regards,
>   Jesper Dangaard Brouer
>   MSc.CS, Sr. Network Kernel Developer at Red Hat
>   Author of http://www.iptv-analyzer.org
>   LinkedIn: http://www.linkedin.com/in/brouer
>



-- 
Dave Täht

Fixing bufferbloat with cerowrt:
http://www.teklibre.com/cerowrt/subscribe.html

[-- Attachment #2: Type: text/html, Size: 3286 bytes --]

Re: [Bloat] ADSL/ATM linklayer tc shaping regression fix commits for stable

[David Miller]

From: Jesper Dangaard Brouer <brouer@redhat.com>
Date: Wed, 21 Aug 2013 14:03:50 +0200

> 
> On Tue, 20 Aug 2013 08:16:50 -0700
> Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
>> On Tue, Aug 20, 2013 at 01:26:29PM +0200, Jesper Dangaard Brouer wrote:
> [...]
>  
>> If there are issues with 3.10, that's a different story.
>> 
>> > Refactor improvements (v3.11-rc1):
>> >  commit 130d3d68b52 (net_sched: psched_ratecfg_precompute() improvements)
> 
> Needed because it fixes the accuracy of the rate calc.
> 
>> > The linklayer ATM/ADSL fix, reached 3.11-rc6:
>> >  commit 8a8e3d84b17 (net_sched: restore "linklayer atm" handling)
> 
> Needed because we broke userspace interface.
> 
>> David sends me the networking patches for the stable tree, and if he
>> thinks these are applicable, then I'll take them.
> 
> DaveM, please?  What do you want me to do, submit this req/patches somehow?
> 
> (Can see that is not currently on your stable queue via:
> http://patchwork.ozlabs.org/bundle/davem/stable/?state=*)

I'll put it there, you just have to formally ask me to do it that's
all.

Asking Greg to do it directly is a complete waste of time. :-)