[Cerowrt-devel] better ingress shaping somehow

[Cerowrt-devel] better ingress shaping somehow

[Dave Taht]

On Thu, Oct 2, 2014 at 4:54 AM, David Lang <david@lang.hm> wrote:
> On Thu, 2 Oct 2014, Alpha Sparc wrote:
>
>> How good is the throughput on CeroWrt compared to OpenWrt ?
>
>
> The focus of CeroWrt is on reducing latency, not increasing throughput. If
> you run into really badd bufferbloat problems without these scrips, then
> these scripts can result more more 'goodput' (useable data as opposed to
> 'throughput' bits on the wire) getting through, but in the usual case there
> will be a (slight) reduction in the peak throughput.
>
> This is especially so on the inbound side of things because the router is
> having to work indirectly to throttle the senders so that they don't
> overload the router at the other end of the connection.
>
> I beleive that on the WNDR3800, it's able to work up to about 50Mb with the
> existing configurations. A faster CPU would do better, a slower one worse.

Actually it appears that cache is very important on fixing inbound rate shaping.

The octeon in the edgerouter lite peaks out at above 60mbits also, but the
bigger, fatter pro product actually manages quite a bit better (with increasing
inaccuracy however). See below...

> The re-write that Dave is talking about is hoting to improve this. From the
> pastebin link Dave listed below, they have it up to ~80Mb now

Oh, no, that's an x86 result. It's barely a blip on that cpu use though, so I am
encouraged so far. Haven't got around to porting it to mips, too many
non-working ideas elsewhere in it, ENOTIME.

But: the real killer, on ingress shaping, is the call to skb_clone in
sch_ingress->act_mirred path, I think.

If it were possible to have sch_cake replace sch_ingress, that would
be a huge win, but following the call path for why that must be cloned
has thus far resisted my archeological expertise. An network ninja
is needed here....


>>> And Jonathon morton has been pouring it all into
>>> pure C - with an integral bandwidth shaper that we
>>> hope will be faster and more efficient than htb.
>>>
>>> See an early result:
>>>
>>> http://pastebin.com/zz06WhJr
>>>
>>> It takes much of the heavy lifting out of the existing
>>> sqm scripts.
>>>
>>> tc qdisc add dev eth1 root cake bandwidth 80mbit
>>>
>>>
>>> So I don't know where to go. Certainly I'd like to
>>> see the battle hardened sqm scripts (which are more
>>> flexible than the C code above) get more widely used
>>> and in BB.
>>>
>>> openwrt users can do that today by adding the ceropackages repo to their
>>> build system.
>>> or just installing the sqm-scripts and luci-app-sqm.
>>>
>>> or we can clean it up further for openwrt mainline.
>>>
>>> But I haven't seen one core openwrt dev say, yes, we want this mainlined,
>>> here's what you need to fix, so I'm inclined to go back to my cave, get
>>> more sleep, and work on the successor.



-- 
Dave Täht

https://www.bufferbloat.net/projects/make-wifi-fast

Re: [Cerowrt-devel] better ingress shaping somehow

[William Katsak]

Do you have a working x86 build for Cero again? Or just playing with a 
workstation?

-Bill


On 10/02/2014 07:07 PM, Dave Taht wrote:
> Oh, no, that's an x86 result. It's barely a blip on that cpu use though, so I am
> encouraged so far. Haven't got around to porting it to mips, too many
> non-working ideas elsewhere in it, ENOTIME.

-- 
****************************************
William Katsak <wkatsak@gmail.com>
****************************************

Re: [Cerowrt-devel] better ingress shaping somehow

[Dave Taht]

On Thu, Oct 2, 2014 at 5:57 PM, William Katsak <wkatsak@gmail.com> wrote:
> Do you have a working x86 build for Cero again? Or just playing with a
> workstation?

No. I AM fiddling with the x86 build of openwrt, a bit. To me it seems easier to
port the sqm system to openwrt, and kill bridging, than go to the trouble
of porting cero...

I have several nuc boxes in the yurtlab that are used for load
testing, and recently
as part of the BQL rework, started building modern x86_64 kernels again.

A "plan", if I have it, is to try and clear my plate enough to make a
run at improving wifi.

Still buried from all the things I ignored for the last year...

There's an enormous backlog of bufferbloat.net work left to remedy, also.

Jonathon morton jumping in to write cake is a nice bonus, but unexpected.

>
> -Bill
>
>
>
> On 10/02/2014 07:07 PM, Dave Taht wrote:
>>
>> Oh, no, that's an x86 result. It's barely a blip on that cpu use though,
>> so I am
>> encouraged so far. Haven't got around to porting it to mips, too many
>> non-working ideas elsewhere in it, ENOTIME.
>
>
> --
> ****************************************
> William Katsak <wkatsak@gmail.com>
> ****************************************
> _______________________________________________
> Cerowrt-devel mailing list
> Cerowrt-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/cerowrt-devel



-- 
Dave Täht

https://www.bufferbloat.net/projects/make-wifi-fast

Re: [Cerowrt-devel] better ingress shaping somehow

[Sebastian Moeller]

Hi Dave,


On Oct 3, 2014, at 01:07 , Dave Taht <dave.taht@gmail.com> wrote:
[cut a lot...]
> But: the real killer, on ingress shaping, is the call to skb_clone in
> sch_ingress->act_mirred path, I think.

	So I thought that we basically have around 50Mbps “shaping-capability” on the wndrs which hew can distribute between egress and ingress, but if your analysis is correct wouldn’t that mean that shaping egress is cheaper? (Not that this has much relevance in the real world due uplink << downlink asymmetry the ISPs push onto us endusers)

> 
> If it were possible to have sch_cake replace sch_ingress, that would
> be a huge win, but following the call path for why that must be cloned
> has thus far resisted my archeological expertise. An network ninja
> is needed here....
[…]

	;) 

Best Regards
	Sebastian

Re: [Cerowrt-devel] better ingress shaping somehow

[Richard A. Smith]

On 10/02/2014 07:07 PM, Dave Taht wrote:

>> I beleive that on the WNDR3800, it's able to work up to about 50Mb with the
>> existing configurations. A faster CPU would do better, a slower one worse.
>
> Actually it appears that cache is very important on fixing inbound rate shaping.
>
> The octeon in the edgerouter lite peaks out at above 60mbits also, but the
> bigger, fatter pro product actually manages quite a bit better (with increasing
> inaccuracy however). See below...

Ugh.  Got any hard numbers on the top speed of the edgerouter with shaping?

At work we recently upgraded our link from 60/10 to 100/15 and the WNDR 
I have running the show can no longer keep up.  I've been looking for a 
replacement and the edgerouter lite was looking like a possibility.

I tried to use a TPLink AC1750 thinking that a newer setup with a faster 
cpu might be able to keep up.  If I don't have shaping enabled it can do 
200+ Mbit but with shaping it drops considerably.  It's still faster 
than the WNDR but seems to max out at about 85 Mbit.

-- 
Richard A. Smith  <richard@laptop.org>
Former One Laptop per Child

Re: [Cerowrt-devel] better ingress shaping somehow

[Dave Taht]

On Sun, Oct 5, 2014 at 9:26 AM, Richard A. Smith <richard@laptop.org> wrote:
> On 10/02/2014 07:07 PM, Dave Taht wrote:
>
>>> I beleive that on the WNDR3800, it's able to work up to about 50Mb with
>>> the
>>> existing configurations. A faster CPU would do better, a slower one
>>> worse.
>>
>>
>> Actually it appears that cache is very important on fixing inbound rate
>> shaping.
>>
>> The octeon in the edgerouter lite peaks out at above 60mbits also, but the
>> bigger, fatter pro product actually manages quite a bit better (with
>> increasing
>> inaccuracy however). See below...
>
>
> Ugh.  Got any hard numbers on the top speed of the edgerouter with shaping?
>
> At work we recently upgraded our link from 60/10 to 100/15 and the WNDR I
> have running the show can no longer keep up.  I've been looking for a
> replacement and the edgerouter lite was looking like a possibility.

Shaping the uplink only still helps a lot.

> I tried to use a TPLink AC1750 thinking that a newer setup with a faster cpu
> might be able to keep up.  If I don't have shaping enabled it can do 200+
> Mbit but with shaping it drops considerably.  It's still faster than the
> WNDR but seems to max out at about 85 Mbit.

The fastest-looking thing I've played with of late is the netgear x4.
It's dynamic QoS feature
is basically fq_codel, near as I can tell. I'll try pushing it to
120Mbit this week.

I do not have a high impression of it's overall firmware quality, and
haven't seen the gpl drop yet.

Aside from that, there's always x86.

> --
> Richard A. Smith  <richard@laptop.org>
> Former One Laptop per Child



-- 
Dave Täht

https://www.bufferbloat.net/projects/make-wifi-fast

Re: [Cerowrt-devel] better ingress shaping somehow

[Richard A. Smith]

On 10/05/2014 01:57 PM, Dave Taht wrote:

>> Ugh.  Got any hard numbers on the top speed of the edgerouter with shaping?
>>
>> At work we recently upgraded our link from 60/10 to 100/15 and the WNDR I
>> have running the show can no longer keep up.  I've been looking for a
>> replacement and the edgerouter lite was looking like a possibility.
>
> Shaping the uplink only still helps a lot.

Hmmm... At work I use OpenWRT barrier breaker and not ceroWRT.
I'm just using the stock qos script and luci.  How would I go about 
setting it to only do uplink?

> is basically fq_codel, near as I can tell. I'll try pushing it to
> 120Mbit this week.

Thanks.  That would be very helpful. If it works I'll get one.

We need QoS for our VoIP system.  If I don't run QoS the VoIP system is 
very unhappy.


> I do not have a high impression of it's overall firmware quality, and
> haven't seen the gpl drop yet.

There several pages that say its based on OpenWRT?  But I don't see any 
mention of that on OpenWRT page website and searches for r7500 or AC2350 
  or nighthawk don't seem to turn up anything.  Just a few hits for the 
r7000 and that the radio is not supported.

> Aside from that, there's always x86.

Yeah. That was going to be my fall back option if the edgerouter did not 
work.

I was thinking about getting one of these:

http://www.newegg.com/Product/Product.aspx?Item=N82E16856205007

and running OpenWRT x86 on it.

-- 
Richard A. Smith  <richard@laptop.org>
Former One Laptop per Child

Re: [Cerowrt-devel] better ingress shaping somehow

[Richard A. Smith]

On 10/05/2014 10:16 PM, Richard A. Smith wrote:

> Hmmm... At work I use OpenWRT barrier breaker and not ceroWRT.
> I'm just using the stock QoS script and luci.  How would I go about
> setting it to only do uplink?

Sorry. I didn't complete my thought here. The options in luci are minimal.

How would I go about tweaking the QoS config file such that it only does 
the uplink?

-- 
Richard A. Smith  <richard@laptop.org>
Former One Laptop per Child