[Bloat] documentation review request and out of tree cake builds for openwrt/etc.

[Bloat] documentation review request and out of tree cake builds for openwrt/etc.

[Dave Taht]

1) I have burned much of the morning updating this:

http://www.bufferbloat.net/projects/codel/wiki/Cake

Reviewers wanted.

2) I am not sure of the right configuration to push stuff to the
iproute2-cake repo (removing sfq_codel, and there is an important
bugfix in the mainline iproute2 that needs to be pushed into it)

3) I have pushed support for building cake out of tree in openwrt to
the ceropackages repo (for now), but have not created a new
iproute2-cake for it due to item 2. My intent would be to get both
into openwrt chaos calmer for further testing... in some repo or
another that wasn't ceropackages. the routing repo?

4) http://www.dslreports.com/forum/news,133536~fmode=flat~days=2000
has some good convo going on also.

-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Bloat] [Cake] documentation review request and out of tree cake builds for openwrt/etc.

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 469 bytes --]

> An integral shaper (that can be on or off or tuned dynamically)
> ---
> is much "tighter" than htb - works on current low end hardware without
running out of cpu. See attached graphs.

This seems to imply that "tighter" means "uses less CPU". In fact they are
two separate benefits; "tighter" means "bursts less".

Also, what graphs?

As for installing kernel headers, on Debian based distros the right package
should be linux-headers-`uname -r` .

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 623 bytes --]

Re: [Bloat] [Cake] documentation review request and out of tree cake builds for openwrt/etc.

[Dave Taht]

On Tue, Apr 28, 2015 at 1:52 PM, Jonathan Morton <chromatix99@gmail.com> wrote:
>> An integral shaper (that can be on or off or tuned dynamically)
>> ---
>> is much "tighter" than htb - works on current low end hardware without
>> running out of cpu. See attached graphs.
>
> This seems to imply that "tighter" means "uses less CPU". In fact they are
> two separate benefits; "tighter" means "bursts less".

OK, will fix.

> Also, what graphs?

I did not put them up because I was fiddling with the dslreports test
at the same time as a rrul test and I got a result that I do not
understand.

http://snapon.lab.bufferbloat.net/~d/dsl/vsdslreports_at_the_sametime.png

(cake and pie results in the same dir, unplotted)

dslreports test result: http://www.dslreports.com/speedtest/384651

I have not fully internalized what a 10x1 ratio of down to up
bandwidth "looks like" - where a full download stream would basically
fill half the uplink with acks (every other ack) or the whole thing
(on tcp's acking on every packet), and I have internalized that aqms
have little effect on pure ack traffic... but....

What I had expected to see was about a 2ms (tops!) further increase in
measured inbound latency here. A single full mtu packet at 100Mbit is
130us, and I was under the impression the dslreports cable test was 16
inbound flows, so 130us * 16 = 2ms. The number of acks in flight
should have been constant, but got a great deal more service time, but
that is not evident from the uplink...

Where is the extra latency coming from at T+50? It is repeatable no
matter the inbound qdisc (tried pie, fq_codel, cake) This bump from 4
flows to 20 should ultimately have been handled, AND with FQ in place,
the measure at 100mbit should actually have stayed pretty flat, just
as it did for outbound. Instead, 10ms is added.

A) I have generally worried about the effects of pacing on packet
aggregation in cable modems, etc. HTB is doing a bit of pacing that
might be affecting media access opportunities here, and/or we are
running out of room to wedge acks into an aggregate.
B) Also DRR tends to release bunches of acks per TCP flow which in
turn releases a bunch in response...
C) NAPI?
D) Servicing tons of acks eats cpu
E) Something else?

Ah, well, I guess I gotta go try a test on ethernet at the same rates
and setup, and fiddle with owamp...

The topology here was:

OSX - wifi          - rangeley box - cablemodem - internet
Linux - ethernet - rangeley box

>
> As for installing kernel headers, on Debian based distros the right package
> should be linux-headers-`uname -r` .
>
> - Jonathan Morton



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67

Re: [Bloat] [Cake] documentation review request and out of tree cake builds for openwrt/etc.

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

It took me a while to get around to thinking about this, partly because my
phone inexplicably refuses to believe snapon exists.

I have two possible explanations for these results. Maybe both apply to
some extent.

Dropping packets rather than marking them results in an increase in ack
density in the reverse direction, because delayed acks get temporarily
disabled. The strength of this effect depends on the BDP and the depth of
delayed acks.

Increasing the number of simultaneous flows might increase the CPU load of
connection tracking for NAT. Are you shaping and doing NAT on the same box?
I think this might be the basic reason for increased latency.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 766 bytes --]

Re: [Bloat] [Cake] documentation review request and out of tree cake builds for openwrt/etc.

[Jonathan Morton]

[-- Attachment #1: Type: text/plain, Size: 760 bytes --]

I'm not concerned about aggregation effects on cable, because it's not
station specific as it is on Wi-Fi. It might be a source of one extra
access grant delay at most; after that there'll be enough packets in the
modem's FIFO to justify a full sized grant. Here the modem's buffer really
does exist for a good reason, and we can rely on it to do the job.

I'm also not concerned about ack bunching, because realistically that isn't
really caused by FQ. Given a 10:1 bandwidth ratio, and a 3:1 delayed ack
factor, there'll be 3.33 acks for each data packet in the slow direction,
while at our default 300 quantum the DRR will cycle five times per data
packet. So acks for a given flow will only be delivered bunched if they
arrived bunched.

- Jonathan Morton

[-- Attachment #2: Type: text/html, Size: 851 bytes --]

Re: [Bloat] [Cake] documentation review request and out of tree cake builds for openwrt/etc.

[Dave Taht]

On Thu, Apr 30, 2015 at 7:38 AM, Jonathan Morton <chromatix99@gmail.com> wrote:
> It took me a while to get around to thinking about this, partly because my
> phone inexplicably refuses to believe snapon exists.

It has an old dnssec signed dns tree, using the isc dlv, which turned
out to somewhat break older versions of dnsmasq, if that helps any.
Dire need for some sysadmin help on bufferbloat.net, the cruft has accumulated.

> I have two possible explanations for these results. Maybe both apply to some
> extent.
>
> Dropping packets rather than marking them results in an increase in ack
> density in the reverse direction, because delayed acks get temporarily
> disabled. The strength of this effect depends on the BDP and the depth of
> delayed acks.

Hmm.

>
> Increasing the number of simultaneous flows might increase the CPU load of
> connection tracking for NAT. Are you shaping and doing NAT on the same box?
> I think this might be the basic reason for increased latency.

That particular setup (yurtlab) offloads the NAT to the cable modem.
The 110_11Mbit shaping was of course, done on the rangeley. It is
possible given the older kernel that perhaps I could be hitting the
same power mgmt issue that someone else here ran into earlier, or
merely the kernel is weird, or something new like xmit_more is messing
up life.

I returned to another location (gf) and tested 50mbit down/5mbit up
(cerowrt 3.10.50-1 in front of a live cablemodem, same two boxes
driving the test) up for much of the past 2 days, and on the same test
(rrul starting, then doing the dslreports test) observe expected
results, with the download having the normal brief spike, the upload
impact being nearly immeasurable, and the observed latency increase in
either direction, negligible.

http://snapon.lab.bufferbloat.net/~d/dslreports2/dslreports_vs_rrul_50_5mbit.png

verses the puzzling one:

http://snapon.lab.bufferbloat.net/~d/dsl/vsdslreports_at_the_sametime.png

So the next step for me is to get cake working in openwrt on hardware
fast enough to run at 110Mbit and returning to the yurtlab to try
it... but that won't be til sunday at best. Tho I almost got it built,
at least, last night. Still sorting through patches....





>
> - Jonathan Morton



-- 
Dave Täht
Open Networking needs **Open Source Hardware**

https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67