[Cake] inbound cake or fq_codel shaping fails on cable on netflix reno

[Dave Taht <dave.taht@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2813 bytes --]

This is something I noticed years ago, inspired me to think about a
"bobbie" policer, and I decided I could live with, and never poked
into further. After our successes with shaping inbound cable at
then-typical 20mbit rates, I was happy, although never satisifed with
the 85% magic number we use to come up with a set rate for inbound
shaping.

Simultaneously with all that work we did on sqm, linux added tcp tsq,
pacing, etc, etc and in general inbound shaping cable seems to work ok
against one or more linux tcp flows. We end up with some persistent
queuing delay (at the cmts in the 5-15ms range) which I've generally
assumed was unavoidable that fq cannot cut through (oh, I dream of FQ
at the CMTS!)

BUT: In testing fast.com's test, I see it fail to shape it to anything
sane, with spikes of up to 160ms.

So, anyway, I've seen this pathology before with netflix flows. What I
didn't realize then was that it was independent of the shaped rate.
(see plots) It's independent of the rtt setting in cake too (at least,
down to 25ms). My assumption is the netflix (freebsd) traffic + the
cable is so bursty as to not let codel keep improving its drop rate.

I'm curious if it's also the case on other link layer techs (dsl, fiber)?

The structure of my test is simple: shape inbound to 80% of the
cablemodem rate, setup irtt (not strictly needed), start this,

root # flent -H flent-fremont.bufferbloat.net -t 'your_location' -s .02 ping

, let it run a few sec, then run the fast.com test.

I tried to verify this using linux reno on a recent kernel, but that
seemed healthy. Assuming it actually got reno. Or that this weirdness
is a function of the RTT.

1) Can someone else on a cablemodem (even without the latest cake,
this happens to me on older cake and fq_codel) try this test?
2) Can someone with a dsl or fiber device try this test?
3) Is there a freebsd box "out on the net", 45ms or so from me, we can
setup netperf/irtt/etc on to run flent with? (I can donate a linode in
LA I but we'd need someone that can setup freebsd)

Some pics attached, flent data files at:
http://www.taht.net/~d/fast_vs_cable.tgz

PS I also have two other issues going on. This is the first time I've
been using irtt with a 20ms interval, and I regularly see single 50+ms
spikes (in both ping and irtt) data and also see irtt stop
transmitting. On this front, it could merely be that my (not tested in
months!) test cablemodem setup is malfunctioning also! Or we're
hitting power save. Or (finally) seeing request-grant delays. Or
scheduling delay somewhere in the net-next kernel I'm using... Or....
(regardless, this seems independent of my main issue, and I've not had
such high res data before)
-- 

Dave Täht
CEO, TekLibre, LLC
http://www.teklibre.com

[-- Attachment #2: ping_-_flent-newark-reno.png --]
[-- Type: image/png, Size: 134097 bytes --]

[-- Attachment #3: ping_-_shape_fail_60mbit.png --]
[-- Type: image/png, Size: 51478 bytes --]

[-- Attachment #4: ping_-_shape_fail_40mbit.png --]
[-- Type: image/png, Size: 61707 bytes --]