[Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Dave Täht]

On 2/26/16 3:23 AM, De Schepper, Koen (Nokia - BE) wrote:
> Hi Wes,
> 
> Just to let you know that we are still working on AQMs that support scalable (L4S) TCPs.
> We could present some of our latest results (if there will be a meeting in Buenos Aires, otherwise in Berlin?)
> 
> * Performance of HTTP Adaptive Video Streaming (HAS) with different TCP's and AQMs
>    o HAS is currently ~30% of Internet traffic, but no AQM testing so far has included it

I am aware of several unpublished studies. There was also something that
compared 1-3 HAS flows from several years back from stanford that I've
longed to be repeated against these aqm technologies.

https://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-14-confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/

>    o the results are very poor with a particular popular AQM

Define "very poor". ?

> Presenter: Inton Tsang
> Duration: 10mins
> Draft: Comparative testing of draft-ietf-aqm-pie-01, draft-ietf-aqm-fq-codel-04, draft-briscoe-aqm-dualq-coupled



> 
> For experiment write-up, see Section 3 of https://riteproject.files.wordpress.com/2015/12/rite-deliverable-3-3-public1.pdf

At the risk of sounding grumpy and pedantic, partially driven by trying
to read through 58 pages of text on a b/w kindle (before I switched to
something higher res)

No source, can't reproduce.

The diagrams of the topology, etc, were quite helpful. Insight into the
actual complexity of BT's network was quite useful. I would love to know
the actual usage of the various QoS queues in terms of real traffic...

Has DCTP's not responding to drop properly problem been fixed in linux
yet? (sec 3.3.2). That remains a scary bug....

More detailed comments:

The tests tested download traffic only and seem to have assumed that the
uplink would never have been a bottleneck. Home connections are
asymmetric and increasingly so - comcast, for example is now putting out
links with 75Mbit down, 5.5Mbit up. It is not clear what the uplink rate
of the emulated or real network is in this paper.

2) Section 2 - would have been tons more interesting had it evaluated
the effects of other workloads against actual "twitch" games such as any
of the quake series, call of duty, or starcraft. The game chosen was
quite uninteresting from "needing a low latency and jitter to win" on,
front.

Applying interesting background network workloads while game bots
competed as in here http://sscaitournament.com/ would have been great
fun! - injecting random jitter and delay into the network stack has been
what Karl Auerbach has been doing to the competitors at the DARPA robot
competitions, and he's probably been the cause of quite a few of those
real-life robots falling over (I can hear his evil chuckles now).

Artificially applying a wifi loss rate of 1.5% is a far cry from what
actually happens on wifi, where what you see is more typically large
spikes in l2 delay, nowadays.

3) As for section 3 - it uses a "bottleneck of 20Mbps and RTT of 20ms,
ten TCP long file download flows as background traffic and one HAS
request, which launch two TCP flows." I will try to restrain myself, but:

A) In a cable modem scenario, the typical uplink varies from 2mbit to
4mbit on a 20mbit link. I am not aware of many 20mbit symmetric links,
aside from FIOS. I would love symmetry on home links, is BT deploying that??

B) this is a workload that has very little resemblance to any reality I
can think of and seems expressly designed to show the burst tolerance
and additional latency induced by pie and dualQ to an advantage, and
fq_codel at a local minimum.

For the sake of science, varying the number of long running downloads
from, say, 2,4,8 and 16, would have provided a more rounded picture.
Still I am really hard pressed to think of any case where home HAS
traffic will be competing with more than 2-3 long running full rate
downloads in your typical home. I HAVE seen 3 android devices attempt to
update themselves all at once, but that's it....

Using bittorrent to generate 5-15 flows down, perhaps? Even then, most
real world torrents are not running very fast, there's utp based
congestion control, and torrenters spend way more time in the upload
phase than the download.

...

Using web access bursts (10-100 flows lasting for 2-8 seconds), chock
full of dns accesses, seems to be a good test against HAS traffic. I see
there is a section doing that, but it is against 5 background flows,
also and I'm pretty sure there will be a pause between bursts, and
although the "The web requests followed an exponential arrival process,
while the size of the downloaded files were designed to represent actual
Internet web objects following a Pareto distribution with size between 1
Kbyte and 1 Mbyte" is documented...

I could do the math assuming that distribution * the number of files...
but I would prefer the total size of the web transfer be documented, and
it's time to complete a load presented... nor is it clear if dns lookups
are enabled or counted. Unless I missed it somewhere? If you are going
to measure two loads - one HAS, one web, presenting the results for both
for contrast, seems logical.

"Figure 3.22 shows that HAS-DCTCP can maintain its performance even with
the additional 100 web traffic request per second as background traffic.
It consistently delivered segments of 2056 Kbps encoding
bit rate. In contrast, under the same network condition, the additional
web traffic deteriorate further the performance of HAS-CTCP as the
delivered segment alternated qualities between 688 Kbps and
991 Kbps encoding rates. This had a lower overall quality when compared
with the previous experiment, which without the additional web traffic
could achieved a sustainable delivery of segment with 991 Kbps
encoding rates, as shown in Figure 3.18"

This kind of stuff really gets under my skin. What was the web page
completion time? Which matters more to the users of the household? a
slight decline in video quality, or a web page load?

Sacrificing all other network applications on the crucible of 4k video
is really not my goal in life, I would like to see videoconferencing,
gaming, web, in particular, work well in the presence of HAS.

Please, please, please, put web page completion times in this document?

...

Also that sort of web traffic would probably cycle on a 30-60 second
basis at most - people just can't read that fast! Secondly, a search
engine query often precedes an actual web page lookup.

 (I note that web page size growth has slowed dramatically since this wg
started, it is now well below the projections we had a few years ago
(7mbit by 20XX, more like 4 now, but I'd have to go redo the math -
growth went from exponential-looking in 2012 to linear now...). There's
also some data presented recently by some googlers at netconf 1.1 that
showed the impact and commonality of dns failures across their
subscriber base.....

I freely confess that maybe i'm out of touch, that maybe having perfect
television quality while waiting seconds for web pages to load, is what
most internet users want, and they deserve to get it, good and hard,
along with their supersized mcgreasy burgers and calming drugs delivered
to their door.

C) And no reverse traffic whatsoever in section three.

In your typical HAS video streaming home scenario, the worst behaviors I
can think of would be bittorrent generated *on the upload*, but the
simplest case of a single upload happening at all - tends towards
starving the ack streams and ramp up times on the video download flows.
Which was not tested. Sigh. How long have I banged on this drum?

D) It is unproven from a QOE perspective (I think) that having a video
stream change qualities on a "better average" basis is of benefit to the
user - seeing a stream never pause "for buffering", and consistent
quality, seems saner. I notice when the quality changes down, but rarely
up. "buffering" - that I seriously notice.

I would have liked reference data for drop tail vs the various
experiments in this paper. Typical amounts of head end buffering on
docsis 3.0 arris cmtss seem to be in the 800ms range at 20mbit.

E) A workload that I would like to see tested with some rigor is a
family of four - one doing a big upload (facebook/instagram), another
browsing the web, another doing a phone call or videoconference, and a
fourth attempting to watch a movie at the highest possible resolution.

The bar for the last has moved to SD or HD quality in the past few years
- 18mbits is a number I keep seeing more and more. Someone with a 20Mbit
link IS going to try for the highest quality, and showing the dynamic
range of that (1-18mbits) would be more interesting than 1mbit to 2, as
in in this paper.

I also would welcome 2-3 HAS testing on downloads against these AQMs,
attempting those rates, along the lines of the stanford thing I
mentioned first.

I petered out before reading sections 4 and 5. I will try to get to it
this week.

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Alan Jenkins]

On 27/02/2016, Dave Täht <dave@taht.net> wrote:
>
>
>
> On 2/26/16 3:23 AM, De Schepper, Koen (Nokia - BE) wrote:
>> Hi Wes,
>>
>> Just to let you know that we are still working on AQMs that support
>> scalable (L4S) TCPs.
>> We could present some of our latest results (if there will be a meeting in
>> Buenos Aires, otherwise in Berlin?)
>>
>> * Performance of HTTP Adaptive Video Streaming (HAS) with different TCP's
>> and AQMs
>>    o HAS is currently ~30% of Internet traffic, but no AQM testing so far
>> has included it


> I am aware of several unpublished studies. There was also something that
> compared 1-3 HAS flows from several years back from stanford that I've
> longed to be repeated against these aqm technologies.
>
> https://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-14-confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/

Wow.


>>    o the results are very poor with a particular popular AQM
>
> Define "very poor". ?

Heh.  After skimming the same sections as you, I think my restraint
must be vastly inferior to yours.

I didn't like to compain on the AQM list.  I think your point about
the measurements made for background web traffic are more interesting.
But it looks a bit weird and I can imagine the results being used
inappropriately.

>> Presenter: Inton Tsang
>> Duration: 10mins
>> Draft: Comparative testing of draft-ietf-aqm-pie-01,
>> draft-ietf-aqm-fq-codel-04, draft-briscoe-aqm-dualq-coupled
>
>
>
>>
>> For experiment write-up, see Section 3 of
>> https://riteproject.files.wordpress.com/2015/12/rite-deliverable-3-3-public1.pdf

I wouldn't complain that I can't sustain 2056Kbps goodput when my fair
share of the shaped bandwidth is 2000Kbps.  The results might be
showing a significant degradation, or it could be a marginal one that
pushes over the boundary (between the 2056k and 1427k encodes).  Which
of those conclusions you start from might be influenced by whether
you're developing a different AQM, hmm.

The HAS over TCP must be more bursty than a simple TCP download.  All
this could be explained by a competitive advantage in queues without
fair scheduling.  There's no effort to rule it out here, at least.

You can see in the first link above, HAS used to basically die in
competition with a simple TCP download.  Clearly it's been fixed for
this case - assuming current commonly deployed queues.  SFQ looks
*very* different to these queues, as they point out, and you see in
the throughput graphs.  So it is possible there's an issue with the
closed-source HAS being tested.  Somehow I doubt the Google Video's of
this world would remain degraded, if there's a real issue here.  (I
know, game theory of incremental deployment, sigh).

It can't be a co-incidence that the variation below 1.5Mbps happens
when the HAS decides to switch the roles of video and audio TCP
streams (!).  That it's below 1.5Mbps for a the first 30s is a second
pointer, towards the handling of bursts related to "slow start".

It's strange to just say "the upside with a possibility to peak in
throughput cannot happen".  What that means is you want a new HAS
stream to inflict this quality drop on an established one, so your new
stream doesn't have to start at the lower quality in order to build up
a buffer.

Sure, that's even more contrived.  And it's probably what you'd prefer
in practice.  Either you've got enough bandwidth for two streams, or
the established one is going to visibly downgrade anyway.  But that's
a path-dependent result, on SFQ [1990] not being deployed when the HAS
was being optimized.

It doesn't seem to be a clear demonstration of the fundamental
advantages of L4S / DualQ.

Alan

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Toke Høiland-Jørgensen]

Alan Jenkins <alan.christopher.jenkins@gmail.com> writes:

> I wouldn't complain that I can't sustain 2056Kbps goodput when my fair
> share of the shaped bandwidth is 2000Kbps.  The results might be
> showing a significant degradation, or it could be a marginal one that
> pushes over the boundary (between the 2056k and 1427k encodes).  Which
> of those conclusions you start from might be influenced by whether
> you're developing a different AQM, hmm.

Exactly. And note how they just so happen to pick 11 total flows (10
competing, one video) to share the bottleneck, putting the per-flow
throughput just below the rate needed to go up one quality level. What a
coincidence. At least it shows how difficult it is to design experiments
that put fairness queueing in a bad light ;)

Oh, and of course HAS is in itself a hack to work around badly managed
queues in the network. In a nicely fairness queued world, we could do
away with HAS entirely and just, y'know, stream things at the desired
rate...

-Toke

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Alan Jenkins]

On 28/02/16 13:39, Toke Høiland-Jørgensen wrote:
> Alan Jenkins <alan.christopher.jenkins@gmail.com> writes:
>
>> I wouldn't complain that I can't sustain 2056Kbps goodput when my fair
>> share of the shaped bandwidth is 2000Kbps.  The results might be
>> showing a significant degradation, or it could be a marginal one that
>> pushes over the boundary (between the 2056k and 1427k encodes).  Which
>> of those conclusions you start from might be influenced by whether
>> you're developing a different AQM, hmm.
> Exactly. And note how they just so happen to pick 11 total flows (10
> competing, one video) to share the bottleneck, putting the per-flow
> throughput just below the rate needed to go up one quality level. What a
> coincidence. At least it shows how difficult it is to design experiments
> that put fairness queueing in a bad light ;)
Ug. I try not to assume malice.  It does indeed come across as motivated 
absence of curiosity.

We ran a test where Product A scores better than Product B.  Buy Product 
A today!

* difference in presented results is within margin of error
** did you notice our "pass" threshold was literally over 100%? That the 
technology commonly assumed to provide fairness breaks down in this 
test?  No?  <simpsons character="Mr. Burns> _Excellent_.

> Oh, and of course HAS is in itself a hack to work around badly managed
> queues in the network. In a nicely fairness queued world, we could do
> away with HAS entirely and just, y'know, stream things at the desired
> rate...
>
> -Toke
Nah.  You do want to handle a variable number of users, without making 
them fiddle with rates when that number changes.

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Juliusz Chroboczek]

>> Oh, and of course HAS is in itself a hack to work around badly managed
>> queues in the network. In a nicely fairness queued world, we could do
>> away with HAS entirely and just, y'know, stream things at the desired
>> rate...

Perhaps I'm missing something -- how do you to pick the right rate for
a given user?

> Nah.  You do want to handle a variable number of users, without making
> them fiddle with rates when that number changes.

Right.

-- Juliusz

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Toke Høiland-Jørgensen]

Juliusz Chroboczek <jch@pps.univ-paris-diderot.fr> writes:

>>> Oh, and of course HAS is in itself a hack to work around badly managed
>>> queues in the network. In a nicely fairness queued world, we could do
>>> away with HAS entirely and just, y'know, stream things at the desired
>>> rate...
>
> Perhaps I'm missing something -- how do you to pick the right rate for
> a given user?

Congestion control? With something like fairness queueing that keeps
bandwidth very stable at the fair share, you could conceivably get away
with a much shallower playback buffer in the application and stream at
something very close to real time, rather than using the chunk-based
retrieval of HAS.

However, that remark was mostly meant to be tongue-in-cheek... not sure
if there exists any really good congestion control algorithms for
real-time video.

-Toke

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Toke Høiland-Jørgensen]

Alan Jenkins <alan.christopher.jenkins@gmail.com> writes:

>> Exactly. And note how they just so happen to pick 11 total flows (10
>> competing, one video) to share the bottleneck, putting the per-flow
>> throughput just below the rate needed to go up one quality level. What a
>> coincidence. At least it shows how difficult it is to design experiments
>> that put fairness queueing in a bad light ;)

> Ug. I try not to assume malice. It does indeed come across as
> motivated absence of curiosity.

Oh, didn't necessarily mean it was malicious. Can totally picture the
discussion landing on 10 competing flows (a nice round number). However,
it does just so happen to result in there being a bit too little
bandwidth for the 2Mbps video flow...

-Toke

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Dave Taht]

On Sun, Feb 28, 2016 at 5:33 AM, Alan Jenkins
<alan.christopher.jenkins@gmail.com> wrote:
> On 27/02/2016, Dave Täht <dave@taht.net> wrote:
>> On 2/26/16 3:23 AM, De Schepper, Koen (Nokia - BE) wrote:
>> I am aware of several unpublished studies. There was also something that
>> compared 1-3 HAS flows from several years back from stanford that I've
>> longed to be repeated against these aqm technologies.
>>
>> https://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-14-confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/
>
> Wow.

Updated for 2015, here:
https://reproducingnetworkresearch.wordpress.com/2015/05/31/cs244-15-confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/

I love this class over at stanford. I've read everything they've
published. With huge percentages of research papers being shown to not
be reproducible, and networking being a moving target rather than
rooted in the laws of physics, chemistry or biology, CS244 is
desperately needed.

If *every* university doing networking work had a second year class
that took on classic and recently popular papers and updated them for
current conditions we'd make short work of the backlog of stuff I'm
dubious about and have better assumptions to stand on going forward.
There are decades of work, in particular, that finds results at
<4mbits that I do not think hold at > 20mbit or > 100mbit.

Actually, more work spent reproducing papers (and having reproducible
papers) would be great across all fields of science.....

>
>
>>>    o the results are very poor with a particular popular AQM
>>
>> Define "very poor". ?
>
> Heh.  After skimming the same sections as you, I think my restraint
> must be vastly inferior to yours.

I wrote this a month ago. Hit save. Edited saturday. Shouldn't have
left in as much grumpiness as I did.

That said, we could build on this experimental design.  That
experiment would be more like:

phase 1)

20Mbit link, 4ms (google fiber), 10ms (cable), 20ms(dsl) (assuming
co-location of the video server), 40ms(random source) base RTT to the
video server. If pressed for time, just pick one of the above.

Provide a reference for actual measured behavior first for the
technology under emulation and emulate the actual bandwidth asymmetry.

HAS app shooting for 18mbit video. How long does it take the flow to ramp up?

Add *one* long running upload flow - How long does the HAS app take to ramp up?

Vary the AQMs on both sides of the link. Run with ECN and without.

Apply web traffic as background - what rates are chosen, what is the
PLT? I'd lean towards a web test that emulated a search, decision, web
page fetch loop.

(honestly I have a low bar for HAS traffic quality personally - merely
never stopping playback to buffer is it. I am hard pressed to remember
the last non-wifi-based interruption I've had on a video download
using sqm-scripts)

phase 2) trying 1-3 HAS flows on that link (as per the stanford
experiment but shooting for 18mbits per flow), varying the AQMs. Then
add in a single upload flow. Then try web traffic. Then all three....

phase 3) Adding in web traffic, a twitch game, a videoconference/voip
session, a (set of) uploads, a (set of) downloads.


To look further forward, pick an even higher encoding rate for the HAS
video. Run the downlink at rates like 75-200mbit. Try it over wifi,
too....

Getting to a high quality fast and staying there is *hard*, not just
for mixed HAS traffic. The videoconferencing servers I've looked at
thus far (jitsy, freeswitch) are doing *no* congestion control to
speak of, although there are some patches in flight - I am seeing
8mbits udp down, 2mbit up from freeswitches's conference server....

>
> I didn't like to complain on the AQM list.

Well, the EU commission is unlikely to read that list also, but
perhaps they will read that paper and other rite outputs.  I have no
idea what impact these outputs they will have.

I look forward (with some dread) as to what sorts of head end aqm/fq
technologies are adopted in the future on cable, fiber, cellular, etc,
if any, ever. At this point I am kind of expecting 5G to more or less
roll out first with better aqm/fq technologies as that's where the
heavy hardware/software investment seems to be going. It looks like
we'll see DOCSIS 3.1 trials in a few cities this year, but there's no
public data on what, if any, improvements were made to the CMTSes.

And gfiber (and services like comcast's blast) are so freaking fast
that nearly all the bloat moves to the wifi. At the same time, the
only things that seriously stress out wifi are multiple devices, and
trying to watch HAS traffic over it!

I did do some testing recently on the latest/greatest of the ethernet
over powerline AV/1200 devices from netgear. It was another one of
those - write my comments, hit save for a month - sort of exercises.

>
> The HAS over TCP must be more bursty than a simple TCP download.  All
> this could be explained by a competitive advantage in queues without
> fair scheduling.  There's no effort to rule it out here, at least.

The typical pattern is to try and fetch 10seconds of video in under
2-4(?) sec, otherwise degrade to a lower rate.

>
> You can see in the first link above, HAS used to basically die in
> competition with a simple TCP download.  Clearly it's been fixed for
> this case - assuming current commonly deployed queues.  SFQ looks
> *very* different to these queues, as they point out, and you see in
> the throughput graphs.  So it is possible there's an issue with the
> closed-source HAS being tested.  Somehow I doubt the Google Video's of
> this world would remain degraded, if there's a real issue here.  (I
> know, game theory of incremental deployment, sigh).

Google switched to using sch_fq (host fq + pacing) pretty universally
a year or two back, youtube switched first.

There was a tcp patch that landed recently in the mainline kernel that
didn't slam a connection after an idle period with the current size of
the congestion window. That looked pretty important... (I have no idea
how long it was in production)

Re: [Bloat] [aqm] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[De Schepper, Koen (Nokia - BE)]

Hi Dave,

Thanks for reviewing our results. But probably I should not have referred to this document, as it contained indeed preliminary results, not showing all experiments. More elaborate results will be presented.
 
Just to be clear: we agree that FQ_ is currently the only network-"fix" to make sure that every flow gets an equal and stable rate. But I think you agree that FQ_ has some disadvantages too (mainly how to correctly identifying "flows", requiring still a classic self-queue, and not allowing deviation both up and down from the fair rate).

The goal of L4S is to solve the problem in the end-system allowing a neutral, transport layer independent and simple network. And note that our rate fairness benchmark is FQ_! If we can reach results that are close to FQ_ (disregarding its disadvantages here), we have achieved our goal. We don't expect L4S to be better than the FQ_ for its default rate fairness, except where FQ_ cannot fix the classic TCP problems in the end-system.
 
Related to the results in this document, the absolute quality selected is indeed skewed by both the Windows CTCP and DCTCP achieving a higher throughput than its Linux Reno and DCTCP counterparts. So the absolute values of quality should not be compared, but rather the stability of the quality. In the border cases where the available bitrate is close to the switchover points, the quality of DCTCP HAS stays stable at the expected quality, the FQ and PIE start fluctuating more heavily, PIE due to classic TCP rate variability and PIE's bigger queue, and FQ_ because the HAS flows cannot regain lost scheduling opportunities during TCP time-outs and restarts (referred to as "the peeks are cut off").
 
Of course feel free to reproduce our results, Windows server is free available for evaluation including the IIS server, Linux has DCTCP and the AQMs (the immediate step function for DCTCP can be made with RED, so no need for DualQ), and also other (open source) adaptive streaming servers can be used.

More inline too...

Regards,
Koen.


> -----Original Message-----
> From: aqm [mailto:aqm-bounces@ietf.org] On Behalf Of EXT Dave Täht
> Sent: zaterdag 27 februari 2016 20:05
> To: aqm@ietf.org; bloat@lists.bufferbloat.net
> Subject: [aqm] review: Deployment of RITE mechanisms, in use-case trial
> testbeds report part 1
> 
> 
> 
> 
> On 2/26/16 3:23 AM, De Schepper, Koen (Nokia - BE) wrote:
> > Hi Wes,
> >
> > Just to let you know that we are still working on AQMs that support
> scalable (L4S) TCPs.
> > We could present some of our latest results (if there will be a
> meeting in Buenos Aires, otherwise in Berlin?)
> >
> > * Performance of HTTP Adaptive Video Streaming (HAS) with different
> TCP's and AQMs
> >    o HAS is currently ~30% of Internet traffic, but no AQM testing so
> far has included it
> 
> I am aware of several unpublished studies. There was also something that
> compared 1-3 HAS flows from several years back from stanford that I've
> longed to be repeated against these aqm technologies.
> 
> https://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-14-
> confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/
> 
> >    o the results are very poor with a particular popular AQM
> 
> Define "very poor". ?
> 

"Very poor" compared to what can be expected from a perfect per flow scheduler in the network ;-). Of course it is not necessarily due to FQ_'s performance, rather the combination of FQ_ and Classic TCP's shortcomings. If something is not functioning well in the end-systems, it cannot always be fixed in the network only. So this is certainly not a criticism on the FQ_ implementation.

> > Presenter: Inton Tsang
> > Duration: 10mins
> > Draft: Comparative testing of draft-ietf-aqm-pie-01, draft-ietf-aqm-
> fq-codel-04, draft-briscoe-aqm-dualq-coupled
> 
> 
> 
> >
> > For experiment write-up, see Section 3 of
> https://riteproject.files.wordpress.com/2015/12/rite-deliverable-3-3-
> public1.pdf
> 
> At the risk of sounding grumpy and pedantic, partially driven by trying
> to read through 58 pages of text on a b/w kindle (before I switched to
> something higher res)

I appreciate your effort in reviewing the full document, but I mentioned section 3 as the relevant section here.
Note that this deliverable is not a like a peer reviewed publication, but a write-up of research at a certain moment in time, and therefore is mostly work in progress. It has been reviewed before by independent assigned experts by the EU, and their some of their comments were in line with yours.

> 
> No source, can't reproduce.
> 
> The diagrams of the topology, etc, were quite helpful. Insight into the
> actual complexity of BT's network was quite useful. I would love to know
> the actual usage of the various QoS queues in terms of real traffic...
> 
> Has DCTP's not responding to drop properly problem been fixed in linux
> yet? (sec 3.3.2). That remains a scary bug....
> 
> More detailed comments:
> 
> The tests tested download traffic only and seem to have assumed that the
> uplink would never have been a bottleneck. Home connections are
> asymmetric and increasingly so - comcast, for example is now putting out
> links with 75Mbit down, 5.5Mbit up. It is not clear what the uplink rate
> of the emulated or real network is in this paper.
> 
> 2) Section 2 - would have been tons more interesting had it evaluated
> the effects of other workloads against actual "twitch" games such as any
> of the quake series, call of duty, or starcraft. The game chosen was
> quite uninteresting from "needing a low latency and jitter to win" on,
> front.
> 
> Applying interesting background network workloads while game bots
> competed as in here http://sscaitournament.com/ would have been great
> fun! - injecting random jitter and delay into the network stack has been
> what Karl Auerbach has been doing to the competitors at the DARPA robot
> competitions, and he's probably been the cause of quite a few of those
> real-life robots falling over (I can hear his evil chuckles now).
> 
> Artificially applying a wifi loss rate of 1.5% is a far cry from what
> actually happens on wifi, where what you see is more typically large
> spikes in l2 delay, nowadays.
> 
> 3) As for section 3 - it uses a "bottleneck of 20Mbps and RTT of 20ms,
> ten TCP long file download flows as background traffic and one HAS
> request, which launch two TCP flows." I will try to restrain myself,
> but:
> 
> A) In a cable modem scenario, the typical uplink varies from 2mbit to
> 4mbit on a 20mbit link. I am not aware of many 20mbit symmetric links,
> aside from FIOS. I would love symmetry on home links, is BT deploying
> that??

We define these experiments with the purpose to analyze and improve, not necessarily to resemble realistic scenario's. Of course we try to cover the range of today's realistic cases and finally evaluation of realistic test cases are needed as well. But when mixing too many mechanisms together we find it too complex to analyze. The current Internet is working most of the time correctly, it are only the short moments of coincidental conditions that make users experience the Internet as unreliable. We tried to create these conditions in our testcases frequently enough so we can evaluate the impact of different mechanisms (AQMs, schedulers and CCs). Also detecting and solving "none-used" problems allow new applications to become realistic...


> 
> B) this is a workload that has very little resemblance to any reality I
> can think of and seems expressly designed to show the burst tolerance
> and additional latency induced by pie and dualQ to an advantage, and
> fq_codel at a local minimum.
> 
> For the sake of science, varying the number of long running downloads
> from, say, 2,4,8 and 16, would have provided a more rounded picture.
> Still I am really hard pressed to think of any case where home HAS
> traffic will be competing with more than 2-3 long running full rate
> downloads in your typical home. I HAVE seen 3 android devices attempt to
> update themselves all at once, but that's it....
> 
> Using bittorrent to generate 5-15 flows down, perhaps? Even then, most
> real world torrents are not running very fast, there's utp based
> congestion control, and torrenters spend way more time in the upload
> phase than the download.
> 
> ...
> 
> Using web access bursts (10-100 flows lasting for 2-8 seconds), chock
> full of dns accesses, seems to be a good test against HAS traffic. I see
> there is a section doing that, but it is against 5 background flows,
> also and I'm pretty sure there will be a pause between bursts, and
> although the "The web requests followed an exponential arrival process,
> while the size of the downloaded files were designed to represent actual
> Internet web objects following a Pareto distribution with size between 1
> Kbyte and 1 Mbyte" is documented...
> 
> I could do the math assuming that distribution * the number of files...
> but I would prefer the total size of the web transfer be documented, and
> it's time to complete a load presented... nor is it clear if dns lookups
> are enabled or counted. Unless I missed it somewhere? If you are going
> to measure two loads - one HAS, one web, presenting the results for both
> for contrast, seems logical.
> 
> "Figure 3.22 shows that HAS-DCTCP can maintain its performance even with
> the additional 100 web traffic request per second as background traffic.
> It consistently delivered segments of 2056 Kbps encoding
> bit rate. In contrast, under the same network condition, the additional
> web traffic deteriorate further the performance of HAS-CTCP as the
> delivered segment alternated qualities between 688 Kbps and
> 991 Kbps encoding rates. This had a lower overall quality when compared
> with the previous experiment, which without the additional web traffic
> could achieved a sustainable delivery of segment with 991 Kbps
> encoding rates, as shown in Figure 3.18"
> 
> This kind of stuff really gets under my skin. What was the web page
> completion time? Which matters more to the users of the household? a
> slight decline in video quality, or a web page load?
> 
> Sacrificing all other network applications on the crucible of 4k video
> is really not my goal in life, I would like to see videoconferencing,
> gaming, web, in particular, work well in the presence of HAS.
> 
> Please, please, please, put web page completion times in this document?

For our purpose, we prefer the statistical mix of downloads with different sizes and the representation of completion times for these different sizes. Again creating sporadic events frequently enough to study them. The representation allows clearly to compare the impact of the different mechanisms, as we repeat the same scenarios over the different combinations of AQMs, schedulers and CCs.

> 
> ...
> 
> Also that sort of web traffic would probably cycle on a 30-60 second
> basis at most - people just can't read that fast! Secondly, a search
> engine query often precedes an actual web page lookup.
> 
>  (I note that web page size growth has slowed dramatically since this wg
> started, it is now well below the projections we had a few years ago
> (7mbit by 20XX, more like 4 now, but I'd have to go redo the math -
> growth went from exponential-looking in 2012 to linear now...). There's
> also some data presented recently by some googlers at netconf 1.1 that
> showed the impact and commonality of dns failures across their
> subscriber base.....
> 
> I freely confess that maybe i'm out of touch, that maybe having perfect
> television quality while waiting seconds for web pages to load, is what
> most internet users want, and they deserve to get it, good and hard,
> along with their supersized mcgreasy burgers and calming drugs delivered
> to their door.
> 
> C) And no reverse traffic whatsoever in section three.
> 
> In your typical HAS video streaming home scenario, the worst behaviors I
> can think of would be bittorrent generated *on the upload*, but the
> simplest case of a single upload happening at all - tends towards
> starving the ack streams and ramp up times on the video download flows.
> Which was not tested. Sigh. How long have I banged on this drum?
> 
> D) It is unproven from a QOE perspective (I think) that having a video
> stream change qualities on a "better average" basis is of benefit to the
> user - seeing a stream never pause "for buffering", and consistent
> quality, seems saner. I notice when the quality changes down, but rarely
> up. "buffering" - that I seriously notice.
> 
> I would have liked reference data for drop tail vs the various
> experiments in this paper. Typical amounts of head end buffering on
> docsis 3.0 arris cmtss seem to be in the 800ms range at 20mbit.
> 
> E) A workload that I would like to see tested with some rigor is a
> family of four - one doing a big upload (facebook/instagram), another
> browsing the web, another doing a phone call or videoconference, and a
> fourth attempting to watch a movie at the highest possible resolution.
> 
> The bar for the last has moved to SD or HD quality in the past few years
> - 18mbits is a number I keep seeing more and more. Someone with a 20Mbit
> link IS going to try for the highest quality, and showing the dynamic
> range of that (1-18mbits) would be more interesting than 1mbit to 2, as
> in in this paper.
> 
> I also would welcome 2-3 HAS testing on downloads against these AQMs,
> attempting those rates, along the lines of the stanford thing I
> mentioned first.
> 
> I petered out before reading sections 4 and 5. I will try to get to it
> this week.
> 
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Fred Baker (fred)]

[-- Attachment #1: Type: text/plain, Size: 1811 bytes --]


> On Feb 27, 2016, at 11:04 AM, Dave Täht <dave@taht.net> wrote:
> 
> https://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-14-confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/
> 
>>   o the results are very poor with a particular popular AQM
> 
> Define "very poor". ?

Presuming this is Adaptive Bitrate Video, as in Video-in-TCP, we (as in Cisco engineers, not me personally; you have met them) have observed this as well. Our belief is that this is at least in part a self-inflicted wound; when the codec starts transmission on any four second segment except the first, there is no slow-start phase because the TCP session is still open (and in the case of some services, there are several TCP sessions open and the application chooses the one with the highest cwnd value). You can now think of the behavior of the line as repeating a four phase sequence: nobody is talking, then one is talking, then both are, and then the other is talking. When only one is talking, whichever it is, its cwnd value is slowing increasing - especially if cwnd*mss/rtt < bottleneck line rate, minimizing RTT. At the start of the "both are talking" phase, the one already talking has generally found a cwnd value that fills the line and its RTT is slowly increasing. The one starting sends a burst of cwnd packets, creating an instant queue and often causing one or both to drop a packet - reducing their respective cwnd values. Depending on the TCP implementation in question at the sender, if the induced drop isn't a single packet but is two or three, that can make the affected session pause for as many RTO timeouts (Reno), RTTs (New Reno), or at least retransmit the lost packets in the subsequent RTT and then reduce cwnd by at least that amount (cubic) and maybe half (SACK).

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Alan Jenkins]

On 02/03/16 18:09, Fred Baker (fred) wrote:
>> On Feb 27, 2016, at 11:04 AM, Dave Täht <dave@taht.net> wrote:
>>
>> https://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-14-confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/
>>
>>>    o the results are very poor with a particular popular AQM
>> Define "very poor". ?
> Presuming this is Adaptive Bitrate Video, as in Video-in-TCP, we (as in Cisco engineers, not me personally; you have met them) have observed this as well. Our belief is that this is at least in part a self-inflicted wound; when the codec starts transmission on any four second segment except the first, there is no slow-start phase because the TCP session is still open (and in the case of some services, there are several TCP sessions open and the application chooses the one with the highest cwnd value). You can now think of the behavior of the line as repeating a four phase sequence: nobody is talking, then one is talking, then both are, and then the other is talking. When only one is talking, whichever it is, its cwnd value is slowing increasing - especially if cwnd*mss/rtt < bottleneck line rate, minimizing RTT. At the start of the "both are talking" phase, the one already talking has generally found a cwnd value that fills the line and its RTT is slowly increasing. The one starting sends a burst of cwnd packets, creating an instant queue and often causing one or both to drop a packet - reducing their respective cwnd values. Depending on the TCP implementation in question at the sender, if the induced drop isn't a single packet but is two or three, that can make the affected session pause for as many RTO timeouts (Reno), RTTs (New Reno), or at least retransmit the lost packets in the subsequent RTT and then reduce cwnd by at least that amount (cubic) and maybe half (SACK).

Interesting!  Just as Dave reminds us that Google avoid the bursts you 
describe, using pacing. (See end of message 
https://lists.bufferbloat.net/pipermail/bloat/2016-February/007205.html)

You can call the result a disadvantage of FQ in the real world if you 
want.  But you can also say it provides some necessary alignment of 
incentives.  Incentives for applications to develop more 
network-friendly behaviour :).  I was surprised that a project with 
large ISP involvement seems to take the first point of view.

(Also the part about connections being chosen by cwnd helps explain the 
fq_codel throughput graph.  You can see the audio and video connections 
switch roles several times.  The same times as the bitrate fluctuates, I 
notice)

I was just skimming PANDA[1], which does AIMD for adaptive streaming. So 
they decrement the interval between chunk fetches, until the observed 
throughput _over the full on-off cycle_ is sufficient to sustain the 
next quality level.  <handwave>It could just as easily pace the fetch 
over the full period. <utopian>No more on-off cycle, no damaging bursts 
of packets?

Alan

[1] http://arxiv.org/pdf/1305.0510.pdf

Re: [Bloat] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Fred Baker (fred)]

[-- Attachment #1: Type: text/plain, Size: 573 bytes --]


> On Mar 2, 2016, at 12:53 PM, Alan Jenkins <alan.christopher.jenkins@gmail.com> wrote:
> 
> Google avoid the bursts you describe, using pacing.

I have been an advocate of pacing during the first transmission burst (including after an idle period) for quite some time, and also for delay-based algorithms. There would be no need for AQM if TCPs would maximize throughput while at the same time minimizing mean end to end delay, which is a matter of choosing for cwnd the largest value that maintains the smallest current RTT.

Those Google folk are pretty smart.

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Bloat] [aqm] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Ingemar Johansson S]

Hi

Thanks for the pointer to the RITE paper, will read it carefully. 
Some comments on HAS or DASH
The HAS behavior when subject to different AQMs and other competing background traffic, depends heavily the rate control algorithm. 
Investigations we have done and also e.g. the Netflix papers and lately also the BOLA paper (link below) shows that rate based algorithms are more easily starved out by competing traffic than the buffer level based algorithms. The bufferbased algorithms (Netflix, BOLA) are more opportunistic and TCP is more allowed to work like large file downloads when the links are fully utilized. Rate based algorithms on the other hand can more easily end up in a vicious circle, a low download rate is detected, so the next segment requested is with a reduced rate, other traffic will grab a larger share, and this repeats itself until the lowest rate is reached.
This is elaborated upon in the paper "A Buffer-Based Approach to Rate Adaptation: Evidence from a Large Video Streaming Service" by Te-Yuan Huang et.al.
I don't have full insight how MS Silverlight operates so I cannot quantify it is rate based or buffer based.

BOLA : http://arxiv.org/pdf/1601.06748.pdf 

/Ingemar

> -----Original Message-----
> From: Fred Baker (fred) [mailto:fred@cisco.com]
> Sent: den 2 mars 2016 19:09
> To: Dave Täht
> Cc: aqm@ietf.org; bloat@lists.bufferbloat.net
> Subject: Re: [aqm] [Bloat] review: Deployment of RITE mechanisms, in use-
> case trial testbeds report part 1
> 
> 
> > On Feb 27, 2016, at 11:04 AM, Dave Täht <dave@taht.net> wrote:
> >
> > https://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-
> 14-confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/
> >
> >>   o the results are very poor with a particular popular AQM
> >
> > Define "very poor". ?
> 
> Presuming this is Adaptive Bitrate Video, as in Video-in-TCP, we (as in Cisco
> engineers, not me personally; you have met them) have observed this as
> well. Our belief is that this is at least in part a self-inflicted wound; when the
> codec starts transmission on any four second segment except the first, there
> is no slow-start phase because the TCP session is still open (and in the case of
> some services, there are several TCP sessions open and the application
> chooses the one with the highest cwnd value). You can now think of the
> behavior of the line as repeating a four phase sequence: nobody is talking,
> then one is talking, then both are, and then the other is talking. When only
> one is talking, whichever it is, its cwnd value is slowing increasing - especially
> if cwnd*mss/rtt < bottleneck line rate, minimizing RTT. At the start of the
> "both are talking" phase, the one already talking has generally found a cwnd
> value that fills the line and its RTT is slowly increasing. The one starting sends
> a burst of cwnd packets, creating an instant queue and often causing one or
> both to drop a packet - reducing their respective cwnd values. Depending on
> the TCP implementation in question at the sender, if the induced drop isn't a
> single packet but is two or three, that can make the affected session pause
> for as many RTO timeouts (Reno), RTTs (New Reno), or at least retransmit
> the lost packets in the subsequent RTT and then reduce cwnd by at least that
> amount (cubic) and maybe half (SACK).

Re: [Bloat] [aqm] review: Deployment of RITE mechanisms, in use-case trial testbeds report part 1

[Luca De Cicco]

[-- Attachment #1: Type: text/plain, Size: 5344 bytes --]

Dear Ingemar and all,

I hope not to hijack the topic, but I would like to add some bits to the
interesting HAS/DASH
discussion you bootstrapped.
Regarding the performance of HAS/DASH adaptive streaming control algorithm,
the reason for the
poor performance of rate-based algorithms is due to the ON/OFF behaviour of
the clients (i.e. video clients
insert idle periods to control the buffer and concurrent TCP flows can take
advantage of this).
This phenomenon was first uncovered in the IMC 2012 paper that Te-Yuan et
al. named "the downward
spiral effect" and is also experimentally shown in the following paper in
the case of several "rate based" algorithms
(sorry for the advertisement ;)) where we proposed a buffer-based adaptive
streaming control algorithm:

http://c3lab.poliba.it/images/a/a1/Elastic-pv2013.pdf
L. De Cicco, V. Caldaralo, V. Palmisano, and S. Mascolo
ELASTIC: a Client-side Controller for Dynamic Adaptive Streaming over HTTP
(DASH)
Proc. of Packet Video Workshop 2013, San Jose, CA, USA, December 2013

A more theoretical explanation of rate-based and buffer-based approaches
can be found here
 where some properties of hysteresis buffer-based controllers are shown:

http://c3lab.poliba.it/images/b/b1/Acc2015.pdf
Giuseppe Cofano, Luca De Cicco, Saverio Mascolo
Characterizing Adaptive Video Streaming Control Systems
Proc. of American Control Conference (ACC 2015), Chicago, USA, July 1-3 2015

Cheers,
--
Luca De Cicco
Assistant Professor
Politecnico di Bari


On Thu, Mar 3, 2016 at 1:13 PM Ingemar Johansson S <
ingemar.s.johansson@ericsson.com> wrote:

> Hi
>
> Thanks for the pointer to the RITE paper, will read it carefully.
> Some comments on HAS or DASH
> The HAS behavior when subject to different AQMs and other competing
> background traffic, depends heavily the rate control algorithm.
> Investigations we have done and also e.g. the Netflix papers and lately
> also the BOLA paper (link below) shows that rate based algorithms are more
> easily starved out by competing traffic than the buffer level based
> algorithms. The bufferbased algorithms (Netflix, BOLA) are more
> opportunistic and TCP is more allowed to work like large file downloads
> when the links are fully utilized. Rate based algorithms on the other hand
> can more easily end up in a vicious circle, a low download rate is
> detected, so the next segment requested is with a reduced rate, other
> traffic will grab a larger share, and this repeats itself until the lowest
> rate is reached.
> This is elaborated upon in the paper "A Buffer-Based Approach to Rate
> Adaptation: Evidence from a Large Video Streaming Service" by Te-Yuan Huang
> et.al.
> I don't have full insight how MS Silverlight operates so I cannot quantify
> it is rate based or buffer based.
>
> BOLA : http://arxiv.org/pdf/1601.06748.pdf
>
> /Ingemar
>
> > -----Original Message-----
> > From: Fred Baker (fred) [mailto:fred@cisco.com]
> > Sent: den 2 mars 2016 19:09
> > To: Dave Täht
> > Cc: aqm@ietf.org; bloat@lists.bufferbloat.net
> > Subject: Re: [aqm] [Bloat] review: Deployment of RITE mechanisms, in use-
> > case trial testbeds report part 1
> >
> >
> > > On Feb 27, 2016, at 11:04 AM, Dave Täht <dave@taht.net> wrote:
> > >
> > > https://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-
> > 14-confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/
> > >
> > >>   o the results are very poor with a particular popular AQM
> > >
> > > Define "very poor". ?
> >
> > Presuming this is Adaptive Bitrate Video, as in Video-in-TCP, we (as in
> Cisco
> > engineers, not me personally; you have met them) have observed this as
> > well. Our belief is that this is at least in part a self-inflicted
> wound; when the
> > codec starts transmission on any four second segment except the first,
> there
> > is no slow-start phase because the TCP session is still open (and in the
> case of
> > some services, there are several TCP sessions open and the application
> > chooses the one with the highest cwnd value). You can now think of the
> > behavior of the line as repeating a four phase sequence: nobody is
> talking,
> > then one is talking, then both are, and then the other is talking. When
> only
> > one is talking, whichever it is, its cwnd value is slowing increasing -
> especially
> > if cwnd*mss/rtt < bottleneck line rate, minimizing RTT. At the start of
> the
> > "both are talking" phase, the one already talking has generally found a
> cwnd
> > value that fills the line and its RTT is slowly increasing. The one
> starting sends
> > a burst of cwnd packets, creating an instant queue and often causing one
> or
> > both to drop a packet - reducing their respective cwnd values. Depending
> on
> > the TCP implementation in question at the sender, if the induced drop
> isn't a
> > single packet but is two or three, that can make the affected session
> pause
> > for as many RTO timeouts (Reno), RTTs (New Reno), or at least retransmit
> > the lost packets in the subsequent RTT and then reduce cwnd by at least
> that
> > amount (cubic) and maybe half (SACK).
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
>

[-- Attachment #2: Type: text/html, Size: 7094 bytes --]

[Bloat] HAS traffic behaviors

[Dave Täht]

On 3/3/16 4:31 AM, Luca De Cicco wrote:
> Dear Ingemar and all, 
> 
> I hope not to hijack the topic, 

No worries, but I changed the subject line to suit. I strongly support
those writing papers to submit them for discussion to the lists... I
read google scholar regularly but missed these.

>but I would like to add some bits to the
> interesting HAS/DASH 
> discussion you bootstrapped.
> Regarding the performance of HAS/DASH adaptive streaming control
> algorithm, the reason for the 
> poor performance of rate-based algorithms is due to the ON/OFF behaviour
> of the clients (i.e. video clients
> insert idle periods to control the buffer and concurrent TCP flows can
> take advantage of this). 
> This phenomenon was first uncovered in the IMC 2012 paper that Te-Yuan
> et al. named "the downward 
> spiral effect" and is also experimentally shown in the following paper
> in the case of several "rate based" algorithms
> (sorry for the advertisement ;)) where we proposed a buffer-based
> adaptive streaming control algorithm:
> 
> http://c3lab.poliba.it/images/a/a1/Elastic-pv2013.pdf
> L. De Cicco, V. Caldaralo, V. Palmisano, and S. Mascolo
> ELASTIC: a Client-side Controller for Dynamic Adaptive Streaming over
> HTTP (DASH)
> Proc. of Packet Video Workshop 2013, San Jose, CA, USA, December 2013

This experiment can be easily repeated with various AQM,AQM/fq
technologies in the link.

The size of the drop tail queue is not documented...

Is this the netshaper codebase? http://netshaper.sourceforge.net/ ?

"NetShaper that performs bandwidth shaping and allows
propagation delays to be set. This tool uses the nfqueue
library provided by Netfilter1
in order to capture and
redirect the packets arriving at the client host to a user space
drop-tail queue, where traffic shaping and measurement are
performed."

This is in a place where we would just plug the various algorithms into
the sqm-scripts and not do it in userspace.

The emulated base RTT is not documented. Arguably HAS traffic would
often have a lower base RTT than a random tcp link.

The dynamic range of the selected rates seems low.

I am always harping on the need to test against an upload flow, and to
emulate asymmetric network links common in the home, in general.

> A more theoretical explanation of rate-based and buffer-based approaches
> can be found here
>  where some properties of hysteresis buffer-based controllers are shown:
> 
> http://c3lab.poliba.it/images/b/b1/Acc2015.pdf

Read this also. I guess a key thing I keep wanting to see is the effect
of a web traffic burst - and the reaction time changes with an upload
going on at the same time. The baseline RTT is tiny compared to the
enormous buffers in CMTSes, in particular.

> Giuseppe Cofano, Luca De Cicco, Saverio Mascolo
> Characterizing Adaptive Video Streaming Control Systems
> Proc. of American Control Conference (ACC 2015), Chicago, USA, July 1-3 2015
> 
> Cheers,
> --
> Luca De Cicco
> Assistant Professor
> Politecnico di Bari
> 
> 
> On Thu, Mar 3, 2016 at 1:13 PM Ingemar Johansson S
> <ingemar.s.johansson@ericsson.com
> <mailto:ingemar.s.johansson@ericsson.com>> wrote:
> 
>     Hi
> 
>     Thanks for the pointer to the RITE paper, will read it carefully.
>     Some comments on HAS or DASH
>     The HAS behavior when subject to different AQMs and other competing
>     background traffic, depends heavily the rate control algorithm.
>     Investigations we have done and also e.g. the Netflix papers and
>     lately also the BOLA paper (link below) shows that rate based
>     algorithms are more easily starved out by competing traffic than the
>     buffer level based algorithms. The bufferbased algorithms (Netflix,
>     BOLA) are more opportunistic and TCP is more allowed to work like
>     large file downloads when the links are fully utilized. Rate based
>     algorithms on the other hand can more easily end up in a vicious
>     circle, a low download rate is detected, so the next segment
>     requested is with a reduced rate, other traffic will grab a larger
>     share, and this repeats itself until the lowest rate is reached.
>     This is elaborated upon in the paper "A Buffer-Based Approach to
>     Rate Adaptation: Evidence from a Large Video Streaming Service" by
>     Te-Yuan Huang et.al <http://et.al>.
>     I don't have full insight how MS Silverlight operates so I cannot
>     quantify it is rate based or buffer based.
> 
>     BOLA : http://arxiv.org/pdf/1601.06748.pdf
> 
>     /Ingemar
> 
>     > -----Original Message-----
>     > From: Fred Baker (fred) [mailto:fred@cisco.com
>     <mailto:fred@cisco.com>]
>     > Sent: den 2 mars 2016 19:09
>     > To: Dave Täht
>     > Cc: aqm@ietf.org <mailto:aqm@ietf.org>;
>     bloat@lists.bufferbloat.net <mailto:bloat@lists.bufferbloat.net>
>     > Subject: Re: [aqm] [Bloat] review: Deployment of RITE mechanisms,
>     in use-
>     > case trial testbeds report part 1
>     >
>     >
>     > > On Feb 27, 2016, at 11:04 AM, Dave Täht <dave@taht.net
>     <mailto:dave@taht.net>> wrote:
>     > >
>     > > https://reproducingnetworkresearch.wordpress.com/2014/06/03/cs244-
>     > 14-confused-timid-and-unstable-picking-a-video-streaming-rate-is-hard/
>     > >
>     > >>   o the results are very poor with a particular popular AQM
>     > >
>     > > Define "very poor". ?
>     >
>     > Presuming this is Adaptive Bitrate Video, as in Video-in-TCP, we
>     (as in Cisco
>     > engineers, not me personally; you have met them) have observed this as
>     > well. Our belief is that this is at least in part a self-inflicted
>     wound; when the
>     > codec starts transmission on any four second segment except the
>     first, there
>     > is no slow-start phase because the TCP session is still open (and
>     in the case of
>     > some services, there are several TCP sessions open and the application
>     > chooses the one with the highest cwnd value). You can now think of the
>     > behavior of the line as repeating a four phase sequence: nobody is
>     talking,
>     > then one is talking, then both are, and then the other is talking.
>     When only
>     > one is talking, whichever it is, its cwnd value is slowing
>     increasing - especially
>     > if cwnd*mss/rtt < bottleneck line rate, minimizing RTT. At the
>     start of the
>     > "both are talking" phase, the one already talking has generally
>     found a cwnd
>     > value that fills the line and its RTT is slowly increasing. The
>     one starting sends
>     > a burst of cwnd packets, creating an instant queue and often
>     causing one or
>     > both to drop a packet - reducing their respective cwnd values.
>     Depending on
>     > the TCP implementation in question at the sender, if the induced
>     drop isn't a
>     > single packet but is two or three, that can make the affected
>     session pause
>     > for as many RTO timeouts (Reno), RTTs (New Reno), or at least
>     retransmit
>     > the lost packets in the subsequent RTT and then reduce cwnd by at
>     least that
>     > amount (cubic) and maybe half (SACK).
>     _______________________________________________
>     aqm mailing list
>     aqm@ietf.org <mailto:aqm@ietf.org>
>     https://www.ietf.org/mailman/listinfo/aqm
> 
> 
> 
> _______________________________________________
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
> 

Re: [Bloat] [aqm] HAS traffic behaviors

[Luca De Cicco]

[-- Attachment #1: Type: text/plain, Size: 3822 bytes --]

Dear Dave,

thanks a bunch for your feedback. My reply is inline.

On Thu, Mar 3, 2016 at 5:05 PM Dave Täht <dave@taht.net> wrote:

> On 3/3/16 4:31 AM, Luca De Cicco wrote:
> > Dear Ingemar and all,
> >
> > I hope not to hijack the topic,
>
> No worries, but I changed the subject line to suit. I strongly support
> those writing papers to submit them for discussion to the lists... I
> read google scholar regularly but missed these.
>

Great. Thanks for taking time to browse the papers :)


> >but I would like to add some bits to the
> > interesting HAS/DASH
> > discussion you bootstrapped.
> > Regarding the performance of HAS/DASH adaptive streaming control
> > algorithm, the reason for the
> > poor performance of rate-based algorithms is due to the ON/OFF behaviour
> > of the clients (i.e. video clients
> > insert idle periods to control the buffer and concurrent TCP flows can
> > take advantage of this).
> > This phenomenon was first uncovered in the IMC 2012 paper that Te-Yuan
> > et al. named "the downward
> > spiral effect" and is also experimentally shown in the following paper
> > in the case of several "rate based" algorithms
> > (sorry for the advertisement ;)) where we proposed a buffer-based
> > adaptive streaming control algorithm:
> >
> > http://c3lab.poliba.it/images/a/a1/Elastic-pv2013.pdf
> > L. De Cicco, V. Caldaralo, V. Palmisano, and S. Mascolo
> > ELASTIC: a Client-side Controller for Dynamic Adaptive Streaming over
> > HTTP (DASH)
> > Proc. of Packet Video Workshop 2013, San Jose, CA, USA, December 2013
>
> This experiment can be easily repeated with various AQM,AQM/fq
> technologies in the link.
>
> The size of the drop tail queue is not documented...
>
>
IIRC the DT queue size was equal to the BDP. We have also run experiments
with tc/netem but we didn't study the impact of AQMs on performance.


> Is this the netshaper codebase? http://netshaper.sourceforge.net/ ?
>
> "NetShaper that performs bandwidth shaping and allows
> propagation delays to be set. This tool uses the nfqueue
> library provided by Netfilter1
> in order to capture and
> redirect the packets arriving at the client host to a user space
> drop-tail queue, where traffic shaping and measurement are
> performed."
>
> This is in a place where we would just plug the various algorithms into
> the sqm-scripts and not do it in userspace.
>
> The emulated base RTT is not documented. Arguably HAS traffic would
> often have a lower base RTT than a random tcp link.
>
>
IIRC the baseRTT was set equal to 50ms.


> The dynamic range of the selected rates seems low.
>

What do you mean by selected rates?


> I am always harping on the need to test against an upload flow, and to
> emulate asymmetric network links common in the home, in general.
>
> > A more theoretical explanation of rate-based and buffer-based approaches
> > can be found here
> >  where some properties of hysteresis buffer-based controllers are shown:
> >
> > http://c3lab.poliba.it/images/b/b1/Acc2015.pdf
>
> Read this also. I guess a key thing I keep wanting to see is the effect
> of a web traffic burst - and the reaction time changes with an upload
> going on at the same time.


This can be easily set up. We have released a tool which is able to
generate a relatively high number of HAS flows using a single machine
(decoding/rendering of the video can be turned off but the dynamics of the
control algorithm is not impacted). The tool is also useful to
implement/compare different adaptive streaming control algos.
Please look at:

https://github.com/ldecicco/tapas
and the companion paper:
http://c3lab.poliba.it/images/f/f3/Tapas-videonext.pdf

(And now I swear I'll stop with the advertisement!)

Cheers,
Luca

[-- Attachment #2: Type: text/html, Size: 5436 bytes --]