[Cerowrt-devel] Ubiquiti QOS

Wed May 28 12:34:47 EDT 2014

Ok, I am not understanding your proposal then.

I thought you were claiming that since the optimum buffer length is 1-2 packets, 
the endpoints should be adjusting their sending speeds to try and make that 
happen on all switches and routers in the path.

The endpoints do know what their latency budget is and can have up to that much 
data in flight. They don't know if that data is sitting in router buffers, or is 
in transit on a high-speed-high-latency link (high speed satellite links can 
have a LOT of data that's left the transmitter, but not arrived at the receiver 
yet, this is will look exactly like data sitting in a buffer to the endpoints)

the endpoints don't know the state of all the intermediate connections, so 
unless they get feedback (ECN or dropped packets) they have to assume that there 
is no congestion.

David Lang

On Wed, 28 May 2014, dpreed at reed.com wrote:

> Interesting conversation.   A particular switch has no idea of the "latency budget" of a particular flow - so it cannot have its *own* latency budget.   The switch designer has no choice but to assume that his latency budget is near zero.
> 
> The number of packets that should be sustained in flight to maintain maximum throughput between the source (entry) switch and destination (exit) switch of the flow need be no higher than
> 
> the flow's share of bandwidth of the bottleneck
> 
> multiplied by
> 
> the end-to-end delay (including packet forwarding, but not queueing).
> 
> All buffering needed for isochrony ("jitter buffer") and "alternative path selection" can be moved to either before the entry switch or after the exit switch.
> 
> If you have multiple simultaneous paths, the number of packets in flight involves replacing "bandwidth of the bottleneck" with "aggregate bandwidth across the minimum cut-set of the chosen paths used for the flow".
> 
> Of course, these are dynamic - "the flow's share" and "paths used for the flow" change over short time scales.  That's why you have a control loop that needs to measure them.
> 
> The whole point of minimizing buffering is to make the measurements more timely and the control inputs more timely.  This is not about convergence to an asymptote....
> 
> A network where every internal buffer is driven hard toward zero makes it possible to handle multiple paths, alternate paths, etc. more *easily*.   That's partly because you allow endpoints to see what is happening to their flows more quickly so they can compensate.
> 
> And of course for shared wireless resources, things change more quickly because of new factors - more sharing, more competition for collision-free slots, varying transmission rates, etc.
> 
> The last thing you want is long-term standing waves caused by large buffers and very loose control.
> 
>
>
> On Tuesday, May 27, 2014 11:21pm, "David Lang" <david at lang.hm> said:
>
>
>
>> On Tue, 27 May 2014, Dave Taht wrote:
>> 
>> > On Tue, May 27, 2014 at 4:27 PM, David Lang <david at lang.hm> wrote:
>> >> On Tue, 27 May 2014, Dave Taht wrote:
>> >>
>> >>> There is a phrase in this thread that is begging to bother me.
>> >>>
>> >>> "Throughput". Everyone assumes that throughput is a big goal - and
>> it
>> >>> certainly is - and latency is also a big goal - and it certainly is
>> -
>> >>> but by specifying what you want from "throughput" as a compromise
>> with
>> >>> latency is not the right thing...
>> >>>
>> >>> If what you want is actually "high speed in-order packet delivery" -
>> >>> say, for example a movie,
>> >>> or a video conference, youtube, or a video conference - excessive
>> >>> latency with high throughput, really, really makes in-order packet
>> >>> delivery at high speed tough.
>> >>
>> >>
>> >> the key word here is "excessive", that's why I said that for max
>> throughput
>> >> you want to buffer as much as your latency budget will allow you to.
>> >
>> > Again I'm trying to make a distinction between "throughput", and "packets
>> > delivered-in-order-to-the-user." (for-which-we-need-a-new-word-I think)
>> >
>> > The buffering should not be in-the-network, it can be in the application.
>> >
>> > Take our hypothetical video stream for example. I am 20ms RTT from netflix.
>> > If I artificially inflate that by adding 50ms of in-network buffering,
>> > that means a loss can
>> > take 120ms to recover from.
>> >
>> > If instead, I keep a 3*RTT buffer in my application, and expect that I have
>> 5ms
>> > worth of network-buffering, instead, I recover from a loss in 40ms.
>> >
>> > (please note, it's late, I might not have got the math entirely right)
>> 
>> but you aren't going to be tuning the retry wait time per connection. what is
>> the retry time that is set in your stack? It's something huge to survive
>> international connections with satellite paths (so several seconds worth). If
>> your server-to-eyeball buffering is shorter than this, you will get a window
>> where you aren't fully utilizing the connection.
>> 
>> so yes, I do think that if your purpose is to get the maximum possible in-order
>> packets delivered, you end up making different decisions than if you are just
>> trying to stream a HD video, or do other normal things.
>> 
>> The problem is thinking that this absolute throughput is representitive of
>> normal use.
>> 
>> > As physical RTTs grow shorter, the advantages of smaller buffers grow
>> larger.
>> >
>> > You don't need 50ms queueing delay on a 100us path.
>> >
>> > Many applications buffer for seconds due to needing to be at least
>> > 2*(actual buffering+RTT) on the path.
>> 
>> For something like streaming video, there's nothing wrong with the application
>> buffering aggressivly (assuming you have the space to do so on the client side),
>> the more you have gotten transmitted to the client, the longer it can survive a
>> disruption of it's network.
>> 
>> There's nothing wrong with having an hour of buffered data between the server
>> and the viewer's eyes.now, this buffering should not be in the network devices, it
>> should be in the
>> client app, but this isn't because there's something wrong with bufferng, it's
>> just because the client device has so much more available space to hold stuff.
>> 
>> David Lang
>> 
>> >>
>> >>> You eventually lose a packet, and you have to wait a really long
>> time
>> >>> until a replacement arrives. Stuart and I showed that at last ietf.
>> >>> And you get the classic "buffering" song playing....
>> >>
>> >>
>> >> Yep, and if you buffer too much, your "lost packet" is actually still in
>> >> flight and eating bandwidth.
>> >>
>> >> David Lang
>> >>
>> >>
>> >>> low latency makes recovery from a loss in an in-order stream much,
>> much
>> >>> faster.
>> >>>
>> >>> Honestly, for most applications on the web, what you want is high
>> >>> speed in-order packet delivery, not
>> >>> "bulk throughput". There is a whole class of apps (bittorrent, file
>> >>> transfer) that don't need that, and we
>> >>> have protocols for those....
>> >>>
>> >>>
>> >>>
>> >>> On Tue, May 27, 2014 at 2:19 PM, David Lang <david at lang.hm>
>> wrote:
>> >>>>
>> >>>> the problem is that paths change, they mix traffic from streams,
>> and in
>> >>>> other ways the utilization of the links can change radically in a
>> short
>> >>>> amount of time.
>> >>>>
>> >>>> If you try to limit things to exactly the ballistic throughput,
>> you are
>> >>>> not
>> >>>> going to be able to exactly maintain this state, you are either
>> going to
>> >>>> overshoot (too much traffic, requiring dropping packets to
>> maintain your
>> >>>> minimal buffer), or you are going to undershoot (too little
>> traffic and
>> >>>> your
>> >>>> connection is idle)
>> >>>>
>> >>>> Since you can't predict all the competing traffic throughout the
>> >>>> Internet,
>> >>>> if you want to maximize throughput, you want to buffer as much as
>> you can
>> >>>> tolerate for latency reasons. For most apps, this is more than
>> enough to
>> >>>> cause problems for other connections.
>> >>>>
>> >>>> David Lang
>> >>>>
>> >>>>
>> >>>>  On Mon, 26 May 2014, David P. Reed wrote:
>> >>>>
>> >>>>> Codel and PIE are excellent first steps... but I don't think
>> they are
>> >>>>> the
>> >>>>> best eventual approach.  I want to see them deployed ASAP in
>> CMTS' s and
>> >>>>> server load balancing networks... it would be a disaster to
>> not deploy
>> >>>>> the
>> >>>>> far better option we have today immediately at the point of
>> most
>> >>>>> leverage.
>> >>>>> The best is the enemy of the good.
>> >>>>>
>> >>>>> But, the community needs to learn once and for all that
>> throughput and
>> >>>>> latency do not trade off. We can in principle get far better
>> latency
>> >>>>> while
>> >>>>> maintaining high throughput.... and we need to start thinking
>> about
>> >>>>> that.
>> >>>>> That means that the framing of the issue as AQM is
>> counterproductive.
>> >>>>>
>> >>>>> On May 26, 2014, Mikael Abrahamsson <swmike at swm.pp.se>
>> wrote:
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, 26 May 2014, dpreed at reed.com wrote:
>> >>>>>>
>> >>>>>>> I would look to queue minimization rather than "queue
>> management"
>> >>>>>>
>> >>>>>>
>> >>>>>> (which
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> implied queues are often long) as a goal, and think
>> harder about the
>> >>>>>>> end-to-end problem of minimizing total end-to-end
>> queueing delay
>> >>>>>>
>> >>>>>>
>> >>>>>> while
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> maximizing throughput.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> As far as I can tell, this is exactly what CODEL and PIE
>> tries to do.
>> >>>>>> They
>> >>>>>> try to find a decent tradeoff between having queues to
>> make sure the
>> >>>>>> pipe
>> >>>>>> is filled, and not making these queues big enough to
>> seriously affect
>> >>>>>> interactive performance.
>> >>>>>>
>> >>>>>> The latter part looks like what LEDBAT does?
>> >>>>>> <http://tools.ietf.org/html/rfc6817>
>> >>>>>>
>> >>>>>> Or are you thinking about something else?
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> -- Sent from my Android device with K-@ Mail. Please excuse
>> my brevity.
>> >>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Cerowrt-devel mailing list
>> >>>> Cerowrt-devel at lists.bufferbloat.net
>> >>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> >>>>
>> >>>> _______________________________________________
>> >>>> Cerowrt-devel mailing list
>> >>>> Cerowrt-devel at lists.bufferbloat.net
>> >>>> https://lists.bufferbloat.net/listinfo/cerowrt-devel
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>
>> >
>> >
>> >
>> >
>>