[Cake] Latency target curiosity

[Cake] Latency target curiosity

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 877 bytes --]

Hi All,

As some of you are aware, I’ve been working on ’near live’ graphing of Cake’s stats under Openwrt, basically taking the output of ’tc -s -j qdisc foo’ and putting in relevant graphs.

A curiosity has arisen:  I use diffserv4 mode on a 20Mbit egress link.  Bulk tin has ‘capacity’ threshold of 1.2Mbit and because it’s a slow ’tin', the default target & interval values get overridden to 14.6ms and 109.6ms respectively.  The 3 other tins are 5ms & 100ms defaults.

I have a backup job that bulk uploads 5 simultaneous flows to Onedrive.  The sparse_delay, average_delay & peak_delay figures settle on 32, 38 & 43 ms respectively with around 9 drops per second on that tin.

I’m curious as to why the reported delays are over double the target latency?

Cheers,

Kevin D-B

gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Cake] Latency target curiosity

[Jonathan Morton]

> On 7 May, 2020, at 10:58 am, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
> 
> A curiosity has arisen:  I use diffserv4 mode on a 20Mbit egress link.  Bulk tin has ‘capacity’ threshold of 1.2Mbit and because it’s a slow ’tin', the default target & interval values get overridden to 14.6ms and 109.6ms respectively.  The 3 other tins are 5ms & 100ms defaults.
> 
> I have a backup job that bulk uploads 5 simultaneous flows to Onedrive.  The sparse_delay, average_delay & peak_delay figures settle on 32, 38 & 43 ms respectively with around 9 drops per second on that tin.
> 
> I’m curious as to why the reported delays are over double the target latency?

It's likely that there's a minimum cwnd in your sender's TCP stack, which may be as large as 4 segments.  In Linux it is 2 segments.  No matter how much congestion signalling is asserted, the volume of data in flight (including retransmissions of dropped packets) will always correspond to at least that minimum per flow.  If the path is short, most of that volume will exists in queues instead of on the wire.

Fortunately, backups are unlikely to suffer from a small amount of extra latency, and Cake will isolate their influence from other flows that may be more sensitive.

 - Jonathan Morton

Re: [Cake] Latency target curiosity

[Kevin Darbyshire-Bryant]

[-- Attachment #1: Type: text/plain, Size: 2415 bytes --]



> On 7 May 2020, at 09:09, Jonathan Morton <chromatix99@gmail.com> wrote:
> 
>> On 7 May, 2020, at 10:58 am, Kevin Darbyshire-Bryant <kevin@darbyshire-bryant.me.uk> wrote:
>> 
>> A curiosity has arisen:  I use diffserv4 mode on a 20Mbit egress link.  Bulk tin has ‘capacity’ threshold of 1.2Mbit and because it’s a slow ’tin', the default target & interval values get overridden to 14.6ms and 109.6ms respectively.  The 3 other tins are 5ms & 100ms defaults.
>> 
>> I have a backup job that bulk uploads 5 simultaneous flows to Onedrive.  The sparse_delay, average_delay & peak_delay figures settle on 32, 38 & 43 ms respectively with around 9 drops per second on that tin.
>> 
>> I’m curious as to why the reported delays are over double the target latency?
> 
> It's likely that there's a minimum cwnd in your sender's TCP stack, which may be as large as 4 segments.  In Linux it is 2 segments.  No matter how much congestion signalling is asserted, the volume of data in flight (including retransmissions of dropped packets) will always correspond to at least that minimum per flow.  If the path is short, most of that volume will exists in queues instead of on the wire.

This is a Qnap NAS box running (a form of) Linux but to say I don’t trust the IP stack as being vanilla is an understatement.  I’ve seen it do some really odd things when enabling ECN on egress…I don’t know if that’s a qnap or onedrive issue.
> 
> Fortunately, backups are unlikely to suffer from a small amount of extra latency, and Cake will isolate their influence from other flows that may be more sensitive.

Absolutely!  That’s why the traffic is in the Bulk tin, I don’t care about its ‘interactivity’, I just want the data transferred eventually :-)  Cake does really well, I’ve had these backups running for days, maxing out the line but I simply couldn’t tell.  I can also reasonably reliably classify facetime calls, so I put those in ‘Video’, so just for fun, whilst the other half was engaged on a 2 hour(!) facetime call with family I started backups, ran speed tests etc to try to disturb that call.  Cake just kept that video call running smoothly all the time, brilliant!  I know that’s more of a tin isolation test rather than a flow isolation test but it’s still great!



Cheers,

Kevin D-B

gpg: 012C ACB2 28C6 C53E 9775  9123 B3A2 389B 9DE2 334A


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]