From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 42FC43B2A4 for ; Sun, 12 Nov 2017 14:59:03 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:References:Cc:To:Subject:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=SqW3G/eDlb1rgVrPC5g8sRaPMYxnYPJRYWak+sAiaqY=; b=BfGgVbi0SvGDBBIN37RN1kHTw LrWsIXXXAv39j7EGLlr2sZJtg12Hk9UnugXFKgKCZSg/Fjc0D4Mj+XJYzsPsRYlbRB72QeYgqIESl BUrmgFZNuZOTMaqB98ajJKG7iGPqMyJo+hqCy74EYEZWWWFx5mTcwsQBl+pPLC/2FmN/Lvoul4oCH kmkcnUtWFkM0hNgNGMZU5A/AOT2zS16dRkxNgvcEhCiqZ8HOtLUzWARSWT0RflgvinrGolDpMf7td y4xDUH6Uv4MF9xlDYhH5URVPzlbQg9fOBGAP8T4KQ2DkmwofGJOPuXFkEBILJb3gnjbp5hotnCIWC nBtQfv5NA==; Received: from [101.127.176.77] (port=51926 helo=[192.168.0.106]) by server.dnsblock1.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.89) (envelope-from ) id 1eDyPF-0008P6-Pa; Sun, 12 Nov 2017 19:59:02 +0000 From: Bob Briscoe To: Matthias Tafelmeier Cc: bloat@lists.bufferbloat.net References: <4d54f24f-ce83-34a0-41f3-9f728420d548@gmx.net> <87shdr0vt6.fsf@nemesis.taht.net> <79f4d92c-74f4-8cd0-9d38-e51a668cb9b6@gmx.net> Message-ID: <796aa11e-9e35-cf34-e456-6ae98d1875d6@bobbriscoe.net> Date: Mon, 13 Nov 2017 03:58:26 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <79f4d92c-74f4-8cd0-9d38-e51a668cb9b6@gmx.net> Content-Type: multipart/alternative; boundary="------------FDBB57726A7B19B559355185" Content-Language: en-GB X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server.dnsblock1.com X-AntiAbuse: Original Domain - lists.bufferbloat.net X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - bobbriscoe.net X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net Subject: Re: [Bloat] DETNET X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Nov 2017 19:59:03 -0000 This is a multi-part message in MIME format. --------------FDBB57726A7B19B559355185 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Matthias, Dave, The sort of industrial control applications that detnet is targeting require far lower queuing delay and jitter than fq_CoDel can give. They have thrown around numbers like 250us jitter and 1E-9 to 1E-12 packet loss probability. However, like you, I just sigh when I see the behemoth detnet is building. Nonetheless, it's important to have a debate about where to go to next. Personally I don't think fq_CoDel alone has legs to get (that) much better. I prefer the direction that Mohamad Alizadeh's HULL pointed in: Less is More: Trading a little Bandwidth for Ultra-Low Latency in the Data Center In HULL you have i) a virtual queue that models what the queue would be if the link were slightly slower, then marks with ECN based on that. ii)  a much more well-behaved TCP (HULL uses DCTCP with hardware pacing in the NICs). I would love to be able to demonstrate that HULL can achieve the same extremely low latency and loss targets as detnet, but with a fraction of the complexity. *Queuing latency?* This keeps the real FIFO queue in the low hundreds to tens of microseconds. *Loss prob?* Mohammad doesn't recall seeing a loss during the entire period of the experiments, but he doubted their measurement infrastructure was sufficiently accurate (or went on long enough) to be sure they were able to detect one loss per 10^12 packets. For their research prototype, HULL used a dongle they built, plugged into each output port to constrict the link in order to shift the AQM out of the box. However, Broadcom mid-range chipsets already contain vertual queue hardware (courtesey of a project we did with them when I was at BT: How to Build a Virtual Queue from Two Leaky Buckets (and why one is not enough) ). *For public Internet, not just for DCs?* You might have seen the work we've done (L4S ) to get queuing delay over regular public Internet and broadband down to about mean 500us; 90%-ile 1ms, by making DCTCP deployable alongside existing Internet traffic (unlike HULL, pacing at the source is in Linux, not hardware). My personal roadmap for that is to introduce virtual queues at some future stage, to get down to the sort of delays that detnet wants, but over the public Internet with just FIFOs. Radio links are harder, of course, but a lot of us are working on that too. Bob On 12/11/2017 22:58, Matthias Tafelmeier wrote: > On 11/07/2017 01:36 AM, Dave Taht wrote: >>> Perceived that as shareworthy/entertaining .. >>> >>> https://tools.ietf.org/html/draft-ietf-detnet-architecture-03#section-4.5 >>> >>> without wanting to belittle it. >> Hope springs eternal that they might want to look over the relevant >> codel and fq_codel RFCS at some point or another. > > Not sure, appears like juxtaposing classical mechanics to nanoscale > physics. > > -- > Besten Gruß > > Matthias Tafelmeier > > > > _______________________________________________ > Bloat mailing list > Bloat@lists.bufferbloat.net > https://lists.bufferbloat.net/listinfo/bloat --------------FDBB57726A7B19B559355185 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

Matthias, Dave,

The sort of industrial control applications that detnet is targeting require far lower queuing delay and jitter than fq_CoDel can give. They have thrown around numbers like 250us jitter and 1E-9 to 1E-12 packet loss probability.

However, like you, I just sigh when I see the behemoth detnet is building.

Nonetheless, it's important to have a debate about where to go to next. Personally I don't think fq_CoDel alone has legs to get (that) much better.

I prefer the direction that Mohamad Alizadeh's HULL pointed in:
Less is More: Trading a little Bandwidth for Ultra-Low Latency in the Data Center

In HULL you have i) a virtual queue that models what the queue would be if the link were slightly slower, then marks with ECN based on that. ii)  a much more well-behaved TCP (HULL uses DCTCP with hardware pacing in the NICs).

I would love to be able to demonstrate that HULL can achieve the same extremely low latency and loss targets as detnet, but with a fraction of the complexity.

Queuing latency? This keeps the real FIFO queue in the low hundreds to tens of microseconds.

Loss prob? Mohammad doesn't recall seeing a loss during the entire period of the experiments, but he doubted their measurement infrastructure was sufficiently accurate (or went on long enough) to be sure they were able to detect one loss per 10^12 packets.

For their research prototype, HULL used a dongle they built, plugged into each output port to constrict the link in order to shift the AQM out of the box. However, Broadcom mid-range chipsets already contain vertual queue hardware (courtesey of a project we did with them when I was at BT:
How to Build a Virtual Queue from Two Leaky Buckets (and why one is not enough) ).

For public Internet, not just for DCs? You might have seen the work we've done (L4S) to get queuing delay over regular public Internet and broadband down to about mean 500us; 90%-ile 1ms, by making DCTCP deployable alongside existing Internet traffic (unlike HULL, pacing at the source is in Linux, not hardware). My personal roadmap for that is to introduce virtual queues at some future stage, to get down to the sort of delays that detnet wants, but over the public Internet with just FIFOs.

Radio links are harder, of course, but a lot of us are working on that too.



Bob

On 12/11/2017 22:58, Matthias Tafelmeier wrote:
On 11/07/2017 01:36 AM, Dave Taht wrote:
Perceived that as shareworthy/entertaining ..

https://tools.ietf.org/html/draft-ietf-detnet-architecture-03#section-4.5

without wanting to belittle it.

Hope springs eternal that they might want to look over the relevant
codel and fq_codel RFCS at some point or another.

Not sure, appears like juxtaposing classical mechanics to nanoscale physics.

-- 
Besten Gruß

Matthias Tafelmeier




_______________________________________________
Bloat mailing list
Bloat@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/bloat

--------------FDBB57726A7B19B559355185--