From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 333313B29E for ; Mon, 13 Nov 2017 12:56:56 -0500 (EST) Received: from [192.168.10.50] ([93.233.79.61]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MMTZa-1eI7uf1B8A-008L7n; Mon, 13 Nov 2017 18:56:11 +0100 To: Bob Briscoe Cc: bloat@lists.bufferbloat.net, ken@cs.cornell.edu References: <4d54f24f-ce83-34a0-41f3-9f728420d548@gmx.net> <87shdr0vt6.fsf@nemesis.taht.net> <79f4d92c-74f4-8cd0-9d38-e51a668cb9b6@gmx.net> <796aa11e-9e35-cf34-e456-6ae98d1875d6@bobbriscoe.net> From: Matthias Tafelmeier Message-ID: Date: Mon, 13 Nov 2017 18:56:09 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <796aa11e-9e35-cf34-e456-6ae98d1875d6@bobbriscoe.net> Content-Type: multipart/signed; micalg=pgp-sha384; protocol="application/pgp-signature"; boundary="cPUPLc7Xnto8vWCrD24dtNwPnKvH56p6O" X-Provags-ID: V03:K0:CTTtez0Hb01cNruS+1VYnZLdBPbRvhmTuuhYl4SjtEfgBu9PLpY Mnp3OEbhlqnWQbwF8t0L6QIwP3DMfbUGMJsrGQxCZ1a9JDkof5VJqvitwVmYSAU2CyjLHI6 FMKqSnDJv9y0CIlG6wYCTK6oN7UBdH6lUKkT1SUzUEITYpykpyITZySIyLvRtndy9RMduyT TaszAj/LNjlcU9NIyOMPg== X-UI-Out-Filterresults: notjunk:1;V01:K0:nO9oTU82XoU=:2Vq6GOnmo+lg1XfNTSIETo v38uJaAFO0fFbFtiwPnUD4jDod0ssHaGEAunAav3gN+Dh8WZLtIQrM3w/c7YK/mDoSPAJtFh1 xf7EE97oVK6w267zF66QXxkw/4PCuOLvj1G/pR5VKh1xAjOTwr8RfTu0FiXkZmfwNJYs8o97w srrYaLMRxkgk70ihON9alMfZOPyFp4JlJI5gbrzT/dzt0kj0swGtCmwiEI1kGQG8nTkwSVMqO xItBK1whzNqUuRE8k57kEOoYqmWH4TBIpQ+fCZs482ACGUJKr4sN1+WUB+rBSHN070thEzUEj rj/OhVMTspMVC7oCaZqqwZMfFLLVnZqQHKk+uxjxW4KfJaBqQ3pa1D8NsX8uSPuOS5+nxMi/P yy4ibI2evAfPl1rhPHFPpolqZR1sgADAedm+XRtBdjArDXOG1bLUfGzoszUekns8Or/XwjQti UHwgoJ1+4kmcfzuYpwRg43ad9FtPuVosqZjEuhMZ5YSlu6/el8PrgDQuDeiDMY0IrI3qgMKrY cAC5LTDykrN8QGyfBgMzGQyH0ibE7GbfxG05bMp+HErPDIjeaCAf013vbvAxqkkWGK76fti+F GBFF+hYw7ymQ5wAc+dGMy3QEJBcAQPGtifVFLMWwZP+gEvfvhQLpOu1qqCwredmXpgX0eNM/O R1hrg4BMsZit0ymNyVWZWiI+OpevTqv59JstIDbWVh3vIi+T/7VzJh8+DDKbnd+42WBUokV0q tKOwTObbBOVFnhicF5Hj5L3cNxChx+X0J21iGLUYo5G+zom9kcck5DI0WCmEcIz3psrhowHSP lwnpHLwa6ZHXHHbVRdy77zOwX8pUQ== Subject: Re: [Bloat] DETNET X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2017 17:56:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --cPUPLc7Xnto8vWCrD24dtNwPnKvH56p6O Content-Type: multipart/mixed; boundary="vmC1OAcVgL9QRuRQjbJeri8sh1PtaxS08"; protected-headers="v1" From: Matthias Tafelmeier To: Bob Briscoe Cc: bloat@lists.bufferbloat.net, ken@cs.cornell.edu Message-ID: Subject: Re: [Bloat] DETNET References: <4d54f24f-ce83-34a0-41f3-9f728420d548@gmx.net> <87shdr0vt6.fsf@nemesis.taht.net> <79f4d92c-74f4-8cd0-9d38-e51a668cb9b6@gmx.net> <796aa11e-9e35-cf34-e456-6ae98d1875d6@bobbriscoe.net> In-Reply-To: <796aa11e-9e35-cf34-e456-6ae98d1875d6@bobbriscoe.net> --vmC1OAcVgL9QRuRQjbJeri8sh1PtaxS08 Content-Type: multipart/mixed; boundary="------------2362BD2813048BD987309465" Content-Language: de-DE This is a multi-part message in MIME format. --------------2362BD2813048BD987309465 Content-Type: multipart/alternative; boundary="------------9229CA662074707276C5C3A6" --------------9229CA662074707276C5C3A6 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > However, like you, I just sigh when I see the behemoth detnet is buildi= ng. > Does it? Well, so far the circumference seems justififiable for what they want to achieve, at least according to what I can tell from these rather still abstract concepts. =20 >> The sort of industrial control applications that detnet is targeting >> require far lower queuing delay and jitter than fq_CoDel can give. >> They have thrown around numbers like 250us jitter and 1E-9 to 1E-12 >> packet loss probability. >> > Nonetheless, it's important to have a debate about where to go to > next. Personally I don't think fq_CoDel alone has legs to get (that) > much better. > Certainly, all you said is valid - as I stated, I mostly wanted to share the digest/the existance of the inititiative without judging/reproaching/peaching ... > I prefer the direction that Mohamad Alizadeh's HULL pointed in: > Less is More: Trading a little Bandwidth for Ultra-Low Latency in the > Data Center > > In HULL you have i) a virtual queue that models what the queue would > be if the link were slightly slower, then marks with ECN based on > that. ii) a much more well-behaved TCP (HULL uses DCTCP with hardware > pacing in the NICs). > > I would love to be able to demonstrate that HULL can achieve the same > extremely low latency and loss targets as detnet, but with a fraction > of the complexity. > Well, if it's already for specific HW, then I'd prefer to see RDMA in place right away with getting rid of IRQs and other TCP/IP specific rust along the way, at least for DC realms :) Although, this HULL might has a spin for it from economics perspective. > *For public Internet, not just for DCs?* You might have seen the work > we've done (L4S ) to get queuing delay > over regular public Internet and broadband down to about mean 500us; > 90%-ile 1ms, by making DCTCP deployable alongside existing Internet > traffic (unlike HULL, pacing at the source is in Linux, not hardware). > My personal roadmap for that is to introduce virtual queues at some > future stage, to get down to the sort of delays that detnet wants, but > over the public Internet with just FIFOs. > > Thanks for sharing, that sounds thrilling - especially the achieved latencies and the non-spec. HW needs. All the best with it, again, maybe more an economical quarrel to overcome then again. --=20 Besten Gru=C3=9F Matthias Tafelmeier --------------9229CA662074707276C5C3A6 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

However, like you, I just sigh when I see the behemoth detnet is building.

Does it? Well, so far the circumference seems justififiable for what they want to achieve, at least according to what I can tell from these rather still abstract concepts.
=C2=A0

=

The sort of industrial control applications that detnet is targeting require far lower queuing delay and jitter than fq_CoDel can give. They have thrown around numbers like 250us jitter and 1E-9 to 1E-12 packet loss probability.

Nonetheless, it's important to have a debate about where to go to next. Personally I don't think fq_CoDel alone has legs to get (that) much better.

Certainly, all you said is valid - as I stated, I mostly wanted to share the digest/the existance of the inititiative without judging/reproaching/peaching ...

I prefer the direction that Mohamad Alizadeh's HULL pointed in:
Less is More: Trading a little Bandwidth for Ultra-Low Latency in the Data Center

In HULL you have i) a virtual queue that models what the queue would be if the link were slightly slower, then marks with ECN based on that. ii)=C2=A0 a much more well-behaved TCP (HULL uses DCTCP with hardware pacing in the NICs).

I would love to be able to demonstrate that HULL can achieve the same extremely low latency and loss targets as detnet, but with a fraction of the complexity.

Well, if it's already for specific HW, then I'd prefer to see RDMA in place right away with getting rid of IRQs and other TCP/IP specific rust along the way, at least for DC realms :) Although, this HULL might has a spin for it from economics perspective.

For public Internet, not just for DCs? You might have seen the work we've done (L4S) to get queuing delay over regular public Internet and broadband down to about mean 500us; 90%-ile 1ms, by making DCTCP deployable alongside existing Internet traffic (unlike HULL, pacing at the source is in Linux, not hardware). My personal roadmap for that is to introduce virtual queues at some future stage, to get down to the sort of delays that detnet wants, but over the public Internet with just FIFOs.


Thanks for sharing, that sounds thrilling - especially the achieved latencies and the non-spec. HW needs. All the best with it, again, maybe more an economical quarrel to overcome then again.

--=20
Besten Gru=C3=9F

Matthias Tafelmeier
--------------9229CA662074707276C5C3A6-- --------------2362BD2813048BD987309465 Content-Type: application/pgp-keys; name="0x8ADF343B.asc" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0x8ADF343B.asc" -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2 mQENBFJ0FzIBCADZ/hkwcprVGydMOqeqM+2k6v5e5kb4YDMKU7nMbCVmH4sn01T7 Yh9kDwG5LOMLD06BB2txjLBvTY+c0mpK+hE4pWr+i3qhU5CbVvx7jppJqCD6ZT/T A3I7NxsdixRvLIF4UXgKQOMKPIx+aw/sp86NqzCLAMse7F0vXUjAP5YANtJid2rf r/B37BGKhqDGhi4Appz4UZOzpRov/v8JD4XScuvJnl09/oi5cDj3Mn2uqOc/G6hA t7HXsbHh4dKxd3AftqPPzEkJAmm+9Z4ASG9hy8IXms8Czimr+BGL0CnfsJlX6DCU m6mVDqT1GJyzmP4zkWcPi+2fOI4KtpV+C7+bABEBAAG0Ok1hdHRoaWFzIFRhZmVs bWVpZXIgKHByaXZhdCkgPG1hdHRoaWFzLnRhZmVsbWVpZXJAZ214Lm5ldD6JAUEE EwECACsCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAhkBBQJaAMzNBQkLTxyN AAoJEOAWT1uK3zQ7u+4IAL+W82wbz1FwGfNHhOgOheCh/wlTLssgQ7XVGRduJ/m3 k22aodOKSV5aH3AUy9c9zkgkRHUU5XCG9FRujVeVYhvLP1JTG97oEjk8YGBAOqN7 D4hUHh0c3ZBpTqeE9cndXr504GMauh5mY74qdNl9nL+Gcv7CekENML1nLWBnoDWV NTpvZktPZpHozQHPBV6lk09ICxOocb7VHl+lyorStqiUcLciHTdOByC35ekJebv9 dDZ9oloI8tvLytyle1kuVQLJj0LrpkUjcjLSYoa7ZFVKCNK6FM1pWd1XRHmWqhAn i4+4l92+UHU2TASBFIUVO3kPEDOXn7kK4q6tQ2pRexuJARwEEwECAAYFAlP4qFwA CgkQc1YJs62PXiMJwggAgwa8bM1DVdB5wdWVbsEvjDWgoD4CZOH+3/nCAKFv+eKf d3GrJUtOh3T/QVpmbVgNwnyqqNLGlyIOHltVkrn9WqSC33kuXsIStR6KM+LXnA99 FjyAiTcVbzbfl/XNlIQrgV9+niSSUFCUge5242itPjBBCtlYHUkQ5Y9hsNwV9Hb7 dpVxUf01CJcNKlWscC7lTt2FqjJrIOw2NHxgHWxRlDqo9dFg4uwI+O90orKAyJ6N Sowu2Ca6DXB3jHgoG7WbAh1nEVus/JkyVsnTIMCsfOpwhJNd1fvy6JMVAe7+/p6/ JGZcMjUTadmsHeJHwqOJSVOoX3Y8CZkV8/PHIqmYvokBPgQTAQIAKAUCU/hT+gIb AwUJAeEzgAYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQ4BZPW4rfNDvHZAf7 BpqfTpLdR25q72DZ4T6SrkcZCOJ8jJQhQ1K9cAc0snpK+jcWg9iUgCpV8QJiXpGL dAkux/YCu7SRstOSbMv4G4Qb/g8y2bowFI/mAzK1o6s6CYt3URNBe7zRLK6sJbK0 f5fDpWoRufW9/Ppj1/S7dki5JpkUlyGa/y2O+X4C/P0Rh3HfL5HicRHamc7PVElh z/8nVA+KUkcA9ksVJqe50LahTbDyqOmd8cjSdUKlH2dsP/cAmZfU3IAa26UBKWVn rxnQ61VV5QLpcYvW3jTfzBy7xv1s7YSj2rFpIC5WKgPC/p+2gqVMSGI1hRlrNaSA qH9XgWNPCa5NlOxuN4mSN4kBQQQTAQIAKwIbAwYLCQgHAwIGFQgCCQoLBBYCAwEC HgECF4AFCQSrhqYFAlRXtDwCGQEACgkQ4BZPW4rfNDvqmQf/Wa2LXaOvzftegDI+ LAiFOB/Dq5yhFp6urk5+yC+YzCFin8HfP+LVXR8Xkei6fMmFMjfRU0MrNLBxFd3I UrIgRrtmJGaHB+vkIqNGgU8LcpHBdd6nprtIF53IhtOINwkmCgLzWi3sGYJ4yQyj 9OSNnh7j7ENFeZd8LgN/FgB5GjPisN3zJD19z065jlfeXvHIZOL90PaTqih90x6n oTr4dbKhk1t9zZYhY5W812gCMVn2g4wLLO+iijKOe8uNrOw22xDGckoL5UFRE8Vj Twup3eYyzb/2TVpAmM5GhnI+PodZ6GGcQRVKGMYwYyFVLDEcDRxAUpwfXfUzHwvD op2fQ7Q/TWF0dGhpYXMgVGFmZWxtZWllciAoSGFwbG9ycmhpbmkpIDxtYXR0aGlh cy50YWZlbG1laWVyQGdteC5uZXQ+iQEfBDABAgAJBQJWPZtPAh0AAAoJEOAWT1uK 3zQ7FrIH/3OFr/bZ2UQeJrn9n67G9o9neJv4ES9Lcq6xnCIc+ZRHqrBTwsYkfYC5 MEMTF7TMFNUJTr2Np3OG7iKcHVePpeMpHicXppJ4hUsIQ0kwXlynRAScrAqoQHBD IKzu5qHDME0UKIWr9iTASFHJgZGyH6OoPh7LIifV8cGdVPQ/5FF1kqM4YMZ3IygO C3CaYtEaOz0B1L00zJan8rbEnpsI1msZ3hjacGB2SD5kFAUMDbpoXVOOE07GSLF0 KKhMv02WdrKO8iedStubO9BON9Vf7IIq21RpDEhhjAzt4Ui2q0UEodTvnBX4ifFM UEU/+NdC3deuRwdxOq1ozSQlUTzVFASJATQEEwECAB4CGwMCHgECF4ACCwkCFQoC FgMFAlRXs+UFCQSrhqYACgkQ4BZPW4rfNDs/HQf+K5swcPreRRBXQbTBCgTQAoAI JtvG+TLlPPnpYMqkQoKIhw/USN2Je4Gqm3DhRcCteA5wbmhlHj9DbapbCOwE7vfK 3YC/hpntvnmgCl6atT2QbE4Ak7xeT2ljLiRYD1re7oE8fAUqkI2S+vePiK1+b8Cc OKPmuAJYmgAMmNVMKcknryNoFc7xseNEy58T+AoyCKcxV9ZJdyd+6Ye48LkRlmyf lfRnCvgfS74TEq7Gr5uCJPgqcjrl8SS3G6jgUrzPcV2mFROt9EH3d80T+GOIy4pB SeYGdfkeqUbflj0CeRIyazzAZurllCQWjpaeh009Y/wuzLm91zrVWwADP/5oRIkB HAQTAQIABgUCU/ioXAAKCRBzVgmzrY9eI23xB/461lM4c/08tEwmd0oC1jdwyidO ZRCj/vOqZ+Af6oB3FdpWseuKWdJ7zb8NR+BcEUQRqbaF/677cCrKIEnRoq7IzNsw KiqK9K5cFHLtm9TNZ0Mf5QP/PG47Jrex5l59LMMz+LW1Rv/uXJjTQDeQYsrYDAPK mJx2c0OyzZnr+CRrHJKUH0P+oVBvSQEnDTbCT2W9wuLrDIHF6H3YQLAuCS2sslq5 teAinTjTGnPmkP7hKcK3CC0BdiUgFFybIZOGthFm+bTG+V3qGUiam95dqDmQ2VTX QxCetGDTjUrvKdODh5qLFUM+StWLNEP+QtMcymZfrRIHayKS3GyjKBOPoCZpiQE0 BBMBAgAeAhsDBQkB4TOAAh4BAheABQJSdB8LAgsJAhUKAhYDAAoJEOAWT1uK3zQ7 5nMH/iscBMT7fEnIBeYZlFaxiJmFobRVWFP/A2IfzvKVIdY9vDqjN5M5chrRfsk9 HI0EPbiF3kmVwmRdl8J5fgN7O8QFbhW4ojda8UXBAsgF50kurqk2NrdgM+xLy2TI jVZdwQYK+R11SpO9xaf+nqLKV8r9VkQ5mzb+BEzDiaX7z9IBrm28v0BfelDZVRzW cOrskVnwX1PySt3xvCwwo3cwY39yno7H4AlgTXhAvhwI6DQMxZXm4MZcugSfB5b4 2uyslOOxMkgvLW1CmpJxzbWXYT+40vW0DbQAUC1VIr7hiPrunrRAWfbV/RWZl6lr 6gMJJXkLGN70MUX40FF0IhZw5Ja5AQ0EUnQXMgEIAPICr+5yNyuVcmsv5xpmRKnz KoTjJ7xt0EPiru895LEUPN25tJyi8PZmLciNJnEoQ231jjAloQvx1pb3cr35zzGX PTPJ5fEECZDxMWMjVvCMb4XK0YjqCF9i/uKic5zqjwRNAPEGTO/ZgS+e21lUJSmu KR6m5WQcKgBH+tqS3rodgjunnIN4UiNMxbq/VVGICWPjdgoTkqWE3r8QthKLg4Lj zILEH3HbG39l+vTwEKaP4q5xShFZjRUrZC2anBP+gQx/FbBff3ufpCL9LF5dkywZ P3NHVyaa/D8T66CTj7Rynd4NZy/qdqGMjMjEuf0RkgXF7Uar4GXmuOfrIcf/X6EA EQEAAYkBJQQYAQIADwIbDAUCVj2bwwUJB4vrkQAKCRDgFk9bit80O/0ZCACoWtov fl7vH3YNW0K3xWil9wj43X2OxwKiGBdfbI48bW+b6LJQwNwFePFQ/RQCBgg1eerU Oys0ymcmp0VeMdwpw27qWcMcbsDn3Pucqp1C2IUuXesbUcRo+QDqhl96KxAAWY5O JO16dfRrIxyX6Pb0uImqpDetT4Kbr1doTF8cIfRH2rszHKU9BEWag/us9V8H5S8h F7Ws2wH2JlWbQpP8E7/Z5kVM9psdqX9rwbrUAqpyNhtILoC0+zXkdnOxz1WZaBpr ckEYQS4/CQOliYJyd5nXYcwVXCNpdy1Vt53ArN9j/EIvfFxOPCPoBQj1b9nkHqNn qnlCR0LVNOnSE3Eb =3Droc5 -----END PGP PUBLIC KEY BLOCK----- --------------2362BD2813048BD987309465-- --vmC1OAcVgL9QRuRQjbJeri8sh1PtaxS08-- --cPUPLc7Xnto8vWCrD24dtNwPnKvH56p6O Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCQAGBQJaCdy5AAoJEOAWT1uK3zQ77M8H/2S0yrHvSz4OUsqnJXDpw6pg fiJHbsDuoLGkB4LEQhBKNOfb38qbkFjPqbRV/SowEfuWoautAOTd/c0KXufNy7e6 vf+bl5IxGmn/azna+GeYxHy3ObmRN9Q4qHpqSxpTIhpUYp7d66kQs9eWOF6om6bQ D+wpECfEuYlkMsZDY4kXD+rpuiStumfgo1iSO9hmt+AxDeULjCOO9S5suVgo/fLY G09cyLE8G6nPkT1jmNtiiDfWDou/G/kS6xc75wyazbnTqfCwOSO18sRTdNUMFKHS oBVrj4Hcv9a2kyctkqVpJstRikzueRw/e1JS1cjv8Lc0/4/f8UCqKkzi+aPRCI0= =k44B -----END PGP SIGNATURE----- --cPUPLc7Xnto8vWCrD24dtNwPnKvH56p6O--