From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp99.iad3a.emailsrvr.com (smtp99.iad3a.emailsrvr.com [173.203.187.99]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DE2193CB35; Sat, 28 Mar 2020 19:15:43 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=g001.emailsrvr.com; s=20190322-9u7zjiwi; t=1585437343; bh=eftFmnd/pmhGp5uGk3dBJ0Y8T3AC8lohXAjC0gj85v4=; h=Date:Subject:From:To:From; b=zcz0YWyjuWj0DlyxRYHy93qEajjq3RhQHuZf5bKWyD0kTWAyTjq9sQ0j1+ypHWQCa rI1Erk/4agXwmcgn4uEOFVw9uI1QoL7dLZI56uP01yvnmCcWEvcTGq8Ujgi2DQLYC6 FnXUpCJoCBupj/nrStcQA3JIVvnzI7k+8YGZ/hfg= Received: from app55.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by smtp21.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 845C323EB1; Sat, 28 Mar 2020 19:15:43 -0400 (EDT) X-Sender-Id: dpreed@deepplum.com Received: from app55.wa-webapps.iad3a (relay-webapps.rsapps.net [172.27.255.140]) by 0.0.0.0:25 (trex/5.7.12); Sat, 28 Mar 2020 19:15:43 -0400 Received: from deepplum.com (localhost.localdomain [127.0.0.1]) by app55.wa-webapps.iad3a (Postfix) with ESMTP id 277A761436; Sat, 28 Mar 2020 19:15:40 -0400 (EDT) Received: by apps.rackspace.com (Authenticated sender: dpreed@deepplum.com, from: dpreed@deepplum.com) with HTTP; Sat, 28 Mar 2020 19:15:40 -0400 (EDT) X-Auth-ID: dpreed@deepplum.com Date: Sat, 28 Mar 2020 19:15:40 -0400 (EDT) From: "David P. Reed" To: "=?utf-8?Q?Toke_H=C3=B8iland-J=C3=B8rgensen?=" Cc: "Dave Taht" , "Make-Wifi-fast" , "Anthony Minessale II" , "Cake List" , "Ken Rice" , "cerowrt-devel" , "bloat" MIME-Version: 1.0 Content-Type: text/plain;charset=UTF-8 Content-Transfer-Encoding: quoted-printable Importance: Normal X-Priority: 3 (Normal) X-Type: plain In-Reply-To: <878sjkl0a7.fsf@toke.dk> References: <1585335604.839628636@apps.rackspace.com> <878sjkl0a7.fsf@toke.dk> Message-ID: <1585437340.158425591@apps.rackspace.com> X-Mailer: webmail/17.3.3-RC X-Classification-ID: 226e5f71-bcc6-4ae2-a8fc-6eaadc290b6c-1-1 Subject: Re: [Cake] =?utf-8?q?mo_bettah_open_source_multi-party_videoconfernci?= =?utf-8?q?ng_in_an_age_of_bloated_uplinks=3F?= X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Mar 2020 23:15:43 -0000 Regarding EDF.=0A=0AI've been pushing folks to move latency sensitive compu= ting in ALL OS's to a version of EDF since about 1976. This was when I was = in grad school working on distributed computing on LANs. In fact, it is whe= re I got the idea for my Ph.D. thesis (completed in 1978) which pointed out= a bigger idea - that getting ACID consistency [ACID hadn't been invented t= hen as a term, we called it atomic actions] on data in a distributed system= being processed by concurrent distributed transactions can be done by usin= g timestamps that behave like the "deadlines" in EDF. In fact, the scheduli= ng of code in my thesis was a generalized version of EDF, approximated beca= use of the impossibility of perfect synchronization.=0A=0AThe Croquet syste= m, which was a real-time edge based decentralized system, with no central s= erver, that we demonstrated with a Second-Life style virtual world that wor= ed entirely on a set of laptops that could be across the country from each = other was based on an OS implemented in a variant of the Squeak programming= language, where the scheduling and object model was not process based, but= message based with replicated computation synchronized via a shared "times= tamp" that was used for execution scheduling (essentially distributed EDF).= The latency requirements for this distributed virtual world were on the or= der of 100 msec. simultaneity for mouse clicks affecting all participating = nodes across the country in a virtual 3D world, with sound, etc.=0A=0ACroqu= et was built in 2 years by 3 people (starting from scratch). And schedulin= g was never a problem, nor was variable network delay (our protocol was bas= ed on UDP frames synchronized by the same timestamps used to synchronize ea= ch object method execution.=0A=0AThe operating system model is one I create= d within that modified Squeak environment as part of its base "interpreter"= , which wasn't a loop, but a scheduler using EDF.=0A=0ATo make this work pr= operly, the programming model has to be unified around this kind of schedul= ing.=0A=0AAnd here's why I am mentioning this. To put EDF *only* into the n= etworking stack, but leave the userspace applicaiton living with the stupid= Linux timesharing system scheduler, optimized for people typing commands o= n terminals every few seconds and running batch compilation is the *worst o= f all possible ways to use EDF*.=0A=0ABecause it creates a huge mess bridgi= ng those two ideas.=0A=0ACroquet is a much more complicated thing that a te= leconferencing system, because it actually lets end users write simple prog= rams that control the user interactive experience, 30 frames per second acr= oss the entire US, replicated on each computer, in the Squaak variant of Sm= alltalk. And we did it with 3 coders in a couple of years. (yes, they are s= ckilled people - me, David A. Smith, and the late Andreas Raab, who died wa= y too young).=0A=0AIn contrast, trying to bridge between EDF and regular Li= nux processes running under the ordinary scheduler, even with "nice" and al= l kinds of hacks, just to do a video conferencing system with fixed, non-pr= ogrammable behavior, would take far more design, far more lines of code, et= c.=0A=0ASo this is why I think timesharing OS's are really obsolescent for = modern distributed interactive systems. Yeah, "rsync" and "git" are nice fo= r batch replication of files. ANd yeah, EDF can help make them perform fast= er in their file transferring.=0A=0ABut to make an immersive, real-time exp= erience (which is what computing today is all about, on all time scales, ev= en in the servers other than HPC) it is ALL wrong, and incrementally patchi= ng little pieced of Linux ain't gonna get there. Windows or BSD (macOS) ain= 't gonna do it either.=0A=0AI'm old. Why is Linux living in the idea space = of operating systems that precededed networking, distributed computing, med= ia sharing?=0A=0AMy opinion, and it is only an opinion based on experience,= is that it really is time for networking to stop focusing on file transfer= s, and OS's to stop focusing on timesharing behavior. The world is "live" a= nd time-based. It may not be hard-real-time. But latency is what matters.= =0A=0ASince networking will remain separate from OS's, the interface concep= ts in both really need to be matched to get to that future.=0A=0AIt's why I= pushed so hard for UDP, not reliable in-order streams alone. And in my vie= w, though no one every implemented it, those UDP packets will be carring ti= mes, essential for synchronization of coordinated operations at all the end= points of the computation.=0A=0AI'd love to see that happen before this old= guy dies. I think it will make it a whole lot easier to make networked pro= grams work.=0A=0ADecentralization isn't "blockchain". My thesis, in 1978, t= alked about one way to decentralize computation, not just data structures. = And timing is critical.=0A=0ASorry for the rant. I'm tired of waiting for "= backwards compatibility" with Unix version 1 to allow us to go forward. To = me, Linux is a great version of a subset of the operating systems I worked = on in the early 1970's. And little more.=0A=0A=0A=0A=0A=0A=0A=0A=0A=0AOn Sa= turday, March 28, 2020 3:58pm, "Toke H=C3=B8iland-J=C3=B8rgensen" said:=0A=0A> Dave Taht writes:=0A> =0A>>> So= : 1. We really should rethink how timing-sensitive algorithms are=0A>>> exp= ressed, and it isn't gonna be good to base them on semaphores and=0A>>> thr= eads that run at random rates. That means a very different OS=0A>>> concept= ual framework. Can this share with, say, the Linux we know and=0A>>> love -= yes, the hardware can be shared. One should be able to=0A>>> dedicate virt= ual processors that are not running Linux processes, but=0A>>> instead anot= her computational model (dataflow?).=0A>>=0A>> Linux switched to an EDF mod= el for networking in 5.0=0A> =0A> Not entirely. There's EDT scheduling, and= the TCP stack is mostly=0A> switched over, I think. But as always, Linux e= volves piecemal :)=0A> =0A>>> 2. EBPF is interesting, because it is more se= cure, and is again=0A>>> focused on running code at kernel level, event-dri= ven. I think it=0A>>> would be a seriously difficult lift to get it to the = point where one=0A>>> could program the networked media processing in BPF.= =0A>>=0A>> But there is huge demand for it, so people are writing way more = in it=0A>> than i ever ever thought possible... or desirable.=0A> =0A> Tell= me about it.=0A> =0A> We have seen a bit of interest for combining eBPF wi= th realtime, though.=0A> With the upstreaming of the realtime code, support= has landed for=0A> running eBPF even on realtime kernels. And we're starti= ng to see a bit=0A> of interest for looking specifically at latency bounds = for network=0A> processing (for TSN), including XDP. Nothing concrete yet, = though.=0A> =0A> -Toke=0A> =0A> =0A