[Make-wifi-fast] [Cake] mo bettah open source multi-party videoconferncing in an age of bloated uplinks?

[Toke Høiland-Jørgensen]

"David P. Reed" <dpreed at deepplum.com> writes:

> Regarding EDF.
>
> I've been pushing folks to move latency sensitive computing in ALL OS's to a version of EDF since about 1976. This was when I was in grad school working on distributed computing on LANs. In fact, it is where I got the idea for my Ph.D. thesis (completed in 1978) which pointed out a bigger idea - that getting ACID consistency [ACID hadn't been invented then as a term, we called it atomic actions] on data in a distributed system being processed by concurrent distributed transactions can be done by using timestamps that behave like the "deadlines" in EDF. In fact, the scheduling of code in my thesis was a generalized version of EDF, approximated because of the impossibility of perfect synchronization.
>
> The Croquet system, which was a real-time edge based decentralized system, with no central server, that we demonstrated with a Second-Life style virtual world that wored entirely on a set of laptops that could be across the country from each other was based on an OS implemented in a variant of the Squeak programming language, where the scheduling and object model was not process based, but message based with replicated computation synchronized via a shared "timestamp" that was used for execution scheduling (essentially distributed EDF). The latency requirements for this distributed virtual world were on the order of 100 msec. simultaneity for mouse clicks affecting all participating nodes across the country in a virtual 3D world, with sound, etc.
>
> Croquet was built in 2 years by 3 people (starting from scratch).  And scheduling was never a problem, nor was variable network delay (our protocol was based on UDP frames synchronized by the same timestamps used to synchronize each object method execution.
>
> The operating system model is one I created within that modified Squeak environment as part of its base "interpreter", which wasn't a loop, but a scheduler using EDF.
>
> To make this work properly, the programming model has to be unified around this kind of scheduling.
>
> And here's why I am mentioning this. To put EDF *only* into the networking stack, but leave the userspace applicaiton living with the stupid Linux timesharing system scheduler, optimized for people typing commands on terminals every few seconds and running batch compilation is the *worst of all possible ways to use EDF*.
>
> Because it creates a huge mess bridging those two ideas.
>
> Croquet is a much more complicated thing that a teleconferencing system, because it actually lets end users write simple programs that control the user interactive experience, 30 frames per second across the entire US, replicated on each computer, in the Squaak variant of Smalltalk. And we did it with 3 coders in a couple of years. (yes, they are sckilled people - me, David A. Smith, and the late Andreas Raab, who died way too young).
>
> In contrast, trying to bridge between EDF and regular Linux processes running under the ordinary scheduler, even with "nice" and all kinds of hacks, just to do a video conferencing system with fixed, non-programmable behavior, would take far more design, far more lines of code, etc.
>
> So this is why I think timesharing OS's are really obsolescent for modern distributed interactive systems. Yeah, "rsync" and "git" are nice for batch replication of files. ANd yeah, EDF can help make them perform faster in their file transferring.
>
> But to make an immersive, real-time experience (which is what computing today is all about, on all time scales, even in the servers other than HPC) it is ALL wrong, and incrementally patching little pieced of Linux ain't gonna get there. Windows or BSD (macOS) ain't gonna do it either.
>
> I'm old. Why is Linux living in the idea space of operating systems that precededed networking, distributed computing, media sharing?
>
> My opinion, and it is only an opinion based on experience, is that it really is time for networking to stop focusing on file transfers, and OS's to stop focusing on timesharing behavior. The world is "live" and time-based. It may not be hard-real-time. But latency is what matters.
>
> Since networking will remain separate from OS's, the interface concepts in both really need to be matched to get to that future.
>
> It's why I pushed so hard for UDP, not reliable in-order streams alone. And in my view, though no one every implemented it, those UDP packets will be carring times, essential for synchronization of coordinated operations at all the endpoints of the computation.
>
> I'd love to see that happen before this old guy dies. I think it will make it a whole lot easier to make networked programs work.
>
> Decentralization isn't "blockchain". My thesis, in 1978, talked about one way to decentralize computation, not just data structures. And timing is critical.
>
> Sorry for the rant. I'm tired of waiting for "backwards compatibility" with Unix version 1 to allow us to go forward. To me, Linux is a great version of a subset of the operating systems I worked on in the early 1970's. And little more.


This was a fascinating read, thank you! Are you aware of any
contemporary systems implementing such a unified EDF scheduling model?

-Toke