From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-74.mimecast.com (us-smtp-delivery-74.mimecast.com [63.128.21.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id DC7483CB3B for ; Mon, 30 Mar 2020 06:58:04 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1585565884; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qeTakWPJToMQdUIddHIkP4xKOCfpSQJiOYr7qwspWag=; b=UotGjNEtdRtf943cNYiHeL5wX6dn7ThJ6fp0Q6duBN0m90YhI8qqd8YJq+8gu+8ckV0CU8 Rd28mgOab4GcjZ1XTWMmNPdhk/1ggXjXA0WI4garXVQ7/xA5YmrCTMU8Gxki2ZCQ6xOyrD 3SS4fFEbLy0TU7ttFRsyRmiFdSwraMQ= Received: from mail-lf1-f70.google.com (mail-lf1-f70.google.com [209.85.167.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-250-olJX-SpMNL-vs1YFM3rESg-1; Mon, 30 Mar 2020 06:58:01 -0400 X-MC-Unique: olJX-SpMNL-vs1YFM3rESg-1 Received: by mail-lf1-f70.google.com with SMTP id l5so2818006lfg.3 for ; Mon, 30 Mar 2020 03:58:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=O12YIZiLmVjiT7xxb/fYz6+zASFVdwjE2b1U7uK56yI=; b=Z9vso4Ifvn0ES7vyncjN6jL71fneF/jYoAHmwEBnvAebYd3cukTOFDHVOnglnQhyrZ bEqtd9O56bzBfdRNwjxtU9zAQuRDIpaPpHuI4kP/l2+2xrL+UA7MGM/AETOe1hPVgyLU THO77SgP6EPKwA059ycJPHD4B+GOys3MzL8srhiR530zSrBtUP76rwsoYsTGmOXJGW/Z Wkmx5MsrJ2wavx+ExIh6YtlXQD5w5trUwjy6FG4F8SMweSC7KD5trqc2KPHzKtzYZ+qN xwdzuDT/G5L4ndMG+QQYxfFTbM7/bdkM6+M4s+2Ych/1gn41GPr8LZhMRR3gpzk8omjI 9YVA== X-Gm-Message-State: AGi0PuYE6i9DslpIHdjQDYuvsptJQkrDWCfSdq/Am6/aJrQBEpUuMhAE siZKRx4M0a5bV44isM10oEyU7wKrxA4oZbq9hJ94H++cUbvEAq9XzUH5Byj3LPs6vL1OGToorlW eX/92ic+iIY8u7lXCayFs8A== X-Received: by 2002:a19:4a50:: with SMTP id x77mr7697953lfa.159.1585565880132; Mon, 30 Mar 2020 03:58:00 -0700 (PDT) X-Google-Smtp-Source: APiQypJ3TO/Yj0gViMTGbqAjQMEol4MyvOrRVw9rOPcE7gNrFpABO8nECYS6BKy/qm0Ip8/lyohykQ== X-Received: by 2002:a19:4a50:: with SMTP id x77mr7697910lfa.159.1585565879590; Mon, 30 Mar 2020 03:57:59 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id c22sm6624619lja.86.2020.03.30.03.57.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Mar 2020 03:57:58 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id D6A8718158B; Mon, 30 Mar 2020 12:57:57 +0200 (CEST) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= To: "David P. Reed" Cc: Dave Taht , Make-Wifi-fast , Anthony Minessale II , Cake List , Ken Rice , cerowrt-devel , bloat In-Reply-To: <1585437340.158425591@apps.rackspace.com> References: <1585335604.839628636@apps.rackspace.com> <878sjkl0a7.fsf@toke.dk> <1585437340.158425591@apps.rackspace.com> X-Clacks-Overhead: GNU Terry Pratchett Date: Mon, 30 Mar 2020 12:57:57 +0200 Message-ID: <87d08ujeiy.fsf@toke.dk> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Subject: Re: [Cake] mo bettah open source multi-party videoconferncing in an age of bloated uplinks? X-BeenThere: cake@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: Cake - FQ_codel the next generation List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2020 10:58:05 -0000 "David P. Reed" writes: > Regarding EDF. > > I've been pushing folks to move latency sensitive computing in ALL OS's t= o a version of EDF since about 1976. This was when I was in grad school wor= king on distributed computing on LANs. In fact, it is where I got the idea = for my Ph.D. thesis (completed in 1978) which pointed out a bigger idea - t= hat getting ACID consistency [ACID hadn't been invented then as a term, we = called it atomic actions] on data in a distributed system being processed b= y concurrent distributed transactions can be done by using timestamps that = behave like the "deadlines" in EDF. In fact, the scheduling of code in my t= hesis was a generalized version of EDF, approximated because of the impossi= bility of perfect synchronization. > > The Croquet system, which was a real-time edge based decentralized system= , with no central server, that we demonstrated with a Second-Life style vir= tual world that wored entirely on a set of laptops that could be across the= country from each other was based on an OS implemented in a variant of the= Squeak programming language, where the scheduling and object model was not= process based, but message based with replicated computation synchronized = via a shared "timestamp" that was used for execution scheduling (essentiall= y distributed EDF). The latency requirements for this distributed virtual w= orld were on the order of 100 msec. simultaneity for mouse clicks affecting= all participating nodes across the country in a virtual 3D world, with sou= nd, etc. > > Croquet was built in 2 years by 3 people (starting from scratch). And sc= heduling was never a problem, nor was variable network delay (our protocol = was based on UDP frames synchronized by the same timestamps used to synchro= nize each object method execution. > > The operating system model is one I created within that modified Squeak e= nvironment as part of its base "interpreter", which wasn't a loop, but a sc= heduler using EDF. > > To make this work properly, the programming model has to be unified aroun= d this kind of scheduling. > > And here's why I am mentioning this. To put EDF *only* into the networkin= g stack, but leave the userspace applicaiton living with the stupid Linux t= imesharing system scheduler, optimized for people typing commands on termin= als every few seconds and running batch compilation is the *worst of all po= ssible ways to use EDF*. > > Because it creates a huge mess bridging those two ideas. > > Croquet is a much more complicated thing that a teleconferencing system, = because it actually lets end users write simple programs that control the u= ser interactive experience, 30 frames per second across the entire US, repl= icated on each computer, in the Squaak variant of Smalltalk. And we did it = with 3 coders in a couple of years. (yes, they are sckilled people - me, Da= vid A. Smith, and the late Andreas Raab, who died way too young). > > In contrast, trying to bridge between EDF and regular Linux processes run= ning under the ordinary scheduler, even with "nice" and all kinds of hacks,= just to do a video conferencing system with fixed, non-programmable behavi= or, would take far more design, far more lines of code, etc. > > So this is why I think timesharing OS's are really obsolescent for modern= distributed interactive systems. Yeah, "rsync" and "git" are nice for batc= h replication of files. ANd yeah, EDF can help make them perform faster in = their file transferring. > > But to make an immersive, real-time experience (which is what computing t= oday is all about, on all time scales, even in the servers other than HPC) = it is ALL wrong, and incrementally patching little pieced of Linux ain't go= nna get there. Windows or BSD (macOS) ain't gonna do it either. > > I'm old. Why is Linux living in the idea space of operating systems that = precededed networking, distributed computing, media sharing? > > My opinion, and it is only an opinion based on experience, is that it rea= lly is time for networking to stop focusing on file transfers, and OS's to = stop focusing on timesharing behavior. The world is "live" and time-based. = It may not be hard-real-time. But latency is what matters. > > Since networking will remain separate from OS's, the interface concepts i= n both really need to be matched to get to that future. > > It's why I pushed so hard for UDP, not reliable in-order streams alone. A= nd in my view, though no one every implemented it, those UDP packets will b= e carring times, essential for synchronization of coordinated operations at= all the endpoints of the computation. > > I'd love to see that happen before this old guy dies. I think it will mak= e it a whole lot easier to make networked programs work. > > Decentralization isn't "blockchain". My thesis, in 1978, talked about one= way to decentralize computation, not just data structures. And timing is c= ritical. > > Sorry for the rant. I'm tired of waiting for "backwards compatibility" wi= th Unix version 1 to allow us to go forward. To me, Linux is a great versio= n of a subset of the operating systems I worked on in the early 1970's. And= little more. This was a fascinating read, thank you! Are you aware of any contemporary systems implementing such a unified EDF scheduling model? -Toke