From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 8A71B3B29D for ; Sun, 12 Mar 2023 17:10:36 -0400 (EDT) Received: by mail-ed1-x530.google.com with SMTP id da10so41238253edb.3 for ; Sun, 12 Mar 2023 14:10:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678655435; h=to:subject:message-id:date:from:references:in-reply-to:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=SGboTnYGQxb9q/0Wwftbl607jIeCIBOxogGz2ZRA6zI=; b=JSgcrofU5J85e0xKxQx2bqUuwO1vSuk4eyDMgYcXUsGMKL3w0TaOKC6gXn3HPsHPr+ CEVZC9ZIRSE2qUVQ7a1KZxp8Z3Lo0Ph6HQVviPMDmJodgeF8dZeJ+J9hdocEGXF1I/lT kg/gamC+KKj/v7AKobZyK3/Zhl9XJpGHGHJCBAJWg77W+lo2Tfm6tcommhB8lGJVJ6c9 pMLkoIByD35PSzc65mYxYFORP5kMTRrFMGr/oAUlYGRzTeoTfiASNjolC7qo9TO3WzED cWIgCFfUhWDB/22dxfIh6joe7UI4JCUVAeyLWlaUkHkP6GoxTcP1lRpDCZyGhhego8MD 6fsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678655435; h=to:subject:message-id:date:from:references:in-reply-to:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SGboTnYGQxb9q/0Wwftbl607jIeCIBOxogGz2ZRA6zI=; b=txyoaj4HY7/v5x9Wx74Jp7U/hwFQ67fjhRyPB6JTHS/+llOxZ8NhyAbshC31pCqW1c +464XM6kT+GdGRPmMwXgZZHMXtSWlDYGEN8ixi10pr0Qweqlz+z6Gbax8+nabv95tqyC lE78BIGqHqPQ0AyBBJiecQC9k7eWXEzvubXT9o3ROHtEfwBY7YYMa3RJF7BRR6MN9hp5 eucbPEyC+opWkqAGlGXt9v9aqc7xxf/a5aWK3KKy/F1+fL6nYPgvLuBKEXxj/8PQvKjI nCLEedjrHjcck8fIXFABBTWqfQ92WnbsPzD0It95+rie+6ds4HlnqXQDWiWghLlJbHl1 wKMg== X-Gm-Message-State: AO0yUKVfkbS9I76PEG30HCp7J7hB+g6lLFUZ4bGYwyU3YcKKNgfxDxqT 8XR5Nfw/MNpIztZs+uhbK6ZY5Lxj+QPsU4ejVsY1KG4v X-Google-Smtp-Source: AK7set/YFNUfYe4OnA9UYABA6Mbq0cnCYuJaIDLYuFXOQ5u6R2wwH/V2uM9YzOlto+GPjGFEW4djv7hMHs5GNBb+je4= X-Received: by 2002:a50:8e5d:0:b0:4fc:6494:81c3 with SMTP id 29-20020a508e5d000000b004fc649481c3mr612654edx.1.1678655435217; Sun, 12 Mar 2023 14:10:35 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a05:6f02:423:b0:49:ad23:480 with HTTP; Sun, 12 Mar 2023 14:10:34 -0700 (PDT) In-Reply-To: <1678647418.092929775@apps.rackspace.com> References: <1678647098.508830273@apps.rackspace.com> <1678647418.092929775@apps.rackspace.com> From: Sauli Kiviranta Date: Sun, 12 Mar 2023 23:10:34 +0200 Message-ID: To: starlink@lists.bufferbloat.net Content-Type: text/plain; charset="UTF-8" Subject: Re: [Starlink] RFC: bufferbloat observability project (Dave Taht) X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Mar 2023 21:10:36 -0000 David, you make a valid point. I have a corner case in mind that you maybe did not intend, forgive me if this was beyond the scope of your response to Dave. Unbounded queueing is perfectly fine when two conditions are met: an object in the queue is assumed to be valid until infinity, never expires -and- consumption is always higher or equal to production of items in the queue. Thus, delays can not accumulate causing us to need infinitely large buffers and infinite time to ultimately empty the queue. Order to me is just vertical dependency between objects (continuity) and timing is horizontal dependency (referential integrity), both valid. So even in the first case you outlined, I think we are still dealing with finite and bounded scenario, just of a vertical type. Obviously infinite space nor time do not exist with the real world systems. I am a bit nitpicking here, but there is a real world example of this assumption, that it exists, and that is the serious design flaw of TCP. We do not have the infinity and what we thought as infinity, relative infinity that I think you thought of with Haskell example. In my opinion this premise should never exist. It should not exists in the design "we assume our objects to have infinite lifetime validity and for the consumption to always be larger than production of objects or else we just increase our buffers to make sure things average out until infinity". The moment someone pronounced "guaranteed delivery" it was game over for TCP in dealing with real-time like systems, a step to infinity. Once you draw a boundary, bound with time constraint to spoil an object or space by selectively discarding objects, effectively reducing rate, then you avoid infinite growth of the queue. I think this is bothering Dave deep inside, the reality, what he is not seeing considered enough? We need some form of queue management if we can not have feedback loop to the production source about consumption rate or if production can not provide TTL information to the processing. There is no escape, as far as I am concerned. If our set of possibilities includes a moment that your consumption falls below production rate and the objects have infinite like lifetime, we will be in trouble sooner or later. Bufferbloat. Even queue management is slightly wrong, as it is just trying to resolve symptoms instead of tooling the root cause out: ability to signal back to production about consumption rate so that the use-case can make decisions what to do on its side (e.g. video transmission use case can reduce bitrate) or alternatively use-case must be given possibility to say about time validity of data. If those are not tooled in, we have no other option but to start throwing objects to trash at our convenience when buffers overflow. This is a fundamental issue, and should be treated as such. If you think about any kind of control system that has some form of parallelism e.g. multiple sensors like in a self-driving car, you need queues if those signals are converging at any point down the chain for the purpose of decision making (continuity in time for a given signal and parallel referential integrity in time). You end up with same mess that we have with streaming today: artisan like rules of thumb instead of doing rigorous engineering and try to be scientific literally, properly analyzing the beast and taking control of it. Jitterbuffers / queues are fundamental necessity, and the management of those queues must be properly understood and accepted premise, not afterthought. Again, if there is any dependency between the objects being queued (temporal continuity or spatial referential integrity). TCP has this flaw, under the premise of "guaranteed delivery" without mentioning the asterisk "... with infinite scope". In TCP, reality of is then being handled by buffer overflows and timeouts, instead of accepting the reality and dealing with it like we deal with everything else (Can I have redundancy too? Maybe that is too much to ask...). Summary: 1. When we have concurrency (with vertical or horizontal dependencies), queue is a must 2. When we have queue, rate control is a must 3. Rate control can be proactive with TTL (payment with space) or reactive with feedback loop (payment with time) 4. Only then we can set safeguards to prevent bufferbloat as our world is not infinite, also we better communicate clearly how we do the housekeeping, as it really should be exception case when use-case is doing stupid things regardless of our best effort to inform about the conditions. Conforming to reality and being rigorous would be a nice start. I love TCP, it is absolutely beautiful in its nature, the founding fathers did care to large extend. Just not enough... time at their hands back then to take it to finish. We too should care and conform to reality, and hope that we will be given enough time. Just my 2 cents. Best regards, Sauli On 12/03/2023, David P. Reed via Starlink wrote: > > I should have added this: I am aware of a full TCP stack implementation > implemented in Verilog. (In fact, my son built it, and it is in production > use on Wall St.). > > On Sunday, March 12, 2023 2:51pm, "David P. Reed" > said: > > > > Regarding unbounded queues > On Sunday, March 12, 2023 12:00pm, Dave Taht said: > >> Also it increasingly bothers me to see unbounded queues in so many new >> language libraries. > > > I disagree somewhat. Unbounded queueing is perfectly fine in a programming > language like Haskell, where there are no inherent semantics about timing - > a queue is an ordered list with append, and it's a GREAT way to formulate > many algorithms that process items in order. > > Where the problem with queues arises is in finite (bounded) real-time > programming systems. Which include network protocol execution machines. > > It's weird to me that people seem to think that languages intended for > data-transformation algorithms, parsers, ... are appropriate for programming > network switches, TCP/IP stacks, etc. It always has seemed weird beyond > belief. I mean, yeah, Go has queues and goroutines, but those aren't > real-time appropriate components. > > What may be the better thing to say is that it increasingly bothers you that > NO ONE seems to be willing to create a high-level programming abstraction > for highly concurrent interacting distributed machines. > > There actually are three commercial programming languages (which are about > at the level of C++ in abstraction, with the last maybe being at the level > of Haskell). > 1. Verilog > 2. VHDL > 3. BlueSpec > > For each one, there is a large community of programmers proficient in them. > You might also consider Erlang as a candidate, but I think its "queues" are > not what you want to see. > > Why doesn't IETF bother to try to delegate a team to create such an > expressive programming language or whatever? I'd suggest that starting with > Verilog might be a good idea. > > A caveat about my point: I write Verilog moderately well, and find it quite > expressive for modeling networking systems in my mind. I also write Haskell > quite well, and since BlueSpec draws on Haskell's model of computation I > find it easy to read, but I've not written much Haskell. > > To me, those who write networking code in C or C++ are stuck in the past > when protocols were documented by bit-layouts of packets and hand-waving > English "standards" without any way to verify correctness. We need to stop > worshipping those archaic RFCs as golden tablets handed down from gods. > > Who am I to criticize the academic networking gods, though?