From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id BC3B93B2A4 for ; Sun, 26 Dec 2021 14:12:42 -0500 (EST) Received: by mail-io1-xd31.google.com with SMTP id h23so6724364iol.11 for ; Sun, 26 Dec 2021 11:12:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to:cc; bh=OMRM8zcopTpcrtfwZETXKgiUWiRC2iroiih6pzsuySE=; b=eai3NkKE5kyi/U43T1LKY9HZROMaeLZRV3C7E9IadjCANX+mHzOb3cGN7xahJxqSSA WbX20GojqRrDxwUL30OF7J/usZKTiBO1CtE4xq773b5h4ujvkW+w/0k/hguxdvB8GxL4 gN0y2uhExvv6TBmhYCryrO4YbPti9VvvQiEwMq/ZLiIFxpzN8w1pqjp/n0HMqKQxftzZ BDXy6hGr/qju0cSv3Txt6H064vVdaEIxwIbk2INCuniDWC1cuL2zkgZ297MPM4k33S4V jluZgpYtGnASuW7+UIyQe88F/de32CImU3SBYy2hrQ9JIoCQCe1M1KJ30UT3SWQOiHmw FILQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=OMRM8zcopTpcrtfwZETXKgiUWiRC2iroiih6pzsuySE=; b=Usx+6xMSn/re84CiH+BX2q1N/T9VE8g4A/cIp2xKDGfX/eUbAcXOY16mPn8FBqkV6B zTEljpF3V09uKrCU6OS2DE/EJiIXnH7hKg1NaJLGdvY+0vgcHLOkUchrYnibrH6xMFlI 5hq9JYcrkMZOca1UjUFTVopqgbWx0D1wbfIWbPxcePNBuMaWgZswviwfb6Ts8ywbW0Bi qsIUmC5Ge4eJtGAZlpCAKwbtnJ2+MTbcRam7RXkLpnNF8nBPgljYoaVlvP0TELAmMGrj O9AwrMFeX7Vsw2ovBONXy5HjUIpQ2aT1CTpntM3BY42Ur1+a7pxuUPpsuh5erm81iCTi OFjw== X-Gm-Message-State: AOAM532xEhcVlAost6PeWyR1JrkT7xihu+0wZM2pGiXylfrkEzWoTENH mG/c8lgGqxj4yOwRZdFHCAtQeR/S9EPrO3SyGkt3COk+FZ4= X-Google-Smtp-Source: ABdhPJxifCSriNwfPrZLDJZwyrSIwdqFNLaiSIq5Z11ouBrKxdCzBXgnVFdfCXL1+TdsDkkO7tAacEQllXcWiYKhPrk= X-Received: by 2002:a02:23ca:: with SMTP id u193mr5204058jau.38.1640545961846; Sun, 26 Dec 2021 11:12:41 -0800 (PST) MIME-Version: 1.0 From: Dave Taht Date: Sun, 26 Dec 2021 11:12:29 -0800 Message-ID: To: bloat Content-Type: text/plain; charset="UTF-8" Subject: [Bloat] RTMFP improvements from Michael Thornburgh X-BeenThere: bloat@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: General list for discussing Bufferbloat List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Dec 2021 19:12:42 -0000 This is extracted from a chat conversation with Michael Thornburgh with his permission. I too have experimented with OBS and twitch streaming and felt it needed some love, in particular. https://github.com/zenomt/rtmfp-cpp#delay-based-congestion-detection . ... Hi Dave. For some reason I keep working on RTMFP, the real-time protocol I co-created at Adobe ("some reason" == I still think it's a better holistic solution than the mish-mash of stuff that's WebRTC, and QUIC). A while ago I added ECN support. Unfortunately, no networks I regularly use (my android phone + TMobile, Cruzio) seem to have any AQM+ECN deployed. I'm not really a fan of E2E delay-based bufferbloat mitigations because there are too many false signals and there's a better real solution. but because of reality, i held my nose and added delay-based congestion detection to my RTMFP implementation. I had a few insights while doing it that I thought you might find interesting (which of course might be totally obvious, but i didn't find written down before), mostly around clearing/resetting "baseline RTT" in more circumstances to try to avoid permanent starvation. I wrote an explanation (including how it works and how to turn it on) here: https://github.com/zenomt/rtmfp-cpp#delay-based-congestion-detection . I've found it works very well on my TMobile LTE connection when the signal is highly variable (like walking around my house and through known coverage shadows). I haven't done experiments yet with bidirectional traffic and mixes with delay-insensitive traffic, nor have I simulated different scenarios beyond my own real-world testing. I hope to have time for that soon, and to write up findings vs something like SCReAM. this is also an invitation to check out RTMFP (RFC 7016) and my impl!!! more eyes on rtmfp would make me happy. :) There's a bunch of stuff in my RTMFP implementation's CC that might not be obvious without getting deep in the code. I should probably add more exposition in the README. The CC is window-based but with modified AIMD parameters for real-time (cut 1/8 instead of 1/2 on congestion signal, but grow more slowly when things are good to balance) and with slower slow-start for real-time. "congestion signal" is loss or ECN-CE, or extra delay > thresh if enabled. transmission uses a hybrid of ack-clock and a burst-avoiding pacer. i could talk about it all day if you're curious. :) or just point you at relevant parts of the code or RFC. I think one of the big problems with people streaming on twitch (and other services) is bad RTMP implementations. OBS, as far as i can tell, still hasn't heard of TCP_NOTSENT_LOWAT. that, a good ABR for your encoder, and actually-good real-time treatment for data queued in RTMP can make a giant difference for quality and latency. of course none of those things can fix a router that's bloating up! and delay-sensitive ABR algorithms will drive to 0 if there isn't some other reservation (like i saw at that link with trying to allocate 3Mbps of 5Mbps for twitch with DSCP). -miketh