[Bloat] The sad state of MP-TCP

[Bloat] The sad state of MP-TCP

[Juliusz Chroboczek]

Hi,

For those of you who don't remember, MP-TCP is an extension to TCP that
implements multipath, and can be used both for fast roaming (redundancy)
and for bandwidth aggregation.  MP-TCP is able to cross NATs, and it can
reliably detect that TCP extensions are being corrupted by middleboxes and
transparently fall back to plain TCP.

MP-TCP is an IETF standard.  MP-TCP is in the Linux kernel since 5.6, and
it has good performance since 5.12 [1].  It's also in Apple's kernels
(it's used by Siri for fast roaming).

Unfortunately, MP-TCP does not replace TCP in Linux, it's implemented as
a separate transport protocol.  That means that in order to use MP-TCP,
every application needs to be patched to use PROT_MPTCP instead of
PROT_TCP.  Go applications need to call SetMultipathTCP(true) on every
net.Conn and every net.Listener.

There should be a knob in the kernel to transparently replace TCP with
MP-TCP, but I couldn't find one.

And of course no applications are being patched.  Ssh is probably the
application which I'd be most keen to run over MP-TCP [2], but the patch
is being blocked upstream [3].  For what it's worth, I've just added
MP-TCP support to storrent [4], with a view to making it enabled by
default, which means that all three storrent users will get MP-TCP out of
the box.

Dave, are you looking for a new crusade?


[1] https://github.com/multipath-tcp/mptcp_net-next/wiki#changelog

[2] http://blog.multipath-tcp.org/blog/html/2022/07/08/openssh.html

[3] https://github.com/openssh/openssh-portable/pull/335

[4] https://github.com/jech/storrent/commit/c8d95c17f4227da77b97d5b7a9bdf289fea44aca

-- Juliusz

Re: [Bloat] The sad state of MP-TCP

[Toke Høiland-Jørgensen]

Juliusz Chroboczek via Bloat <bloat@lists.bufferbloat.net> writes:

> Unfortunately, MP-TCP does not replace TCP in Linux, it's implemented as
> a separate transport protocol.  That means that in order to use MP-TCP,
> every application needs to be patched to use PROT_MPTCP instead of
> PROT_TCP.  Go applications need to call SetMultipathTCP(true) on every
> net.Conn and every net.Listener.
>
> There should be a knob in the kernel to transparently replace TCP with
> MP-TCP, but I couldn't find one.

There is, sorta. Specifically, a BPF hook that can override the protocol
(added in kernel 6.6):

https://lore.kernel.org/all/cover.1692147782.git.geliang.tang@suse.com/

-Toke

Re: [Bloat] The sad state of MP-TCP

[Simon Leinen]

Toke Høiland-Jørgensen via Bloat writes:
> Juliusz Chroboczek via Bloat <bloat@lists.bufferbloat.net> writes:
>> There should be a knob in the kernel to transparently replace TCP with
>> MP-TCP, but I couldn't find one.

> There is, sorta. Specifically, a BPF hook that can override the protocol
> (added in kernel 6.6):

> https://lore.kernel.org/all/cover.1692147782.git.geliang.tang@suse.com/

So all we need is a small modification to, say, liblzma, to make systemd
install that hook when calling sshd.  Right? ;-)
-- 
Simon.

Re: [Bloat] The sad state of MP-TCP

[Daniel Gröber]

On Tue, Apr 02, 2024 at 12:40:45PM +0200, Simon Leinen via Bloat wrote:
> So all we need is a small modification to, say, liblzma, to make systemd
> install that hook when calling sshd.  Right? ;-)

Too soon :P

Re: [Bloat] The sad state of MP-TCP

[Juliusz Chroboczek]

>> There should be a knob in the kernel to transparently replace TCP with
>> MP-TCP, but I couldn't find one.

> There is, sorta. Specifically, a BPF hook that can override the protocol
> (added in kernel 6.6):
> 
> https://lore.kernel.org/all/cover.1692147782.git.geliang.tang@suse.com/

So we're no longer doing sysctls, we're now monkey patching system calls?
I guess if it works for JavaScript, why shouldn't it work for the kernel.

-- Juliusz

Re: [Bloat] The sad state of MP-TCP

[Toke Høiland-Jørgensen]

Juliusz Chroboczek <jch@irif.fr> writes:

>>> There should be a knob in the kernel to transparently replace TCP with
>>> MP-TCP, but I couldn't find one.
>
>> There is, sorta. Specifically, a BPF hook that can override the protocol
>> (added in kernel 6.6):
>> 
>> https://lore.kernel.org/all/cover.1692147782.git.geliang.tang@suse.com/
>
> So we're no longer doing sysctls, we're now monkey patching system
> calls? I guess if it works for JavaScript, why shouldn't it work for
> the kernel.

If it helps you sleep at night, you can think of it more as a lisp
machine than a javascript runtime ;)

You're not the first to make the comparison, though:

https://dl.acm.org/doi/abs/10.1145/3609021.3609306

-Toke