[Ecn-sane] mosh ecn bits washed out

[Ecn-sane] mosh ecn bits washed out

[Dave Taht]

mosh, which has long had excellent support for ecn, appears to be getting the
ecn bit washed out along my path from california to england.

ecn survives up that way, but not down.

Just a single data point thus far.

-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Ecn-sane] mosh ecn bits washed out

[Pete Heist]

Meaning, the negotiation succeeds and ECT(0) is set going up, but
zeroed on packets by the time they come down?

The negotiation can also be blocked with iptables --ecn-tcp-remove,
which just zeroes out ECE and CWR, preventing negotiation
(https://git.netfilter.org/iptables/tree/extensions/libipt_ECN.c), but
I doubt that's very commonly done.

On Thu, 2021-03-18 at 13:00 -0700, Dave Taht wrote:
> mosh, which has long had excellent support for ecn, appears to be
> getting the
> ecn bit washed out along my path from california to england.
> 
> ecn survives up that way, but not down.
> 
> Just a single data point thus far.
> 

Re: [Ecn-sane] mosh ecn bits washed out

[Sebastian Moeller]

Hi Pete,

> On Mar 19, 2021, at 09:25, Pete Heist <pete@heistp.net> wrote:
> 
> Meaning, the negotiation succeeds and ECT(0) is set going up, but
> zeroed on packets by the time they come down?
> 
> The negotiation can also be blocked with iptables --ecn-tcp-remove,
> which just zeroes out ECE and CWR, preventing negotiation
> (https://git.netfilter.org/iptables/tree/extensions/libipt_ECN.c), but
> I doubt that's very commonly done.

	Oh, that looks interesting, probably not too hard to extend this to also allow to selectively only re-map ECT(1) and or ECT(0). I wonder what TCP Prague would do if its ECT(1) flows are switched to ECT(0) in transit?
	
Best Regards
	Sebastian

> 
> On Thu, 2021-03-18 at 13:00 -0700, Dave Taht wrote:
>> mosh, which has long had excellent support for ecn, appears to be
>> getting the
>> ecn bit washed out along my path from california to england.
>> 
>> ecn survives up that way, but not down.
>> 
>> Just a single data point thus far.
>> 
> 
> 
> _______________________________________________
> Ecn-sane mailing list
> Ecn-sane@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/ecn-sane

Re: [Ecn-sane] re-marking ECT(1) to ECT(0) (was: mosh ecn bits washed out)

[Pete Heist]

...

On Fri, 2021-03-19 at 09:56 +0100, Sebastian Moeller wrote:
> Hi Pete,
> 
> > On Mar 19, 2021, at 09:25, Pete Heist <pete@heistp.net> wrote:
> > 
> > Meaning, the negotiation succeeds and ECT(0) is set going up, but
> > zeroed on packets by the time they come down?
> > 
> > The negotiation can also be blocked with iptables --ecn-tcp-remove,
> > which just zeroes out ECE and CWR, preventing negotiation
> > (https://git.netfilter.org/iptables/tree/extensions/libipt_ECN.c),
> > but
> > I doubt that's very commonly done.
> 
>         Oh, that looks interesting, probably not too hard to extend
> this to also allow to selectively only re-map ECT(1) and or ECT(0). I
> wonder what TCP Prague would do if its ECT(1) flows are switched to
> ECT(0) in transit?

I suspect it would then dominate all traffic in the C queue, or any
other RFC3168 signalling queue
(https://github.com/heistp/l4s-tests/#unsafety-in-shared-rfc3168-queues
). That might be an easy way to game the dual queue for higher
throughput.

Pete

> Best Regards
>         Sebastian
> 
> > 
> > On Thu, 2021-03-18 at 13:00 -0700, Dave Taht wrote:
> > > mosh, which has long had excellent support for ecn, appears to be
> > > getting the
> > > ecn bit washed out along my path from california to england.
> > > 
> > > ecn survives up that way, but not down.
> > > 
> > > Just a single data point thus far.
> > > 
> > 
> > 
> > _______________________________________________
> > Ecn-sane mailing list
> > Ecn-sane@lists.bufferbloat.net
> > https://lists.bufferbloat.net/listinfo/ecn-sane
> 

Re: [Ecn-sane] re-marking ECT(1) to ECT(0) (was: mosh ecn bits washed out)

[Sebastian Moeller]

Hi Pete,

if I were to design a high priority lane cutting through all the clutter of other's flows in the LL-queue, I would create a tunnel that asserts ECT(1) but does not react to CEs and simply would employ some FEC to deal with the expected packetloss more gracefully (to take over the queue it should sufficie to suffer less from packet loss than the rest). 
	Et voila, L4S should effectively ceed the LL-queues 15/16th capacity to my data, mwuahaha...

Best Regards
	Sebastian



> On Mar 19, 2021, at 10:18, Pete Heist <pete@heistp.net> wrote:
> 
> ...
> 
> On Fri, 2021-03-19 at 09:56 +0100, Sebastian Moeller wrote:
>> Hi Pete,
>> 
>>> On Mar 19, 2021, at 09:25, Pete Heist <pete@heistp.net> wrote:
>>> 
>>> Meaning, the negotiation succeeds and ECT(0) is set going up, but
>>> zeroed on packets by the time they come down?
>>> 
>>> The negotiation can also be blocked with iptables --ecn-tcp-remove,
>>> which just zeroes out ECE and CWR, preventing negotiation
>>> (https://git.netfilter.org/iptables/tree/extensions/libipt_ECN.c),
>>> but
>>> I doubt that's very commonly done.
>> 
>>         Oh, that looks interesting, probably not too hard to extend
>> this to also allow to selectively only re-map ECT(1) and or ECT(0). I
>> wonder what TCP Prague would do if its ECT(1) flows are switched to
>> ECT(0) in transit?
> 
> I suspect it would then dominate all traffic in the C queue, or any
> other RFC3168 signalling queue
> (https://github.com/heistp/l4s-tests/#unsafety-in-shared-rfc3168-queues
> ). That might be an easy way to game the dual queue for higher
> throughput.
> 
> Pete
> 
>> Best Regards
>>         Sebastian
>> 
>>> 
>>> On Thu, 2021-03-18 at 13:00 -0700, Dave Taht wrote:
>>>> mosh, which has long had excellent support for ecn, appears to be
>>>> getting the
>>>> ecn bit washed out along my path from california to england.
>>>> 
>>>> ecn survives up that way, but not down.
>>>> 
>>>> Just a single data point thus far.
>>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Ecn-sane mailing list
>>> Ecn-sane@lists.bufferbloat.net
>>> https://lists.bufferbloat.net/listinfo/ecn-sane

Re: [Ecn-sane] mosh ecn bits washed out

[Dave Taht]

On Fri, Mar 19, 2021 at 1:25 AM Pete Heist <pete@heistp.net> wrote:
>
> Meaning, the negotiation succeeds and ECT(0) is set going up, but
> zeroed on packets by the time they come down?

There has never been any "negotiation" for setting the ect(0) bit in mosh.
It's been set since 2012 on all udp packets.

I reasoned at the time, that since mosh is the tool of desparate sysadmins
everywhere coping with excessive latency and jitter on everything (the local
echo of keystrokes is a godsend), that turning further enabling CE would
help, or, if stuff with that bit set was dropped, be the focus of
outrage by sysadmins
everywhere and ecn support quickly reverted.

Nothing went wrong. :)

mosh has an extremely robust response to marks, reducing the rate to 2 frames
per second from as much as 60 fps on receipt of a single CE. Not going
any further is analogous to the  "subpacket window" problem ecn
enabled tcps have, and attempting to
sustain that low frame rate essential for mosh's operation.

Ironically, since I last bothered to look, mosh gained ipv6 support, but the
parsing of cmsg and the setsockopt for ecn capability were not updated for that.

Also ironically, mosh packets were originally marked AF42 but it stopped working
on one of MITs networks, so the dscp setting was removed, but ecn kept.

https://github.com/mobile-shell/mosh/blob/master/src/network/network.cc#L166

It would be trivial to sce enable mosh.

>
> The negotiation can also be blocked with iptables --ecn-tcp-remove,
> which just zeroes out ECE and CWR, preventing negotiation
> (https://git.netfilter.org/iptables/tree/extensions/libipt_ECN.c), but
> I doubt that's very commonly done.
>
> On Thu, 2021-03-18 at 13:00 -0700, Dave Taht wrote:
> > mosh, which has long had excellent support for ecn, appears to be
> > getting the
> > ecn bit washed out along my path from california to england.
> >
> > ecn survives up that way, but not down.
> >
> > Just a single data point thus far.
> >
>
>


-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Ecn-sane] mosh ecn bits washed out

[Pete Heist]

Ah I see, so mosh is a decent way to see if ECT(0) traverses the path
you're using. I haven't used it before, so I assumed TCP.

SCE would be a piece of cake for this, but with L4S you've got the same
problem as elsewhere, which is that if you set ECT(1), you don't know
which type of CE you're getting back. For an interactive protocol like
this that doesn't seek capacity, bottleneck detection seems impossible.

On Fri, 2021-03-19 at 07:08 -0700, Dave Taht wrote:
> On Fri, Mar 19, 2021 at 1:25 AM Pete Heist <pete@heistp.net> wrote:
> > 
> > Meaning, the negotiation succeeds and ECT(0) is set going up, but
> > zeroed on packets by the time they come down?
> 
> There has never been any "negotiation" for setting the ect(0) bit in
> mosh.
> It's been set since 2012 on all udp packets.
> 
> I reasoned at the time, that since mosh is the tool of desparate
> sysadmins
> everywhere coping with excessive latency and jitter on everything
> (the local
> echo of keystrokes is a godsend), that turning further enabling CE
> would
> help, or, if stuff with that bit set was dropped, be the focus of
> outrage by sysadmins
> everywhere and ecn support quickly reverted.
> 
> Nothing went wrong. :)
> 
> mosh has an extremely robust response to marks, reducing the rate to
> 2 frames
> per second from as much as 60 fps on receipt of a single CE. Not
> going
> any further is analogous to the  "subpacket window" problem ecn
> enabled tcps have, and attempting to
> sustain that low frame rate essential for mosh's operation.
> 
> Ironically, since I last bothered to look, mosh gained ipv6 support,
> but the
> parsing of cmsg and the setsockopt for ecn capability were not
> updated for that.
> 
> Also ironically, mosh packets were originally marked AF42 but it
> stopped working
> on one of MITs networks, so the dscp setting was removed, but ecn
> kept.
> 
> https://github.com/mobile-shell/mosh/blob/master/src/network/network.cc#L166
> 
> It would be trivial to sce enable mosh.
> 
> > 
> > The negotiation can also be blocked with iptables --ecn-tcp-remove,
> > which just zeroes out ECE and CWR, preventing negotiation
> > (https://git.netfilter.org/iptables/tree/extensions/libipt_ECN.c),
> > but
> > I doubt that's very commonly done.
> > 
> > On Thu, 2021-03-18 at 13:00 -0700, Dave Taht wrote:
> > > mosh, which has long had excellent support for ecn, appears to be
> > > getting the
> > > ecn bit washed out along my path from california to england.
> > > 
> > > ecn survives up that way, but not down.
> > > 
> > > Just a single data point thus far.
> > > 
> > 
> > 
> 
> 

Re: [Ecn-sane] mosh ecn bits washed out

[Dave Taht]

Plug: I note that I live and breathe and die in mosh, as it keeps the connection
nailed up no matter what happens.... nat timeouts especially.

I had never noticed it "just worked" on ipv6 before now.

Despite spending a few minutes fiddling with mosh trying to make it set tclass,
it didn't work on a modern kernel, (only a few minutes of fiddling
tho) and I was kind of amused to find myself documenting it on the
internet a decade back.

you *can* set the bits any way you want via irtt

irtt client -6 --dscp=0xff tun.taht.net




On Fri, Mar 19, 2021 at 7:59 AM Pete Heist <pete@heistp.net> wrote:
>
> Ah I see, so mosh is a decent way to see if ECT(0) traverses the path
> you're using. I haven't used it before, so I assumed TCP.
>
> SCE would be a piece of cake for this, but with L4S you've got the same
> problem as elsewhere, which is that if you set ECT(1), you don't know
> which type of CE you're getting back. For an interactive protocol like
> this that doesn't seek capacity, bottleneck detection seems impossible.
>
> On Fri, 2021-03-19 at 07:08 -0700, Dave Taht wrote:
> > On Fri, Mar 19, 2021 at 1:25 AM Pete Heist <pete@heistp.net> wrote:
> > >
> > > Meaning, the negotiation succeeds and ECT(0) is set going up, but
> > > zeroed on packets by the time they come down?
> >
> > There has never been any "negotiation" for setting the ect(0) bit in
> > mosh.
> > It's been set since 2012 on all udp packets.
> >
> > I reasoned at the time, that since mosh is the tool of desparate
> > sysadmins
> > everywhere coping with excessive latency and jitter on everything
> > (the local
> > echo of keystrokes is a godsend), that turning further enabling CE
> > would
> > help, or, if stuff with that bit set was dropped, be the focus of
> > outrage by sysadmins
> > everywhere and ecn support quickly reverted.
> >
> > Nothing went wrong. :)
> >
> > mosh has an extremely robust response to marks, reducing the rate to
> > 2 frames
> > per second from as much as 60 fps on receipt of a single CE. Not
> > going
> > any further is analogous to the  "subpacket window" problem ecn
> > enabled tcps have, and attempting to
> > sustain that low frame rate essential for mosh's operation.
> >
> > Ironically, since I last bothered to look, mosh gained ipv6 support,
> > but the
> > parsing of cmsg and the setsockopt for ecn capability were not
> > updated for that.
> >
> > Also ironically, mosh packets were originally marked AF42 but it
> > stopped working
> > on one of MITs networks, so the dscp setting was removed, but ecn
> > kept.
> >
> > https://github.com/mobile-shell/mosh/blob/master/src/network/network.cc#L166
> >
> > It would be trivial to sce enable mosh.
> >
> > >
> > > The negotiation can also be blocked with iptables --ecn-tcp-remove,
> > > which just zeroes out ECE and CWR, preventing negotiation
> > > (https://git.netfilter.org/iptables/tree/extensions/libipt_ECN.c),
> > > but
> > > I doubt that's very commonly done.
> > >
> > > On Thu, 2021-03-18 at 13:00 -0700, Dave Taht wrote:
> > > > mosh, which has long had excellent support for ecn, appears to be
> > > > getting the
> > > > ecn bit washed out along my path from california to england.
> > > >
> > > > ecn survives up that way, but not down.
> > > >
> > > > Just a single data point thus far.
> > > >
> > >
> > >
> >
> >
>
>


-- 
"For a successful technology, reality must take precedence over public
relations, for Mother Nature cannot be fooled" - Richard Feynman

dave@taht.net <Dave Täht> CTO, TekLibre, LLC Tel: 1-831-435-0729

Re: [Ecn-sane] mosh ecn bits washed out

[Mikael Abrahamsson]

On Fri, 19 Mar 2021, Dave Taht wrote:

> I had never noticed it "just worked" on ipv6 before now.

Last I checked it however didn't switching between IPv4 and IPv6. So if I 
use IPv6 and I then end up on a network without IPv6, mosh doesn't work. 
It keeps trying only IPv6. It'd be nice if it bound to both AF and tried 
both all the time.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se