[Ecn-sane] Meanwhile, over on NANOG...

[Toke Høiland-Jørgensen]

Luca Muscariello <muscariello at ieee.org> writes:

> On Tue, Nov 12, 2019 at 2:02 PM Toke Høiland-Jørgensen <toke at toke.dk> wrote:
>
>> Mikael Abrahamsson <swmike at swm.pp.se> writes:
>>
>> > On Tue, 12 Nov 2019, Toke Høiland-Jørgensen wrote:
>> >
>> >> I'm not on the nanog list, but feel free to cross-post; would be good
>> to
>> >> actually get to the bottom of this issue! Marek and I already had an
>> >> off-list back-and-forth after that original thread, and we couldn't
>> find
>> >> anything wrong on the Cloudflare side. And the RSTs have a higher TTL
>> >> than the actual traffic, indicating an in-path problem...
>> >
>> > tcptraceroute supports setting/clearing ECN bits (-E), would be very
>> > interesting to see difference between those tcptraceroutes?
>>
>> No difference. But the RST is not being sent as a response to the SYN;
>> it is sent in response to the first data packet...
>>
>> ... and now that I'm re-testing, things were working for a little while,
>> but now the bug is back. I got an intermittent successful connection
>> with the same TTL that I was previously getting the RST from. And now
>> I'm back to getting RSTed.
>>
>> So I guess there's some kind of multipath issue here; ECMP path,
>> multiple routing upstreams, or a broken load balancer? Any other ideas?
>>
>
>
> It makes me think of some usage of anycast TCP on the cloudflare side.
> What service is this Toke?

Yeah, I did also think about anycast when I said "multiple routing
upstreams". For testing I've just been doing 'curl 1.1.1.1'. But
Cloudflare-hosted sites in general seem to have this problem; for
instance, 'curl -4 bufferbloat.net' also fails (but IPv6 is fine).

-Toke