From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com [IPv6:2607:f8b0:4864:20::32a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id 0940F3B2A4 for ; Thu, 31 Aug 2023 09:42:56 -0400 (EDT) Received: by mail-ot1-x32a.google.com with SMTP id 46e09a7af769-6bca3588edbso621170a34.0 for ; Thu, 31 Aug 2023 06:42:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693489376; x=1694094176; darn=lists.bufferbloat.net; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=/6ZEba0ql5Hi/edaMswNrrQPrLUkKTGZtUUHcUPlWTs=; b=H4Ylstu/aC30SfWrtfBZUbZkz8VNf2ycoR1Mb/ZsWaUPNWmbs+i97DjoHWoKdipQP2 n3ZLg7khf0PgPvLJ6yL8FRoqW++1yse+ah7YdolDzlRWP9RW+ksBeydunq9WprsQi3LP ran0t/GWJMF2kDisgzrYUc6xyPjkf0JAa0+BM5spZY4KycODCWvQgiZk+X47GpUpJzQd J2oYok/NXPkOGmuQJ+e5iDi8lXpZEAUvm2Om9lSbTmFRESGgD3jMwaM4dre0N2ijqpq1 Fs26m/pUvDW2al00qLt/+C2i5qjHADnJZN5VQBFjQ/YDVRJ7uHx1C3VjcUZzLEhSqIDB 8HVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693489376; x=1694094176; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/6ZEba0ql5Hi/edaMswNrrQPrLUkKTGZtUUHcUPlWTs=; b=MzaBjbEhFeGHG0Kwvq/h83w9N/4JT6Z9Fm+NYH7rq3Gg3xV/tOtt8BgJM9KWHQScHq CR8OUtYQEaxDmQFvxPNYindJuRiwcBREN6GcOk+OAnp2657CwNMyLdWgBcSUFVHchh/s 3HD0foSYUWpYnEUK4folSD/hawSRFAX9xsw3WBIbnS1/ObNUsTBuhbMfpPCIlC+Lg3oL 8YGoPHuTf5akh+bYgrbqrH7JCxR6G9Ju8g5nXeWXz1WD/JKUp/38evat9WdXLt8D19R7 +X1fQIrOAHWujmVRpp/EVN9mwATZ3IKoe0hEuSwBKUcdoDwbVBjiW1FIO4h5NsVyAnvk cKcA== X-Gm-Message-State: AOJu0Yw5VPUZi0PrU8nWcnsLnlAmwOm4WRSEr/FGRax2/j8PdSeTj1/B NgVZSmL7YnULFxFobIYhV+7UHeqD5YlhsCPL97Pwqofd X-Google-Smtp-Source: AGHT+IGprEUbYjjpJu/k+JAUM8SniXSjdereCVjX2iYkMqtjejd+XP1xuMmXBY0hk4qu8cwwUkLHqXbbUfn1Mp70FdI= X-Received: by 2002:a05:6870:d211:b0:1ba:64a5:d2d1 with SMTP id g17-20020a056870d21100b001ba64a5d2d1mr5926189oac.20.1693489376001; Thu, 31 Aug 2023 06:42:56 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dave Taht Date: Thu, 31 Aug 2023 06:42:45 -0700 Message-ID: To: Sebastian Moeller Cc: Dave Taht via Starlink Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [Starlink] a puzzling starlink uplink trace X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Aug 2023 13:42:57 -0000 On Wed, Aug 30, 2023 at 12:43=E2=80=AFPM Sebastian Moeller wrote: > > Hi Dave, > > > you probably know already, but > mtr -ezb4 www.heise.de > will report compliant MPLS segments: Despite any reputation for omniscience I may have acquired, I was an ipv6 bigot in the era when MPLS was being specced, and somewhat willfully ignored it, and did not know traceroute and mtr had gained the ability to see it. It would be kind of neat (for business services at least), if starlink offered MPLS to the router. It would make some things easier. Nobody to this day seems to know what the dishy to ground station actually is, although a lot of folk seem to be speculating it is mpls, my bet is still something custom that has a destination and encrypted payload.... > = My traceroute [v0.95] > 123-1234567.local (192.168.42.229) -> www.heise.de (193.99.144.85) = = 2023-08-30T21:39:48+0200 > Keys: Help Display mode Restart statistics Order of fields quit > = = Packets Pings > Host = = Loss% Snt Last Avg Best Wrst StDev > 1. AS??? 192.168.42.1 (192.168.42.1) = = 0.0% 1067 0.9 0.6 0.5 9.1 0.4 > 2. AS6805 loopback1.0003.acln.06.ham.de.net.telefonica.de (62.52.201.2= 00) = 0.0% 1067 12.3 22.5 9.9 172.7 17.5 > 3. AS6805 bundle-ether10.0002.dbrx.06.ham.de.net.telefonica.de (62.53.= 2.98) = 0.7% 1067 12.4 12.0 10.3 16.5 0.7 > 4. AS6805 ae7-0.0001.corx.06.ham.de.net.telefonica.de (62.53.15.0) = = 0.0% 1067 18.7 19.2 17.2 48.9 2.4 > [MPLS: Lbl 16624 TC 0 S 1 TTL 1] > 5. AS6805 ae6-0.0002.corx.02.fra.de.net.telefonica.de (62.53.0.49) = = 0.0% 1067 41.6 19.6 17.2 51.6 3.9 > [MPLS: Lbl 16624 TC 0 S 1 TTL 1] > 6. AS6805 bundle-ether2.0001.cord.01.off.de.net.telefonica.de (62.53.0= .199) = 0.0% 1067 19.0 19.3 17.7 87.1 2.2 > [MPLS: Lbl 16624 TC 0 S 1 TTL 1] > 7. AS6805 bundle-ether1.0002.corp.01.off.de.net.telefonica.de (62.53.2= 8.171) = 0.0% 1067 22.9 19.2 17.5 29.7 0.9 > 8. AS??? ipv4.de-cix.fra.de.as12306.plusline.net (80.81.192.132) = = 0.0% 1066 18.5 19.7 18.2 93.0 2.5 > 9. AS12306 82.98.102.71 (82.98.102.71) = = 0.0% 1066 19.6 19.4 17.7 69.5 1.7 > [MPLS: Lbl 24002 TC 0 S 1 TTL 1] > 10. AS12306 82.98.102.23 (82.98.102.23) = = 0.0% 1066 17.8 19.8 17.4 197.7 9.9 > 11. AS12306 212.19.61.13 (212.19.61.13) = = 0.0% 1066 19.8 19.6 17.5 155.7 6.1 > 12. AS12306 www.heise.de (193.99.144.85) = = 0.1% 1066 19.6 19.1 17.6 63.7 1.5 > > Not that these do not already stick out as hops at different locations (h= ere Hamburg (ham), Frankfurt (fra) and Offenbach)) all with more or less id= entical distance-independent RTTs. > > Other than that quite intriguing puzzle, I wonder whether we could teach = flent to run a bidirectional traceroute/mtr as part of its measurements..? > > Regards > Sebastian > > > > On Aug 30, 2023, at 20:07, Dave Taht via Starlink wrote: > > > > In the attached 5 minute plot from a few days ago (I can supply the > > flent.gz files if anyone wants them), I see a puzzling spike at T+155s > > to nearly 90ms of baseline latency, then down to 20ms. No degree of > > orbital mechanics can apply to this change, even factoring in an over > > the horizon connection, routing packets on the ground through LA to > > seattle, and back, or using a couple ISLs, can make this add up for > > me. A combination of all that, kind of does make sense. > > > > The trace otherwise shows the sawtooth pattern of a single tcp flow , > > a loss (sometimes catastrophic) at every downward bandwidth change. > > > > An assumption I have long been making is the latency staircase effect > > (see T+170) forward is achieving the best encoding rate at the > > distance then seen, the distance growing and the encoding rate falling > > in distinct steps, with a fixed amount of buffering, until finally > > that sat starts falling out of range, and it choses another at T+240s. > > > > But jeeze, a 70ms baseline latency swing? What gives? I imagine > > somehow correlating this with a mpls enabled traceroute might begin to > > make some sense of it, correlated by orbital positions.... > > > > > > -- > > Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof= .html > > Dave T=C3=A4ht CSO, LibreQos > > _______________________________________________ > > Starlink mailing list > > Starlink@lists.bufferbloat.net > > https://lists.bufferbloat.net/listinfo/starlink > --=20 Oct 30: https://netdevconf.info/0x17/news/the-maestro-and-the-music-bof.htm= l Dave T=C3=A4ht CSO, LibreQos