From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from au-smtp-delivery-117.mimecast.com (au-smtp-delivery-117.mimecast.com [103.96.21.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.bufferbloat.net (Postfix) with ESMTPS id A57983B2A4 for ; Sun, 16 Apr 2023 20:52:06 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=auckland.ac.nz; s=mimecast20200506; t=1681692723; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1BlfRvAP7mB6VfhSszr+qEN0KWJDuk+ZBMawbkyN98g=; b=vu6bFYEXyqlx6VtNrvd/h+xWzEF0uaECiB2rk41nGq7ZRfSiHBeXLeRbUBz9JEyg7B7eCc pgIWQEtS0IADs8XeYhUCWN7/K8HBHwpRhJO/b8bPUONtPtz9iA0zUIKdILWBbj9gj9s0Jl mlZ0oucU4LElHNlFk0EyYhWZd5QaG9g= Received: from AUS01-ME3-obe.outbound.protection.outlook.com (mail-me3aus01lp2233.outbound.protection.outlook.com [104.47.71.233]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id au-mta-39-N7xhAQR2OJCqO_GTjVHukw-1; Mon, 17 Apr 2023 10:52:00 +1000 X-MC-Unique: N7xhAQR2OJCqO_GTjVHukw-1 Received: from SY4PR01MB6979.ausprd01.prod.outlook.com (2603:10c6:10:142::13) by SYZPR01MB7268.ausprd01.prod.outlook.com (2603:10c6:10:169::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.45; Mon, 17 Apr 2023 00:51:58 +0000 Received: from SY4PR01MB6979.ausprd01.prod.outlook.com ([fe80::8ab6:3d14:f4e:6fa2]) by SY4PR01MB6979.ausprd01.prod.outlook.com ([fe80::8ab6:3d14:f4e:6fa2%7]) with mapi id 15.20.6298.045; Mon, 17 Apr 2023 00:51:58 +0000 Message-ID: <79dbfe96-b5d5-3475-8da9-a32c2f4d53b7@auckland.ac.nz> Date: Mon, 17 Apr 2023 12:51:56 +1200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 To: David Lang CC: starlink@lists.bufferbloat.net References: <83qps6or-4574-n64r-6r95-71sp695q58p4@ynat.uz> From: Ulrich Speidel In-Reply-To: X-ClientProxiedBy: SYAPR01CA0027.ausprd01.prod.outlook.com (2603:10c6:1:1::15) To SY4PR01MB6979.ausprd01.prod.outlook.com (2603:10c6:10:142::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SY4PR01MB6979:EE_|SYZPR01MB7268:EE_ X-MS-Office365-Filtering-Correlation-Id: ee8ba644-b4ac-4a1a-555f-08db3eddf27f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0 X-Microsoft-Antispam-Message-Info: Y+QwuXG3hBf7zDdigQCYRHAa6M/q5xgJ9CuOp/00vWg89R3NVEQdPTKAkHFWhg6b2VF1QuAFs6bkWqKxBUFGpHTJ4371jOSj81g4Td0K5weQ9gyT5ui2gVPLKBrlTmk/oQFtdry7URqSLyLY8bLSCX3HfEBBWv/6LOxjOMfSkTcSXJ6PKRirlodL/1V4DEk880B+ROrlh6Fy9VrUYiQrttTxuiibyB2Cs9vZmvdCb+kG96tcUJRJ4d51vHXWHvmhNon8w2HsBTTLAe7dK5B/keQ0uzrmZsvOebpXMMN0Sy6WROAtwKerX5Dd7rwwENTIg05IXIityKikZqCRIB+kcJWQk4f9OR6kAPoX3X9xlq9gRus54T7DF4c/w8umjXdgOktJODvzuI9K/Lwb1wZFq/99zg8J0ev25QTwsV3PEBMB34DpL111CyYj3iQ644Aj5CzJFQarmksHZhgtGsKlMQ1DX1RyVUnhmmv1VAg9X7izVWAUgzxDQomaJaeElAJHshDdAqbFS923GUwFcf+hUoOlhoBiCywLqYT6/EXPjaN5Q58qY3O2d2UtwPnERTQuWR15aG69l78egJH9rvnFC+0BV+2c004Tf9Cdv05pwReKfAhAxPF2GkIgwsgYGbhGljC/z8xmNRH2aFPTNQQZsQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SY4PR01MB6979.ausprd01.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(366004)(39860400002)(396003)(136003)(376002)(346002)(451199021)(53546011)(6506007)(6512007)(316002)(786003)(31696002)(86362001)(4326008)(6916009)(66476007)(66556008)(66946007)(478600001)(36756003)(966005)(2906002)(38100700002)(8936002)(5660300002)(8676002)(6486002)(41300700001)(83380400001)(66574015)(2616005)(186003)(31686004)(66899021)(45980500001)(43740500002); DIR:OUT; SFP:1101 X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kMWgwMJ5rGMH5FIClq/4u7/HZodeRYLzLUUyGp6DoZTzUny54SduUQBF3OLD?= =?us-ascii?Q?t2PK1zcfX0AMBG6lY3lb67uTJ1m8ZhtDL2RZOAWa/kNRR+2lKxGfSVXTL9pm?= =?us-ascii?Q?s4v8o++NhBq/opfxcWDj4HsSL3fWot5vAc8oMxIMT7mZeFslxOHl0eC2rn3h?= =?us-ascii?Q?uNZU5XwX0XbZ5WFvxf/ebQHTSZTfXTkk/Axs5J+RbA+W2Y0GxTnai7doXZDr?= =?us-ascii?Q?X/JhPRXU8mPrdU/yLdOpuvDTAB9TuKVPpb+YgAvu5w/DF9F6dF0zlA+5mz4U?= =?us-ascii?Q?8IjIAw48Sxyp4WGVcXDf6dwK4ZVpSQ7QSexLpCLJ05yvVALNWIqS6VclsMES?= =?us-ascii?Q?ylogiu7NsxEUIY0Srn8EzYzf90iAwMCBA9B4w6COuY4ZhaXia/lWp9EN9VhQ?= =?us-ascii?Q?W4iIsfZjjybihbP19fZWvzsXFAfOSjrVl/4qTJY9dh85Av5bjzpVKegbIs16?= =?us-ascii?Q?C/1mnaxv1VFElqKvDCrYqCZXVqovR+axKICk9R2hkp9IeYAvIicGzBl2QrCD?= =?us-ascii?Q?pAsc+Eje1VwGud2AfY0zRZdcpHBw8OGy5YU/3E0s1usjlcMBGVli7Md5lMd8?= =?us-ascii?Q?iJGz/QILrQvjE3Zcg1To5IDoEa0FuFDHV5SulZRvYMgB8JHqEBfjV4bEqHFI?= =?us-ascii?Q?59PMxadnnTmbImFswlnh7QxKgpjOIopVUI/pYKUwMvRUe9yWUaeH+ubrMmNA?= =?us-ascii?Q?mtyy26uX1/fWne+IXFU+EvTbISButxNpxnxIp/hd9VUwhoWaTqQpZk464PZg?= =?us-ascii?Q?DW1Jz+6LaPtZYoDmw2XpMzTVwJ1wX/xZoxNoHqG5cH9a7ikn68bw9A2/tQfz?= =?us-ascii?Q?zY9Cp0tfdDgnKK3FdIpewk2KzuOrjKPL6r1+pD0/trTwGsb3PGs48baxMoiT?= =?us-ascii?Q?zIoUdw059Nw0l45TL+0T6WMNM5DggMoXvELQK04JpvxQA05+1bOFouP0+MhN?= =?us-ascii?Q?ZzufJTKLXX7TaXCppba677XERVm8zoXJdMPJj5c+KdzQUgkBxGuSB8MPFfTF?= =?us-ascii?Q?m4AHDtBFMvwJ9fromz2ZQ4WqNQcKGSPlOuN4inaIEyh4b5iz+on3XtkwcUiu?= =?us-ascii?Q?LAJ4QbieBTjb24R+8MCsPW3HzRj3bCe816i2hYusClOPQygHRiQIeLczODbl?= =?us-ascii?Q?uRxj0k8yp7ejeYQpAMb1uIfrCjfILsIHURSVh1kgkuBmO9EURR51eHFV2Pon?= =?us-ascii?Q?BJfZUdAhfuTrfHzo9fzGw9lNuvEQtRBG6YC2cHoF+EzdXXU/cZY8D1Aoh8Jn?= =?us-ascii?Q?IDDr8oVg4BCir7kR/c+rsxuZSUmHVSE8ao/CznRIVAF5L6gkJeG6EpU+8j6C?= =?us-ascii?Q?P7+WkmSfjpzrRs201ZQa5mPYbfe4XPoiYgDevxLY8n4+602Y8q8Mc6Kbv1lw?= =?us-ascii?Q?q220ZFdgfLR5DjmB8Zv+2t3/k5iddxBeNmU+zKDeO22EjIpwknHEhc9CEm/I?= =?us-ascii?Q?eP04XG+jv9nblIXHTv9/YCc2ycjoG0J0eSRHtfNey1vLoJVvme681pVioCgb?= =?us-ascii?Q?xLh+14rovvfXLByg3UBQz9+ZeBLiy4TnAY18ZGie2jDNi7bzXP1Js6+LgzpZ?= =?us-ascii?Q?37TzEd9oR2u7xpUyy4tHF/IXQNkvmprYuicWiVybn1qmW3wtT54aeOswPf08?= =?us-ascii?Q?HCEHKpLRjWYdyArM8lXYonuM0iSt1q/GeeZ2FE1avPurtXGNyn8cEC4TiEZO?= =?us-ascii?Q?z/ZhWQ=3D=3D?= X-OriginatorOrg: auckland.ac.nz X-MS-Exchange-CrossTenant-Network-Message-Id: ee8ba644-b4ac-4a1a-555f-08db3eddf27f X-MS-Exchange-CrossTenant-AuthSource: SY4PR01MB6979.ausprd01.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Apr 2023 00:51:58.1980 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: d1b36e95-0d50-42e9-958f-b63fa906beaa X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: p037Pe1vSXjZJQKNNcWNfjy+Tl/wVy3faUK1nvZL4bCdRW3IBHI6ZzPLxWcN5ZWsEtsAMf/uNdAZaAJdrQZ55gqofkdt0bt/p32zno3vC5g= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SYZPR01MB7268 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: auckland.ac.nz Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Subject: Re: [Starlink] fiber IXPs in space X-BeenThere: starlink@lists.bufferbloat.net X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Starlink has bufferbloat. Bad." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Apr 2023 00:52:07 -0000 On 17/04/2023 11:22 am, David Lang wrote: > On Mon, 17 Apr 2023, Ulrich Speidel wrote: > >> On 17/04/2023 10:03 am, David Lang wrote: >>> On Mon, 17 Apr 2023, Ulrich Speidel via Starlink wrote: >>> >>>> On 17/04/2023 5:54 am, David Fern=C3=A1ndez via Starlink wrote: >>>>> In case you put a DNS server in the satellite, so that it replies >>>>> instead of a DNS server on ground, the RTT is reduced by half. >>>>> >>>>> The idea would be that the satellite inspects IP packets and when it >>>>> detects a DNS query, instead of forwarding the packet to ground >>>>> station, it just answers back to the sender of the query. >>>> Understood - it's just that the gain you have from this is quite=20 >>>> small. DNS queries only happen the first time a host needs to=20 >>>> resolve a name, and then again after cache expiry much later, so=20 >>>> they account for only a tiny fraction of the traffic, and also for=20 >>>> only a small amount of the total delay in page loads. RTT isn't=20 >>>> really the big issue in Starlink - yes it's larger than it perhaps=20 >>>> needs to be, and bufferbloat seems to be present, but compared to=20 >>>> GEO, it's now in the range seen for terrestrial Internet. >>> >>> DNS time is more significant than you think, due to the fact that so=20 >>> many websites pull data from many different locations, you end up=20 >>> with a lot of DNS queries when hitting a new site for the first time=20 >>> (and many of these queries are serial not parallel) so it adds quite=20 >>> a bit to the first rendering time of a page. >> But most people don't hit new sites most of the time, and a lot of=20 >> cascading loads hit the same CDNs you've seen previously. > > the timeouts on DNS are short enough that they hit them every day when=20 > they wake up That's OK (and has to be that way or else DNS changes would never=20 propagate). But, an end client, typically hits the same site many times within a=20 relatively short time window. For this, it does only need to do do one=20 lookup. The client will then cache the entry. If it needs to look up=20 again 15 minutes plus later doesn't really matter in terms of a LEO=20 system - the client will be talking to a different satellite by then. Also, the percentage of DNS queries relating to CDN servers is very high=20 nowadays, because CDN use is so pervasive that people will claim that=20 there is an "Internet outage" when a CDN goes down. > >>>>> CDNs or even datacenters (Cloud) in GEO or LEO is even more complex. >>>> >>>> Indeed. In so many ways. >>>> >>>> Mind though that CDNs are generally tied in with DNS nowadays, and=20 >>>> there's another snag: Take two users, Alice in the UK and Bob in=20 >>>> New Zealand - pretty much antipodean, using Starlink in bent-pipe=20 >>>> configuration, i.e., their traffic goes through, say, the London=20 >>>> gateway in the UK and the Clevedon gateway in NZ. Now imagine both=20 >>>> trying to resolve the same CDN hostname some time apart, but via=20 >>>> the same satellite DNS as the satellite has moved from the UK to NZ=20 >>>> in the interim. Say Alice resolves first and gets the IP address of=20 >>>> a CDN server in the UK. If the satellite DNS now caches this, and=20 >>>> Bob queries the same hostname, he gets directed to a server in the=20 >>>> UK literally a world away instead of the Auckland one closest to=20 >>>> him. So unless each satellite carries a geolocated copy of the=20 >>>> world's DNS entries with it and makes a decision based on user=20 >>>> location, you have a problem. >>> >>> This is true when the DNS answer is dynamic, but such cases also=20 >>> have short cache timeouts. Even with a 90 min orbit, a 15 min=20 >>> timeout would significantly lessen the impact (and I would expect=20 >>> that an orbital DNS would detect short timeouts and treat them as a=20 >>> signal to shorten the timeout even more) >> >> Timeout where? At the end user client or at the satellite? > > at the DNS cache and at the client. If you are using DNS to redirect=20 > people to the closest/least loaded site, you need to have your DNS=20 > timeouts set short so that you can change where they go with minimal=20 > downtime. Many clients refuse to honor extremely short timeouts (IIRC=20 > about 15 min is the low end) > >> At the end user client, a short timeout makes no sense at all because=20 >> their host-to-CDN-IP server mapping shouldn't really change in bent=20 >> pipe - only the sat hop changes. >> >> If the timeout is meant to be on the satellite, it means that the=20 >> satellite knows nothing about anything when it arrives to assist you,=20 >> and needs to query some sort of (probably ground-based) DNS server=20 >> anyway. >> >> Also, the assumption that a satellite will return to the same spot=20 >> after a full orbital period (of say 90 minutes) only applies to=20 >> satellites in equatorial orbits (or polar orbits, and then only to=20 >> the poles). In all other cases, the Earth's rotation will assure that=20 >> the satellite's return to the same location takes many orbital periods. > > when the satellite first comes into an area, it won't know what's=20 > appropriate to cach for the area, but it will start caching when=20 > people start using it, the first person suffers the full hit, but=20 > everyone after that benefits. Yes, full understood. That's how it works on terrestrial DNS (and even=20 on GEO this would be a really good argument). But on LEO, that benefit=20 only materialises if the second client and any others in the same area=20 get to query the same satellite that handled the first client's query,=20 because that's where the information would be cached if we had a DNS=20 server of sorts on each bird. For this to happen, the second client and=20 any subsequent ones have to query within minutes if not seconds of the=20 first one. What is the probability for this to happen? This depends on=20 the total number of active users hanging off that satellite and the=20 popularity of the target host/site among them. The larger the number of=20 users and the higher the site popularity, the more likely that cached=20 entries will see a second or subsequent query. "Active" in this context=20 means users navigating to new sites during the visibility window of that=20 satellite. Practically speaking, we know from various sources that each Starlink=20 satellite provides - ballpark - a couple of dozen Gb/s in capacity, and=20 that active users on a "busy" satellite see a couple of dozen Mb/s of=20 that. "Busy" means most active users, and so we can conclude that the=20 number of users per satellite who use any site is at most around 1000.=20 The subset of users navigating to new sites among them is probably in=20 the low 100's at best. If we're excluding new sites that aren't dynamic,=20 we're probably down to a couple of dozen new static sites being queried=20 per satellite pass. How many of these queries will be duplicates? Not a=20 lot. If we're including sites that are dynamic, we're still not getting=20 a huge probability of cache entry re-use. > > DNS data is not that large, getting enough storage into the satellites=20 > to serve 90% of the non-dynamic data should not be a big deal. The=20 > dynamic data expires fast enough (and can be detected as being dynamic=20 > and expired faster in the satellite) that I'm not worried about=20 > serving data from one side of the world to the other. Yes, but the only advantage we'd get here is faster resolution for a=20 very small subset of DNS queries. --=20 **************************************************************** Dr. Ulrich Speidel School of Computer Science Room 303S.594 (City Campus) The University of Auckland u.speidel@auckland.ac.nz http://www.cs.auckland.ac.nz/~ulrich/ ****************************************************************