From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f181.google.com (mail-wi0-f181.google.com [209.85.212.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 40EA620191C for ; Mon, 7 May 2012 14:14:26 -0700 (PDT) Received: by wibhn14 with SMTP id hn14so9917wib.10 for ; Mon, 07 May 2012 14:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=SJCICNn4nTsEKzdJyUkeX3n8f/G9j/8Mq3AsV/WX7LU=; b=xkb50V8ATyENigPCMtS8MusBJ6+WQHYqiCekG0YI/MvVSa/WPV3DwbRlv36awOijDX KbW5QDvbJ8KXMhTkS8ZLQQZTk2oBX5K3HmI6VW1xnItSFC4EZgkQVeoU1kWRS+MwzSqi 5njEH8l4jrFV0xPouGeVc76FiOCWeQZVbkCSeF56pXzJKf3l3JucH4m7Ob7yBzrhpwhi diI9tM9jkWJK5RJ4Bu7gaS47kDSzfeuJLHpcyZYy9R1UZRShoEvp+HuOGAUHHdeUzxKM Yzm/llthduGDxFjGk7jIxXywHU/BjsewWcgD/wWchCUzwyNzJqeMfLc7oXq4Z8SHtJa9 YwFQ== MIME-Version: 1.0 Received: by 10.180.105.69 with SMTP id gk5mr44220840wib.3.1336425260529; Mon, 07 May 2012 14:14:20 -0700 (PDT) Received: by 10.223.112.66 with HTTP; Mon, 7 May 2012 14:14:20 -0700 (PDT) In-Reply-To: <20120507205843.GA21259@thyrsus.com> References: <20120501151435.4a6a5373.gem@rellim.com> <4FA43068.2080606@wildgooses.com> <20120504160455.4514ac26.gem@rellim.com> <20120506214138.GB14699@thyrsus.com> <4FA6F396.70307@tmsw.no> <4FA7BDF1.7030708@wildgooses.com> <4FA7CFB2.9080807@tmsw.no> <4FA7E197.9010902@wildgooses.com> <20120507205843.GA21259@thyrsus.com> Date: Mon, 7 May 2012 14:14:20 -0700 Message-ID: From: Dave Taht To: esr@thyrsus.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: thumbgps-devel@lists.bufferbloat.net, Ed W , gpsd-dev@nongnu.org Subject: Re: [Thumbgps-devel] [gpsd-dev] PPS over USB X-BeenThere: thumbgps-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 May 2012 21:14:27 -0000 On Mon, May 7, 2012 at 1:58 PM, Eric S. Raymond wrote: > Ed W : >> I guess I still don't understand the bufferbloat / thumbgps >> requirements, but either you need a very stable clock local, or you >> need a bunch of clocks which are sync'ed to each other (across the >> world). > > The latter. =A0What we're really after is accurate packet transit times. > So it matters less whether the clocks are accurate than whether > they're highly stable to a common timebase. Which in this case is GPS > atomic clock time. > >> I might guess that this is the goal of thumbgps - get the offset to >> zero? =A0However, it appears in principle that you might achieve >> almost the same simply syncing to a carefully chosen pool of stratum >> 1 servers? > > But we're not sure we can trust NTP. =A0That's the whole problem; if buff= erbloat > really is inducing very large, very short-period latency spikes, it may b= e > screwing with the symmetry and statistical-smoothness assumptions that > NTP synchronization relies on. > > One of the explicit goals of the Cosmic Backround Bufferbloat Detector is= to > sanity-check NTP. > -- > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Eric= S. Raymond > :whew: what a thread! When eric and I first started talking about this 11 months ago we stalled out for 9 months on some of the basic problems. Then he blogged... Going from concept to working hw in the under 60 days since is astounding. I've been heads down solving a bunch of other problems of late and can barely keep up here, but having a plan for actual deployment wasn't even a remote possibility 60 days ago, and while some of that is beginning to move, it's going to take a while to sort it out!!!!! To outline some of that: 0) The intent was to establish a baseline of effectively 'stratum 1' boxes 'beyond the edge', worldwide to do end to end measurements. Most measurements done to date are usually to a multitude of central points. Seeing the interconnects between providers misbehave (or not) seems useful. It's also a technique that scales O(n), which for a change, is something desirable. :) We just arbitrarily picked 100 as a good number. More would be better, 2 would be a start. There would be ongoing measurements of the 'cosmic background' noise that ntp filters out. It would be helpful to identify (dynamically) misbehaving ntp servers as well and get them out of the server pool(s) Similarly, trustable measurements between a solid ntp server(s) somewhere would be good... Theres more to it than this... 1) Establish a partnership with a lab to store/analyze/present the data - am talking with both mlabs and onelab 2) Find software worth leveraging. At the moment, the leading candidate is 'scamper'. Multiple other possibilities exist, suggestions wanted. Take a look at what caida does for visualizations, for example - 3) Pull something together that could gather, collect, and analyze rawstats= data 4) Analyze performance under various loads of the software and hardware. Several posters here seem to have the assumption that the hardware/OS/queues in play can't heisenbug the data, and I assure you, it can.... 5) Choose a database backend - leading candidates are map/reduce and postgres. Suggestions wanted. There's way more than all that, but having trustable time everywhere was the first thing required to get anywhere, particularly when doing comparisons between more than 2 devices at the same time, in a centralized db. --=20 Dave T=E4ht SKYPE: davetaht US Tel: 1-239-829-5608 http://www.bufferbloat.net