[Thumbgps-devel] Fwd: [gpsd-dev] PPS over USB

[Thumbgps-devel] Fwd: [gpsd-dev] PPS over USB

[Dave Taht]

[-- Attachment #1: Type: text/plain, Size: 2670 bytes --]

---------- Forwarded message ----------
From: Gary E. Miller <gem@rellim.com>
Date: Tue, May 1, 2012 at 3:14 PM
Subject: [gpsd-dev] PPS over USB
To: gpsd-dev@nongnu.org


Yo All!

As some of you may know, esr has been helping the bufferbloat project
with some gpsd issues.  Their goal is to get good time from a USB
connected GPS.

Esr negotiated with Navisys to special build three units of a ublox 6
and a pl2303 with PPS conencted to USB.  They call them a GR-601 and I
just received the samples.  The preliminary results are pretty good if a
clock stable to about 1 milliSec is your goal.

A preliminary result from a dual core laptop.

# ntpq -p
    remote           refid      st t when poll reach   delay   offset
jitter
==============================================================================
-backup          .SOC1.           1 u   45   64  377    0.233 -0.006   0.067
+fuzzy           .GPS1.           1 u   47   64  377    0.241 -0.047   0.071
+SHM(0)          .GPS.            0 l   10   16  377    0.000 -2.995   1.656
*SHM(1)          .GPS1.           0 l    9   16  377    0.000  0.317   0.428


backup and fuzzy each have a PPS clock directly connected over serial.
The jitter on backup is about 2 microSec and fuzzy about 0.5 microSec.
And they tend to agree over ntp to about 20 microSec or better.

All the NTP servers are adjacent to each other and connected over GigE.
The jitter over the GigE seems to be about 100 microSec or better.

SHM(0) is NMEA time over USB.  SHM(1) is PPS time over USB.

As you can see the NMEA/USB short term jitter is about 2 milliSec and
the PPS/USB time is about 0.5 milliSec jitter.  Long term NMEA/USB drift
is quite a bit larger, maybe 100 milliSec or more.

So the jitter of PPS over USB is about 50x worse than PPS over GigE, but
I think for the purposes at hand pretty good.  Of course I expect the
results to be worse when run on a single core router.

The important part of the ntp.conf:

   # for gpsd
   server 127.127.28.0 minpoll 4 maxpoll 4
   fudge 127.127.28.0 time1 0.142  refid GPS

   # for PPS and gpsd
   server 127.127.28.1 prefer  minpoll 4 maxpoll 4
   fudge 127.127.28.1 time1 0.001500 refid GPS1

I start the daemons this way:

   ntpd -p /var/run/ntpd.pid -N -u ntp:ntp -g
   gpsd -n /dev/ttyUSB0

Any questions or suggestions?

RGDS
GARY
---------------------------------------------------------------------------
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97701
       gem@rellim.com  Tel:+1(541)382-8588


-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://www.bufferbloat.net

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Dave Taht]

I'm a little confused.

There are two threads going by today. One is about a sirfIII and the
other a ublox 6

I have been assuming they are one and the same, but perhaps I'm confused.

Secondly an item that has not been looked into much is the quality of
the antennas, or the quality of the clock source when only a single
sat is available....

Thirdly, wow.... when can I get one of these puppies to play with?

:me looks out his bay window wistfully:


On Tue, May 1, 2012 at 3:14 PM, Gary E. Miller <gem@rellim.com> wrote:
> Yo All!
>
> As some of you may know, esr has been helping the bufferbloat project
> with some gpsd issues.  Their goal is to get good time from a USB
> connected GPS.
>
> Esr negotiated with Navisys to special build three units of a ublox 6
> and a pl2303 with PPS conencted to USB.  They call them a GR-601 and I
> just received the samples.  The preliminary results are pretty good if a
> clock stable to about 1 milliSec is your goal.
>
> A preliminary result from a dual core laptop.
>
> # ntpq -p
>     remote           refid      st t when poll reach   delay   offset
> jitter
> ==============================================================================
> -backup          .SOC1.           1 u   45   64  377    0.233 -0.006   0.067
> +fuzzy           .GPS1.           1 u   47   64  377    0.241 -0.047   0.071
> +SHM(0)          .GPS.            0 l   10   16  377    0.000 -2.995   1.656
> *SHM(1)          .GPS1.           0 l    9   16  377    0.000  0.317   0.428
>
>
> backup and fuzzy each have a PPS clock directly connected over serial.
> The jitter on backup is about 2 microSec and fuzzy about 0.5 microSec.
> And they tend to agree over ntp to about 20 microSec or better.
>
> All the NTP servers are adjacent to each other and connected over GigE.
> The jitter over the GigE seems to be about 100 microSec or better.
>
> SHM(0) is NMEA time over USB.  SHM(1) is PPS time over USB.
>
> As you can see the NMEA/USB short term jitter is about 2 milliSec and
> the PPS/USB time is about 0.5 milliSec jitter.  Long term NMEA/USB drift
> is quite a bit larger, maybe 100 milliSec or more.
>
> So the jitter of PPS over USB is about 50x worse than PPS over GigE, but
> I think for the purposes at hand pretty good.  Of course I expect the
> results to be worse when run on a single core router.
>
> The important part of the ntp.conf:
>
>    # for gpsd
>    server 127.127.28.0 minpoll 4 maxpoll 4
>    fudge 127.127.28.0 time1 0.142  refid GPS
>
>    # for PPS and gpsd
>    server 127.127.28.1 prefer  minpoll 4 maxpoll 4
>    fudge 127.127.28.1 time1 0.001500 refid GPS1
>
> I start the daemons this way:
>
>    ntpd -p /var/run/ntpd.pid -N -u ntp:ntp -g
>    gpsd -n /dev/ttyUSB0
>
> Any questions or suggestions?
>
> RGDS
> GARY
> ---------------------------------------------------------------------------
> Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97701
>        gem@rellim.com  Tel:+1(541)382-8588



-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://www.bufferbloat.net

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Jim Thompson]

Hi Dave,

There is a third option in-play, but I've not spoken about it in public.  

I have an ARM (xscale) board with OpenWRT (amd gpsd) running that has provisions for a Trimble Copernicus-based GPS module with the PPS output tied to a GPIO pin on the Xscale, and the UART on the GPS module connected to the Xscale at TTL signal levels. 

The module is in-transit, so I've not completed the integration or started testing.  I should be able to use the fast-path IRQ stuff on the xscale to significantly reduce the latency & jitter over a USB-based solution. I know it's fast enough to be able to implement a distributed CSMA scheme among multiple 802.11 radios, with timing sufficient to occur in the same timespan as two short OFDM symbols (800ns each, so 1.6 microsec.) 

Outdoor enclosures, POE, etc. are all sourced, so the whole ball of wax could be put outside, with either Ethernet or Wi-Fi connectivity.  Alternatively, an antenna with DC power could be used, (it's available).

When they're ready, of course you can have one to play with.  I'm doing it to support your efforts wrt bufferbloat. 

Jim

On May 1, 2012, at 8:40 PM, Dave Taht <dave.taht@gmail.com> wrote:

> I'm a little confused.
> 
> There are two threads going by today. One is about a sirfIII and the
> other a ublox 6
> 
> I have been assuming they are one and the same, but perhaps I'm confused.
> 
> Secondly an item that has not been looked into much is the quality of
> the antennas, or the quality of the clock source when only a single
> sat is available....
> 
> Thirdly, wow.... when can I get one of these puppies to play with?
> 
> :me looks out his bay window wistfully:
> 
> 
> On Tue, May 1, 2012 at 3:14 PM, Gary E. Miller <gem@rellim.com> wrote:
>> Yo All!
>> 
>> As some of you may know, esr has been helping the bufferbloat project
>> with some gpsd issues.  Their goal is to get good time from a USB
>> connected GPS.
>> 
>> Esr negotiated with Navisys to special build three units of a ublox 6
>> and a pl2303 with PPS conencted to USB.  They call them a GR-601 and I
>> just received the samples.  The preliminary results are pretty good if a
>> clock stable to about 1 milliSec is your goal.
>> 
>> A preliminary result from a dual core laptop.
>> 
>> # ntpq -p
>>     remote           refid      st t when poll reach   delay   offset
>> jitter
>> ==============================================================================
>> -backup          .SOC1.           1 u   45   64  377    0.233 -0.006   0.067
>> +fuzzy           .GPS1.           1 u   47   64  377    0.241 -0.047   0.071
>> +SHM(0)          .GPS.            0 l   10   16  377    0.000 -2.995   1.656
>> *SHM(1)          .GPS1.           0 l    9   16  377    0.000  0.317   0.428
>> 
>> 
>> backup and fuzzy each have a PPS clock directly connected over serial.
>> The jitter on backup is about 2 microSec and fuzzy about 0.5 microSec.
>> And they tend to agree over ntp to about 20 microSec or better.
>> 
>> All the NTP servers are adjacent to each other and connected over GigE.
>> The jitter over the GigE seems to be about 100 microSec or better.
>> 
>> SHM(0) is NMEA time over USB.  SHM(1) is PPS time over USB.
>> 
>> As you can see the NMEA/USB short term jitter is about 2 milliSec and
>> the PPS/USB time is about 0.5 milliSec jitter.  Long term NMEA/USB drift
>> is quite a bit larger, maybe 100 milliSec or more.
>> 
>> So the jitter of PPS over USB is about 50x worse than PPS over GigE, but
>> I think for the purposes at hand pretty good.  Of course I expect the
>> results to be worse when run on a single core router.
>> 
>> The important part of the ntp.conf:
>> 
>>    # for gpsd
>>    server 127.127.28.0 minpoll 4 maxpoll 4
>>    fudge 127.127.28.0 time1 0.142  refid GPS
>> 
>>    # for PPS and gpsd
>>    server 127.127.28.1 prefer  minpoll 4 maxpoll 4
>>    fudge 127.127.28.1 time1 0.001500 refid GPS1
>> 
>> I start the daemons this way:
>> 
>>    ntpd -p /var/run/ntpd.pid -N -u ntp:ntp -g
>>    gpsd -n /dev/ttyUSB0
>> 
>> Any questions or suggestions?
>> 
>> RGDS
>> GARY
>> ---------------------------------------------------------------------------
>> Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97701
>>        gem@rellim.com  Tel:+1(541)382-8588
> 
> 
> 
> -- 
> Dave Täht
> SKYPE: davetaht
> US Tel: 1-239-829-5608
> http://www.bufferbloat.net
> 

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Eric S. Raymond]

Dave Taht <dave.taht@gmail.com>:
> I'm a little confused.
> 
> There are two threads going by today. One is about a sirfIII and the
> other a ublox 6
> 
> I have been assuming they are one and the same, but perhaps I'm confused.

You are.  The first set of prototypes, from UniTraq, were SiRF-III + CP2101.
These won't do - turns out the CP2101 driver can't wait on a handshake state 
change because the vendor never released enough programnming information.

Gary is talking about the second set of prototypes, from NaviSys.
That's the uBlox + PL2303 device.  It works.  The next step is to
get them into volume production.

> Secondly an item that has not been looked into much is the quality of
> the antennas, or the quality of the clock source when only a single
> sat is available....

Quality of antenna will matter for the device's ability to function in
weak-signal conditions, e.g. indoors.  Clock time should be OK even with
a single sat - multiple birds are needed for the spherical trig to compute
position, but atomic time you get directly from each sat sample.

> Thirdly, wow.... when can I get one of these puppies to play with?

Gary has three.  He's keeping one and sending me the second.  Since OpenBSD 
has no TICMIWAIT, Chris Kuethe is out of contention for the third and it
should logically go to you.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Hal Murray]

esr@thyrsus.com said:
> Clock time should be OK even with a single sat - multiple birds are needed
> for the spherical trig to compute position, but atomic time you get directly
> from each sat sample. 

I think it's more complicated than that.

You can get time with only 1 sat if know where you are located.  It takes 
special software, usually called "timing".

The usual approach for finding the location is to do a "survey".  That 
requires more special software which is usually included in the timing 
package.  It collects a 1000 (or whatever) good samples and averages the 
position.  The "good" qualifier means they don't use low-quality samples 
which might be way off.

It's worth asking Navisys if they have the timing mode software.


-- 
These are my opinions, not necessarily my employer's.  I hate spam.

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[tz]

[-- Attachment #1: Type: text/plain, Size: 2129 bytes --]

SkyTraq has a different version of the 638 (not just different firmware)
that emits corrections to the pps edge to go to 10nS, and after a lock it
will work down to 1 satellite and maintain accurate pps.

I don't know of any others that retain pps if they don't have a 3d lock.

The antenna is important but it won't overcome an obscured view of the sky
- at best you will get multipath or other errors with the weak signals.
On May 1, 2012 11:33 PM, "Eric S. Raymond" <esr@thyrsus.com> wrote:

> Dave Taht <dave.taht@gmail.com>:
> > I'm a little confused.
> >
> > There are two threads going by today. One is about a sirfIII and the
> > other a ublox 6
> >
> > I have been assuming they are one and the same, but perhaps I'm confused.
>
> You are.  The first set of prototypes, from UniTraq, were SiRF-III +
> CP2101.
> These won't do - turns out the CP2101 driver can't wait on a handshake
> state
> change because the vendor never released enough programnming information.
>
> Gary is talking about the second set of prototypes, from NaviSys.
> That's the uBlox + PL2303 device.  It works.  The next step is to
> get them into volume production.
>
> > Secondly an item that has not been looked into much is the quality of
> > the antennas, or the quality of the clock source when only a single
> > sat is available....
>
> Quality of antenna will matter for the device's ability to function in
> weak-signal conditions, e.g. indoors.  Clock time should be OK even with
> a single sat - multiple birds are needed for the spherical trig to compute
> position, but atomic time you get directly from each sat sample.
>
> > Thirdly, wow.... when can I get one of these puppies to play with?
>
> Gary has three.  He's keeping one and sending me the second.  Since OpenBSD
> has no TICMIWAIT, Chris Kuethe is out of contention for the third and it
> should logically go to you.
> --
>                <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
> _______________________________________________
> Thumbgps-devel mailing list
> Thumbgps-devel@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/thumbgps-devel
>

[-- Attachment #2: Type: text/html, Size: 2821 bytes --]

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Ron Frazier (NTP)]

On 5/1/2012 11:33 PM, Eric S. Raymond wrote:
> Dave Taht<dave.taht@gmail.com>:
>    
>> I'm a little confused.
>>
>> There are two threads going by today. One is about a sirfIII and the
>> other a ublox 6
>>
>> I have been assuming they are one and the same, but perhaps I'm confused.
>>      
> You are.  The first set of prototypes, from UniTraq, were SiRF-III + CP2101.
> These won't do - turns out the CP2101 driver can't wait on a handshake state
> change because the vendor never released enough programnming information.
>
> Gary is talking about the second set of prototypes, from NaviSys.
> That's the uBlox + PL2303 device.  It works.  The next step is to
> get them into volume production.
>
>    
>> Secondly an item that has not been looked into much is the quality of
>> the antennas, or the quality of the clock source when only a single
>> sat is available....
>>      
> Quality of antenna will matter for the device's ability to function in
> weak-signal conditions, e.g. indoors.  Clock time should be OK even with
> a single sat - multiple birds are needed for the spherical trig to compute
> position, but atomic time you get directly from each sat sample.
>
>    
>> Thirdly, wow.... when can I get one of these puppies to play with?
>>      
> Gary has three.  He's keeping one and sending me the second.  Since OpenBSD
> has no TICMIWAIT, Chris Kuethe is out of contention for the third and it
> should logically go to you.
>    

Hi guys,

Just wondering.  On the USB + PPS + PL2303 concept, has the long term 
variation that has been observed in NMEA data in other similar units 
without PPS been accounted for?  I have personally observed a variation 
in NMEA data on my Globalsat Sirf III USB device of up to +/- 70 ms over 
several days.  Will having the PPS coming though USB solve that problem?

Sincerely,

Ron


-- 

(To whom it may concern.  My email address has changed.  Replying to former
messages prior to 03/31/12 with my personal address will go to the wrong
address.  Please send all personal correspondence to the new address.)

(PS - If you email me and don't get a quick response, don't be concerned.
I get about 300 emails per day from alternate energy mailing lists and
such.  I don't always see new messages very quickly.  If you need a
reply and have not heard from me in 1 - 2 weeks, send your message again.)

Ron Frazier
timekeepingdude AT techstarship.com

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[tz]

[-- Attachment #1: Type: text/plain, Size: 614 bytes --]

On May 2, 2012 4:13 PM, "Gary E. Miller" <gem@rellim.com> wrote:
>
> Yo tz!
>
> On Wed, 2 May 2012 07:09:09 -0400
> tz <tz2026@gmail.com> wrote:
>
> > SkyTraq has a different version of the 638 (not just different
> > firmware) that emits corrections to the pps edge to go to 10nS, and
> > after a lock it will work down to 1 satellite and maintain accurate
> > pps.
>
> Got a model #?

Venus638LPx-T

http://www.skytraq.com.tw/products/Venus638LPx-T_PB_v2.pdf

They have an evk that goes to a serial usb but has allmthe pins broken out,
and maybe they have a firmware sdk - I have it for the more common version.

[-- Attachment #2: Type: text/html, Size: 914 bytes --]

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Hal Murray]

gem@rellim.com said:
> Looking for that check in the gpsd code, I do not see it in the PPS path or
> in the NMEA path.  I wonder when that disappeared?  That may explain some
> things...  I'll go add that back in now. 

I think there is another piece of fine print in that area.  I'm not sure 
where I saw it, probably one of the Garmin docs.

The idea is that you can't trust the first few seconds of data when it 
switches from not-enough satellites to enough.  I've seen it give bogus 
position (way off) marked as valid.  I think they all happened right after 
recovering from not-enough-satellites.

I'll scan log files if anybody wants examples.


-- 
These are my opinions, not necessarily my employer's.  I hate spam.

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[tz]

[-- Attachment #1: Type: text/plain, Size: 984 bytes --]

Depending on the gps, there can be a dead-reckoning mode that will
extrapolate things for some time (often settable).  The pps may or may not
follow.
On May 2, 2012 5:19 PM, "Hal Murray" <hmurray@megapathdsl.net> wrote:

>
> gem@rellim.com said:
> > Looking for that check in the gpsd code, I do not see it in the PPS path
> or
> > in the NMEA path.  I wonder when that disappeared?  That may explain some
> > things...  I'll go add that back in now.
>
> I think there is another piece of fine print in that area.  I'm not sure
> where I saw it, probably one of the Garmin docs.
>
> The idea is that you can't trust the first few seconds of data when it
> switches from not-enough satellites to enough.  I've seen it give bogus
> position (way off) marked as valid.  I think they all happened right after
> recovering from not-enough-satellites.
>
> I'll scan log files if anybody wants examples.
>
>
> --
> These are my opinions, not necessarily my employer's.  I hate spam.
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1364 bytes --]

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Ron Frazier (NTP)]

Hi all,

Gary said his posts are bouncing, and to please post this to the list.  
So, this post serves that purpose, as well as being my reply.

Thanks for the note Gary.  I guess you're right, as long as PPS is in 
control.  I presume you could still get the numeric value for the second 
from NMEA and start time for it from the PPS.

Check what return address shows up when you send a message to 
thumbgps-devel and make sure it matches what you signed up with.  You 
may have to put a custom identity in your email, like I did with my 
return address above, to get it to accept your posts.

Sincerely,

Ron


On 5/2/2012 3:42 PM, Gary E. Miller wrote:
> Yo Ron!
>
> On Wed, 02 May 2012 09:40:41 -0400
> "Ron Frazier (NTP)"<timekeepingntplist@techstarship.com>  wrote:
>
>    
>> Just wondering.  On the USB + PPS + PL2303 concept, has the long term
>> variation that has been observed in NMEA data in other similar units
>> without PPS been accounted for?
>>      
> Of course, it is an unavoidable side effect of the variable length NMEA
> protocol and the variable time for the GPS to compute depending on
> satellite configuration.
>
>    
>>   I have personally observed a
>> variation in NMEA data on my Globalsat Sirf III USB device of up to
>> +/- 70 ms over several days.
>>      
> Yup, sounds right.  If you look closely, the time jump is when the
> number of sats used in the cmopuatation changes.
>
>    
>>   Will having the PPS coming though USB
>> solve that problem?
>>      
> Nope.  NMEA is NMEA and PPS is PPS and they are not related except
> to the partial second.  So the NMEA jitter will still be there, but who
> cares when the PPS is stable to 0.5 milliSec.
>
> BTW, for some reason thumbgps-devel rejects my posts, so pass this on
> to the list.
>
> RGDS
> GARY
> ---------------------------------------------------------------------------
> Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97701
> 	gem@rellim.com  Tel:+1(541)382-8588
>    


-- 

(To whom it may concern.  My email address has changed.  Replying to former
messages prior to 03/31/12 with my personal address will go to the wrong
address.  Please send all personal correspondence to the new address.)

(PS - If you email me and don't get a quick response, don't be concerned.
I get about 300 emails per day from alternate energy mailing lists and
such.  I don't always see new messages very quickly.  If you need a
reply and have not heard from me in 1 - 2 weeks, send your message again.)

Ron Frazier
timekeepingdude AT techstarship.com

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Eric S. Raymond]

Hal Murray <hmurray@megapathdsl.net>:
> 
> esr@thyrsus.com said:
> > Clock time should be OK even with a single sat - multiple birds are needed
> > for the spherical trig to compute position, but atomic time you get directly
> > from each sat sample. 
> 
> I think it's more complicated than that.
> 
> You can get time with only 1 sat if know where you are located.  It takes 
> special software, usually called "timing".

That's if you care about precision to less than the variation in
lightspeed-from-orbit and ionosphere delays.  Physicists do; most
other people don't, which is why uBlox can sell a device with a
time-service mode that (as I read the datasheet) doesn't require
a pre-survey.

Theoretical discussion only, since most GPses won't report time
without a (3-sat) fix and GPSD doesn't 

> It's worth asking Navisys if they have the timing mode software.

No chance.  I can ask, but I'm certain they'll not evenn know it exists.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Eric S. Raymond]

Ron Frazier (NTP) <timekeepingntplist@techstarship.com>:
> Just wondering.  On the USB + PPS + PL2303 concept, has the long
> term variation that has been observed in NMEA data in other similar
> units without PPS been accounted for?

No.  Hal Murray and I studied this extensively and were unable to
get a handle on the cause.

>                                      I have personally observed a
> variation in NMEA data on my Globalsat Sirf III USB device of up to
> +/- 70 ms over several days.  Will having the PPS coming though USB
> solve that problem?

Yes.  The measured jitter of 1PPS over PL2303 USB is 0.5 milliseconds.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Dave Taht]

On Mon, May 7, 2012 at 1:58 PM, Eric S. Raymond <esr@thyrsus.com> wrote:
> Ed W <lists@wildgooses.com>:
>> I guess I still don't understand the bufferbloat / thumbgps
>> requirements, but either you need a very stable clock local, or you
>> need a bunch of clocks which are sync'ed to each other (across the
>> world).
>
> The latter.  What we're really after is accurate packet transit times.
> So it matters less whether the clocks are accurate than whether
> they're highly stable to a common timebase. Which in this case is GPS
> atomic clock time.
>
>> I might guess that this is the goal of thumbgps - get the offset to
>> zero?  However, it appears in principle that you might achieve
>> almost the same simply syncing to a carefully chosen pool of stratum
>> 1 servers?
>
> But we're not sure we can trust NTP.  That's the whole problem; if bufferbloat
> really is inducing very large, very short-period latency spikes, it may be
> screwing with the symmetry and statistical-smoothness assumptions that
> NTP synchronization relies on.
>
> One of the explicit goals of the Cosmic Backround Bufferbloat Detector is to
> sanity-check NTP.
> --
>                <a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
>

:whew: what a thread!

When eric and I first started talking about this 11 months ago we
stalled out for 9 months on some of the basic problems. Then he
blogged... Going from concept to working hw in the under 60 days since
is astounding.

I've been heads down solving a bunch of other problems of late and can
barely keep up here, but having a plan for actual deployment wasn't
even a remote possibility 60 days ago, and while some of that is
beginning to move, it's going to take a while to sort it out!!!!!

To outline some of that:

0) The intent was to establish a baseline of effectively 'stratum 1'
boxes 'beyond the edge', worldwide to do end to end measurements.

Most measurements done to date are usually to a multitude of central
points. Seeing the interconnects between providers misbehave (or not)
seems useful.

It's also a technique  that scales O(n), which for a change, is
something desirable. :)

We just arbitrarily picked 100 as a good number. More would be better,
2 would be a start.

There would be ongoing measurements of the 'cosmic background' noise
that ntp filters out.
It would be helpful to identify (dynamically) misbehaving ntp servers
as well and get them out of the server pool(s)

Similarly, trustable measurements between a solid ntp server(s)
somewhere would be good...

Theres more to it than this...

1) Establish a partnership with a lab to store/analyze/present the
data - am talking with both mlabs and onelab
2) Find software worth leveraging. At the moment, the leading
candidate is 'scamper'. Multiple other possibilities exist,
suggestions wanted. Take a look at what caida does for visualizations,
for example -
3) Pull something together that could gather, collect, and analyze rawstats data
4) Analyze performance under various loads of the software and hardware.

Several posters here seem to have the assumption that the
hardware/OS/queues in play can't heisenbug the data, and I assure you,
it can....

5) Choose a database backend - leading candidates are map/reduce and
postgres. Suggestions wanted.

There's way more than all that, but having trustable time everywhere
was the first thing required to get anywhere, particularly when doing
comparisons between more than 2 devices at the same time, in a
centralized db.

-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
http://www.bufferbloat.net

Re: [Thumbgps-devel] [gpsd-dev] PPS over USB

[Eric S. Raymond]

Dave Taht <dave.taht@gmail.com>:
> When eric and I first started talking about this 11 months ago we
> stalled out for 9 months on some of the basic problems. Then he
> blogged... Going from concept to working hw in the under 60 days since
> is astounding.

In truth, I'm a little astounded myself.  Maybe I really *am* Manfred Macx.

For the rest of you: Manfred Macx is a character in Charles Stross's
novel "Accelerando" who describes himself as a "venture
philanthropist" - wanders around seeding product concepts for free and
connecting people who need each other to make business ideas work.
Charlie once told me, after I voiced a suspicion, that Macx was not in
fact me but was designed as sort of a composite of several people he
knows, two of them being RMS and myself.

Novel here: http://www.jus.uio.no/sisu/accelerando.charles_stross/sisu_manifest.html

> We just arbitrarily picked 100 as a good number. More would be better,
> 2 would be a start.

And I took the scaling issue very seriously.  It's why I zeroed in on
"Plain Jane" and mostly ignored all the clever DIY ideas.

> Several posters here seem to have the assumption that the
> hardware/OS/queues in play can't heisenbug the data, and I assure you,
> it can....

Yup.  Anyone who's done OS-level work ought to have known that going in.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>