[Thumbgps-devel] Jitter and latency in a linux system might be a problem. Minimal BoM crosspost.

[tz]

The PPS change detect thread in gpsd might have latency problems if
the kernel is busy and/or it isn't a high priority - a plain userland
program shows about 1mS jitter - a loop with the ioctl to block until
a pin-change followed by gettimeofday will vary over about a 1000 uS
band on my dual-core netbook.  If you really need 100nS, you might
need to use the kernel PPS, preferably with a dual core or better CPU
so the interrupt can capture the nanosecond timer reliably.

I'm not sure how much of the jitter might be due to  the USB latency
to detect an interrupt condition at the far end, but USB 2.0 ought to
be fairly fast.  However it might debounce or otherwise delay the
detection of a serial line change to eliminate noise, so might be
introducing a latency, e.g. CTS might need to be in the new state for
tens or hundreds of microseconds before an event happens to avoid
noise spikes.

If it is a USB latency you  might need to feed the 1PPS into a GPIO
pin (maybe the WPS if it isn't being utilized by the router if it is
on an interrupt).

But to state the problem clearly, you want the clock to have 100nS
accuracy, but there is no way in userland and/or USB to get better
than 1mS jitter even if the edges going into a serial device are
precise.  There is a variable latency - error - introduced.  If we
need to do this via Linux-PPS we should decide now - it should be
adaptable to the open router software as it just grabs the nanosecond
timestamp upon an interrupt.  But it would require updating the
kernel.

If this is doing more than the timing measurement, e.g. routing, the
jitter might end up being much greater.  Using an embedded linux
device only running the timing stuff would likely be much more well
behaved, but there are some kernel latencies no matter what - most
routers aren't multi-core.  If you don't grab the timestamp in the
interrupt like Linux-PPS, you won't come near the target accuracy or
precision.

Many routers have a "serial console", but few have anything more than
tx and rx broken out.  They would be direct serial hardware so have
less device-to-kernel latency.  Many are on standard 0.1" spaced
headers so it might be possible to do an adapter board that would
plug-in to the header and connect to the GPS - you would have to open
the router and install the adapter/gps and run the antenna wire out
(maybe drilling a hole for it).  And adjust the serial console kernel
setting so it didn't think it would be a login.

There are off the shelf $30 USB GPS units which say they are using a
SkyTraq chipset but without (visible indication of support for) PPS -
I assume one or more is actually using SkyTraq - the descriptions say
SkyTraq but also some large number of tracking channels which SkyTraq
doesn't do (it has under 20, 2 or 4 for search), but the description
says 50+.

Any SkyTraq GPS (I have the Venus 624 and 628, and their GLONASS/GPS
modules), the "$" character in the first sentence can be synced to the
UTC second and comes out at a fixed delay from the PPS edge, just over
1mS to stop bit with no visible jitter so technically it would be as
accurate as the PPS itself, and the latency could be compensated for.
At 115kBaud there will be over 500mS of serial silence before the $,
so it can be detected fairly easily, perhaps even in a Linux-PPS
interrupt timestamp (if $, grab temp timestamp, if temptimestamp is
over 500mS from last timestamp, update - treat as PPS).  If the
CTS/DCD line switch ioctl jitter is acceptable, this is about the same
and requires no modifications.

Also using just the NMEA mode in an unmodified GPSD, the latency error
for the time was very consistent - about 17 mS with the standard
sentence set,  there will be slightly more jitter since it invokes the
PPS routine after the final sentence in the NMEA set
(GPGGA/GSA/GSVx3/GPRMC), the sentences vary in length.  If you just
counted the characters (they are a continual burst), you could divide
by the baud rate to get the position of the start bit of the '$', but
it would likely have the same userland or USB jitter.

Another problem is even if you get the clock at 100nS accuracy, how do
you send packets out or record their arrival at an exact time or
calculate the jitter there - the network stack and drivers?  You might
need to instrument part of the network stack to record the nanosecond
timestamps, or will have to live with the error from the network stack
variable latency.

Perhaps a possibility would be an external clock that you could
trigger a capture-time event when the packet arrived and read out
slower.  The external timer could be synced to UTC and very precise
(I've played with syncing signals from an Atmel chip to PPS), so the
internal ntp synced clock might just be approximate.  It doesn't fix
the kernel jitter but would act like an external precise and accurate
stopwatch.

Does this have to be an actual router, or could it sit on the network
right at the gateway?  If you just need to measure "router-to-router",
it might be easier to use something like an arduino with ethernet
shield and emit a UDP or ICMP packet EXACTLY at the UTC second
(assuming no ethernet collisions, but a good switch should be able to
avoid them).   There are embedded servers inside what appear to be
large ethernet connectors which are fairly inexpensive.  Perhaps a UTC
synchronized pingbot - ping it and it will do the echo exactly at the
UTC second.  Two pingbots could measure to well under 1mS precision
and accuracy.  Basically an embedded system that would only know UDP,
but you could tell it to send a packet to a destination, but it would
do so within microseconds of the PPS. and tag anything in and/or
return an echo with the incoming timestamp, e.g. over a command line
interface:
uutcping 1.2.3.4
(arrives at 67.890mS after UTC)
(remote returns in a packet, some ID header, 67.890, and the GPRMC
sentence from the previous second)
(output is RTT, remote time, remote GPRMC)
115.234 - 67.890 - $GPRMC...

I suggest a pure embedded timing unit with really limited function
because it would be easy to get it down to 100uS precision.  Otherwise
you are back to Linux-PPS if you want it.

-----

Cross post from "Calling all Openhardware...", edited.  This is the
best I can come up with, but others might know of some other devices
which might work better.

Assuming you compile GPSD with PPS on CTS it might be done with the
following three main parts:
http://www.sparkfun.com/products/464 (Magnetic Mount active antenna
with long cable)
http://www.sparkfun.com/products/11058 (SkyTraq breakout)
http://www.sparkfun.com/products/9873 (FTDI 3.3v Basic)
$13, 50, and 15 respectively for $78, Quantity 1.  100+ drops 20%.
You would need a simple single-sided board with some headers to attach
- all at 0.1" centering, very easy to layout and build but would need
headers, e.g. http://www.sparkfun.com/products/553 and
http://www.sparkfun.com/products/116 - the PPS goes to CTS, the FTDI
provides 3.3v @ 50mA - just enough for the GPS, and then just do TX -
RXI and RX to TXO.  Then some kind of packaging.  You might have to
configure the SkyTraq the first time, and AGPS helps (I have code for
that - ftp download, then serial packetized upload with acks).
SkyTraq has a fast search mode that uses more power so until it locks
might pull over 50mA, that is one of the configurations that have to
be changed.

I have an unpackaged version running now using jumper wires, but
haven't recompiled GPSD to use PPS_CTS yet.  I have tested the
consistency of the CTS edge.

The SkyTraq PPS is accurate to 60nS mainly due to the 16Mhz internal clock rate.

Even if you get a $30 GPS unit you would have to break it open and do
some soldering under a stereo microscope, assuming you can attach the
PPS to DCD or CTS or another pin easily.