From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yx0-f171.google.com (mail-yx0-f171.google.com [209.85.213.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (verified OK)) by huchra.bufferbloat.net (Postfix) with ESMTPS id 8E06A200627 for ; Mon, 12 Mar 2012 12:09:27 -0700 (PDT) Received: by yenl11 with SMTP id l11so4523445yen.16 for ; Mon, 12 Mar 2012 12:09:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:from:date:x-google-sender-auth:message-id :subject:to:content-type; bh=/XKqqns5QlYaByZSXi556Zj2F8wHUeVJ5z8FVQAA8qY=; b=dkOV4SAiV9/Gj+C42ENuhA2VDQgix8Qrq6NetpPtsyROC/iflelUizVhhFyTXZSHUd r+GvLpv5/m3LsA3hBM7R+d8Y3ZNDs1FMxoMiq4ygAbUpJEIuLne2Es9ZhKaiIYw0NLZ6 1/LWOsvRwmubTx81ztMG0YxuTki7/4i+B8+d5BbYZrpozIP8VlFDSS0yS1uJA+RJ2uUz EcBFKyVnx+GFbnKGQCafVSFqYxqDWxvfKt2j01bV0XZth01XuyFIoRI5eMuxsgR6uMeD iKFBSOHoUFSo7A0QwiULS5Bg9Yt4kHG8rJrQZiJ5N9KwqVtTEmMkOQhaCrJKYjhxLlzj SLEg== Received: by 10.182.121.101 with SMTP id lj5mr9007277obb.39.1331579366155; Mon, 12 Mar 2012 12:09:26 -0700 (PDT) MIME-Version: 1.0 Sender: tz2026@gmail.com Received: by 10.182.68.164 with HTTP; Mon, 12 Mar 2012 12:09:06 -0700 (PDT) From: tz Date: Mon, 12 Mar 2012 15:09:06 -0400 X-Google-Sender-Auth: L8aGu6f0sNqS-4iHovLjsBPeino Message-ID: To: thumbgps-devel@lists.bufferbloat.net Content-Type: text/plain; charset=ISO-8859-1 Subject: [Thumbgps-devel] Jitter and latency in a linux system might be a problem. Minimal BoM crosspost. X-BeenThere: thumbgps-devel@lists.bufferbloat.net X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Mar 2012 19:09:27 -0000 The PPS change detect thread in gpsd might have latency problems if the kernel is busy and/or it isn't a high priority - a plain userland program shows about 1mS jitter - a loop with the ioctl to block until a pin-change followed by gettimeofday will vary over about a 1000 uS band on my dual-core netbook. If you really need 100nS, you might need to use the kernel PPS, preferably with a dual core or better CPU so the interrupt can capture the nanosecond timer reliably. I'm not sure how much of the jitter might be due to the USB latency to detect an interrupt condition at the far end, but USB 2.0 ought to be fairly fast. However it might debounce or otherwise delay the detection of a serial line change to eliminate noise, so might be introducing a latency, e.g. CTS might need to be in the new state for tens or hundreds of microseconds before an event happens to avoid noise spikes. If it is a USB latency you might need to feed the 1PPS into a GPIO pin (maybe the WPS if it isn't being utilized by the router if it is on an interrupt). But to state the problem clearly, you want the clock to have 100nS accuracy, but there is no way in userland and/or USB to get better than 1mS jitter even if the edges going into a serial device are precise. There is a variable latency - error - introduced. If we need to do this via Linux-PPS we should decide now - it should be adaptable to the open router software as it just grabs the nanosecond timestamp upon an interrupt. But it would require updating the kernel. If this is doing more than the timing measurement, e.g. routing, the jitter might end up being much greater. Using an embedded linux device only running the timing stuff would likely be much more well behaved, but there are some kernel latencies no matter what - most routers aren't multi-core. If you don't grab the timestamp in the interrupt like Linux-PPS, you won't come near the target accuracy or precision. Many routers have a "serial console", but few have anything more than tx and rx broken out. They would be direct serial hardware so have less device-to-kernel latency. Many are on standard 0.1" spaced headers so it might be possible to do an adapter board that would plug-in to the header and connect to the GPS - you would have to open the router and install the adapter/gps and run the antenna wire out (maybe drilling a hole for it). And adjust the serial console kernel setting so it didn't think it would be a login. There are off the shelf $30 USB GPS units which say they are using a SkyTraq chipset but without (visible indication of support for) PPS - I assume one or more is actually using SkyTraq - the descriptions say SkyTraq but also some large number of tracking channels which SkyTraq doesn't do (it has under 20, 2 or 4 for search), but the description says 50+. Any SkyTraq GPS (I have the Venus 624 and 628, and their GLONASS/GPS modules), the "$" character in the first sentence can be synced to the UTC second and comes out at a fixed delay from the PPS edge, just over 1mS to stop bit with no visible jitter so technically it would be as accurate as the PPS itself, and the latency could be compensated for. At 115kBaud there will be over 500mS of serial silence before the $, so it can be detected fairly easily, perhaps even in a Linux-PPS interrupt timestamp (if $, grab temp timestamp, if temptimestamp is over 500mS from last timestamp, update - treat as PPS). If the CTS/DCD line switch ioctl jitter is acceptable, this is about the same and requires no modifications. Also using just the NMEA mode in an unmodified GPSD, the latency error for the time was very consistent - about 17 mS with the standard sentence set, there will be slightly more jitter since it invokes the PPS routine after the final sentence in the NMEA set (GPGGA/GSA/GSVx3/GPRMC), the sentences vary in length. If you just counted the characters (they are a continual burst), you could divide by the baud rate to get the position of the start bit of the '$', but it would likely have the same userland or USB jitter. Another problem is even if you get the clock at 100nS accuracy, how do you send packets out or record their arrival at an exact time or calculate the jitter there - the network stack and drivers? You might need to instrument part of the network stack to record the nanosecond timestamps, or will have to live with the error from the network stack variable latency. Perhaps a possibility would be an external clock that you could trigger a capture-time event when the packet arrived and read out slower. The external timer could be synced to UTC and very precise (I've played with syncing signals from an Atmel chip to PPS), so the internal ntp synced clock might just be approximate. It doesn't fix the kernel jitter but would act like an external precise and accurate stopwatch. Does this have to be an actual router, or could it sit on the network right at the gateway? If you just need to measure "router-to-router", it might be easier to use something like an arduino with ethernet shield and emit a UDP or ICMP packet EXACTLY at the UTC second (assuming no ethernet collisions, but a good switch should be able to avoid them). There are embedded servers inside what appear to be large ethernet connectors which are fairly inexpensive. Perhaps a UTC synchronized pingbot - ping it and it will do the echo exactly at the UTC second. Two pingbots could measure to well under 1mS precision and accuracy. Basically an embedded system that would only know UDP, but you could tell it to send a packet to a destination, but it would do so within microseconds of the PPS. and tag anything in and/or return an echo with the incoming timestamp, e.g. over a command line interface: uutcping 1.2.3.4 (arrives at 67.890mS after UTC) (remote returns in a packet, some ID header, 67.890, and the GPRMC sentence from the previous second) (output is RTT, remote time, remote GPRMC) 115.234 - 67.890 - $GPRMC... I suggest a pure embedded timing unit with really limited function because it would be easy to get it down to 100uS precision. Otherwise you are back to Linux-PPS if you want it. ----- Cross post from "Calling all Openhardware...", edited. This is the best I can come up with, but others might know of some other devices which might work better. Assuming you compile GPSD with PPS on CTS it might be done with the following three main parts: http://www.sparkfun.com/products/464 (Magnetic Mount active antenna with long cable) http://www.sparkfun.com/products/11058 (SkyTraq breakout) http://www.sparkfun.com/products/9873 (FTDI 3.3v Basic) $13, 50, and 15 respectively for $78, Quantity 1. 100+ drops 20%. You would need a simple single-sided board with some headers to attach - all at 0.1" centering, very easy to layout and build but would need headers, e.g. http://www.sparkfun.com/products/553 and http://www.sparkfun.com/products/116 - the PPS goes to CTS, the FTDI provides 3.3v @ 50mA - just enough for the GPS, and then just do TX - RXI and RX to TXO. Then some kind of packaging. You might have to configure the SkyTraq the first time, and AGPS helps (I have code for that - ftp download, then serial packetized upload with acks). SkyTraq has a fast search mode that uses more power so until it locks might pull over 50mA, that is one of the configurations that have to be changed. I have an unpackaged version running now using jumper wires, but haven't recompiled GPSD to use PPS_CTS yet. I have tested the consistency of the CTS edge. The SkyTraq PPS is accurate to 60nS mainly due to the 16Mhz internal clock rate. Even if you get a $30 GPS unit you would have to break it open and do some soldering under a stereo microscope, assuming you can attach the PPS to DCD or CTS or another pin easily.