Friday, July 18, 2014

The 5 GPS Comparison

I received two additional Canmore GP102+ GPS units today. With not much wind in the forecast for the next few days, I decided to drive around a bit to test them. I mounted a total of 5 GPS units right next to each other on the dashboard of my car: two Locosys GT-31s, a FlySight, and the two new GP102+ units.

I drove around for about 20 minutes at speed similar to typical windsurfing speeds - 30-50 mph. Here's a Google Earth overlay of all 5 tracks:
I uploaded all tracks to ka72.com, so you can download the tracks or look at the Google overlays there (filter by track type "Driving", and look for "Osterville, MA, USA" tracks from July 18, 2014).

Most GPS units do well when you sail or drive in a straight line at constant speed, but some have accuracy problems during turns, for example in jibes. I drove around in circles in a parking lot to check for problems - here's a blowup of this section:
Tracks from the two GT-31 units are in yellow; from the Canmore GP102+ units in green; and from the FlySight in purple. All tracks are reasonably close, except for one of the GT-31 tracks. This track happens to be from the unit I have been using for a couple of years (the other GT-31 track is from my wife's unit, which has been used a lot less). A look at the speed graph also shows big differences between these two units:
For the better unit, speed and doppler speed tracks are smooth and close to each other. For the other unit (light green tracks), the speed tracks have lots of spikes. The doppler speed tracks look better, but still show some minor spikes.

In comparison, the speed tracks for the GP-102 units are more similar:
In this case, the speed calculated from position data and the doppler speed are very close, but the doppler data show fewer artifact peaks. This illustrates why we definitely want to use doppler speeds for higher accuracy.

To see what caused the differences between the two GT-31 units, I looked at the trackpoints. The better unit tracked anywhere between 8 and 10 satellites; the "spikey" unit tracked only 5-6 satellites. Both units had a virtually identical view of the sky. The differences also existed when I checked the satellite screen on both units in my backyard.

No surprise here - the more satellites a GPS device can track, the more accurately it can triangulate the position. Five satellites is the bare minimum (according to the settings suggested by GPS-speedsurfing.com). However, even the "bad" unit always tracked at least 5 satellites, and the error estimate (HDoP) was 2.2 or less for all points (GPS-speedsurfing.com suggests to remove points with HDoP above 5.0).

I do not know why one of the GT-31 units was able to track 8-10 satellites, while the other was only able to track 5-6 satellites. But whatever the cause, the difference in accuracy was quite pronounced. This is quite visible when we compare the speed results that ka72.com gives for the tracks:
The first GT-31 units has substantially higher top speeds than the other units tested (I suggest to ignore the "alpha 500" values, since I did not simulate a alpha run while driving - this value includes multiple full circles from the blowup image above). Taking the values from all 5 units in the test to calculate a standard deviation allows us to see which units give results that fall within one standard deviation of the average. Both Canmore GP-102 units do for all values, but the GT-31_1 unit consistently outside this range.

In this test of two GT-31 units and two GP102+ units, the GP102-units gave overall more accurate results. This is by no means surprising, since the GP-102 units use a newer GPS chip (the Sirf IV), which contains multiple improvements over the Sirf III chip used in the ancient GT-31. One of these improvements is the number of "channels" for tracking satellites: the Sirf III in the GT-31 has only 20 channels, the Sirf IV in the GP102+ has 48 (more about this here). This alone means that the GP102+ should generally be able to track more satellites, resulting in more accurate data.

The results I describe above creates an interesting dilemma: should I keep using my old GT-31, which I know to be less accurate, or should I start using the GP102+? The GPS Team Challenge currently lists the GT-31 as a device "valid for the purposes of the challenge", along with a number of Garmin GPS devices that are known to be less accurate than the GT-31. But no new devices have been approved since the GT-31 was released many years ago (the "GT-35" that is listed was never commercially available).

If I play by the rules, I have to post results from the GT-31. I cannot post results from the FlySight, which is clearly much more accurate than the GT-31, or from the GP102+, which is at least as accurate as a "good" GT-31. From the results I have shown here and in previous blog posts, I will be "rewarded" with higher speeds, since the lower accuracy of the GT-31 tends to result in higher speeds.

Or I could ignore the rules, and post results from one of the more accurate GPS units I have. Sure, these units are not perfect - we'd ideally want more raw data, including number of satellites tracked and error estimates. However, it does appear that the newer units always will track more than the number of required satellites, and provide more accurate data that a GT-31 unit that still gives data points within the limits specified on GPS-speedsurfing.com.

I know two local racers and slalom sailors who are looking for a GPS so that they can be part of our team. I'd really love it if they'd post their speeds - it would be a sure way keeps our team from fighting for the second-to-last place in the monthly rankings. Who knows, maybe we can even dream of making it into the top half in a lucky month! Both of them would buy a GT-31, but they are not available in the US anymore. Should they really pay 2x or 5x as much for an approved Garmin unit, instead of buying a more accurate Canmore G-Porter GP102+? Maybe it is time to change the rules!

6 comments:

  1. Very nice test !, it think it's time to open the gps-speedsurfing.com website for new devices, your blog has been picked up in the netherlands and will be read by technical guys behind gps-speedsurfing.com for sure ! Keep up the good work for now

    ReplyDelete
  2. Good research. :-)

    The Flysight GPS is accepted by the WGPSSRC. (Data from a GPS with the UBlox chip it uses is accepted in amendments to the record rules). 10hz results from those chips have already been posted and accepted on GPS-SS for quite some time. That it is not listed is just an oversight.
    GPS Record rules: http://www.gps-speedsurfing.com/media/uploadimages/GPS_record_Rules%20VA1%202009-09-01.pdf
    .
    Did you check all the settings on your #1 GT-31? If they are the same as the other one, it is clearly faulty. This is one reason why the WGPSSRC requires at least x2 GT-31's for record ratification.If one is off, if the results are too far apart, the results are rejected. (a good reason to use at least 3 GPS if you are going for a record).

    GPS-TC is investigating allowing the Canmore GP102+ for possible inclusion as we write. I will also be prompting GPS-SS to do the same in the light of you testing.

    Regards, Andrew Daff
    Chairman, WGPSSRC

    ReplyDelete
    Replies
    1. Thanks, Andrew. I looked at the GPS-SS rules, but the only thing I found was allowing 10 Hz u-blox data. The FlySight only records at 5 Hz.

      The only possibly relevant difference I found on the units was that the "bad" unit uses firmware V1.4(B0315T), while the accurate unit uses V1.4(B0803T). But the "bad" unit finds satellites slower, and sees fewer of them.

      Great to hear you're looking into the Canmore GP102+. I think it is fine for normal postings to both sides, but for records, a firmware that saves accuracy data would probably be a must.

      Delete
    2. You nailed it! I did more digging, and found that the "bad" unit was set to "Low Power". That apparently limits tracking to 5-6 satellites. Switching to "Normal" power mode tracks 7-8 satellites within a few minutes. Tried it on both GT-31s, got the same results on both.

      I had switched my GT-31 to low power a while back when I went for a distance PB, and never switched it back. Nor did I think it would make such a difference!

      Delete
  3. Very nice test. Yes, I also think that is time to change some rules. I use GT31 for GTC and all other sites, but GT31 is EOL. We must take care for new members and new generation who can't buy GT31 anymore. This tested device is very impressive, but will be nice to test and compare GT31 with new Locosys product GW31 http://www.locosystech.com/product.php?zln=en&id=122

    ReplyDelete
    Replies
    1. Thanks for the link to the GW31. Technical specifications do not give enough information, e.g. about the chip that is used, recording rates, level of waterproofing, and data format. Two numbers that stand out on the specifications as inferior to the GT-31 are recording time (14 h instead of 40 h) and number of data points (120,000 vs. practically unlimited with SD card on the GT-31). None of the images show max speed (only current speed).

      Delete