This all started when our "United Speedsailors of America" team member Denis posted results from a recent session. He had uploaded his results to ka72.com, and it showed that he had reached 2 second speed of 59.88 knots:
Furthermore, Denis often sails with Boro, who just recently had his own problems with GPS accuracy, so he was very skeptical about what his watch tried to tell him. He had also worn his old trusted GT-31 GPS unit, and that had reported a top speed of only around 30 knots.
World-famous speed surfer and GPS guru Roo (yes, he is from Australia!) immediately looked at the tracks, noticed that during the time of the supposed record speeds the "sat count dropped to 6/7", and came to the conclusion that due to "the surrounding mountains the sats that were just above the horizon produced poor signals which caused the anomoly".
Well, this seems to make sense - Denis and Boro are windsurfing on a mountain lake, so the mountains around it must be too high! Since almost everyone else is windsurfing in flat terrain, maybe we can just ignore their problems? That's simple! Simple is great! Right?
Not so fast, cowboy! Sure, there are 10,000 ft (3,000 m) mountains around Washoe Lake. But the lake itself is at 5000 ft elevation, measures 2 x 4 km, and is in a valley that extends more than 10 km from each end of the lake. So a lot of the sky and the GPS satellites will be visible. So let's look at the tracks:
The doppler speed graph from his GW-60 watch is shown on the top in red; for comparison, the GT-31 data are shown in blue at the bottom. The GW-60 data show 3 spikes: 2 above 50 knots, and one above 70 knots! Before we jump to any hasty conclusions, let's zoom in on the largest spike:
We can see that the blue (GT-31) graph shows a speed close to zero around the time of the spike. In other words, Denis was most likely in the water at that time. There are few things that kill GPS reception as well as a few inches of water! We can also bet that Denis wore his watch on his wrist, and the GT-31 on his upper arm, so at times, the watch will have been deeper under water. Sure, the watch should not show a spike like this .. but I have seen similar spikes in GT-31 data in the past (and do not usually see them in GW-60 data, even when I fall a lot). As Forest Gump said: "It happens".
But let's not stop there - let's look at the actual data points:
If you wondered why the +/- numbers never got higher than 4.957, give yourself a geek gold star! That's simply because the SDoP numbers are stored as a single byte in the .sbp files, and the highest number possible corresponds to 4.957 knots. In plain English, an SDoP of 4 or higher indicates complete junk.
So, the watch actually knows that these data points are junk! Any analysis software should now, too! As guru Roo correctlu pointed out, GPSResults completely ignores the problematic data points, and gives top speed results that are practically identical to the results from the GT-31 unit. GPSResults has an SDoP filter, and by default refuses to include any regions with SDoP values above 3.0 (for 5 Hz data). The big question remains: what about other software?
Many Australian speed surfers analyze their results on ka72.com, and Denis did. However, it appears that ka72.com either does not use SDoP filters, or does not use them correctly. If we assume that Denis' problem had more to do with putting the watch under water than with being surrounded by mountains (which was not a problem when he was not swimming), then this problem might affect many other speed surfers, and needs to be addressed (there's also a problem with the alpha 500 numbers being wrong for this track).
Now, how about my favorite GPS analysis software, GPS Action Replay Pro (GPSAR, which I used to create all the screen shots above)? Unfortunately, the results are not pretty:
So, here's the bottom line:
- Accuracy estimates are fantastic to automatically identify bad data can lead to very large over-estimates - even fake "world records"
- ka72.com and GPS Action Replay Pro do not appear to have functioning SDoP filters at this point in time; any data analyzed with either software should be carefully checked!
- GPSResults has SDoP filters that can correctly identify artifacts such as the ones described here; however, as I have shown in a previous blog entry, the default settings may not be adequate to catch smaller errors, which can overstate speeds by a knot or two
--
As mentioned above, Denis's data files are available on ka72.com. I also made zip files with the data from my previous post about GW-60 accuracy for Boro's data and for the 6-GPS test drive.
--Added May 28, 2017:
In a comment to this post made today, Mathew made the bold statement "You dont need SDoP - just have a sensible acceleration filter". Well, acceleration filter can indeed be helpful to identify some problems, but they can not identify all problems that SDoP filters can find. The program GPS Action Replay does have an acceleration filter, but using it (which is an extra step that most users won't do) does not fix the problem. Applying a 3 m/s2 acceleration filter in the current version of GPSAction Replay (5.23) removed 801 points from Denis' track:
Here, the speed increases from less than 1 knot to 20.5 knots in 0.2 seconds, and then to 24.7 knots. After that, the speed remains around 23-25 knots for more than 2 seconds. The acceleration filter identifies the first 2 points, and also a few points at the end of the artifact (shown in red); but the points in the middle have low acceleration and look fine to the filter, which is why they are shown in green. Even after applying the acceleration filter, this data set still gives a 2 second "top speed" that is from an artifact, and 4 knots higher than the real top speed in this track. But the SDoP values in this region, which are all above 3, make it very clear that the data cannot be trusted and should be discarded.
Another example of errors which acceleration filters cannot catch, but SDoP filter can, are the more subtle 1-2 knot errors that can result from using an underhand grip in speed runs, with the watch facing downward and the lower arm blocking the GPS signal. It remains to be seen, however, what the best thresholds and implementation details for SDoP filters are that can catch such artifacts, without throwing out "good" data where just a single data point has higher-than-usual error values. That requires looking at a bunch of GPS files in detail to see how to separate the good from the bad, and will take a while.
