Tuesday, February 13, 2018

The Disappointment

No, I'm not bitching about the weather in Texas. Sure, it was near freezing yesterday, and about 15ºF colder than on Cape Cod .. but on a couple of days, we'll have 70 degrees here, and Cape Cod will be cold.

This is another GPS post. One about some major frustration - it took me 10 days to get ready to write about hit. This is about the results from a windsurfing test, comparing the USB GPS dongle to the GW-60. Here are the first numbers I looked at (from GPSResults):
The numbers are for 2 seconds, with GW-60 on the left and the dongle on the right. The dongle was 1.45 knots higher than the GW-60 .. very suspicious! A quick look at the tracks showed that the dongle was clearly wrong:
The dongle track is blue, the GW-60 dark green. The dongle has a clear artifact peak at the "top speed" region that is about 5 knots too high!

It did not end there. When I compared the results for the nautical mile, things also looked bad:
Again, the dongle gave substantially higher numbers - about 0.9 knots for the nautical mile!

Why did I suddenly get such bad results, after all my driving tests had indicated that the USB dongle would likely be more accurate than the GW-60?

The first hint came from the tracks that GPSLogit had recorded for the phone GPS. For several runs, the phone GPS tracks (in red below) also showed large artifacts:
The phone had slipped on my armband several times during the session; with the phone and USB dongle pointing down instead of up, and my upper arm and body blocking the GPS signal, the data quality suffered .. a lot. I verified this in the next two sessions, as I reported in my two previous posts. With the phone properly fixed so that it could not slide down, I got much more accurate, artifact-free results in two sessions.

I finally went back and gave the "bad" data a closer lock. The nautical mile results looked suspicious, so I looked at the numbers in GPS Action Replay. First the GW-60:
Next the USB dongle:
Finally, the phone:
All the numbers are very close - within 0.1 knots of each other, and within 0.03 knots between the GW-60 and the USB dongle! An examination of the data showed that, once again, missing data points and GPSResult's approach to "filtering" were the cause. At the beginning of the fastest nautical mile run, the GW-60 skipped quite a number of data points; instead of recording a point every 0.2 seconds, it recorded at irregular intervals for about 50 seconds: mostly one point every second, but also at 0.6, 0.8 second, and 3 second intervals. Here's a screen shot of the track point table from the start of the nautical mile run:
Since GPSResults will generally ignore sections with missing data points, it picked a slower section of the track as the "fastest" nautical mile. So this time, the observed differences in the nautical mile results were due to (a) the GW-60 missing data points, and (b) GPSResults not using this section. The effect was that the top speed for the nautical mile was understated by 0.9 knots because the GW-60 dropped points. The GPS dongle and the phone both gave the more accurate results!

This was quite different to the 2-second results, which are much more prone to distortions from artifacts (this is even more so for the "top speed", which is very often too high by 0.5-1 knots!). Here's a look at the data from the USB dongle in this region:
Note that eight of the ten points have error estimates above 1 knot, and 5 points have error estimates above 1.2 knots. Almost a year ago, when I looked at artifacts in GW-60 tracks, I had come to the conclusion that 2 second results with an average SDoP above 1.4 should not be considered for GW-60 data, since they are quite likely to be highly inaccurate. It is safe to assume that the u-blox GPS chip inside the USB dongle calculates the "+/-" numbers a bit differently than the GPS chip inside the GW-60; it appears that a slightly lower threshold may be necessary. When applying a SDoP filter with a 1.2 knot threshold in GPSResults to the dongle data from this session, the artifact peak is filtered out, and the top speed numbers for the dongle are very close to the GW-60 numbers.

One problem with accuracy filtering is that GPS Action Replay currently does not support it; another issue is that it can easily be turned off in GPSResults. However, there is a simple solution for the USB dongle recording software: it could simply set apply the filter before saving the data to a file, for example by setting the speed in the affected region to 0. The app could then also alert the user about the issue with a message on the screen and audio feedback, which could serve as a little reminder to check if the phone and dongle are positioned correctly. I'll look into this, but first I plan to collect more data with the USB dongle while windsurfing, to see if such a filter would possibly through out "good" data.

Friday, February 9, 2018

More GPS Comparison Data

Below are more results from comparing three GPS devices: the Locosys GW-60 watch, an Android phone (Moto E4) with GPS Logit, and the USB GPS dongle I had been testing. These results are from a Kona One session 2 days ago, as reported by GPSResults:
No surprise for the 2 and 10 second results. All differences for the phone are well within the +/- ranges for the watch, with maximum differences of 0.2 knots for 2 seconds, and 0.09 knots for 10 seconds. Numbers for the phone GPS are similar, but slightly higher. Note that for this session, the GW-60 watch and the GPSLogit data were recorded at 1 Hz, while the USB dongle recorded at 5 Hz.

The results for 500 m are similar, except that the phone GPS now does as well as the dongle. For the nautical mile, the results reported by GPSResults differ a lot for the phone. This is because GPSResults filtered out single data points in the middle of a few one-mile runs:
The sixth point is shown in brackets, meaning GPSResults filtered it. Since the GPSLogit data do not have accuracy estimates, and both acceleration and HDoP around this point is low, we have to assume that GPSResults considers data from 7 GPS satellites as insufficient (unfortunately, the Mac version of GPSResults does not show a setting for minimum number of satellites). Unchecking the "On" checkbox for all filters leads to results that are more similar to the watch and dongle:
However, the last 3 nautical mile runs for the phone show larger differences even without filters. This, however, appears to be an artifact of the analysis: the entire session was only about 20 nautical miles long, with just 12.9 nautical miles at speeds above 12 knots. Looking at the sections in GPS Action Replay shows that the speeds over the aligned nautical mile regions only differ by 0.01-0.05 knots; the reported larger changes are caused by slightly different regions being picked for the different units by the analysis programs. For the GPSLogit tracks, GPSResults was able to "squeeze in" a extra nautical mile run of 13.49 knots, while the same region between previously picked (higher speed) nautical miles was just a tad shorter than 1 nautical mile in the data from the two other devices. For speedsurfing competitions, this kind of artificial difference is entirely irrelevant; it only shows that some care has to be taken when analyzing the data.

Wednesday, February 7, 2018

GPS Dongle Test Results

We finally got some wind to test the USB GPS dongles while windsurfing, and the initial results were frustrating. In the driving tests, both the USB dongle and the phone GPS (through GPSLogit) had given results that always were very close to the results from the"gold standard" GW-60 GPS watch. But in two successive windsurf sessions, the results differed quite a bit more, and the tracks showed some disturbing artifacts. Here is an example of a particular bad section in GPSLogit data:
The GW-60 speeds are shown in blue, the phone GPS data are shown in read. Over the selected region, which is a bit more than 1 km long, the phone gave an average speed that was 0.81 knots lower than the GW-60 speed. The cause is obvious: more than 10 times during this run, the phone GPS speed showed a sudden drop by several knots that often lasted 2-4 seconds.

What was puzzling about this was that in other runs of the same session, the phone data did not show any "sudden drop" artifacts; instead, they mirrored the GW-60 data quite closely, as shown in the image below:
What was going on? When I wrote about the negative results on the Seabreeze forum, several experts posted useful hints. The GPS antennas in the USB dongle and the phone are quite small, and therefore sensitive to poor reception. In the driving tests, the antenna orientation was always near-optimal and essentially fixed. But during my windsurfing tests, the waterproof bag on the armband I used often slipped from the top of my upper arm to the underside or the inside of my arm, which effectively block a lot of the GPS signal. At the root of the problem was the kind of armband I used, which allowed the bag quite a bit of movement along the armband. Other armbands, like the more expensive Aquapac, don't allow this kind of movement.

After having spent more than $100 for Android phones and USB dongles, and after getting very good results in driving tests, I was not quite ready to give up. Instead, I attacked a piece of velcro to the backside of my waterproof bag, which keeps the bag from slipping on the armband, and the phone and USB dongle always looking upwards towards the GPS satellites. I did two more tests - yesterday on a Kona One for a couple of hours, and a shorter session today on a speedboard in confused chop and light rain. In both sessions, the speed results from the phone GPS and the USB dongle were much closer to the GW-60 data, and the tracks did not show weird artifacts. Here are some data from today's session, analyzed by GPSResults:
The 2-second results are most prone to distortions from "noise" in the data, but for all 8 runs with speeds above 20 knots, the numbers for the phone and USB dongle are within about 0.2 knots of the GW-60 numbers: well within the estimated accuracy of +/- 0.3 knots. On average, the absolute difference is just 0.11 knots.

For 10 seconds and longer runs, the expectation is that the numbers should be even closer, since distortions from random noise should largely cancel each other out. Here are the numbers for 10 seconds:
For the USB dongle, all numbers are within 0.1 knots of the GW-60 numbers, and also within the estimated accuracy window - great! The differences for the phone GPS, as recorded by GPSLogit, are slightly larger, which may be partially due to the slower measuring rate (1 Hz vs. 5 Hz for GW-60 and USB dongle).

The 500 m data look similar:
 Again, all the dongle data are within the (rather small) error range of the GW-60 results. The same is true for 6 of the 8 GPSLogit results, but 2 are slightly outside of the error range.

When I looked at the results for the nautical mile, I was in for a surprise:
GPSResults gave 4 nautical mile results, the fastest at 18.6 knots, for the USB dongle; but for the watch, it gave only 2 results, the fastest at  only 17.8 knots. A closer look at the data revealed the cause:
My nautical mile runs all had a jibe in the middle, which usually was wet (I still have to learn how to jibe the 72 l speedboard in chop). In the middle of the fastest nm run, this caused the GW-60 to drop three points, probably because the watch was under water. I wore the watch on my wrist, while the phone and USB dongle were on my upper arm, and therefore may not have even been under water. With default settings, GPSResults will not include any region with any missing data points in any speed results up to the nautical mile, so it did not report the fastest nautical mile run for the watch. Similarly, it dropped the 4th-fastest run, and selected a different (and slower) region for the second-fastest run it reported.

Fortunately, GPS Action Replay is not as strict about missing data points, and gave results that included the regions with missing points:
Again, the results are very close to each other, with a slight advantage going to the USB dongle.

These results indicate that the USB dongle and the phone GPS in the Moto E4 can give very accurate results while windsurfing, if it is ensured that the phone is facing up, and cannot slip down. The dongle seem to be slightly more accurate, although the data set presented here is very small; additional tests would be needed to see if the observed difference holds up. The more important issue, however, is that the u-blox based USB dongle provides accuracy estimates, which can be used to automatically identify artifacts caused by poor reception.

More tests to come .. whenever the wind comes back. Today was wicked cold for Texas standards, with air temperatures dropping below 10ºC during our session. We almost could have stayed on Cape Cod!

Thursday, February 1, 2018

South Padre Island

What do you do if you've got a new board but it's not windy in Corpus Christi?
You drive to South Padre Island! Well, at least that's what we did last Sunday, when the wind forecast for SPI was about 5 knots better than for Corpus Christi. Nina really wanted to try her brand-new Skate 86! As you may have guessed from Nina's expression, the trip was a big success - two days of wind, flat and shallow water, and even a freestyler showing Spocks, Funnels, and Switch Konos to keep Nina motivated (perhaps I should rather say "even more motivated").

Our first stop was at a kite boarding shop to get a pass for the SPI North Flats - an area a couple of miles from town that the local windsurf and kiteboard association has leased. Here's a picture:
It looks flat because it is flat - the water is about hip-deep for at least 3 miles out (and probably also three miles or more to the left and right). A perfect training ground for the OBX-Wind long distance race! On Sunday, Nina took out her Skate, but on Monday morning, she joined for a bit of slalom practice, and to improve our monthly numbers on the GPS Team Challenge. I had already tried to improve my personal best in the 1 hour category on Sunday, but a broken harness line after 55 minutes stopped that (neither did it help that I got a bit disoriented and ended up a mile upwind of where I had started). But on Monday, things went smoothly, and we ended the session with three personal bests - 1 hour for both of us, and nautical mile for Nina. The session also got our team into the top-10 rankings for the month of January in the 1 hour and distance categories. Nice!

After finishing the 1 hour, I tried to go out on my 72 l speed board, but just then, the wind took a little break: meter readings dropped from 18 mph averages down to 15. Even considering that the meter reads low, that was too little for the small board, and I switched to using my freestyle gear for the first time here in Texas. Using a light 5.6 m freestyle sail after the 7.8 m 3-cam freerace sail felt so easy!

After a much-needed break day, we were back on the water yesterday. I rented a Goya Bolt 115 (nice board!), Nina had a tuning and jibe practice session on the Falcon 99 / Racing Blade 7.0 combo. Perhaps I'm biased, but I think she looked good on the water:

There are a couple more things I want to talk about, including the new "Do it yourself" kit from Ianovated that lets you convert any wetsuit into a "heated tubes" wetsuit for cold weather windsurfing - but the forecast for tomorrow predicts great NE wind in the morning, so I'll leave you with one more picture of Nina - this time from the "I got my Skate!" celebration at a local brewery: