The Difference Between Good and Bad

No, this is not a philosophy post. It's about GPS reception. Surprised?

When I looked at GPS test results from yesterday and today, one thing puzzled me. Look at the data and take a guess:

Why was the phone GPS so inaccurate on 3/11, with large errors in every GPSTC discipline, but quite accurate on 3/12? I'll give you some clues, so you can see how early you can guess the answer.

Clue #1: A cold front pulled in yesterday afternoon. Temperatures dropped from the 80s into high 60s yesterday evening, and to 50s today.

Clue #2: That made me wear a 4 mm wetsuit today, while I was sailing in a lycra top yesterday.

The ardent reader of my blog may have a suspicion on what was going on by now (ha!).

Clue #3: The arm band I use for my phone tends to slip on the lycra, but much less so on the wetsuit. Yesterday, I stopped several times just to re-orient the phone so that it was pointing to the sky; it often was on the side of my arm.

Clue #4: Compare the track points from one run where the phone reported about 1 knot less speed than the GW-60 shown here:

to a run where both devices gave almost the same speed:

Clue #5: Check the "Sats" column for the phone data (the numbers on the right side) for the picture above.

So: yesterday, the armband with the phone slipped often, so that the phone was on the side of my arm, instead of pointing up. The runs that deviate most from the GW-60 data had reception from fewer GPS satellites, and changes in the number of satellites during the run. It seems that having the phone facing sideways on the arm during speed runs screws things up. That's not really very surprising - perhaps the more surprising thing is that the USB dongle gave accurate results yesterday (it was in the same bag, lying on the screen of the phone).

Can we verify that turning the phone sideways and blocking one side screws up GPS reception? Sure! I used GPSLogit, which has a graph that shows satellite reception. Here's a screen shot showing the reception with the phone facing up:

The phone used GPS 13 satellites, and had a decent horizontal accuracy of 4.6 meters. Just turning the phone onto its side did not change much, but then putting a hand close to the top of the phone on the back side did:

Now, the phone had good signal (green bars and dots) from only 7 satellites, and the estimated positional accuracy was 19.7 meters - four times worse!

If you want to reproduce this, but don't have GPSLogit, there are plenty of free GPS test apps on the Google Play Store available that could be used instead. The phone's GPS antenna is typically located near one of the top corners of the phone. You'll have to wait a few seconds to see changes in the satellite reception (probably because that information is not updated as often as position and speed info).

So, if you want to get accurate speeds from your phone with GPSLogit or Windsport Tracker, make sure the phone is facing to the sky! Of course, that's the same for the GW-60.

Dongles Beats Phone

Here's a graph that compares the accuracy of the GPS from an Android phone to the USB GPS dongle that I have been testing:

The bars show the (absolute) difference between the results of the GPS units to the results from the "Gold Standard" GW-60 GPS in the six GPS Team Challenge disciplines over 10 different windsurfing sessions. The unit is knots. Shorter bars are better.

The results show that the phone GPS (red bars) can give quite accurate results, but sometimes does not. In 5 of the 10 sessions, at least one number (usually for 2 second average speed) was off by 0.3 knots or more. In three sessions, the observed difference was larger than 0.5 knots, which is definitely unacceptable for competition (but still good enough for just recording your sessions, and getting an idea how fast you were).

In contrast, the USB dongle had a maximum error below 0.4 knots in all 10 sessions, with a maximum deviation of 0.2 knots in 9 out of 10 sessions. The one session with a higher error was the one I reported about previously, where the arm band with the phone and dongle had slipped for large parts of the session. In all other session, the accuracy was very good. For comparison, the observed differences when wearing two GW-60 watches (one on each hand) are often in the 0.1-0.2 knot range.

Compared to the phone GPS, the USB dongle has another advantage (in addition to the higher accuracy): the u-blox GPS chip in it can provide accuracy estimates, which allows the automatic detection and elimination of artifacts, for example those that can happen during crashes or when swimming.

Anyone interested more detail can look at the data in this PDF file. You may notice that not all sessions have entries for all 6 GPS disciplines. There are various reasons for this: some sessions are shorter than 1 hour, so GPSResults did not give 1 h results; other sessions are different length in the different files (for various reasons, including a little bug that stopped the dongle logging in earlier versions), so distance differences where meaningless; and so on.

If you're really interested and want to look at the raw GPS data yourself, you can download them from here. It's a 32 MB ZIP archive with about 35 files. Note that you may need to define the time range in GPSResults for some of the sessions where the different units recorded different length sessions if you want to get meaningful results. Also, a few of the GW-60 session contain data from multiple sessions, so make sure to select just the session you want to compare.

Dongle Update

We tried to escape the cold weather on Cape Cod, and drove three days to Texas. But after one decent day of sailing, temperatures dropped to below freezing, and stayed at low for the rest of the week.  Too low to go sailing .. maybe we're too spoiled already. But at least it gave me time to play around with the GPS dongles some more.

The first goal was to use two dongles at the same time so that I could compare the results - any difference is definite noise. I was able to record from two dongles on one phone with a USB hub, but the hub itself introduced some electronic interference which degraded the signal. So a second phone was needed.

Finding a cheap Android phone with USB OTG support was not easy. The first two phones I tried, which had been described as having OTG on some web sites, did not work. I finally found a Moto E4 for $40 at Best Buy that worked, and did a few test drives. After reading that the Bavarian Speedkini contest now allows the use of phones with GPSLogit or Windsport Tracker, I decided to record at 1 Hz, so that I could compare the phone GPS and the USB dongle to the "gold standard" GW-60. Here is a speed track from the latest test drive:

The graph shows tracks from 2 GW-60 phones in blue and red - but since the data are almost identical, only the red track is visible most of the time. The tracks for the two phones (Samsung Galaxy J3 and Moto E4) and for the two GPS dongles look pretty much the same.

A typical way of comparing GPS units is to simply compare the speed results, for example for 2 second and 10 second speeds. That is intuitive, but it has the disadvantage that only a very small subset of the data is used; everything that's not part of the top-speed regions is effectively ignored. To answer the more interesting question about how close the data are in general, I wrote a little program that calculates the differences between the tracks over all points above a minimum speed of 15 knots. It gives the average of the absolute differences, which is a good indicator of average reproducibility.  Here are the results for this test:

• Average absolute differences between 2 tracks:
• GW-60: 0.035 knots
• Phone GPS: 0.051 knots
• USB dongle: 0.025 knots

All these differences are quite small - so far, so good. How about the largest differences?

• Maximum observed differences between 2 tracks:
• GW-60: 1.07 knots
• Phone GPS: 0.796 knots
• USB dongle: 0.318 knots

Let's look at these areas - first the GW-60 watches:

Next the phones:

Finally, the USB dongles:

Visual examination of the traces confirmed what the numbers indicated: the phone traces have noticeable more diverging areas than the USB dongles, and the GW-60 is somewhere in the middle. The fact that the USB dongles and not the GW-60s had the most consistent data is somewhat surprising, considering that the GW-60 data are actually averages build from higher-frequency data, while the dongle data are (presumably!) single-point measurements. Theoretically, the averages data should have about 2-fold less noise than single-point measurements.

Comparing two identical units only gives us a lower limit of errors in the data, since it measures only truly random noise. Comparing the different GPS types to each other is more interesting, and did indeed give larger numbers for observed differences. However, a closer look revealed that there are some systematic error sources that need to be addressed first - take a look at the image below for an example:

The graph shows a comparison of a GW-60 track in blue and a USB dongle track in red. The GW-60 track is time-shifted relative to the dongle track - it's about 0.6 to 0.8 seconds behind. The likely cause for this particular shift is that the GW-60 reports averages of the last five 5-Hz measurements; the shift is not present when comparing data recorded at 5 Hz. However, I have noticed similar time shifts when comparing tracks from different phones; in this case, the underlying cause is probably different (e.g. the GPS chips in the phones take measurements at different times).

For a thorough comparison of data from different devices, the observed time shifts would need to be corrected. That is algorithmically quite simple, but will require a few hours of programming and testing, so it may be a while before I get to it. However, some of the data can already be analyzed at least qualitatively, as shown in the graph below:

The speeds shown are from the GPS chip in the Moto E4 (blue) and two USB dongles (green and red). In this region, the phone GPS was quite inaccurate. The two values at and before the vertical line were about 1 knot too low, and soon followed by 2 values that were about 1/2 knot too high, followed by another down-up cycle. If this artifact had occurred in the  top-speed region, the measured 2-second speed might have been 1 knot too low, or 0.5 knots to high.

The data above are just from one 15-minute test drive, with about 800 data points used for the analysis, so this is just anecdotal evidence. It's definitely way too early to claim that the USB dongle has better accuracy than the GW-60, that would require a lot more tests. However, it is once again an encouraging sign, and the observations are in agreement with what I have seen in other tests. Hopefully, we'll get some wind tomorrow for a windsurfing test!

Dead Toys and Lots of Noise

This is a two-part post. If you're a geek, keep reading to the end. If you just want an update about how things were recently, by all means stop after the first part.

Last Friday had looked so promising in the forecast: wind all day, with mid-20s in the afternoon! Sunny and warm! But it started badly enough, by giving me the finger - a blue finger:

I am still thinking I should sue Chinook for failing to put a warning label on their booms that reads "Remove fingers before closing". Fortunately, it's been too windy to find a lawyer. There's no fun to be had in court rooms!

Briefly, here are the other things that happened:

• My GoPro broke. Dead. Kaputt. 
• When the chop and crowds became unpleasant in front, I sailed to Lewis Bay on my Skate 110/Idol 5.6 combo, and loved the flat water I found. Came back to switch to slalom gear and drag Chris along. Had an absolutely crazy, overpowered, out-of-control ride through nasty chop to make it back to Egg Island - the wind had picked up to averages above 30. Chris promptly declared Lewis Bay as "absolutely unsailable" and sailed back, leaving me alone. Considering that I had barely made it there, I had doubts that I'd make it back, so I stayed. At least there were no crowds. But there was the security guard now posted at Egg Island that makes sure nobody but birds set foot there. 
• My Android phone that I use to announce my speeds through GPSLogit got wet and broke. This was the first time I had not double-bagged it, and also the first time I did not close the waterproof bag properly. It's terminally dead. Drying did not revive it. 

So, here's a pop quiz: what do I call a day of sailing like that? Think about it for a minute. I'll even post a picture to create some distance from the answer.

The answer is (of course): A great day of sailing!

In the morning, I had a few hours of freestyle on what I count as a small sail. I even made one or two wet Upwind 360s - the "Andy Brandt" variety, where you leave the sail forward when you carve, and backwind before loosing all speed. Still had to waterstart out of it, but getting closer.

In the afternoon, I was eventually joined at Egg Island by my lovely wife, who was nicely powered on 3.7. That made me feel a bit better about thinking that my 5.8 was a tad big, especially since the wind had dropped by then. I watched here hack away at the Flaka, with a few good tries. She now has tried Flakas on sails from 3.7 to 5.6 m .. impressive. I still think she'll get her first Flaka before the Cape Cod ABK camp in September.

I did not try anything, but practicing slalom sailing was fun, and I later got some perfectly flat spots for full-speed jibes - so cool! I ended the day with a final session on my 3S 96 with my Idol 5.0 in front, bumping and jumping. 125 kilometers of fun! Who would complain about a blue thumb and a couple of broken toys?

But I wanted to replace the GPSLogit phone, and picked up a Samsung Galaxy J1 for $30 at the local Best Buy. First order of business was checking how well it would work in comparison to the "standard" GPS units.

Warning: here begins the geeky part of the post. Continue reading at your own risk!

So I taped the phone and two Locosys GPS units to the dash board of our Nissan NV 2500 high roof van, and took the babies for a ride. The GPSLogit speech announcement worked as expected, and the speeds always agreed well with the speedometer. I even stopped in the middle to switch the GW-52 from 5 Hz recording to 1 Hz recording, just to see what happened.

Let's start by comparing the phone data with the GT-31 data. Here's a part of the doppler speed graph (click on the image for a large view):

GT31 (blue) vs. Samsung Galaxy J1 with GPSLogit (red)

The first impression was "pretty good", but a few small spikes are visible in the phone data. Here's an enlarged section:

The red phone data are a bit noisier, and have a few spikes that may overstate the speed by a knot or two. Indeed, the speed results in the GPS Team Challenge categories showed that the phone speeds were a bit higher than the GT-31 and GW-52 speeds (up to about 1/2 knot) - something that was seen before for other phones. But while the accuracy is not good enough for posting to the GPS Team Challenge, it's definitely good enough to hear how fast you are going while you're sailing. Great!

When I compared speed results from the GT-31 and the GW-52, the differences were minimal, as expected. However, there was a very noticeable difference in the 5 Hz data from the GW-52, compared the the 1 Hz data from the GT-31:

1 Hz GT-31 data (blue) compared to 5 Hz GW-52 data (red)

The GW-52 were collected every 0.2 seconds, so we have five times as many data points as for the GT-31 data which were collected every full second. The GW-52 data have a lot more spikes - little ups and downs. That had been reported by others before for windsurfing data, and the question arose whether these spikes are real differences in speed, or noise that was caused by the higher measurement rates. We'll get back to that question in a minute, but let's look at a comparison when the GW-52 unit is set to 1 Hz:

1 Hz GPS data from GT-31 (blue) and GW-52 (red) 

The data look very similar. 

So, let's have a closer look at the 5 Hz data. There are valid arguments that can be made why collecting data at a higher frequency is better. If (a) the errors in the speed measurement are purely random, and (b) the error per data point is the same at 1 Hz and at 5 Hz, then measuring at 5 Hz would give about 2.2-fold higher accuracy. But this is only true if both of these conditions are met.

Let's look at the data points (I used GPS Action Replay's "Trackpoint Table" to create the screen shots):

GW52 "raw" 5 Hz data from a small subregion

For comparison, here at GT-31 data from the same region:

GT-31 data (1 Hz)

A picture says more than 100 words, so here is the doppler speed graph for this region:

1 Hz GT-31data (blue) and 5 Hz GW-52 data (red)

When we seen spikes like in the red curve about in windsurf data, we may believe that they are real: we constantly hit chop, gusts, and lulls, and it seems quite possible that our speed changes several times within a second. But the data above are not from windsurfing, but from driving a big, heavy van.  The Nissan NV2500 High Roof van is quite a beast. It weighs in at about 3 tons (6250 lb), and we've loaded it up with interior racks, 6 boards, 15 sails, masts, booms, etc. Does it change speeds several times a second like the GW-52 data indicate? Allow me to cite from The Princess Bride:

"Inconceivable!"

We can get a bit more formal in analyzing this. We can start by looking at the acceleration - the change in speed from data point to data point. The acceleration is shown in the tables above as "Linear m/s2". In the GT-31 data, the values change little from data point to data point; but in the GW-52 data, the acceleration jumps from 0.4 to 2.9 in 200 milliseconds! That's almost no acceleration to 1/3 g. I'm sorry, but I don't drive that crazy!

The change in acceleration is actually a pretty good indicator of noise in the data. It's pretty easy to calculate, and if you're mathematically inclined, you can call it the second derivative (the acceleration being the first derivative of the speed). Here's a graph of the acceleration and noise for GW-52 data:

For comparison, here is what this looks like for the GT-31 data:

That's a lot less noise in the G-31 data! The 1 Hz data do reflect reality better: I accelerated somewhat evenly over about 15 seconds, kept the speed more or less constant for about 1/2 minute, and then slowed down constantly.

This is some rather clear evidence that collecting GPS data at higher frequencies does indeed introduce addition measurement error, which negates any potential advantages of the additional data points at least partially. More data is just that - more data. It's not automatically better data. Nor does a lower error number that some software spits out necessarily mean that the data are indeed more accurate - if basic underlying assumptions are invalid, then the error numbers will also be invalid.
--
Some of my readers may wonder: "Who are you that you think you can analyze this, and go against what everyone else says?". Well, that is a valid question. I admit that windsurfing is my primary addiction, but I have some scientific background. I got a M.S. in Biophysical Chemistry, a Ph.D. in Experimental Sciences, and have worked in scientific data analysis and software development for Bioinformatics for last 3 decades. I certainly enjoy developing algorithms for large-scale (and small-scale) data analysis... especially when it's not windy. 

Voices In My Head

Maybe I windsurf too much. Today was the 4th day in a row. Maybe that's nothing for Corpus Christi or the Gorge, but for Cape Cod, that's a lot. So today, I had a little voice in my head that told me how fast I was going. It only made me want to go faster! It worked, too - I finally broke the 60 km/h barrier (32.4 knots, 37.3 mph).

Meter readings today showed averages around 29 mph, with gusts up to 36 mph. There was no sand flying, though, making the readings appear a bit high. Nina wanted to go to Egg Island to work on Flakas using a 4.0 m sail, so I took my 90 l slalom board and 5.8 m speed sail and tried to go fast. The voice in my head? That was GPSLogit. Very useful. Just like Roo said.

Conditions were near-perfect for speed runs (and Flaka tries, says Nina). I heard "32" (knots) several times, which is faster than I had ever sailed before. 33 seemed so close.. but I later discovered that I switched from top speed runs to nautical mile runs just as the wind got strongest (and then dropped quickly). But my sail was probably too small for "real" speed, anyway: going back upwind was easy. That means I did not have much power in the downwind speed runs. Oh well. I'm happy with a new 2 second personal best. And it was fun! Yes, fun.

The way back was interesting. The wind had dropped, which seems to be the rule for Egg Island sessions. Without enough wind, the 90 l board becomes a submarine. Not a problem - if you don't want wet feet and knees, you should not windsurf. I opted for the easy way home, aiming for the Lewis Bay side beach at Kalmus. I even got a lucky gust that let me plane through the ferry lane. Submarine schlogging practice was fun, too. When I hit the wind shadow near the yacht club, I was almost on shore, and the swim was short. As warm as the water is, that was fun, too.

Yesterday?  Fun on a 6.5 in south wind - big, orderly swell, than flat. Fog to keep us cool. Fun.

2 days ago? Another Egg Island/Lewis Bay session, again on 6.5 (Nina on 5.0). Sailed 135 km, jibed 150 times. Fun!

3 days ago? SSE wind at West Dennis, high 20s, low tide. Fun waves outside, nice and flat inside. Nina on 4.0, I used a 5.0 Idol. Found that it worked well with the slalom board. Fun!

But today was the best day - lots of wind, lots of speed, lots of sun. And for the first time, my ankle did not hurt at all for the entire session. That means I have run out of excuses not to do freestyle. Bummer! Not.

Nina is following Brendon's advice and is hacking away at the Flaka, and nothing else. Sure, she'll throw in a Clew First Push Tack or a 360 every now and then, but those don't count because she can do them already. She's making slow but steady progress. Could happen any day now!

I'll leave you with a picture of today's GPS tracks:

GPS Speedsurfing with your phone

Can you use your phone for GPS speedsurfing? Some GPS pioneers like Roo have said "absolutely!" for a few years already. With the recent production stop of the "gold standard" GT-31 GPS unit, this question has become more urgent. Several replacements are on the horizon, but none are available yet.

So Manfred Fuchs, author of the widely used program GPSResults, has developed an app for Android phones for speedsurfing called GPSLogIt. It not only logs GPS data to a file for later analysis, but also shows results for the last and best run of the session, and even tracks:

As a geeky wanna-be speedsurfer, I had to try it, so I bought a cheap Android phone on Amazon.com (RCA M1 4.0 for $67), installed the GPSLogIt trial, and tried it out as soon as we got some wind. For comparison, I used my GT-31. Results were quite encouraging - most numbers were very close, although the long distance numbers and the alpha 500 where a bit lower with the phone. With the current trial (1.3), I was not able to read the numbers while sailing, or even when taking short breaks on the beach. Part of the problem was that I double-bagged the phone, using a ziplock bag inside a waterproof armband. I had to take the phone out of the armband at a shady place to see all the numbers and tracks. Future versions of the software will probably have a screen with bigger numbers that can easily be read while sailing.

After the initial promising results, a more thorough test was warranted. I decided to compare the phone with GPSLogIt to five GPS units I have: two Locosys GT-31s, two Canmore GP-102s, and the Flysight GPS. Here's a table of the results (raw data are here):

The GT-31 and the Flysight data include accuracy estimates (SDOP), so I included the +/- numbers as shown by GPSResults (for 5x10 seconds, I used a "typical" number for the 10-second runs).

All speed results obtained with the Android phone and GPSLogIt are within 0.6 knots of the numbers the other units gave. The 5x10 second numbers, which are often viewed as the most important top speed numbers, are within 0.1 knots, and well within the accuracy range of the GT-31 and Flysight units. However, the numbers for the nautical mile and especially alpha 500 are significantly lower. A close look at the doppler speed graph and the tracks provides some clues. Let's start with a large-scale view:

Mostly, the data for all 6 units are very close, but there are a few areas in the speed graph where the phone data drop below the other units. Here is a closeup:

There are several short areas in the track where the phone speed is artificially low; this explains why the long distance numbers are a bit lower than for the other units. It is fortunate that the error is on the low side - from the view of a competition, GPS units that understate true speed are acceptable, but units that overstate speed are not.

For the alpha, a look at the tracks when I drove around in circles is illuminating:

Blue: GT-31; red: Flysight; purple: phone & GPSLogIt

The tracks for the Flysight, which uses an UBX GPS chip and records at 5 Hz, is most accurate. The GT-31 tracks are close, but the phone tracks show some obvious deviations. Clearly, the phone had the most problems to accurately track direction changes. This may well be due to processing limitations in the GPS chip firmware that are related to limits on how much battery the GPS chip can drain. As it is, the accuracy is certainly sufficient for the common uses - navigation and photo tagging.

Overall, the GPS accuracy of the phone I used was certainly acceptable, albeit not "perfect". Other phones may use different GPS chips and give different results, but several others have also reported decent accuracy with other phones. Nice to have another option for tracking your windsurfing sessions - and one that's less than $100, even if you buy a phone just for this purpose!