More Noise and Errors

Perhaps I should apologize to 95% of my windsurfing friends for this post. It's another geeky post about analyzing GPS data. But I won't. Nobody forces you to read this post. If you do not care about important things like whether or not high-frequency GPS data are really better, stop now. Go windsurfing or watch TV, or whatever.

I recently posted here about noise in GPS data, specifically about the spikes we see in 5 Hz data from  Locosys GW-52 units. This led to a discussion on the "GPS and Speed talk" section of the Australian windsurf forum, where I posted this picture (click on it to enlarge):

This is a screen shot from the GPSResults program where I analyzed GW-52 data. On the left, the data were collected at 5 Hz (every 0.2 seconds); on the right, the data were collected at 1 Hz (every second). The most interesting thing about this picture is near the bottom: the error estimates (+- numbers) for 10-second runs were significantly lower in the 1 Hz data. This was unexpected. Conventional wisdom says 5 Hz data are more accurate.

On the forum, sailquick quickly pointed out that the error estimates should be very similar, since (as far as we know) the 1 Hz data are simply averages of the 5 Hz data. So I looked at this again. Much to my surprise, I now got different error estimates for the 1 Hz data:

Now, the +/- numbers were about 60% higher! Why would I get different results this time?

It took me a few hours to figure this out. The error estimate numbers in the lower picture are simply the average of the data point errors for a given run; in the upper picture, the numbers are significantly lower. Looking at the GPSLogit web site, I discovered that the Windows version of the program has two options for calculation the error estimates for runs ("Error Propagation"): "Average" and "Gaussian". I also noticed that the program would automatically adjust some "filter" parameters every time I loaded a new file. For example, the "Max SDoP" number would be set to 1.5 for 1 Hz data, but to 3.0 for 5 Hz data. It seemed that the software also changed the setting for "Error propagation"! Unfortunately, the Mac version of the program does not have any way of checking or changing this - only the windows version does. For Mac users, this is an entirely hidden change in the way the analysis is done. I must have accidentally changed the setting by switching back and forth between different GPS files.

So I installed the Windows version of GPSResults, and looked at the files again. After loading a file, I went to the "Extras" menu, selected "Filter Settings..", and checked the "error propagation" settings in the  dialog. Sure enough, whenever I loaded a file with 1 Hz data, "Average" was selected; but when I loaded a 5 Hz data file, "Gaussian" was selected.

So what happens if we analyze both data sets with the same settings? For "Gaussian" error propagation, we get basically the same results as shown in the first figure. For "Average", this is what we get:

The estimate error for 10 second runs using averages range from 0.75 to 1.5 for 5 Hz data range, and from 0.19 to 0.35 for 1 Hz data. The conclusion is clear:

When analyzing 1 Hz and 5 Hz data from the GW-52 using the same math for both data sets, the 1 Hz data are more accurate.

This raises the question why GPSResults uses different default settings for 1 Hz data and higher Hz data. The "Average" error propagation is really a "worst case" model: it assumes that errors are all in the same direction - that all speed measurements in a given period are either too low or too high. The "Gaussian" error propagation assumes that errors are random, and cancel each other out to some extend. Under the Gaussian model, the error gets smaller the more measurements we take; this is a primary reason behind the drive to higher data acquisition rates. 

When the error estimates were initially developed, the quality of GPS chips and units was a lot worse than it is now; back then, a Gaussian model may have been too optimistic for routine use, which would explain why it still is the default for 1 Hz data. But when newer, better GPS units with higher Hz rates became available, this was no longer true, and the Gaussian model was needed to get more accurate estimates of the speed errors. Hence it became the standard for high frequency data.

So is there a practical relevance of this for most speedsurfers? Not really. How the errors are calculated has no influence on the speed numbers, unless you are going for "official" records. Most of us never look at the error numbers; you cannot even get error estimates in two of the most popular ways of analyzing GPS speedsurfing data (GPS Action Replay Pro and ka72.com).

Perhaps the most practical consequence is for GW-52 users: there is no harm done if you record your speed sessions at 1 Hz. If we put any faith in the error estimates at all, then the 1 Hz data are more accurate than the 5 Hz data - at least for the GW-52. For other GPS units that have better antenna, use better GPS chips, and/or have better interference shielding, the opposite may be true - but no such units are currently commercially available.

--
Added a few hours later:
I went back to some windsurfing tracks to see what the calculated errors are in 1 Hz and 5 Hz data when using the Gaussian error propagation in GPSResults. I looked at three sessions each, and the 8 fastest 10 second runs in each session. Here are the data for the 1 Hz sessions:

 The average error for the 24 10-second runs is 0.15 knots, with a standard deviation of 0.037.

The 5 Hz data have slightly higher error estimates:

The average is 0.193 knots, standard deviation 0.025.

Based on just these 6 sessions, the 1 Hz data are slightly more accurate, although the difference is not statistically significant.

Interestingly, the Gaussian error propagation numbers do not seem to follow the formulas given in Tom Chalko's paper from 2009 ("Estimating Accuracy of GPS Doppler Speed Measurement using Speed Dilution of Precision (SDOP) Parameter"); instead, the error estimates are about 2x higher than expected. But perhaps I am missing something here.

Upwind 360

The planing upwind 360 is a pretty basic trick. Many ABK campers who can do it in light wind learn it in a day. Not me. I worked on it during several camps, and never got a dry one. Until today!

I really like this move when Andy Brandt does it the "sail forward way" - keep the sail luffed in front, instead of moving it to the back. Looks so cool when he snaps around!

I had a couple of tries that I water started out during the last few sessions. Today, after the wind had calmed down a bit from "Surprise!" 28 mph averages, I got a dry one. And another one right after. Cool! Of course, the camera battery had given up by then. The image above is from an earlier try that ended wet. Maybe I should have charged it after the last session.

I knew I was close, but I think it helped to watch Mike Burns doing Flaka's with a mast mount camera yesterday evening (no, I'm not working on Flakas - that's Nina!). Very nice of him to demo such an "easy" move for Nina. I had the image of his fully extended front arm in my mind today, and that helped. He also was the wind sacrifice today. Forecast was 10, we got mid- to high 20s for a while. Many thanks, Mike!

Dead Toys and Lots of Noise

This is a two-part post. If you're a geek, keep reading to the end. If you just want an update about how things were recently, by all means stop after the first part.

Last Friday had looked so promising in the forecast: wind all day, with mid-20s in the afternoon! Sunny and warm! But it started badly enough, by giving me the finger - a blue finger:

I am still thinking I should sue Chinook for failing to put a warning label on their booms that reads "Remove fingers before closing". Fortunately, it's been too windy to find a lawyer. There's no fun to be had in court rooms!

Briefly, here are the other things that happened:

• My GoPro broke. Dead. Kaputt. 
• When the chop and crowds became unpleasant in front, I sailed to Lewis Bay on my Skate 110/Idol 5.6 combo, and loved the flat water I found. Came back to switch to slalom gear and drag Chris along. Had an absolutely crazy, overpowered, out-of-control ride through nasty chop to make it back to Egg Island - the wind had picked up to averages above 30. Chris promptly declared Lewis Bay as "absolutely unsailable" and sailed back, leaving me alone. Considering that I had barely made it there, I had doubts that I'd make it back, so I stayed. At least there were no crowds. But there was the security guard now posted at Egg Island that makes sure nobody but birds set foot there. 
• My Android phone that I use to announce my speeds through GPSLogit got wet and broke. This was the first time I had not double-bagged it, and also the first time I did not close the waterproof bag properly. It's terminally dead. Drying did not revive it. 

So, here's a pop quiz: what do I call a day of sailing like that? Think about it for a minute. I'll even post a picture to create some distance from the answer.

The answer is (of course): A great day of sailing!

In the morning, I had a few hours of freestyle on what I count as a small sail. I even made one or two wet Upwind 360s - the "Andy Brandt" variety, where you leave the sail forward when you carve, and backwind before loosing all speed. Still had to waterstart out of it, but getting closer.

In the afternoon, I was eventually joined at Egg Island by my lovely wife, who was nicely powered on 3.7. That made me feel a bit better about thinking that my 5.8 was a tad big, especially since the wind had dropped by then. I watched here hack away at the Flaka, with a few good tries. She now has tried Flakas on sails from 3.7 to 5.6 m .. impressive. I still think she'll get her first Flaka before the Cape Cod ABK camp in September.

I did not try anything, but practicing slalom sailing was fun, and I later got some perfectly flat spots for full-speed jibes - so cool! I ended the day with a final session on my 3S 96 with my Idol 5.0 in front, bumping and jumping. 125 kilometers of fun! Who would complain about a blue thumb and a couple of broken toys?

But I wanted to replace the GPSLogit phone, and picked up a Samsung Galaxy J1 for $30 at the local Best Buy. First order of business was checking how well it would work in comparison to the "standard" GPS units.

Warning: here begins the geeky part of the post. Continue reading at your own risk!

So I taped the phone and two Locosys GPS units to the dash board of our Nissan NV 2500 high roof van, and took the babies for a ride. The GPSLogit speech announcement worked as expected, and the speeds always agreed well with the speedometer. I even stopped in the middle to switch the GW-52 from 5 Hz recording to 1 Hz recording, just to see what happened.

Let's start by comparing the phone data with the GT-31 data. Here's a part of the doppler speed graph (click on the image for a large view):

GT31 (blue) vs. Samsung Galaxy J1 with GPSLogit (red)

The first impression was "pretty good", but a few small spikes are visible in the phone data. Here's an enlarged section:

The red phone data are a bit noisier, and have a few spikes that may overstate the speed by a knot or two. Indeed, the speed results in the GPS Team Challenge categories showed that the phone speeds were a bit higher than the GT-31 and GW-52 speeds (up to about 1/2 knot) - something that was seen before for other phones. But while the accuracy is not good enough for posting to the GPS Team Challenge, it's definitely good enough to hear how fast you are going while you're sailing. Great!

When I compared speed results from the GT-31 and the GW-52, the differences were minimal, as expected. However, there was a very noticeable difference in the 5 Hz data from the GW-52, compared the the 1 Hz data from the GT-31:

1 Hz GT-31 data (blue) compared to 5 Hz GW-52 data (red)

The GW-52 were collected every 0.2 seconds, so we have five times as many data points as for the GT-31 data which were collected every full second. The GW-52 data have a lot more spikes - little ups and downs. That had been reported by others before for windsurfing data, and the question arose whether these spikes are real differences in speed, or noise that was caused by the higher measurement rates. We'll get back to that question in a minute, but let's look at a comparison when the GW-52 unit is set to 1 Hz:

1 Hz GPS data from GT-31 (blue) and GW-52 (red) 

The data look very similar. 

So, let's have a closer look at the 5 Hz data. There are valid arguments that can be made why collecting data at a higher frequency is better. If (a) the errors in the speed measurement are purely random, and (b) the error per data point is the same at 1 Hz and at 5 Hz, then measuring at 5 Hz would give about 2.2-fold higher accuracy. But this is only true if both of these conditions are met.

Let's look at the data points (I used GPS Action Replay's "Trackpoint Table" to create the screen shots):

GW52 "raw" 5 Hz data from a small subregion

For comparison, here at GT-31 data from the same region:

GT-31 data (1 Hz)

A picture says more than 100 words, so here is the doppler speed graph for this region:

1 Hz GT-31data (blue) and 5 Hz GW-52 data (red)

When we seen spikes like in the red curve about in windsurf data, we may believe that they are real: we constantly hit chop, gusts, and lulls, and it seems quite possible that our speed changes several times within a second. But the data above are not from windsurfing, but from driving a big, heavy van.  The Nissan NV2500 High Roof van is quite a beast. It weighs in at about 3 tons (6250 lb), and we've loaded it up with interior racks, 6 boards, 15 sails, masts, booms, etc. Does it change speeds several times a second like the GW-52 data indicate? Allow me to cite from The Princess Bride:

"Inconceivable!"

We can get a bit more formal in analyzing this. We can start by looking at the acceleration - the change in speed from data point to data point. The acceleration is shown in the tables above as "Linear m/s2". In the GT-31 data, the values change little from data point to data point; but in the GW-52 data, the acceleration jumps from 0.4 to 2.9 in 200 milliseconds! That's almost no acceleration to 1/3 g. I'm sorry, but I don't drive that crazy!

The change in acceleration is actually a pretty good indicator of noise in the data. It's pretty easy to calculate, and if you're mathematically inclined, you can call it the second derivative (the acceleration being the first derivative of the speed). Here's a graph of the acceleration and noise for GW-52 data:

For comparison, here is what this looks like for the GT-31 data:

That's a lot less noise in the G-31 data! The 1 Hz data do reflect reality better: I accelerated somewhat evenly over about 15 seconds, kept the speed more or less constant for about 1/2 minute, and then slowed down constantly.

This is some rather clear evidence that collecting GPS data at higher frequencies does indeed introduce addition measurement error, which negates any potential advantages of the additional data points at least partially. More data is just that - more data. It's not automatically better data. Nor does a lower error number that some software spits out necessarily mean that the data are indeed more accurate - if basic underlying assumptions are invalid, then the error numbers will also be invalid.
--
Some of my readers may wonder: "Who are you that you think you can analyze this, and go against what everyone else says?". Well, that is a valid question. I admit that windsurfing is my primary addiction, but I have some scientific background. I got a M.S. in Biophysical Chemistry, a Ph.D. in Experimental Sciences, and have worked in scientific data analysis and software development for Bioinformatics for last 3 decades. I certainly enjoy developing algorithms for large-scale (and small-scale) data analysis... especially when it's not windy. 

Kalmus in NE

The picture above shows Kalmus on July 3rd. At times, there were close to 30 windsurfers on the water at the same time. Over the course of the day, more than 50 windsurfers sailed. Almost like old times.

Yesterday had good wind, too, but nobody sailed. Yes, it was Friday, instead of the middle of a long weekend. Yes, it was cloudy instead of sunny, and air temperatures where in the mid-60s instead of the mid-70s. But the bigger difference was the wind direction: north to northeast instead of the usual southwest. That's straight offshore at Kalmus. Even the iWindsurf meter readings looked gusty:

Lulls below 10, gusts in the mid-20s? That sounds like the Gorge, not like Kalmus!

But we know that wind meters can be deceitful. No way it would really be that bad! But gusts near 25 - there's a reason to go windsurfing! So we went.

The beach was almost empty - two or three vacationers braved the "cold", and no life guard was to be seen when we checked out the wind. Not many white caps to be seen, either, but my wind meter showed averages of 17, and gusts above 20 - good enough. By the time we had rigged, the life guards had shown up, and informed us that we absolutely could not launch on the Lewis Bay side. It did not matter that there were about 800 feet of completely empty beach next to the swimming area - "windsurfers must launch in front".  So we did, and sailed over to Egg Island. Here are my GPS tracks:

You may notice that I was planing the entire time. The lulls in Lewis Bay were nowhere near as bad as near the wind meter. The wind was a bit up and down, from just powered to slightly overpowered on my 6.5 - better than I had expected, and plenty of fun. The chop was small, mostly orderly, without any of the voodoo chop Kalmus is famous for.  The tide was high, so the inner sandbar was submerged completely, allowing runs that were the same length as typical runs in front of Kalmus beach. I played around with a few freestyle tricks, only to discover that the smaller (5.6) freestyle sails I have been able to use recently spoiled me a bit too much to enjoy wrestling the 6.5. Nor did it help that the chop was just big enough to keep the speed down a bit, and that gusts seemed to hit me every time I wanted to oversheet the sail. I kept thinking that slalom gear would have been the better choice for the conditions ... next time!

This weekend will see cloudy and colder weather with a chance of rain, but not much wind. But the forecast currently predicts a couple of nice southwest days for the middle of the week - see you on the water!