The Milk Jug Experiment

 My last post ended with this graph:

Before I tell you what I did to generate it, let's first look at the "why". Here's a short section from a recent windsurfing session that illustrates the problem:

I compared three different GPS units in this test. At the highlighted section, the three GPS units disagree from each other by roughly 0.5 knots for a few points in a row. Areas like this are easy to find in many GPS traces - but what causes them? The GPS units can be much more accurate that this, as stationary tests show, where typical errors are about 10-fold lower. 

One potential cause of the larger errors are short-term changes in the satellite signals received by the GPS; specifically, changes in the signal and/or noise for one or more satellites, and in which satellites the GPS uses to compute speed and position. So the experiment was to induce controlled changes, without moving the GPS, and see what effect they had on the observed error (the speed, since the GPS was stationary) and the error estimates given by the GPS.

To disturb the signal, I took a one gallon milk jug and filled it with water. I then moved it around the GPS and on top of the GPS, keeping a distance of at least a few centimeters, for about 30 seconds. I did that twice, with 30 second control periods where I kept the jug away before, in between, and after. The periods where I moved the jug around the GPS are highlighted in the first graph.

The speeds that were reported by the GPS because of the distorted satellite signal were around 0.5 knots multiple times, and near 0.8-0.9 knots a few times.  I was a little surprised to see such "speeds" just because the signal was partially blocked - after all, the GPS should still have been able to get a perfectly clean signal from most of the 20-25 satellites it was tracking. But apparently, having just a few signals removed or distorted is sufficient to cause errors in the range that we can often see in windsurf tracks.

Now I don't usually windsurf closely to people waving milk jugs, but it's just an example of a sudden change in the satellite signal cause by external factors. During windsurfing, that could be something as simple as changing the position of the arm that the GPS is on, or the body position, so that a different part of the sky is blocked by the arm or the body. The more things move around, the more likely that is to happen - and if you don't have the luxury of windsurfing at a flat spot like Lake George or La Franqui, chop might just do the moving for you.

In a similar experiment, I looked at what happened when moving the GPS up and down rapidly, and the results looked similar, even when looking at directional speeds (velN and velE). But my lovely wife pointed out that I could not really be sure that I was moving the GPS straight up and down, without side-to-side movement, so this experiment would need to be repeated in a more controlled fashion. For now, I'll just say that it is possible that rapid up-down movements of the GPS, which are typical when sailing in small to medium chop, might also have an effect on GPS accuracy.

One interesting question arises when comparing the results from different GPS units that are very close together, and technically identical (or very similar). They both should "see" the same distorted signal, so why would the not compute exactly the same result?

Rather than answering this question directly, or speculating about why this may not be the case, I'll show a graph from a stationary experiment where I had two identical units positioned very close to each other, with a clear view of the sky:

One interesting result shown in the table above is in the "sats" column, which shows how many satellites each unit tracked. Since the units were very close to each other and technically identical, it is reasonable to expect that they would use exactly the same satellites. But at the start of the highlighted region, one GPS used 17 satellites, while the other used 20 - that's a pretty substantial difference! Here is a graph of the satellites tracked by the two Motion units over a time of about 1000 seconds (starting a few minutes into the test, after both units showed the same number of satellites):

For large parts of the tests, the two GPS units differed in how many satellites they used - which means that blockage or distortions of one or a few satellites could affect the two units differently, thus possibly leading to comparatively large discrepancies in the results.

How does all this relate to speedsurfing, you might ask? The blocking experiment is an example of non-random noise in the data. If you look back at the first graph, you may notice that the error estimates increase much less than the speeds. This means that there are multiple regions where the error estimates are significantly lower than the observed errors for more than 10 points in a row - here is an example:

Statistically, we would expect to see some points where the speed is higher than the error estimates, but we should practically never see this many points where the error is larger than the error estimate in a row - if the error is random. But if the error is not random, then we can not use Gaussian error propagation, which makes collecting at high data rates entirely pointless.

GPS Noise and Data Rates

I'm noise sensitive, so perhaps it is quite fitting that I spent some time looking at "noise" in GPS units, and the relation between noise and data rates. Between the relatively cold water around Cape Cod and Cape Cod's dubious distinction to be the hot spot for the Brazilian P.1 variant of the COVID virus, on-the-water testing will have to wait a while. So the tests I report here are all stationary tests.

One big advantage about stationary tests is that we know exactly what speed the GPS should report: 0.00. Everything else is an error. As Tom Chalko did with Locosys GPS units many years ago, we can use this to learn about the error characteristics of GPS chips. There's one little caveat about the directionality of speed in speedsurfing, and the non-directionality of measured speed when the actual speed is 0, but we can ignore this for now.

For this series of tests, I am using a Sparkfun Neo M9N GPS chip with an active ceramic antenna (25 x 25 x 2 mm from Geekstory on Amazon). I'm logging either via USB to the a Surface computer, or with the Openlog Artemis I described in my last post. Compared to the M9 chip with the onboard chip antenna I used before, the current combo gets a lot better reception.

Let's start with an utterly boring test, where the GPS had a perfect, unobstructed view of the sky on a sunny day (the GPS was on top of the roof rack on our high roof Nissan NV van):

At the beginning, there's a little bit of speed when I switched the GPS on, and moved it to the top of the van. I had used the GPS just a few minutes earlier, so this was a "hot start" where the GPS very quickly found more than 20 satellites. After that, it recorded a speed close to zero (the graph on top), with an error estimate around 0.3 knots (the lower graph).

Let's switch to a more interesting example. The next test was done inside, over a period of almost 2 hours. The GPS was positioned right next to a wide glass door, so it had a clear view in one direction. Here's the graph:

At three different times, the GPS recorded speeds of more than 1 knot, even though it was not moving at all. With a typical estimated accuracy of about +/- 0.5 knots for each point, that number is actually a bit lower than expected. But what raises some red flags is that each point with a speed above one knot is closely surrounded by several other points that are also much higher than average. This is reflected in the top speeds averaged over 2 and 10 seconds: 0.479 knots and 0.289 knots. In fact, all top 5 speeds over 10 seconds are near or above 0.2 knots.

Let's look at one more test run - this one done at 8 Hz overnight, for about 10 hours:

The GPS was at exactly the same spot for this test. The overall results is similar, with a bunch of spikes between 0.5 and 0.9 knots. The top results for 2 second and 10 second averages are a bit lower, but we still see a couple of 10 second "speeds" of 0.2 knots.

Now wait a minute - I just said that the recording at the 3-fold lower data rate that covered a 5-fold longer observation period had lower observed errors. That is exactly the opposite of what we would expect! At first glance, the errors appear random, or at least mostly random. For random errors, statistic tells us that if we measure more often, the measured error will go down. Going from 25 samples per second down to 8 samples per second should reduce the observed error by about 77% - but that's not what we see!

The explanation for what we see is that the error is not entirely random - in fact, it has a substantial non-random element. That's quite obvious in the 25 Hz graph, where we can see a wave pattern in the error estimates, and clusterings of high speed points when the "error wave" is higher.

To have a closer look at the (non-)randomness of the observed errors, I copied the data from GPS Speedreader into a spreadsheet program, and then shuffled the observed speeds randomly. Next, I looked at all 2 second periods in the shuffled data, and compared the results to the results from the original data set. With completely random data, shuffling should not have an effect - we'd see the same top speeds. But if the original reported speeds (= errors) were not randomly distributed, we should see a difference. Here's a screen shot that shows the results for a test recorded at 18 Hz:

The average 2 second speed is 0.062 knots for the original and the 4 different shuffle tests, as expected. But in the shuffled sets, the maximum observed 2-second speed was between 0.093 and 0.097 knots - more than 3-times lower than in the original data set. Essentially, this proves that the error in the data set was not randomly distributed.

For comparison, here is the same analysis for a test recorded at 5 Hz:

For the 5 Hz data, we also observe a difference, but it is much smaller: the original data had a 2-second maximum that was about 50% higher than the shuffled data sets.

In my next post, I'll look into what's behind the non-randomness of the error. However, I'll leave you with a little puzzle and show the results of one of the tests I did:

Can you figure out what I did here? Here are a few hints: it involved a gallon plastic jug filled with water, and the numbers 2, 3, 5, and 30. Good luck, Sherlock!

Olapap: Why We Need Ceramic Antennas

 This is a post about the Olapap GPS, a "plug and play" GPS logger put together with the Openlog Artemis (OLA), u-blox GPS chips, and a couple of cables. I'm sharing some windsurf test results that I have obtained with the "SparkFun GPS Breakout - NEO-M9N, Chip Antenna" board. This board has a powerful GPS chip (the M9), but a GPS antenna that is quite weak - similar to antennas used in phones.

I compared the "Olapa M9 chip" GPS to three other GPS devices that have been approved for use in the GPS Team Challenge:

1. A Motion GPS (the original one with a screen), worn on my upper right arm.
2. An Openlog prototype that uses the Beitian BN880 chip, worn on top of my helmet in a GoPro housing.
3. Another Openlog Prototype that uses the (discontinued) Beitian BN280 chip, worn in a waterproof armband on my left arm

The Olapap GPS was in the same armband as the BN280 GPS, slightly above it. Both of these prototypes were in separate ziplock bags, in case the armband developed a leak. 

Here's a screenshot from a GPS Speedreader comparison of the top speeds for the 4 units:

If you look at the top row that shows the fastest speed over 2 seconds, you see that the three approved GPS devices all show 31.3 knots; they differ only in the second digit after the period. But the Olapap M9 chip GPS (in the last set of columns) shows a top speed of only 30.487 knots - almost a knot slower than the others! Furthermore, the M9 chip GPS also reports the highest error estimates in the "+/-" column: 0.85 knots, compared to 0.139 knots to 0.239 knots for the other units.

Let's have a look at the speed graph here:

The curves for the three approved devices (in blue, red, and green) are all close together, with just small point-to-point variations that look random. But curve for the M9 chip GPS (in magenta) looks quite different, sometimes being lower and at other times being higher than the approved GPS units.

When we look at the data points for this section, we can identify a likely cause of the problem:

The columns with the rectangles show the number of satellites that the GPS units used: about 18 for the Motion, 24 for the GPS on top of the helmet, 20 for the BN280, and only 12 for the M9 chip GPS (which was very close to the BN280, in the same armband).

The differences between the first 3 GPS units are primarily due to the GPS position: the unit on the head had the clearest view of the sky, and was able to use 4-6 more satellites than the units worn on the arm. The lower number of satellites for the M9 chip unit is due to the smaller, weaker antenna. I did a number of tests where I looked at the signal-to-noise ratios for the satellites in the Ucenter software, and the values for the M9 chip antenna were always significantly worse than for other GPS units. No real surprise here - it is common knowledge that larger ceramic antennas provide superior GPS reception.

The end result of this test is that the NEO-M9 board with the chip antenna does not provide data that are of adequate quality for speedsurfing. Since there is no easy way to replace the antenna, I returned the chip to Amazon. I have started looking at the Sparkfun SAM-M8 chip that has an older (M8) GPS chip, but a larger and more sensitive antenna, and will probably add other M9 chips with external antennas in the future. The initial M8 results look promising, but more test sessions on the water are needed. I'll post results here when I get them.

The Olapap GPS

 This is a geeky post about making a GPS. I think it might be of interest to about five windsurfers in the world. If you're not one of them, I suggest you find something else to read. If you are not curious about what the picture shows, you are definitely not one of them!

Why make another GPS?

That's a good question. The short answer is that it is very hard to get a good GPS for speedsurfing competition - specifically, for the GPS Team Challenge (GPSTC). There are currently two GPS devices you can buy that are approved for the GPSTC. The first one is a watch: the Locosys GW60. Unfortunately, it has a few annoying habits, like wrist bands that just fall apart without any warning; buttons that you should NEVER press when the watch or your fingers are wet; and batteries that die if you don't use the watch for a while, and that require advanced soldering skills of you want to replace them. 

The second one is the "Motion Mini" logger. It's a fantastic little device, but very hard to get. Some people have waited many months to get their order, which is hand-assembled by one guy who can't keep up with the demand. He's also loosing money on his quest to provide a great GPS, so we don't know how long he'll be able to keep making the Mini Motions.

The question about alternatives comes up on a very regular basis on the Australian speedsurfing forum. Unfortunately, many GPS watches out there work well enough most of the time, but don't provide any accuracy estimates that allow software to automatically identify artifacts. Within the context of a competition where about 500 results determine the ranking every month, that's not good enough.

I have written about making a simple little GPS logger with a u-blox GPS chip, an Openlog datalogger, and a couple of other bits and pieces in the past. While this thing was easy enough to build, and is at least as accurate as "approved" devices, it requires some soldering - which means most windsurfers won't even consider making one. But what if we can make a good GPS buy just buying parts and putting them together with a cable or two? That seems to be possible now!

Olapap GPS? Really?

Well, I had to give the thing a name, right? After many hours of searching copyright databases (just kidding!), I based the name on a critical component and the basic idea behind it. "Ola" stands for "OpenLog Artemis", the data logger part of the device. "PAP" stands for "Plug And Play" - no soldering! 

Parts needed

The Olapap GPS logger requires exactly three parts and one cable.

1. The Openlog Artemis from Sparkfun ($50). This is a nifty little data logger, nicknamed "OLA",  that can write to a micro SD card. It's similar to the Openlog I have used in my previous prototypes, but with two major difference: a much more powerful processor, and a "Quiic" connector to hook up a GPS chip. The Artemis processor on the chip is much faster than the one used by the old Openlog, and has several hundred times more memory. No more trying to optimize every single byte of memory and every CPU cycle! Sparkfun show the device on backorder right now, but you can probably find one on Amazon or your favorite electronics supplier.

2. A u-blox GPS chip with a Quiic connector from Sparkfun ($40 - $70). The u-blox chips are the only generally available GPS chips that (a) use Doppler for speed calculations, and (b) provide estimates of the speed accuracy based on the quality of the GPS signals received. We need one with the Quiic connector so that we can hook it up to the OLA, and Sparkfun offers a bunch of different types. So far, I've used two of them:

- The SAM-M8Q board ($40). This is an older (8th) generation GPS chip with a ceramic antenna that seems adequate. It is what I am currently using.

- The NEO-M9N board with a chip antenna ($70). This is a newer (9th) generation GPS chip that offers a higher data rate and use of the Chinese BeiDou satellite system, two functions that could lead to more accurate speed data. However, the chip antenna on this board is very small and inferior to the ceramic antennas that are typically used. In my tests, the reception quality of the antenna was so poor that the speed data were inaccurate. If you want to use an M9 chip, I strongly suggest that you get a NEO-M9 board with an antenna connector instead - either the Sparkfun board with the U.FL connector (which is a bit tricky to use!) or the Sparkfun board with the larger SMA connector. 

3. A battery. I am using a 1 Ah Lithium Ion battery from Sparkfun, but a smaller or larger battery should also work. If you order a battery from a different supplier, make sure that is has the correct polarity - some have the cables connected the other way around, which will kill your electronics!

4. A Quiic cable to connect the OLA and the GPS chip. I got a set of cables from Amazon.

5. A micro SD card (32 GB or less). Make sure to get a class 10 or better card!

6. A USB type C cable.

Getting started

Connect the pieces together (check the hookup guide at the Sparkfun site), press the reset button on the OLA - and voila, you start logging GPS data! But we're quite there yet, since the data will be logged as text files, with some of the essential data points missing. We need binary data (".ubx" files)!

Fortunately, the Sparkfun tutorial provides a link to a firmware specifically for GPS logging. Updating the firmware is pretty easy with the Sparkfun firmware update application. If you are using an M9 logger, you're all set to create .ubx files with the required "NAV-PVT" sentences. You just need to edit a settings file on the micro SD card that the OLA creates the first time it starts after installing the GNSS firmware to increase the logging rate to 5 Hz (or higher).

If you're using an M8 chip, though, you are a bit out of luck. The GNSS firmware only works with the newer M9 chips, not with the SAM-M8 chip I listed above. But the firmware is open source, so modifying it is straightforward. Well, at least if you know how to program in C/C++ and know your way around Arduinos. I've done both in the past, but rarely and reluctantly, so it took me quite a few hours to get things to work properly. The current version of my modified GNSS firmware is not pretty, but at least is works with the SAM-M8 chip. 

The next steps will be some more on-the-water testing of the prototype to see if the accuracy is adequate, and perhaps some testing with an M9-based GPS chip and an external antenna. If the results are positive, I'll clean up the modifications I made a bit and post the firmware so that anyone else interested can download and use it.