HERE GPS, Bad GPS Health

#1

Morning all,
With a couple of aircraft we are getting the “Bad GPS Health” warning. I know that this comes from a less the 5Hz update rate (specifically a delta of greater the 250ms between messages) as the flight controller sees them.

I have identified two different symptoms which may or may not be related.

The delta raises above the 250ms limit if the satellite count on said GPS is above 18. Scoped the data lines when we have had this happening and the bus isn’t anywhere near full.


(from a bench test unit with a GPS that was showing the same issue in the aircraft)

And this odd occurrence where the message freq simply just halves.


Both the cable and the GPS unit has been replaced in the aircraft with the same results.
That GPS unit has shown the same slow message freq on a bench test cube that had a clean install of latest stable on it.
Message freq has been upped from the default 5Hz to 10Hz and once the sat count was over 18 there again was delays in the message freq over the threshold.

This has all been captured when not in flight. To continue with other testing we have relied on the second (CAN) GPS only.

I have run some more bench tests with a brand new HERE and a HERE2, the HERE shows the same halving in freq symptom but the HERE2 does not.

#2

Firmware? I would expect this to happen at some point with any receiver. The more sv’s the more data to transfer. I think you could raise the elevation mask and patch the issue. I do not mean to point out the obvious I was just thinking as I read your post.

#3

GPS’s are running 3.01 as from factory, bench test cube is arducopter 3.6.9.
I was initially thinking I could just not report on some of the constellations to reduce the satellite count but unless the root cause is to many messages then reducing the amount of data being transmitted doesn’t really fix the problem.
Scoping the serial line when the issue is actively causing pre-arm warnings shows roughly 30% bus usage so I’m pretty confident that is not the problem. The cube for same reason may not be phrasing the data fast enough or the data being sent is not readable/relevant.

We have been using cubes and HERE GPS’s for quite a while now and haven’t seen this issue before which made me originally think it may have been a hardware issue with these specific units.

I am going to run an extended test on the HERE2 to confirm if it does or does not show the same issue after a certain time period.

#4

Are you running CAN or serial?

#5

The Zubax GNSS 2 that is the second GPS in the aircraft is CAN, both the HERE and HERE2 are being run with serial. The bench test setup only has one GPS on it at a time to narrow down the variables at fault.

#6

Left there HERE2 attached for a couple of hours yesterday and had none of the previous issues.
Sat count never got as high as previous test so a little inconclusive given the >~20 satellite then it fails hypothesis.

–Will update post with similar log from the HERE unit once I get the log–

#7

Found an original HERE to test as well as the new HERE. Difference externally is the LED lenses are frosted white on the early ones and clear on the latter ones.
I did this to test weather the fault was present with both units, in the past using the original units we had not seen this fault before.
Both units where set up exactly the same and run through the bench test. Both units failed with the same Bad GPS Health message which only occurred once the GPS count was greater then ~20.
Looking through historical logs to see if this had been happening to any degree in the past I discovered the GPA Delta field was not a field that was recorded until recently.
I’m leaning towards the conclusion that the GPS message freq has always suffered when the satellite count got high, but until recently the FC did not act on that metric and just continued as if nothing was amiss.

To work around this I limited the GNSS channels in the HERE GPS (using U-Centre) to 17 which is the minimum that can be set.
After this change I have not had any Bad GPS Health messages but will continue to monitor.

This doesn’t solve the root cause of the problem as the GPS module and FC should be compatible out of the box and nerfing the GNSS count is just limiting otherwise useful functionality. On the flip side “it
doesn’t, not work” now.

#8

In u-center the default for the elevation sv mask is 5 deg. That is far too low use 10 deg min, I use 13 deg. The data from the low sv’s will not help position mostly because of ionosphere interference.

#9

@jschall @Michael_Oborne

#10

do you have raw logging or something else enabled in the ardu firmware?

given you are running serial, my guess is your flooding the serial link, based on one of the ardupilot settings