DUAL Here3+ Prearm: GPS2 Not Healthy when connected them on SAME CAN PORT

Hi @philip @Michael_Oborne
After long iteration i have just found Dual Here3+ connected on Single CAN port(either CAN 1 or CAN2) on any flight controller which introduce the Prearm ERROR :GPS2 NOT HEALTHY even HERE3+ power was isolated.

TEST 1:
below link is the already discussed topic where the Philips Know the issue.
here i wanted to discuss deeply about this issue that is the reason i have created new topic.

TEST 2:
Then i come up with CUAV Neo V2 pro ( CAN GPS) dual Gps connected with Cube orange+ mini carrier board CAN2 port. as a result it doesn’t report the any prearm error except compass Inconsistent which i didn’t do the proper Compass calibration on it.

here is the GPA delta graph for Both the GPS:

here is the LOG_Disarmed log file for that :https://drive.google.com/file/d/1OL7Yno3bZRh2E_CaLHV91bsPXptzT5u8/view?usp=sharing

TEST 3:
Dual Here3+ connected on CUAV V5+ CAN port 1 as a result again prearm:GPS 2 Not healthy which similar to the TEST case 1 with Cube orange+ mini carrier board.

Here is the GPS delta graph for the both the GPS:

here is the Log file for the same:CUAV V5+ with Dual Here3+.bin - Google Drive

TEST 4:
Dual HERE3+ connected on standard carrier board CAN1 Port or Dual Here3+ connected on standard carrier board CAN2 port both of them resulted Same prearm error :GPS 2 Not Healthy.
where as connected them on each CAN port single GPS doesn’t have this problem.
sorry i forgot to enable the log for Standard carrier board.

Hence I suspect there is issue in the HERE3+ GPS.

Hi,
I was able to connect two Here3+ on a single CAN Port (CAN1), using Cube Orange+ and standard Pixhawk Carrier (but on ArduPlane firmware). I tested RTK operations and rover only operations. Please find example log attached.
I can do more tests if you need more details.
Cube_Pixhawk_DualHERE3_singleCAN.BIN

Can you test it with arducopter firmware on both CAN 1 and CAN2 ports individually with dual here3+ gnss configuration.

Do you have Mini carrier board?
I m very curious to get test results on mini carrier board.

One of my friend drone also facing similar issue with mini carrier board and dual here3+ .

i think you too had same issue where it hided by not changing flight mode to GPS aided mode like loiter, guided or auto etc…there have been Multiple failsafe function was triggered which may be the reason you could not able to see the GPS not healthy issue. when you clear all the failsafe function then you came to know the actual error still present.

here is your GPA delta for both gps and you can see the GPS 2 is not maintain the GPS2_RATE_Ms = 200 ms.

1 Like

Try to improve the GPA.Delta by setting
GPS_GNSS_MODE,5 or GPS_GNSS_MODE,65 by limiting the constellations to two.
If you have SBAS then also add that, usually only in USA.

I was also used that one too in here3+ .

I have also been experiencing this issue with duel here3+ along with a cube orange+ with a standard carrier board. If the two GPSs are plugged into different ports there is no problem, but if they are plugged into the same port then I have the not healthy issue.

I noticed that in the UAVCAN inspector the UAVCAN_EQUIPMENT_GNSS_FIX2 would often fall below 5Hz for the node ID of 121 as shown here:

To eliminate any chance it is a flight controller or mission planner issue I used a slcan tool and the Drone CAN gui to inspect the bus by subscribing to the message. I found that most of the time the message comes through at 5 Hz, but there are spurts where messages stop coming through and you can see the time delta of the messages is higher than expected. (they should be at 200000).

Finally I checked the Bus monitor to see if I could see what was causing it. I found that there were missing frames on the uavcan.equipment.gnss.Fix2. The missing frames were causing the message to not be decodable. See the message: “Transfer could not be decoded: EOT not found”. This was only ever happening on the uavcan.equipment.gnss.Fix2 message sent from the GPS with node ID 121. The other thing to note is that it happens quite a few times in succession when it does happen. I have screenshots of 3 successive incidents of this happening. The fact that it only every happens on the one message from the one node makes me pretty sure it is not a problem with the tool.



This leads me to believe that the problem could be happening in 2 places:

  1. It is a bug in the Here3+ that causes it to terminate sending long DroneCAN messages when a there are collisions on the bus (possible due to the node ID 120 trying to also send a uavcan.equipment.gnss.Fix2 at the same time but having a higher priority due to having a lower node ID) (I am thinking that maybe the message being delayed due to conflicts causes the memory on the here3+ to fill)
  2. Since everything in these tests basically uses the same DroneCAN library maybe there is a problem in the library that can’t handle so many rapid long messages since the uavcan.equipment.gnss.Fix2 is quite a long message definition.

The one thing I am pretty sure of is that it is not just a problem with the mini carrier board.

Unfortunately at this time I don’t have time to dig into this much further and I am hoping I can get some help with this. @philip any idea on who can help?

Yes i also came to know that its not mini carrier board issue.
@ceng what a debug process you have done here to explain the issue on Here3+.
We are all hope on @philip let see this and solve the issue.

@sidbh here….

Please confirm terminator status on both here3+

The issue is related to over termination, as Here3+ has terminators enabled by default, and old firmware we haven’t had an option to disable the terminators. I have pushed a new Here3+ beta firmware where its added to the parameter and set to off by default. Please use Mission Planner to update the firmware (select Beta) and see if it fixes your issue. @ceng @Kalai_Selvan . You should see the firmware version to be 1.10 after update.

@sidbh thank you for your clarification and fix towards the issue on the Here3+ Gps .
If i upload the beta firmware then terminator will disabled by default or we have to manually disable the parameter.
If yes what will the parameter name if that?

After working on this issue last month with @ceng, we found that FDcan also allows this error message to go away. However, the GPS units cannot be taken out of FDmode. Is there a way to factory reset the GPS units which are in FDcan mode, in order to update them to the new beta software?

The issue is discussed here:

EDIT: It looks like the GPS that was switched was simply D.O.A, as it never got sats with any software. The GPS was switched for a new one, and the new firmware successfully let us turn off the terminating resistor and removed the Bad Health message


The updated GPS cannot get a fix anymore.

@ceng and I use two GPS units in series, and updated the middle GPS to the new firmware to remove the terminating resistor. We then calibrated the GPS units.
After applying the firmware update to 1.1, the GPS health issue goes away, but the GPS no longer gets a fix. Here is an image of the parameters:

We took out the regular GPS and tried to run the Updated GPS (FW 1.10) by itself, and it was still unable to find satellites.
image

Does something else have to be changed in the parameters due to the new firmware?

@sidbh i have just carry out the HERE3+ firmware update as you said but seems after FW update also appears same version as previous.


i think after updated HERE3+ with latest Beta version solved the issue.

@Kalai_Selvan did you see the correct software version in the end?

@CjF @ceng regarding the Here3+ that’s not getting fix. Can you let me know how recently was it purchased. Also if you get a chance, there is a feature in new firmware that allows you to connect u-center and you can list SVINFO and other settings which can give further insight. You can follow the steps here Here 4 Manual - CubePilot

The GPS module in Here3+ is M8P not F9P so just the firmware update bit will be different rest of the connection process should be same.

Yes …we have to do and repeat the process N number of time to get it work firmware update correctly.

Regarding connected through the U- center i have followed Here3+ manual.
Manual says set pass through parameter to appropriate number like 0 ,1 and 2.but when i see Here3+ parameters i couldn’t find what parameter I have to use it for to connect U center.
Because manual parameters and original here3+ parameters are not similar.