I am running 2 Here 2 GPS Compass systems over CAN on Arudupilot 4.05. On cold boot I consistenly get AHRS not healthy even though I have a 3d lock on both GPS’s. If I wait about 3.5 minutes it will start alternating between healthy AHRS and unhealthy AHRS. Once I reboot the issue goes away and AHRS is healthy immediately. Every reboot after the cold start AHRS is healthy. Compasses have been calibrated and I amy only running the 2 external Here 2 compasses. Any help would be greatly appreciated.
Can you try to recalibrate after the cube has been heated up?
The offsets can become inaccurate in different temperatures.
Thanks for the suggestion. I tried to recalibrate after booting up waiting 4 minutes, which it then started fluctuating between AHRS not healthy and a Ready Healthy beep. I then rebooted immediately go a Healthy ready beep with GPS 3D lock on both GPS’s. I then recalibrated and rebooted. On reboot I got a healthy beep with gps locks again. I powered down waited an hour and booted up again and once again got an unhealthy AHRS, wait 4 minutes and it fluctuates. Note: I am only booting up and not starting engines.
I am not sure if dual CAN GPS is supported.
You may need to confirm this from ardupilot forum.
Can you try to remove one of the GPS, or switch one of them to serial i2c mode? See if the problem would be solved.
https://ardupilot.org/plane/docs/common-gps-blending.html
When reviewing the logs I did see an IMU mismatch which I believe tells me one of the 3 IMU’s I have enabled is producing a slightly different EKF2 calculation which outside acceptable ranges which would be part of the AHRS unhealthy message. I did wonder if the dual CAN bus usage for GPS would cause a performance problem. I will try switching one of the GPS’s over to Serial and see if the problem persists.
So I switched GPS2 to run serial i2c recalibrated the compass’s and tested the cycles again. Same results as when running both GPS’s on CAN. I reviewing my logs and running an auto analyze on it I notice one issue consistently. IMU mismatch occurs on cold boot until about 3-4 minutes when the code resets the IMU’s and restarts the calculations then it will fluctuate between good then bad until I reboot, then it calculates clean and stays accurate. Here are the last two test results from the logs on IMU mismatch.
Cold Start:
Test: Compass = GOOD - mag_field interference within limits (4.00%)
Max mag field length (641.16) > recommended (550.00)
Test: Dupe Log Data = GOOD -
Test: Empty = FAIL - Empty log? Throttle never above 20%
Test: Event/Failsafe = GOOD -
Test: GPS = GOOD -
Test: IMU Mismatch = FAIL - Check vibration or accelerometer calibration. (Mismatch: 405.09, WARN: 0.75, FAIL: 1.50)
Warm Reboot:
Test: Compass = GOOD - mag_field interference within limits (3.23%)
Max mag field length (634.76) > recommended (550.00)
Test: Dupe Log Data = GOOD -
Test: Empty = FAIL - Empty log? Throttle never above 20%
Test: Event/Failsafe = GOOD -
Test: GPS = GOOD -
Test: IMU Mismatch = GOOD - (Mismatch: 0.12, WARN: 0.75, FAIL: 1.50)
It seems like this is a situation where the IMU calculations, which I am running all 3 IMU’s, need to wait until the cube reaches a certain temp then start the calculations, or I have an ardupilot setting set wrong that is causing this that other people are not experiencing.
I found a similar topic on CubePilot referrencing the same problem.
I did as was suggested in this article as a test and disabled IMU2 saved and rebooted. Wham immediately got a clean prearm check and 3d locks on my gps’s. The problem appears to be affecting cube black boards and the heating of IMU2 to operate at a acceptable rate for EKF calculations. Any help in how to get around this? Is is safe to operate without IMU2 running?
It also seems based on the messages produced below that the calculations are wrong on IMU2 cause it cannot get access to the gps until some time later. Which is when the error tries to clear.
11/4/2020 10:06:43 PM : EKF2 IMU1 is using GPS
11/4/2020 10:06:29 PM : PreArm: AHRS not healthy
11/4/2020 10:05:59 PM : PreArm: AHRS not healthy
11/4/2020 10:05:28 PM : PreArm: AHRS not healthy
11/4/2020 10:04:58 PM : PreArm: AHRS not healthy
11/4/2020 10:04:28 PM : PreArm: AHRS not healthy
11/4/2020 10:03:58 PM : PreArm: AHRS not healthy
11/4/2020 10:03:28 PM : PreArm: AHRS not healthy
11/4/2020 10:02:58 PM : PreArm: AHRS not healthy
11/4/2020 10:02:28 PM : PreArm: AHRS not healthy
11/4/2020 10:01:58 PM : PreArm: AHRS not healthy
11/4/2020 10:01:50 PM : EKF2 IMU2 is using GPS
11/4/2020 10:01:50 PM : EKF2 IMU0 is using GPS
11/4/2020 10:01:28 PM : u-blox 2 HW: 00080000 SW: EXT CORE 3.01 (107900)
11/4/2020 10:01:28 PM : PreArm: AHRS not healthy
11/4/2020 10:01:27 PM : EKF2 IMU0 origin set
11/4/2020 10:01:27 PM : EKF2 IMU2 origin set
11/4/2020 10:01:27 PM : EKF2 IMU1 origin set
11/4/2020 10:01:26 PM : GPS: u-blox 2 saving config
11/4/2020 10:01:16 PM : GPS 1: specified as UAVCAN
11/4/2020 10:01:15 PM : EKF2 IMU0 tilt alignment complete
11/4/2020 10:01:15 PM : EKF2 IMU1 tilt alignment complete
11/4/2020 10:01:15 PM : EKF2 IMU2 tilt alignment complete
11/4/2020 10:01:14 PM : Frame: QUAD
11/4/2020 10:01:14 PM : ChibiOS: d4fce84e
11/4/2020 10:01:14 PM : RCOut: PWM:1-12
11/4/2020 10:01:14 PM : CubeBlack 00200027 33375110 37353832
11/4/2020 10:01:14 PM : ArduCopter V4.0.5 (3f6b43e3)
11/4/2020 10:01:14 PM : EKF2 IMU1 initial yaw alignment complete
11/4/2020 10:01:14 PM : EKF2 IMU2 initial yaw alignment complete
11/4/2020 10:01:14 PM : EKF2 IMU0 initial yaw alignment complete
11/4/2020 10:01:12 PM : Frame: QUAD
11/4/2020 10:01:12 PM : RCOut: PWM:1-12
11/4/2020 10:01:12 PM : CubeBlack 00200027 33375110 37353832
11/4/2020 10:01:12 PM : ChibiOS: d4fce84e
11/4/2020 10:01:12 PM : ArduCopter V4.0.5 (3f6b43e3)
11/4/2020 10:01:12 PM : GPS 2: detected as u-blox at 115200 baud
11/4/2020 10:01:11 PM : Barometer 3 calibration complete
11/4/2020 10:01:11 PM : Barometer 2 calibration complete
11/4/2020 10:01:11 PM : Barometer 1 calibration complete
11/4/2020 10:01:10 PM : Calibrating barometer
If the problem came from the CAN Here2, you may try to change BRD_BOOT_DELAY to 5000. This prevent the board start recognizing peripherals before they are ready.
By your latest description, I think the offset had changed after the board heated up.
Can you try to do calibration after the Cube is fully warmed up?
By default the target temperature is 45 degree Celsius, ideally 65 degree celsius.
https://ardupilot.org/copter/docs/parameters.html#brd-imu-targtemp-target-imu-temperature