SB_0000002 Critical service bulletin for Cubes Purchased between January 2019 to July 2019. DO NOT FLY

Mandatory Safety Bulletins
41

What was the reason for the catastrophic IMU failover , (before the fix)?
Why did it not just switch EKF track when it failed?

42

I will do a full briefing on the root cause, and subsequent failures in code that led to the decision to ground all cubes.

43

I have again modified the wording of the bulletin to be even stronger as some people appear to be not understanding the serious nature of the consequences of failing IMUs in flight!

Please stop asking if it affects you, assume the answer is yes, and stay on the ground till the warning becomes more specific.

44

Thanks Philip, I do really appreciate the effort that CubePilot and the dev team are putting on this issue

1 Like
45

Yes thank you bringing attention to this and being straightforward about it.

Thinking about short-term mitigation, I’d like to ask:

  1. This defect may affect IMU 1 or 2, but not IMU 3?
  2. The EKF failover (once it works correctly) will require INS_USE3 = 1 to be able to switch from IMU1 or 2 to IMU 3?
  3. Would having a separate EKF core running on IMU 3 (with EK2_IMU_MASK) be sufficient for safe flying, even without the above EKF patch (since the patch only fixes failover within a core, not switching between cores)?
46
  1. Yes
  2. Yes
  3. No. After many long talks with Paul, the issue is long and complicated, but for now, this will not help.
47

Are there new fixed cubes to buy?

1 Like
48

No, this is a work in progress, please be patient, any new news will be here when it’s available

49

Dear Philip,

Actually I have 2 questions:

1/ What have you changed in the production process between 2018 and 2019 which make sure that 2018 versions are not affected?
2/ Is the Cube Blue also affected?

Best regards,
Cédric

50

We will release information once we finish the investigation, more questions will only delay that.

51

Is the failure showing up on mission planner?
We didn’t see it on 5 different cubes.

We bought 15 CUBE in january. so they could be from last year. they still have Pixhawk on the box.

So is there already a view on a location (serial or production date) on the cube or box to know this one is from before?

thanks

52

If you are running today’s beta release, then you should get information from today’s full release mission planner

53

Is there something that we users can do to help with the investigation?
For example: where, what and how we should report faulty units?

54

Please test today’s Beta code if you are willing, it should safely switch IMU’s if there is a fault, and combined with today’s mission planner, faulty cubes should be detected.

Mission planner will identify the cube after you connect and all parameters are uploaded.

If it detects a bad cube, it will give you the option to send us data. Please do.

55

Can I get some feedback from fellow system integrator on how they are handling this “responsibly”, but realistically. We for only have AP bought from this year. Last years batch is long finished.
There is no alternative to use other item till this is sorted out. We do not have a “backup” supplier.

Will you:

  • Halt all flight testing (and maybe even halt relevant production as who knows it this would mean complete AP unit replacement).
  • Halt all deliveries (delay on promised deadlines).
  • Inform all clients that were shipped units which had Cubes from this years batch to stop flying all together till further notice? Even it means major financial loss (say not being able to do a scheduled stockpile). As this affects both backup units even if available (the whole point of actually why they got a backup unit in the first place).

I am trying to decide how to be reasonable with this. The failure has been very low according to the reports. But should that mean we ignore the warning… or seriously hurt our businesses and do the above?
I think I know what most are doing, but I am curious if anyone would confirm this. Any company producing multiple units per week as we do, will find this a serious setback and thus a hard dilemma.

56

This is Aviation. Put your vehicles on the ground and remember that no matter how pissed off your customers may be that you are stopping them from flying, it’s nowhere near as bad as having them crash.

As soon as we can we will get options out for people to fly.

1 Like
57

I can assure all of you, we are doing Everything in our power to get to the bottom of this and get you back in the air.

But we WILL NOT COMPROMISE ON SAFETY NO MATTER HOW MUCH YOU COMPLAIN.

1 Like
58

BETA TESTING welcome!!!

The ardupilot team has worked tirelessly to put together this Hotfix to get redundancy working again!!! Thankyou team!

We need some brave souls to test on expendable frames…

Please update to the latest mission planner via the .msi method

http://firmware.ardupilot.org/Tools/MissionPlanner/MissionPlanner-latest.msi

It may ask you to install drivers… do…

Then install today’s Beta.

https://discuss.ardupilot.org/t/copter-3-6-8-rc1-available-for-beta-testing/41492

Power cycle the cube.

Connect with mission planner

If you have a known faulty cube, you should get a message telling you this. And an option to send us information, please do.

If you don’t, please test fly, then after flight look for IMUs not matching in fligh.

If you don’t see this, your cube is behaving ok.

This is no guarantee that it will not have issues, but it is the best you can have.

Please report back if you have any issues with the beta on the beta release thread on ardupilot . .Org

Thanks!

3 Likes
59

Not sure who you saw complaining. I did not see any post here with complaints.

About the beta test. I am very happy to assist with any testing. I don’t mind expending any frame either to get this sorted out ASAP. Only problem is we only make fixed wings and vtols. We do not make/have any copters.
Will it be very difficult to make a plane version aswell? If not let us have it so we can do our part.

To be clear… Does this hotfix “fix” the issue. It is unclear based on you last sentence. I mean obviously if it works. What will the testing prove or not?

60

Please be patient.

This is a fix for a bug…

It doesn’t remove the grounding requirement