This issue is most likely related to bootloader and how it initializes. This just happens to cause issue with PSM, because the VDD_5V_PERIPH_EN is pulled up high for short duration. That happens to cause issues on peripheral power.
This happens only on BLUE cubes, NOT on black cubes. Reasons are no yet found. EMI ?!??
This problem has been replicated on multiple units. 5 units so far.
Clear replicable âfailureâ behavior can be triggered with following procedure.
Load PSM BQ2431x output close to its maximum allowed current (0.8A or more), but not over current.
Wait for a short while.
BQ2431x sets itself to a 220-250mA limiting mode until rebooted. Works like constant current powersupply.
BQ2431x will regulate voltage to meet 220-250mA limit.
Other findings:
BQ2431x is not overheating. No hot spots on PSM.
This is definitely hardware failure and only solution for this is to make new revision to cube, carrier board or PSM.
So far custom carrier boards have worked, but will do a bit more testing on those too. Those are using 3DR version of the PSM.
Iâm a bit dissapointed as we have had the black cube issue and now we have this issue with BLUE cubes.
TLDR; Cube BLUE is not usable with the default carrier board.
To start with, this is the place for support, not Facebook.
Please contact your reseller, and ask them to follow up with me.
Please detail exactly what you are connecting to your system.
I have stated this multiple times in multiple forums, the Autopilot is not a power supply, it is an Autopilot. There are many accessories that do not play nice. The Autopilot will protect the FMU over the accessories.
This is by design.
Once you have contacted your reseller, and they contact me, I will investigate the matter.
Please ensure you send them every possible detail of how to replicate the issue, exactly what hardware you are connecting, what itâs peak current draw is (you may need an oscilloscope to determine this)
Quick info on the test setup as it is very simple.
Cube BLUE and Standard Carrier Board
Power Brick Mini and/or lab power supply to supply 5V to autopilot POWER1 port.
Programmable DC load connected to a GPS1 port.
Programmable load is adjusted from 0A to +0.8A and the issue appears.
From that point the BQ2431x starts to regulate current to a maximum of 220-250mA.
There are no current spikes as the DC load is pretty much like a constant resistive load.
A simple power resistor can be used to cause this issue.
All of the above faulty devices are designed by a CubePilot.
I will contact the reseller of the blue cubes, bought 20 pieces .
I will provide more info later through this forum and trough reseller.
Iâll get back tomorrow with more info, but I have to say that if Iâm not able to use HERE2 connected to autopilot with single (20mA/color) Toshiba I2C led with it, the AUTOpilot is no longer AUTO without GPS.
In my opinion the standard carrier board is essential part of the âautopilotâ and it must work as it is specified on its manual. BTW, some might say that âautopilotâ is just software.
I should be able to have peripherals with total max current of 800mA connected to standard carrier board, but NO! this current limiting IC freaks out and starts to limit current to ~250mA after hitting currents over 800mA but less than 1A ( less than OC limit).
The pheripheral can be simplified to a single resistor as a load that dumps 0.8-0.9A @5V and we have the current limiting IC in very strange fault mode. The most strangest thing is that this issue happens only on cube blue with standard carrier board.
In my opinion this is critical issue because, losing GNSS or other vital pheripheral can cause crashing or flyaway.
This is not a power supply issue and I havenât said so. Its more like a power distribution issue on standard carrier board, while using it within itâs specs.
Image from previous Facebook thread. Cyan is voltage at power input (POWER1 connector). Yellow is voltage on GPS1 port with HERE2 + before mentioned Toshiba I2C LED. Voltage drops on pheripheral bus as the IC is in a fault mode and limits current to a ~250mA. When leds on HERE2 and I2C RGB LED flash the current need is above 250mA and thus the IC regulates voltage to meet the 250mA limit.
These test have been done on the desk with minimal amount of connected devices ( battery, power brick, cube blue, standard carrier board, HERE2, RGB I2C LED) using default cables provided with the Cube package.
Electronic programmable DC load is connected to GPS1 port to simulate a pheripheral. Oscilloscope leads are connected next to a GPS1 port to avoid voltage drop affecting measured voltage. Lab powersupply is used to provide stable 5V rail to carrier board and it is connected to POWER1 port. Cube blue is installed on standard carrier board. NOTE: DC load is connected with longish leads and thus voltage on the RIGOL DC load display is not precisely representing the bus voltage (voltage drop over cables)
After cube is booted up a 0.9A load is attached and about 10 seconds later BQ2431x starts to limit current with accelerating pace until a constant 220mA limit is reached. Please see video
Output voltage swings from 5v to 0 after or during bootup as seen on the image below. Autopilot probably switches enable pin to reset the PSM? This âresetâ doesnât happen on every boot up and those times when it doesenât it seems to make the PSM more sensitive to failure. On a short observation period failure doesnât happen if PSM reset is done. Longer testing periods would be needed to confirm this behavior.
On a black cube, there are no fast led flashing at bootup and there are no reset during boot up for PSM. Black cube works properly and we are not able to make it fail with same test setup.
This test should be very easily reproduced and all 5 units tested so far are affected by this issue.
This issue will concern ALL pheripheral equipment powered trough the standard carrier board when cube blue is used. Issue can happen on any CAN bus, I2C, SERIAL or anything else as long as the peripheral uses the 5V bus from the standard carrier board.
This does not affect autopilot (cube) 5V power rail, only peripheral 5V bus.
I donât see any benefit connecting reseller as long as there are no clear fix for the issue. You @philip, are the person who I need to be in contact, like I do now. I will contact the reseller when replacements or refund is necessary.
Further testing revealed that Cube blue sends very short 3V3 pulse to a PSM VDD_5V_PERIPH_EN aka CE pin on BQ24315. The duration of the pulse ranges from 500nsâŚ1us.
The BQ24315 chips seems to fail to reset properly if the pulse is shorter than 800us, but works correctly when the pulse duration is longer than 800us. This was confirmed by using external signal generator to make exact pulse lengths.
The failure mode on BQ24315 is probably there always after improper reset, but this current limiting failure shows up only when using higher currents 600-800mA. Most likely this issue is triggered by a heat/thermals instead of specific amount of current. Outer temperature of the IC doesnât rise measurably at any point.
The cube black doesnât do that pulse at all. Not the new or older ones.
Is this reset pulse unintentional or made on purpose?
Fix?:
This issue could be fixed by resetting PSM with proper long reset pulse >=2us when bootloader is run.
I do not recommend anyone to use cube blue with any carrier board using PSM with BQ24315 IC on it.
@philip, please do replicate these tests so we can continue with this and get things fixed!
Until this is solved, I cannot supply power to peripherals trough carrier board and that is inconvenient
PS. You should be pleased as we did the hard work for you
PPS:
And I will keep complaining until problems are fixed. I donât need distributor for that.
I have asked you to contact your reseller, because your reseller can assist me to get to the bottom of this faster!
From what I can see from your test results, this âdifferenceâ between blue and black meets to have a root cause that is not Hardware related.
Please post the name of the serial port on mission planner that is used to connect to the Blue cube, and please post the same for the black.
Under the messages tab on mission planner, please post the startup information for the blue and black.
As stated before, you can continue to loose your cool, and just deal with me, and get one or two bits of information a day due to time zones, or, you can deal with your distributor, who is one of the best experts in these matters.
The startup information is very interesting.
Logs hopefully will tell us if ardupilot is detecting a fault, which is important here.
ArduCopter 3.6.0 first stable release with NuttX. Will change over to ChibiOS during next few months.
The only significant difference we have found between black and blue cube is following. Blue cube will pull-up randomly CE pin on BQ24315 to 3V3 shortly 500âŚ1000us at bootloader time. Too short <=800us pull-up on CE pin causes undefined fault on BQ24315. This fault with 600-800mA load causes the BQ24315 to go into a current regulating mode. It will continue to regulate current until power rebooted or reset properly with >1ms pull-up on CE pin.
The question is this pull-up caused by hardware on cube blue or does it have different bootloader software that has this issue?
A proper pull-up during bootloader or during ArduCopter boot up can solve this issue. For example a
2ms pull-up on CE (reset) will clear the fault on BQ24315 100% times on our tests.
We use Serial 0 (aka usb port) on both black and blue cubes.
_UPDATE: Before you get to comment on the IO thread heartbeat. We didnât have SD cards on the cube at the time. However this issue happens regardless of the SD card installed or not.
Black cube
Blue cube
In short autopilot wonât detect power fault directly.
Here are the logs from the autopilot. You will find that GPS/compass (HERE2) will die suddenly, when the issue begins. BQ24315 IC is not giving Over Current fault during this problem, unless the current really goes shortly over 1A. The autopilot could make a guess about the issue indirectly if it sees multiple peripherals stopping to work.
I would say that you have pretty much all information you need to replicate this issue.
We have already made changes to power supply infrastructure as this is a critical failure in our view and we cannot use the standard carrier board as part of the power bus for peripherals. We are no longer in a urgent hurry due changes, but in my opinion this should be investigated quickly, so that other users could make actions if necessary.
I have begun the conversation with the distributor, with same kind of conversation delays as we have here. Distributor will try to reproduce the issue. I will still update our findings here.
Canât do that now. Will do that during next few months. Thanks for the caution.
As I have said on my last post, we didnât have SD card attached during taking those images.
We use them always when flying and this should be quite obvious as we had the LOGS there for you to download.
Even tough the software is not most up to date and SD card was not used while taking prints screen of startup info, it has nothing to do with the issue.
Feels like this issue is a joke for you. Thanks for your efforts anyway.
NONE OF THIS IS A JOKE TO US!
You are running code that has known serious bugs that directly affects the safety of your flight!
I have said we are looking into the issues we have raised, and I thank you for finally contacting your reseller, who is actively assisting in the matter.
Our team is looking into the code, and neither Nuttx or Chibios versions of ardupilot do anything but set the pin low.
It is possible that there is a change of timing of when that happens due to the new systems being more efficient, so we are looking at that.
If you could chill with the narrative around the issue, and stick to the exact known facts, we will get to the bottom of this.
I do thank you for bringing it up, it has highlighted that ardupilot is missing some key code on how to safely reset accessories for a transient OC in flight
Thanks for that bit.
Letâs get back to the engineering on this issue, and get some results.
Thanks for taking this under work
I think I have nothing more to add to this for now.
Distributor has my email and I will check this forum frequently if you need something like testing from me.
Hopefully this will further improve the autopilot and itâs resilience to external error sources.
Iâd like to add that we may be experiencing the same issue, although we use Cube Black. I described our experience here, but the tl;dr is that our peripheral supply seems to be getting stuck in a current limiting state, even though our draw is well under the limit (<400 mA) and no overcurrent state is logged.
Any tests we can/should do to determine if itâs the same issue?
In any case, we will likewise be contacting our supplier with our test data.
I have read your description and that sounds similar. During the current limiting the yellow flashing turned to orange.
You could verify this issue by having resistor or other constant load of about 800mA on peripheral output. Remove all other peripherals and connect 6 ohm power resistor or other suitable load across the peripheral power supply (Note: Power 4W). Measure voltage and or current on the resistor with multimeter. Voltage should be about 5V at begin and current about 0.8A, after a while voltage will drop to about 1,5V across the resistor and current to ~250mA.
If you donât mind, I would like to change the title of this to **peripheral power / bootloader/ brownout **
Or something similar, need to get some more software types onto this
.
