Herelink video stops because of companion computer?

HereLink
1

Hello everyone,

This problem is pretty particular to my setup as I use a custom companion computer, although it may have a common solution since it’s related to Herelink and MP.
Any help will be appreciated :slight_smile:

Graphical summary
My attempt to graphically summarize the problem:

image
image799×650 26.1 KB

Usual behavior :
I am using a Cube Orange along with a Herelink and a Firefly Split camera to obtain a video stream.
Firefly cam sends its video stream to Herelink’s airunit via HDMI, Herelink sends the video stream to its RC (with QGroundControl), on which I am able to see the video.
If I connect my Herelink RC to MP via UDPCI or UDP, the params load on MP and I can still see the video on my Herelink RC.
Unusual behavior
When using my custom companion computer on Telem2, I lose the video signal completely. Herelink’s QGC stays in “Waiting for video”. This only happens when I connect to MP using UDPCI.

Companion computer
I am currently developping a companion computer for a specific application. This companion computer communicates with my autopilot via Telem2, using mavlink 2.
On boot, my companion computer sends a lot of TIMESYNC requests in order to get an optimal message travel time. Right before doing so, it sets all Mavlink messages rates to 0 (except Heartbeat etc) to “lighten” the mavlink data stream.
The video stream stops between the moments every rate is set to 0 and the moment I request a lot of timesyncs.

This right here lead me to think I was setting the wrong mavlink messages to 0 and potentially asking the autopilot to stop the video stream.
However I did not think my companion computer would be able to interact with my herelink airunit’s camera on HDMI via the autopilot’s Telem2…
As I learned not long ago, the camera DOES get a Mavlink ID, and Mission Planner also asks if I want to connect to the video stream on startup (only when i first connect the RC to MP at the beginning of the day, I then leave the RC on and MP connected to it for all my next flights and MP does not ask again if i want to connect to the video steam)

What’s strange :
This happens ONLY when my Herelink is connectected to MissionPlanner via UDPCI.
This does not happen when my Herelink is connected to MP via UDP, nor when the Herelink is not connected to MP. Video stream stays on QGC.

Moreover, when connecting to MP using UDPCI, the device list shows a MavID 42-Camera next to my 14-QUADROTOR
image
This MavID is not there when I connect to MP via UDP.
I searched through all Herelink’s, MP, and Arducopter params, I never found any reference to this MavID 42 for the camera.

If you are still reading, thank you so much, I’m looking forward to solving this problem with you

2

do you have a traffic capture? my guess its something to do with timesync, but need to confirm

3

Hi Michael,

Thank you for you reply. I agree with your guess on timesync messages.
I ran some new tests while using MP’s Mavlink Inspector and Wireshark for a traffic capture.
Hope these files will help:
Test Timesync Caméra.zip (1.2 MB)

Summary of my 3 tests (2 of which generated a traffic capture, 3rd one being a USB test)

  1. Using UDPCI :
    In the mavlink inspector, during the Timesync operation of my companion computer, I can see who’s talking to who.
    In the picture below , top cropped part is the Mavlink traffic from my drone (Vehicle 11), then you see my companion computer (Vehicle 167), then the MP GCS PC (Vehicle 255) and finally the camera (Vehicle 42).
    When Timesync messages are being sent back and forth, supposedly between my companion computer and the Cube Autopilot, I would expect to see a frequency increase for the drone (Vehicle 11) and the Companion Computer (Vehicle 167).
    But actually the MP GCS PC (255) seems to be responding to the companion computer’s timesync, as their frequency both increases to 50 Hz (approx).
    MavlinkInspector UDPCI while Timesync
    MavlinkInspector UDPCI while Timesync631×665 143 KB

At the same time as the Timesync frequency reaches 50Hz, my video stream shuts down.

  1. Using UDP :

When looking at MP’s mavlink inspector during a UDP connection, I do not see any Timesync messages on any of the Vehicles from before. Everything else is the same, my companion computer works fine, I see the camera in the mavlink inspector etc… But no vehicle shows any timesync messages.
The camera stream does not stop.

MavlinkInspector UDP while Timesync
MavlinkInspector UDP while Timesync745×515 112 KB

  1. using USB :

Being curious, I tried the same experiment but while connecting my drone to MP via USB.
Protocol : powering up the drone, connecting to Herelink’s RC, waiting for params download on Herelink, connecting drone to PC using USB cable (there is no wireless connection between PC and Herelink whatsoever), starting companion computer.
In this case, I don’t see any camera on the mavlink inspector (vehicle 42). But my video stream on the Herelink RC works perfectly.
When starting the timesync request, same behavior as UDPCI happens : companion computer (167) and MP GCS PS (255) both see an increase in timesync frequency.

MavlinkInspector USB
MavlinkInspector USB742×546 124 KB

Looking forward to discussing this with you

4

what software is on the companion computer?

from the screenshots i see MP responding, and the receiver should not be requesting again because both tc1 and ts1 are set. and MP will only respond when tc1 is set to 0.

5

The companion computer runs on linux but the MCU sending mavlink commands runs on C Bare Metal

6

Hello and Happy New Year,

Just checking if you had any news regarding this issue ?

7

How did you connect your script with mavlink? Did you not specify the source id? I found that not specifying the source id in pymavlink makes Solex unable to connect.

Not sure if your problem come from similar reason.

8

Hi,
We use a mavlink_msg_command_long_pack to communicate between the companion computer and the cube.
This command gets :

  • system_id of companion computer -> 167
  • component id of companion computer -> MAV_COMP_ID_USER67
  • target_system_id -> set depending on the SYSID of the cube
  • target_component_id ->set depending on the component seen in the mavlink inspector

We guessed this was the correct way to communicate with the cube, can you confirm ?

9

This is about the mavlink communication which is kind of out of topic.

But as far as I know, this shouldn’t stop the video.

10

We have also have had the experience that Mavlink Messages send by the companion computer can freeze the airunit. I think that should not be possible and needs an adaption of the firmware which isolates the process causing the issue.

11

Glad to hear I am not the only one having this issue !
Would you mind sharing your configuration with us ?

12

In our case the airunit freezes if we send a timesync request via pymavlink from the companion computer. It only happens with the current stable release, with the original stable release this was no issue. We did not have the time to look into the issue in detail yet and we just dropped the timesync for now.

I have to look into the code for more details on the configuration (I’m not writing the code).
Could you specify which details on the configuration you would like to know?

Not a very sophisticated answer, I’m a little bit tired after a long working day. Je suis désolée.

13

Timesync also causes our airunit to freeze…

As for your configuration I was wondering about your SRXX and Serial Configuration on the Cube, as well as the Planner panel in MP and your Herelink Mavlink settings (in QGC). I guess some of these parameters should influence the behavior of mavlink since it manages the rates of mavlink messages out of the Cube (?)

14
  • We do not use a Cube. We are using Serial 2 @115200 baud and Mavlink 2. We did not change any other settings. Did you change serial pass-through?
  • We have a lot of different HereLinks, some more than 2 years old, some only weeks. The one we detected is more than a year old.
  • We did not change the QGC settings, only the ID in Mission Planner.
  • It did not happen with the first stable firmware and exactly the same configuration.
  • I also have a gimbal connected via Mavlink
  • We have 4 devices showing up in Mission Planner drop down menu
  • In our case the AirUnit freezes exactly when the TimeSync command is send by our companion computer. If you are using pymavlink, could you send me your timesync call?
15 
  • We do not use a Cube. We are using Serial 2 @115200 baud and Mavlink 2. We did not change any other settings. Did you change serial pass-through?

So you are directly going from the Companion Computer into the Airunit ? Or do you use another Flight Controller ?
We have SERIAL_PASS1 = 0 (Serial0) and SERIAL_PASS2=-1 (disabled)

  • We have 4 devices showing up in Mission Planner drop down menu

Herelink airunit, Gimbal , Flight controller and ?

  • In our case the AirUnit freezes exactly when the TimeSync command is send by our companion computer. If you are using pymavlink, could you send me your timesync call?

As far I as know, the MCU responsible for mavlink communication on our computer uses C Bare Metal so it does not use pymavlink but a C library instead. (As I am not the one developping the companion computer directly, I can only give you some partial answers sorry)
Here is a graphical explanation of the algorithm

Timesync
Timesync825×1794 106 KB

17

We are using a flight controller of an american company which I’m not allowed to mention here (due to community rules, my post got flagged).

The fourth device is the camera.

image

18

Hello,
We are still having this issue, which does not stop our flights but lowers security levels as we do not see the video stream while using UDPCI…
Any new leads ?

19

Maybe there is a connection here:
Power Supply? - HereLink - Cubepilot

20

i realy need an idea of what timesync message people are sending, to make sure the herelink can handle this.

21

Hello everyone,

This problem seems to have been solved by new updates (Copter 4.1.5 and newer)

With the exact same setup, only changing the arducopter version, this bug does not occur anymore …

Thanks for the help :+1:

1 Like