Herelink video stops because of companion computer?

Hello everyone,

This problem is pretty particular to my setup as I use a custom companion computer, although it may have a common solution since it’s related to Herelink and MP.
Any help will be appreciated :slight_smile:

Graphical summary
My attempt to graphically summarize the problem:

Usual behavior :
I am using a Cube Orange along with a Herelink and a Firefly Split camera to obtain a video stream.
Firefly cam sends its video stream to Herelink’s airunit via HDMI, Herelink sends the video stream to its RC (with QGroundControl), on which I am able to see the video.
If I connect my Herelink RC to MP via UDPCI or UDP, the params load on MP and I can still see the video on my Herelink RC.
Unusual behavior
When using my custom companion computer on Telem2, I lose the video signal completely. Herelink’s QGC stays in “Waiting for video”. This only happens when I connect to MP using UDPCI.

Companion computer
I am currently developping a companion computer for a specific application. This companion computer communicates with my autopilot via Telem2, using mavlink 2.
On boot, my companion computer sends a lot of TIMESYNC requests in order to get an optimal message travel time. Right before doing so, it sets all Mavlink messages rates to 0 (except Heartbeat etc) to “lighten” the mavlink data stream.
The video stream stops between the moments every rate is set to 0 and the moment I request a lot of timesyncs.

This right here lead me to think I was setting the wrong mavlink messages to 0 and potentially asking the autopilot to stop the video stream.
However I did not think my companion computer would be able to interact with my herelink airunit’s camera on HDMI via the autopilot’s Telem2…
As I learned not long ago, the camera DOES get a Mavlink ID, and Mission Planner also asks if I want to connect to the video stream on startup (only when i first connect the RC to MP at the beginning of the day, I then leave the RC on and MP connected to it for all my next flights and MP does not ask again if i want to connect to the video steam)

What’s strange :
This happens ONLY when my Herelink is connectected to MissionPlanner via UDPCI.
This does not happen when my Herelink is connected to MP via UDP, nor when the Herelink is not connected to MP. Video stream stays on QGC.

Moreover, when connecting to MP using UDPCI, the device list shows a MavID 42-Camera next to my 14-QUADROTOR
This MavID is not there when I connect to MP via UDP.
I searched through all Herelink’s, MP, and Arducopter params, I never found any reference to this MavID 42 for the camera.

If you are still reading, thank you so much, I’m looking forward to solving this problem with you

do you have a traffic capture? my guess its something to do with timesync, but need to confirm

Hi Michael,

Thank you for you reply. I agree with your guess on timesync messages.
I ran some new tests while using MP’s Mavlink Inspector and Wireshark for a traffic capture.
Hope these files will help:
Test Timesync Camé (1.2 MB)

Summary of my 3 tests (2 of which generated a traffic capture, 3rd one being a USB test)

  1. Using UDPCI :
    In the mavlink inspector, during the Timesync operation of my companion computer, I can see who’s talking to who.
    In the picture below , top cropped part is the Mavlink traffic from my drone (Vehicle 11), then you see my companion computer (Vehicle 167), then the MP GCS PC (Vehicle 255) and finally the camera (Vehicle 42).
    When Timesync messages are being sent back and forth, supposedly between my companion computer and the Cube Autopilot, I would expect to see a frequency increase for the drone (Vehicle 11) and the Companion Computer (Vehicle 167).
    But actually the MP GCS PC (255) seems to be responding to the companion computer’s timesync, as their frequency both increases to 50 Hz (approx).

At the same time as the Timesync frequency reaches 50Hz, my video stream shuts down.

  1. Using UDP :

When looking at MP’s mavlink inspector during a UDP connection, I do not see any Timesync messages on any of the Vehicles from before. Everything else is the same, my companion computer works fine, I see the camera in the mavlink inspector etc… But no vehicle shows any timesync messages.
The camera stream does not stop.

  1. using USB :

Being curious, I tried the same experiment but while connecting my drone to MP via USB.
Protocol : powering up the drone, connecting to Herelink’s RC, waiting for params download on Herelink, connecting drone to PC using USB cable (there is no wireless connection between PC and Herelink whatsoever), starting companion computer.
In this case, I don’t see any camera on the mavlink inspector (vehicle 42). But my video stream on the Herelink RC works perfectly.
When starting the timesync request, same behavior as UDPCI happens : companion computer (167) and MP GCS PS (255) both see an increase in timesync frequency.

Looking forward to discussing this with you

what software is on the companion computer?

from the screenshots i see MP responding, and the receiver should not be requesting again because both tc1 and ts1 are set. and MP will only respond when tc1 is set to 0.

The companion computer runs on linux but the MCU sending mavlink commands runs on C Bare Metal