Controller stick input over wifi to orange cube

I’m playing around with a setup where I have a orange cube with a companion computer connected to a wifi network. This works great and I can control it via a computer and mission planner.

I would like to use my herelink controller in the same setup, basically skipping the herelink air unit. When connecting the controller to the same network, it connects just fine and gets telemetry. I can even enable the on-screen stick inputs in QGC to control the quad.

However, I cannot seem to find a way to use the physical sticks on the controller. Is there a way to get these stick inputs to be sent as mavlink messages over wifi?