OpenCV imshow not responding when implemented together with dronekit

Hi, been working on this for quite some time. I have connected my Cube Orange through my R-Pi’s UART. My Raspberry Pi has a connected camera.

So, I want the robot to rotate based on a detected marker from where the camera captures.

However, my issue is that the imshow tab doesn’t open once i’ve ran the program. It always lags and goes to a ‘not responding’.

I’ve run the camera and the detection and it did work. Is there any reason why the opencv camera tab does not open when used together with the dronekit?

Here’s a copy of my code

from dronekit import connect
import cv2
import numpy as np
import math

def ConnectMyCopter():
    print('Connecting to vehicle on:' )#% connection_string)
    vehicle = connect('/dev/ttyAMA0', baud=921600, wait_ready=True)
    return vehicle

def arm ():
    #vehicle.mode = VehicleMode("MANUAL")
    vehicle.armed = True

def turnRight():
    vehicle.channels.overrides = {'2', 1900} #Max signal for turn Right
    
def turnLeft():
    vehicle.channels.overrides = {'2', 1100} #Max signal for turn Left


if __name__ == '__main__':
    vehicle = ConnectMyCopter()
    arm()

    cap = cv2.VideoCapture(0)

    while True:
        success, img = cap.read()
        img = cv2.flip(img,0)
        img = cv2.rotate(img, cv2.ROTATE_180)
        cv2.imshow('img',img)
        
        img_hsv=cv2.cvtColor(img, cv2.COLOR_RGB2HSV)
        mask = cv2.inRange(img_hsv, np.array([110,50,50]), np.array([130,255,255]))

        pixels2 = np.where(mask==[255]) #Pixels for the Detected

        circle_centerX = int((max(pixels2[1]) + min(pixels2[1])) / 2) #Getting the Center of the Marker
        circle_centerY = int((max(pixels2[0]) + min(pixels2[0])) / 2)

        image_centerX = (img.shape[1] / 2) #Getting the Center of the Image
        image_centerY = (img.shape[0] / 2) 

        if (image_centerY-circle_centerY) == 0:
            Angle = 90 #Division by zero is not possible.
        else:
            Angle = math.degrees(abs(math.atan((image_centerX-circle_centerX)/(image_centerY-circle_centerY))))
        
        if circle_centerX > image_centerX:
            turnRight()
        elif circle_centerX < image_centerX:
            turnLeft()
        
        k = cv2.waitKey(30) & 0xff
        if k == 27:
            break
    cap.release()
    cv2.destroyAllWindows()

Did you check:

  • Same baud rate setting on telemetry port of the flight controller that is connected to the RasPi? Perhaps you should start with a lower baud rate first (e.g. 115200 or 230400) on both ends
  • Did you detach the console from the serial port of the RasPi and disconnect it from Bluetooth and swap “mini” UART and “PL011” UART in the configuration? (there are many tutorials out there how to setup ttyAMA0 for serial communication on GPIO 14/15. My recommandation is to use uart3 or uart4 if you use a Rpi 4)
  • Did you cross over RX/TX?
1 Like

I tried fixing the baud rate and it worked. Thanks!