Current branch: GitHub - UWARG/computer-vision-python at houston-testing

Hardware Connections

#TODO: Get pictures and better explanations for these. idk how to explain this well

1. The RPi is connected to the RPi Interface Rev B Comp 2025 Variant End User Manual (I linked the user manual), by sticking all of the pins together. It should be already connected.

2. The RPi gets powered by the 3S LiPo battery (thin/small one). Plug the battery into the main drone plug, and there’s another one that goes from the main plug to the RPi Interface.

3. Connect the UART cable from the RPi Interface onto the TELEM2 or TELEM3 port (depending on what’s setup for that specific drone/Pixhawk). On Houston, it is TELEM2.

4. Connect the killswitch line from the RPi Interface to the Pixhawk’s side GPIO pins (This is the 3 pin connector - J10). There are 8 GPIO ports on the Pixhawk, so it depends on which drone and how those are setup. On Houston, plug it into 8.

Ground Testing

Setup repository

We will clone the Airside System Repository onto the RPi.

Clone Repo

You should know how to clone the repo, see Airside System Repository or Autonomy Workflow Software if you’re confused.

Setting the right configs in config.yaml

Example config file: https://github.com/UWARG/computer-vision-python/blob/houston-testing/config.yaml (As of Mar 20, 2025, this one’s detect_target section is slightly outdated. Please see below for correct settings)

video_input

OpenCV camera (regular RGB USB camera) camera_enum: 0

1. Set width: 1920

2. Set height: 1200

3. Set camera_config.device_index: 0

Picamera2 camera_enum: 1

1. Set width: 1920

2. Set height: 1200

3. Important camera settings for camera_config:

   1. exposure_time: 10 (In us. Min: 36, Max: ?, Default: ~1000-2000?)

   2. analogue_gain: 64.0 (Min: 0, Max: 64, Default: 1)

   3. contrast: 1.0 (Min: 0, Max: 32, Default: 1)

   4. lens_position: null (focal length, in 1/m. 0 for infinity, null for auto-focus. Default: null)

detect_target

YOLO ML model option: 0

1. Download the latest model from Landing Pad Models to tests/model_example/ or anywhere you like

2. Change config.model_path to the new model’s path (relative to the repository’s root).

Bright spot detection option: 1

1. Defaults are in comments of the code, and in Airside System Repository .

2. As of Mar 20, 2025: The defaults are as follows:

   brightspot_percentile_threshold: 99.99 filter_by_color: True blob_color: 255 filter_by_circularity: False min_circularity: 0.01 max_circularity: 1 filter_by_inertia: True min_inertia_ratio: 0.1 max_inertia_ratio: 1 filter_by_convexity: False min_convexity: 0.01 max_convexity: 1 filter_by_area: True min_area_pixels: 160 max_area_pixels: 2000 min_brightness_threshold: 50 min_average_brightness_threshold: 130

flight_interface

1. Change address to /dev/ttyAMA0 for the serial port (this is how RPi is connected to Pixhawk)

2. baud_rate should be 57600

3. Increase timeout if you see a lot of worker died ... messages in the logs during ground tests (60.0 or 120.0 is usually more than enough)

data_merge

1. Increase timeout if you see a lot of worker died ... messages in the logs during ground tests (60.0 or 120.0 is usually more than enough)

geolocation

1. Make sure resolution_x and resolution_y are the same as width and height from video_input

2. Set fov_x and fov_y according to the installed camera’s field of view in degrees

3. Set camera_position_[x,y,z] to the camera’s distance from GPS on the drone (in meters). This is in NED, so positive x is drone’s front, positive y is drone’s right, positive z is drone’s bottom.

4. Set camera_orientation_[yaw,pitch,roll] to the camera’s yaw, pitch, roll relative to the upright drone (pointing forwards), in radians. If it’s pointing straight down, then it should be yaw = 0, pitch = -1.57079632679 (-pi/2), roll = 0.

cluster_estimation

1. Currently, min_activation_threshold must be greater than or equal to 10. For now, set it to 10

2. Set min_new_points_to_run to 3 or 5, as long as it doesn’t make the RPi too slow.

communications

1. Increase timeout if you see a lot of worker died ... messages in the logs during ground tests (60.0 or 120.0 is usually more than enough)

Run integration tests

Assuming you’ve already tested this on your laptop using the Mission Planner simulator like a good developer, these steps should all go smoothly.

Test connection to Pixhawk

First, run common’s connection_test.py to make sure you’ve set up the hardware properly (modify the CONNECTION_ADDRESS to /dev/ttyAMA0 as instructed by the comment).

The output should be CONNECTED, ... and not DISCONNECTED, ....

Worker + camera tests

Next, run all the integration tests and make sure they pass (i.e. doesn’t throw an error and you see the Done! printed at the end).

Run Airside

Assuming you’ve already tested this on your laptop using the Mission Planner simulator like a good developer, these steps should all go smoothly. Please be patient, as it takes a while to start up and get some data (if you’ve run it on the simulator before, you would know. And the RPi is probably slower than your laptop).

Bad signs

• Seeing a lot of worker died ... messages in the console and main.log.

• There are a lot of [ERROR] ... messages in the worker logs (logs/[worker]_[pid].log).

Good signs

• A new folder for the current time appears in the logs folder, and you can see all the pictures the camera took and the pictures with bounding boxes for detections (if you pointed the camera at either landing pad images on your phone or an IR beacon if using IR camera).

• There is a logs/[worker]_[pid].log file for all the workers, and they do not just have 1 line saying logger initialized. Note that you will need at least min_activation_threshold number of detections before cluster estimation even starts, so it may take a while. You can lower this in the config, but remember it must be larger than 10.

Example: a good geolocation_worker_1234.log

This is an example of what a good log looks like after we detect something in detect target, and geolocation worked properly.

Real Flight Tests:

We want the program to start automatically as soon as the drone is powered on, so we do it by using rc.local.

Setting camera focus

CV Camera (OpenCV option)

1. Google webcam tester, and make sure you can find the camera.

2. Set the focus by turning screwing the lens in/out, and focus on something really far away (just hold the drone and point it far)

PiCamera (Picam2 option)

1. If on auto-focus, then no need to do anything

2. If on manual focus, set the config to place focus at 25m. (lens_position is in units of 1/m!)

Setting live camera feed via VTX

Simply run these commands in the terminal:

This should not be required for screens connected via HDMI.

Setup hotspot

A hotspot is needed for SSH, which you need to code the RPi, see logs, and debug.

1. Switch on the pi, connect it to a monitor, and connect a mouse and keyboard. See Raspberry Pi 5 for help.

2. Switch on your phone hotspot and connect to it on the RPi

   1. Note the IP address notification that pops up at the top right (just in case you need it later). Sometimes you can see it on your phone, but other times, you cannot.

3. In the network settings of the RPi (top right), set your hotspot Wifi to Priority 1 (or just any big number, make sure it’s high enough to prioritize over the other networks) to auto-connect it to your hotspot on bootup.

4. Connect to your hotspot on the warg laptop (or your laptop, whichever one you plan on using during flight tests)

SSH

1. In the powershell of the laptop, connect to the RPi: ssh warg@raspberrypi.local.

   1. If it doesn’t work, then you can use the IP address that you noted down earlier instead of raspberrypi.local.

Setting up rc.local

Edit rc.local using

Paste or type the following lines in (after you have already setup the airside repo in the above steps from the Ground Testing section):

Ensure that rc.local is enabled

You should see that it says “enabled” or “active” in green, you may have to scroll down or right using the arrow keys. If it is still currently running, it may say active or running somewhere. You can check the status (if it’s enabled, stopped, running, or disabled) using the following:

Stopping the script, which was started by rc.local

This should stop the program, but still leave the rc-local.service enabled. You can check again to make sure if you want. So, the next time you power on the drone, it will automatically start the program again.

Ground side program

Now that the airside program (main_2025.py) is set up (on the RPi), the ground-side program (recieve_statustext.py) should be started up as well (on the laptop). The airside system will send MAVLink messages to the ground station, and we have a ground side script to receive and parse these messages, generating a KML file. It’s very simple to start if you’re familiar with Mission Planner (Help docs here: https://uwarg-docs.atlassian.net/wiki/x/AYCNhQ?atlOrigin=eyJpIjoiYTUyNThiMzRlYzRmNGIzNTg4OWZlMmE1NDA2YmIyYzciLCJwIjoiYyJ9 )

Details can be found here: https://uwarg-docs.atlassian.net/wiki/x/VQAsqw?atlOrigin=eyJpIjoiZGZkMmY5Yjk2Njc4NGIxMThmZjhkOWFlNTgzNjJhNzQiLCJwIjoiYyJ9 . TLDR is below:

1. Connect the drone/controller to Mission Planner. There should be a little green drone on the map, and the top left should say “Connected”

2. Enable MAVLink forwarding

   1. Press ctrl + f → MAVLink → TCP Host 14550 + Enable Write Access

      Open

      

       

   2. After that, you can close all these windows

3. Start the script