2024-07-06 Houston Autonomy and EFS flight test

Admin Preparation

Requested By

@Yuchen Lin

Goal Summary

Test specific requirement for autonomy geolocation and object avoidance and potentially EFS objective

Status?

DRAFT / SUBMITTED / WAITING FOR SUB-TEAM REVIEW / APPROVED

Desired Airframe

Houston 2

Date(s)

6th July 2024

Testplan

Test 1: Prove basic Flight performance

Procedure

Goals / Objectives

Knockoff Criteria

  1. Battery plugged in

  • Mission planner no pre-arm warning

 

  1. Bring the drone to the flightline

  • toggle safety switch

 

  1. Enable logging

 

 

  1. Take off the drone in Stablize

  • verify drone flight performance

  • Drone unstable

  • unable to establish VLOS

  • Timer threshold:

    • flight time exceeded 5 min.

    VBatt below safety threshold: 10V

    • QLAND immediately, disarm when landed.

    • Recover drone & gather state of batteries

    Lost link

    • Follow lost link procedures. Maintain vlos if possible.

    • Contact necessary local authorities.

  1. Switch Flight mode to Alt Hold

  • verify drone flight performance

  • Drone unstable

  • unable to establish VLOS

  • Timer threshold:

    • flight time exceeded 5 min.

    VBatt below safety threshold: 10V

    • QLAND immediately, disarm when landed.

    • Recover drone & gather state of batteries

    Lost link

    • Follow lost link procedures. Maintain vlos if possible.

    • Contact necessary local authorities.

  1. Switch Flight mode to Loiter

  • verify drone flight performance

  • Drone unstable

  • unable to establish VLOS

  • Timer threshold:

    • flight time exceeded 5 min.

    VBatt below safety threshold: 10V

    • QLAND immediately, disarm when landed.

    • Recover drone & gather state of batteries

    Lost link

    • Follow lost link procedures. Maintain vlos if possible.

    • Contact necessary local authorities.

  1. circling

  • verify drone Flight performance

  • Drone unstable

  • unable to establish VLOS

  • Timer threshold:

    • flight time exceeded 5 min.

    VBatt below safety threshold: 10V

    • QLAND immediately, disarm when landed.

    • Recover drone & gather state of batteries

    Lost link

    • Follow lost link procedures. Maintain vlos if possible.

    • Contact necessary local authorities

  1. transfer control to @Smile Khatri @Manasva Katyal for practice

  • pilot training

  • Drone unstable

  • unable to establish VLOS

  • Timer threshold:

    • flight time exceeded 5 min.

    VBatt below safety threshold: 10V

    • QLAND immediately, disarm when landed.

    • Recover drone & gather state of batteries

    Lost link

    • Follow lost link procedures. Maintain vlos if possible.

    • Contact necessary local authorities

  1. land the drone

  • disarm the drone, Toggle safety switch

 

  1. drone props inspection

  • ensure the locking nuts are working as expected

  • Cancel FT is the Nuts Cannot lock the Props

Test 1: Test Basic Obstacle Avoidance

Procedure

Goals / Objectives

Knockoff Criteria

  1. Set waypoints

  • Create a simple path for the drone to follow.

 

  1. Run main obstacle avoidance script

 

 

  1. Set up the obstacle in course

 

 

  1. Battery plugged in

  • Mission planner no pre-arm warning

  • make sure AUTO is in one of the flight mode

 

  1. Bring the drone to the flightline

  • toggle safety switch

 

  1. Enable logging

 

 

  1. Arm the drone in AUTO

 

 

  1. Let drone start flying through waypoints in AUTO

 

  • Drone unstable

  • unable to establish VLOS

  • Timer threshold:

    • flight time exceeded 5 min.

    VBatt below safety threshold: 10V

    • QLAND immediately, disarm when landed.

    • Recover drone & gather state of batteries

    Lost link

    • Follow lost link procedures. Maintain vlos if possible.

    • Contact necessary local authorities.

  1. Wait for drone to approach the obstacle

 

 

  1. Ensure that drone stops and enters LOITER mode when detect object

  • Ensure the drone stops ~10m away from obstacle.

  • Ensure drone switches to LOITER mode upon detecting obstacle.

If the drone does not stop within 10m of the person and obstacle:

  • The person holding the obstacle steps out of flight path immediately.

  • Pilot manually intervenes and skips to Step 13 (landing and disarming drone).

  1. Remove the obstacle from the flight path and ensure the drone reenters AUTO mode

  • Ensure the drone resumes its mission.

  • Ensure drone switches to AUTO mode upon obstacle removal.

If drone does not resume to landing waypoint

  • Pilot manually takes over and proceeds to Step 13 (landing and disarming drone).

  1. Drone hovers over the 2nd waypoint (the landing waypoint)

  •  Ensure drone completes the mission.

 

  1. Pilot manually lands drone

 

 

  1. Disarm the drone

  • disarm the drone, Toggle safety switch

 

 

Test 1: GEOLocation

Procedure

Goals / Objectives

Knockoff Criteria

  1. Place blue landing pad on ground

  •  

 

  1. Set up waypoints

  • Create path that flies over the landing pad

 

  1. Run airside

  • SSH into the raspberry pi to check logs

  • Ensure no errors

  •  System crashes

  1. Battery plugged in

  • Mission planner no pre-arm warning

  • make sure AUTO is in one of the flight mode

 

  1. Bring the drone to the flightline

  • toggle safety switch

 

  1. Enable logging

 

 

  1. Arm the drone and set mode to AUTO

 

 

  1. Let drone start flying through waypoints in AUTO

 

  • Drone unstable

  • unable to establish VLOS

  • Timer threshold:

    • flight time exceeded 5 min.

    VBatt below safety threshold: 10V

    • QLAND immediately, disarm when landed.

    • Recover drone & gather state of batteries

    Lost link

    • Follow lost link procedures. Maintain vlos if possible.

    • Contact necessary local authorities.

  1. Wait for drone to complete path and hover over the 2nd waypoint (the landing waypoint)

  •  Ensure drone completes the mission.

 

  1. Pilot manually intervenes and lands drone

 

 

  1. Disarm the drone

  • disarm the drone, Toggle safety switch

 

 

Test 1: Gemini Radio

Procedure

Goals / Objectives

Knockoff Criteria

  1. Connect Gemini RF

Configure Radio Failsafe

  • Confirm that Gemini is working

  • Houston doesn’t crash if radio failsafe happens

 

  1. Bring the drone to the flightline

  • toggle safety switch

 

  1. Enable Gemini Logging

  • Starts to fetch that will later be used for Gemini performance evaluation

 

  1. Take off the drone in Loiter with the receiver facing Gemini

  • Testing the Gemini performance when no object between two radio modules

  • Drone unstable

  • Radio Link Lost

  • Low Battery

  1. Fly away and fly back

  • Testing the Gemini performance as distance increase

  • Drone unstable

  • Radio Link Lost

  • Low Battery

  1. Yaw, and receiver facing away from Gemini.

Fly away and fly back

  • Verify the performance as there is obstacles between radio modules

Drone unstable

  • Radio Link Lost

  • Low Battery

  1. Land and repeat if necessary

  • We want to reproduce the link status issue we had last time.

  • Got desired data

  • Houston unable to keep flying

Necessary Preparation

Explain what capacity you need, what needs to be mounted, etc.

Necessary items:

  • RPI

  • Lidar

Flight characteristics needed:

  • Flight stability

Comms / Support needed:

  •  

Mandatory Attendees

This is a section for all attendees from your subteam which will be present for this flight test. (because they are testing their products, or otherwise)

Name

Phone #

Sub-team

Role

Name

Phone #

Sub-team

Role

@Smile Khatri

8076320853

Mech

 

@Yuchen Lin

6474256385

 

 

@Hardy Yu

 

 

 

@Manasva Katyal

2896985018

 

 

@Andrew Shum

6473836886

Auto

 

@Ashish Agrahari

5195898378

Auto

 

@Mihir Gupta

6473319757

Auto

 

 

 

 

 

  • @Daniel Puratich did not attend the flight test in favor of EE session, but supporting debrief efforts.


Flightline Team

Name

Phone #

Role

Reason

Name

Phone #

Role

Reason

 @Smile Khatri

 

 

 

@Yuchen Lin

 

 

 

@Manasva Katyal

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Pre-Flight Checklists

  • Software configuration correct

  • Compare against latest baselines and verify no params changed, or changes approved by comp-ft-team

  • LTE tested

  • Autopilot Orientation checked

  • Motors tested

  • Prop direction check

  • Sensors outputs healthy & within range

  • Controller input mappings correct

  • Controller input choices correct for pilot preferences

  • Wireless trainer tested & working per pilot preferences.

Hardware (wiring)

  • Wires are not too tight or too loose (comfortable, not strained, not flapping around)

    • No cut/exposed wires

    • Shrouding is neat + presentable

    • Solder joints look good

      • no cracks

      • no corrossion

      • no exposed contacts (unless expected)

    • Connectors fully plugged in

    • Connectors in good health

Hardware (airframe)

  • Bolts secure (nothing loose)

    • Carbon fiber not cracked

    • 3D prints not failing / crushed / cracked

    • No loose items

Hardware (groundstation)

  • ELRS Gemini tested & Working

    • Logging correct

    • Data looks correct

    • Control tower tested & working

  • Tracking antenna x2 (control + video) (mechanical)

    • Servos secure

    • Hardware (antennas) secure

    • No cracks/bends/chips in polycarb/aluminum

    • Gears / shafts look healthy

    • Grease if necessary

  • Tracking antenna x2 (control + video) (electrical)

    • Electrical components secure

    • Power delivery / signal wires secure, no cracks/corrosion/damage

  • Video System

    • goggles(?)

    • Monitors(?)

    • Video tower tested & working

  • Laptop w/mission planner configured correctly.

    • LTE Link tested working

Flightline Kit

  • Spare motors

  • Spare props

  • Spare tools

    • Metric allen keys

    • Socket set

    • Pliers (flatnose + needle nose)

  • USB-C cable (short & long)

  • Soldering iron + heat shrink + spare cables

  • Power supplies for ground station computer

  • Ground station computer

  • Spare bolts

  • Radio Controllers

Flight Test Timeline

Updated during debrief with best guess of timings from the testing team.

A flight is denoted by a single time the drone remained in the air, each landing is a new flight.

New log is for every power cycle. We power cycled to restart the Pi.

Time

Event

Time

Event

11:00

departure

11:00 - 11:30

go back and get the controller box

11:30

  • flight validation flight

    • Card One Test One was started, but not completed due to pilot confidence.

    • Then control was handed over as a pilot training flight.

  • Flight #1

  • Pilot: @Yuchen Lin @Manasva Katyal @Smile Khatri

11:45

  • Flight #2

  • Pilot: @Yuchen Lin @Manasva Katyal

  • calibration again (compass and accelerator)

11:50

  • take off and do a flight validation again

  • Flight #2

  • battery swap after this flight

11:55

  • card 2 test 1 auto mission dry run

  • Flight #3

  • Data

    • alt ~ 2

      speed 10m/s

12:00

  • card 2 test 1 auto mission dry run #2

  • Flight #4

  • Data

    • alt ~ 2 to 3

      speed 1m/s

12:00 - 12:15

  • assembly for rpi and lidar onto houston

  • rpi screw mount on the frame

12:15

card 2 test 1 try 1

  • Flight 5

12:20

card 2 test 1 try 2

  • Flight 6

  • battery swap after flight #6

12:50~1:37

  • Ground Testing

  • Not a flight

  • Decided to not try card 2 in the air

1:37 ~1:50

  • mount camera

  • the lidar and RPi remained on the drone

1:50 - 2:00

setup and card 3 test 1

  • Flight 7

2:00 -2:05

setup and card 3 test 1 try 2

  • Flight 8

2:00 -2:05

card 3 test 1 take off at landing pad

  • crash 2

  • Flight 9

 

 

 

 


DEBRIEF

Debrief is done chronologically and aligns roughly with the above timeline table

Lead Up

  • Gemini testing was deleted from this test the evening before after it failed ground validation.

Flight #1 Debrief

  • Took off Altitude Hold

  • Switch to Loiter

  • Controls Check

  • full stick deflection

  • was at ~10m

  • still in loiter

  • pitch and roll input led to oscillations

    • these were not recoverable and led to a crash

  • No damage to Houston observed after this crash

Flight #2

  • Card 1 test 1 #2

    • redid calibration for accelerator and Magnetometer

      • these were done on the ground before the flight

    • Pilot Reports

      • @Yuchen Lin the Loiter control reasonable

      • @Manasva Katyal responsive and flew as expected

      • did not feel different at all after this accel and magnetometer re-calibration.

      • we have not run auto-tune or done any tuning on Houston so far.

  • Decided the aircraft was ready to operate in auto mode.

Flight #3

Card 2 Test 1 #1 dry run

  • No autonomy hardware mounted

  • Flying in Auto mode

  • straight line way point

    • pilot intervene due to uphill slope of terrain and worried about hitting the ground

    • the speed was too fast which was concerning

  • switched to loiter to return

  • uneventful landing

Flight #4

Card 2 Test 1 #2 Dry run

  • No autonomy hardware mounted

  • Fly in auto

  • Straight line way point

    • WP_NAV_Speed reduced to 1m/s

    • altitude is ascending from 2 m to 3 m

  • rtl at the end

  • uneventful landing

Autonomy Hardware Install

  • Mounting of LIDAR and RPi 5

    • smaller and longer Allen key would be useful

    • mounting hole for RPI need to be oversized (right now too tight)

    • need a better placement for RFD900 one screw is not reachable

    • no nylon nuts for the all 3 screws

  • Ground test the system and verified it’s working

    • this is testing for the autonomy system only

Flight #5

Card 2 Test 1 Try 1

  • Take off in auto and @Dylan Finlay to act as the obstacles

    • @Dylan Finlay at the left side of the drone ~5 meter

  • No mode changed observed when the drone should’ve detected an obstacle

    • see card 2 test 1 expect mode change if detect obstacles

  • when landing mode change occurred switching between the two mode(loiter → auto) land in loiter

    • during manual landing it was detecting ground as obstacle

    • RPi was commanding it into auto when trying to land the drone

    • Had to spam the loiter button to force into loiter for long enough to land then disarm

    • need to implement logging

    • Possible Solutions:

      • RPi could try reading the flight mode to know when it shouldn’t issue commands?

      • In Ardupilot we need to look into a way to block the telem port when pushing a button

      • We need a software disable mechanism for the autonomy system before we fly this again. @Daniel Puratich hard requirement. A hardware disable mechanism can come down the line. This is extremely scary imo, we got lucky here.

      • Need to look into state variables to do that in Ardupilot

Flight #6

  • ask @Dylan Finlay to stand in front of the drone to see if the mode change got triggered

  • Tried taking off in auto, but RPi was not allowing it and switching back into loiter

    • The take off mode can only be loiter (auto keep switching out into loiter)

    • there was grass off to the side so it was detecting that as an obstacles

  • Pilot manually took off in loiter and switch to auto

  • Dylan was standing in front of drone and it was not able to recognize him

    • not observing mode change at all throughout the flight test

  • manually switched to loiter

  • landed

  • did not have to battle the RPi for flight mode for this landing.

  • Hotspot phone over heat and need to wait before reconnect

    • this was being used to connect to the RPi

    • had a few phones that could hotspot the RPi

    • router is long term path for this just wasn’t setup

Ground Test

  • after flight #6, ground testing ensued

  • drone was in loiter and auto for this test, but was disarmed the entire time

  • drone stayed in loiter and not switching to auto

    • decide not to flight test obstacle avoidance anymore today

  • the mode change from auto to loiter triggered after picking up the tree when testing

  • it did not switch back to auto when it was in a open field (and was expected to do so)

  • Weather is extremely sunny we suspect interfered with Lidar performance

Camera Mounting

  • Want to remake the camera mount for a simpler installation

  • CV camera, RPI and Lidar are all mounted

  • We did not remove Lidar because we did not have a working Allen key to remove the screws. It was possible to remove it using pliers.

    • Possible Solutions: get more tools!

Flight #7

Card 3 test 1 Try 1:

  • Gelocation Testing flight #1

  • Ground test the drone and verify the Geolocation value make sense

  • Worked well at 1m height with drone being held

  • ran script a second time and saved to log1.txt, will go into flight test folder

  • take off in loiter, remained in loiter for the entire flight

  • 0 degrees north, 9m away, no tape measure, just approximated with walking

  • hovering over the pad for a few minutes

  • flew around the pad for a while

    • was kind of pilot training session here

    • flight lasted a few minutes total

  • landed in loiter.

  • reviewed log files

  • 10m north, 0m east was reported from geolocation

  • confirmed with phone app this seemed correct orientation wise

  • with phone app measuring got about 9m

Flight #8

  • Gelocation Testing flight #2

  • log2.txt

  • same script

  • took off in loiter

  • flew toward landing pad

  • hoverred for a few minutes

  • then landed on the pad in loiter

Flight #9

  • Gelocation Testing flight #3

  • Wanted to takeoff in landing pad to set the drone’s home location to the landing pad

  • logged the home location, stored latitude and longitude, stored in flight interface log on RPi

  • took off in Loiter mode from the landing pad

  • landing pad was flat

  • Flight plan was to take off and then go a square path and hover over the pad

    • When given pitch command, and the drone roll for a full turn 360 degree

  • Ascended to 5m in loiter, always in loiter for this

  • Pitch was given and this resulted in a roll and crash

  • It started pitching too far and control authority was lost so drone was disarmed

    • about a second

    • disarmed from ~5m

  • GCS report EKF failure during this event, logs to confirm

  • Drone was disarmed in the air

Damage Assessment

Crash #1

  • resulted in no observed damage

Crash #2

  • Lidar appears to have some damage

    • testing it off the drone

    • it functions software wise and rotates but it looks a bit beat up

  • One prop broken

  • GPS scratched, needs to be verify

  • telem continued after crash

    • should be inspected

  • RPi looks okay from cursory glance

  • CV camera looks ok

  • lost a nut on the landing gear after this crash

Extra Notes

  • @Hardy Yu observed vibration on Z axis was high for all flights

  • Houston is not stock PIDs, it’s the tune from the old Houston, this is the newer Houston but same frame.

    • Unsure what the aggressiveness was set to from the past Houston tune

    • @Daniel Puratich suspects the high aggressiveness on the tune of previous Houston results in an un-stable tune on this new Houston, especially when the new Houston is loaded with extra weight.

  • EKF warning generally happens during crashes

  • Vibration overall in the logs doesn’t appear to be that bad.

  • Pixhawk is secured with tacky tape and a zip tie over the top

  • This was the pixhawk that was in the pond, possibly its pixhawk?

Root Cause Analyses

Analyzing the log files to determine why this crash occurred and how we can not repeat these mistakes.

Link to logs is here in our Onedrive folder, if that doesn’t work, try here.

Steady crosswind for all tests, got stronger as test progressed. Wind coming from south east, going toward north west.

Crash #1

logs analyzed are from 129

 

image-20240706-184252.png
Flight Path 3D for this flight. Shows Oscillation and Crash. (some of flight is clipped out)

image-20240706-184549.png
GPS, Roll, and Acceleration during the Crash

The above capture confirms GPS was fine throughout this event. GPS.NSats=30 during this and GPS.Status =4 during this.

 

Crash #2

logs analyzed are from 134

 

 

 

 

 

 

Ways to Improve

 

  •