Model 1 - Results

Owned by Yash Gunturi Eshwara Vidya
Last updated: 2024-07-07 by Yue Yang
3 min read

Paths:

Find model in path → C:\Users\warg\Ultralytics\runs

To see training results go to path → C:\Users\warg\Ultralytics\runs\detect\train

Find OG model images here → D:\Downloads\WARG AEAC MODEL 1 TRAINING.yolov8\valid\images

Overview:

Overall the model demonstrates strong performance among several metrics.

Confusion Matrix:

51720217081_.pic-20240705-220443.jpg
Confusion Matrix

  • 1514 (successful detecting landing pad)

  • 21  (fail detecting landing pad put there are landing pads in the image),

  • 32 (predicted  as background but the image has the landing pad),

  • nothing in true negative(all the images have landing pads) .

F1-confidence score:

 

61720217084_.pic-20240705-220448.jpg
f1-confidence curve

  • The f1-score remains high for a large range of threshold(between 0.119 to 0.8).

  • The model might be overfitting, since the curve is decreasing after the confidence of 0.119.

  • There is a sharp drop-off after 0.8 confidence threshold, indicate that the model is lack of performance after this threshold.

Precision-Recall curve:

 

 

  • the model’s mean Average Precision is 99.4% across all classes at the intersection of union (IOU) at the threshold 0.5.

  • The precision achieves both high precision and high recall, the model identify most of the landing pad with few errors.

  • The curve is nearly right-angle, so both high-recall and high-precision.

  • The high area under the curve (AUC) implies that the model performs well in distinguishing between the classes. But we only have only 1 class, so this does not suggest anything.

Precision-confidence curve:

  • confidence 0.05 has 95%.  It has an 100% precision at the confidence threshold 0.875 after 0.06 confidence.

  • Precision is consistently high (close to 1.0) across most confidence thresholds

Recall-confidence curve:

 

  • It has an 100% recall at a lowest confidence threshold.

  • the recall decreases as the threshold for making a positive prediction is raised.

  • After the confidence threshold 0.8, the model will more likely fail to detect the landing pad.

Metrics:

  • Precision and recall metrics on the validation set increase and stabilize, indicating good model performance.

  • The mean Average Precision (mAP) @0.5 and mAP @0.5-0.95 metrics improve over epochs, showing the model's overall effectiveness in object detection tasks.

Box Loss, Classification Loss, and DFL Loss:

 

 

  • Box: bounding box, A lower box_loss means that the predicted bounding boxes are more accurate.

  • classification: pad-blue, measures the error in the predicted class probabilities for each object in the image compared to the ground truth. A lower cls_loss means that the model is more accurately predicting the class of the objects.

  • Dfl: detecting various scales of landing pad. A lower dfl_loss indicates that the model is better at handling object deformations and variations in appearance.

  • All these individual losses decrease over time, showing consistent improvement of the model.

Â