Integration Testing Architecture

Owned by Shrinjay Mukherjee
Last updated: 2021-05-14
2 min read

Goal: Develop an overall approach to integration testing for the current computer vision system architecture.

Architecture Overview:

The current CV system architecture revolves around the following abstractions

  • Programs: Orchestrate the use of modules to perform a specific task

  • Modules: Contain processing logic for specific tasks that can be reused between programs

    • Modules are run using worker functions

  • Pipelines: Python queues that are passed into worker functions, worker functions take data from pipelines and put data into them

How can we test this?

The testing goal will primary be module logic, integration testing will involve either testing logic in a module that spans across multiple units (functions), or a module worker function, or integration between two modules. The testing approach will be loose bottom-up, assuming the functions that can be unit tested have been unit tested, we will now start building up from higher-level module logic up to integration between multiple modules.

Our first focus for testing will be TargetAcquisition, and we’ll start by testing the predict functionality. The rest of this document will show an example test plan for TargetAcquisition.

Tests:

1-module multi-unit tests: DecklinkSrc

  • Test Statement: Given an frame F, dumped into OBS, run the get_frame method in decklinkSrc, denoting the return value as F_cap, assert F == F_cap (with tolerance of maybe 10% of all pixels being failed, or looking at each R/G/B value and ensuring its within 10%)

    • Conditions: OBS running, decklinksrc running, test image in test folder

multi-module tests: DecklinkSrc + targetAcquisition

  • Test Statement: Given a frame F, run the YOLOv5 model on F outside the cv system to get the bounding boxes B_expected. Set up a video stream that OpenCv caputres from in decklinkSrc, known as S. Each frame of S is = F, Si where i = 1, 2, 3 … n: Si = F. Run the decklinksrc and targetacquisition worker functions with the output of the worker process being B_actual (bounding boxes), B will be an array that looks like [[x_1, y_1], [x_2, y_2]]. for each x_i, y_i in i = 1, 2 assert that B_actual[i][x_i and y_i] is within 5% of B_expected[i][x_i and y_i]

    • TLDR: The stream is an array of the same frame, we are making sure that the result of the frame is the same as about what it would be outside of the model.

Â