V&V Automation Test Structure | LePhan Publishing

Automation Software Structure

The automation framework is separated into PC application, configurable test criteria, STM32 firmware, and generated test evidence. This separation allows new test suites and device interfaces to be added while retaining a common automation architecture.

VVAutomation
│
├── PCApplication/
│   │
│   ├── MainWindow.xaml
│   ├── MainWindow.xaml.cs
│   ├── MainWindowViewModel.cs
│   │
│   ├── Communication/
│   │   ├── SerialService.cs
│   │   ├── PacketBuilder.cs
│   │   └── PacketValidator.cs
│   │
│   ├── Temperature/
│   │   ├── TemperatureTestCase.cs
│   │   ├── TemperatureTestResult.cs
│   │   ├── TemperatureTestRunner.cs
│   │   ├── TemperatureCriteriaLoader.cs
│   │   └── TemperatureTestSession.cs
│   │
│   ├── Battery/
│   │   ├── BatteryTestCase.cs
│   │   ├── BatteryTestResult.cs
│   │   ├── BatteryTestRunner.cs
│   │   ├── BatteryCriteriaLoader.cs
│   │   └── BatteryTestSession.cs
│   │
│   ├── Accelerometer/
│   │   ├── AccelerometerTestCase.cs
│   │   ├── AccelerometerTestResult.cs
│   │   ├── AccelerometerTestRunner.cs
│   │   ├── AccelerometerCriteriaLoader.cs
│   │   └── AccelerometerTestSession.cs
│   │
│   ├── AllTests/
│   │   ├── VVAllTestsControl.cs
│   │   └── VVAllTestsSession.cs
│   │
│   ├── Reporting/
│   │   ├── JsonWriter.cs
│   │   ├── PdfWriter.cs
│   │   ├── ExcelWriter.cs
│   │   └── ResultFolderManager.cs
│   │
│   └── Logging/
│       └── FileAppLogger.cs
│
├── TestCriteria/
│   ├── TemperatureCriteria.json
│   ├── BatteryCriteria.json
│   └── AccelerometerCriteria.json
│
├── STM32Firmware/
│   │
│   ├── Communication/
│   │   ├── SerialProtocol.c
│   │   ├── SerialProtocol.h
│   │   └── CommandDispatcher.c
│   │
│   ├── Temperature/
│   │   ├── TemperatureTestCase.c
│   │   ├── TemperatureTestRunner.c
│   │   └── TemperatureCommandHandler.c
│   │
│   ├── Battery/
│   │   ├── BatteryTestCase.c
│   │   ├── BatteryTestRunner.c
│   │   └── BatteryCommandHandler.c
│   │
│   └── Accelerometer/
│       ├── AccelerometerTestCase.c
│       ├── AccelerometerTestRunner.c
│       └── AccelerometerCommandHandler.c
│
└── TestResults/
    ├── Temperature/
    ├── Battery/
    ├── Accelerometer/
    └── AllTests/

Major Automation Components

PC Test Application

The WPF application controls test execution, sends commands to the embedded target, receives test results, evaluates acceptance criteria, displays PASS / FAIL status, and generates verification evidence.

Test Criteria

Test definitions and acceptance criteria are stored separately from the test execution logic. Criteria may include expected values, expected states, tolerances, test identifiers, and enable settings.

Embedded Firmware

The STM32 firmware receives test commands, executes the requested firmware or hardware test operation, captures the resulting measurements or states, and returns a structured response to the PC application.

Automated Reporting

Test results are automatically recorded as structured JSON evidence and summarized into PDF and Excel reports. Execution logs provide additional information for engineering review and troubleshooting.

Main Automated Test Flow

Each automated test follows the same basic execution pattern, allowing different verification suites to use a consistent test framework.

1. Load Test Criteria

↓

2. Select Test Case

↓

3. Build Test Request

↓

4. Send Request to STM32

↓

5. Execute Embedded Firmware Test

↓

6. Return Measured Results

↓

7. Validate Results Against Acceptance Criteria

↓

8. Determine PASS / FAIL

↓

9. Save Verification Evidence

Run All V&V Tests Flow

In addition to running individual test suites, the automation application can execute the complete verification sequence in one V&V session.

Start V&V Test Session

↓

Run Temperature Tests

↓

Run Battery Tests

↓

Run Accelerometer Tests

↓

Calculate Test Totals

↓

Determine Overall V&V PASS / FAIL

↓

Generate Session Reports and Verification Evidence

Generated Verification Evidence

Test evidence is generated automatically during execution so that individual results can be traced through the complete verification session.

Individual test JSON result files
Test-suite session JSON summaries
Complete V&V session summary
PDF verification reports
Excel verification reports
Execution and diagnostic log files
Test identifiers, timestamps, expected values, measured values, and tolerances
Individual and overall PASS / FAIL results

Example Test Result Organization

When all V&V test suites are executed, generated evidence is organized into one session folder with separate subfolders for each verification area.

VVTestResults
│
└── AllTests
    │
    └── VVALL-yyyyMMdd-HHmmss
        │
        ├── Temperature
        │   ├── Test JSON Files
        │   ├── Session JSON
        │   ├── PDF Report
        │   └── Excel Report
        │
        ├── Battery
        │   ├── Test JSON Files
        │   ├── Session JSON
        │   ├── PDF Report
        │   └── Excel Report
        │
        ├── Accelerometer
        │   ├── Test JSON Files
        │   ├── Session JSON
        │   ├── PDF Report
        │   └── Excel Report
        │
        └── All V&V Session Summary