I wanted to add Unit tests to my existing code base for purely logical blocks, running on native.
Target platform is an stm32 F303CC with HAL.
When trying to compile, I get errors that stm32_f3xx.h is not defined. Which is correct, as the native env does not have MCU specific implementations.
But the code I want to test has no dependencies on any native code (trying to test math utils, filters, PI-control etc.).
So why do I need to compile all files, even those not relevant to the code I want to test?
In my opinion, unit testing tests only 1 unit (=source file) of code. So any external dependencies have to be removed and any missing interfaces (defines, function calls etc.) either need to be included on purpose in the test(other logic blocks) or mocked/stubbed.
With this system, I can write unit tests without having to adjust the existing code base and plastering ifndef UNIT_TEST all over the code.
Is this an issue regarding the unity test framework or is this just purely based out of pios test execution?
Indeed, if your business logic doesn’t include the stm32f3 HAL files and they’re not needed for unit testing, it should be possible to unit test.
Note that various options exist for controlling tests and also which tests are compiled / executed (test_filter).
But if you want to unit test you shouldn’t use “Build” to compile but the “Test” button.
Generally, assuming the platformio.ini doesn’t contain the config test_build_project_src = yes, the files compiled during “Test” will be just the libraries (in lib/) that are being used by the unit tests.
If you have a concrete project uploaded, we can take a close look on what’s going on there.
Have you tried moving that file from include/ to lib/BasicSoftware/? I presume since the source code in src/ is not automatically built, the headers in include/ are ignored likewise. So, all the to-best-tested software should be standalone and complete for themselves in lib/.
This file is used in mutliple modules and stores the config of the project. So I cannot move it.
And why should external modules be in the same folder as the Code piece I want to test. That does not make sense for unit testing.
And this is not what a unit test is. All interfaces outside of that module should need mocking, stubbing or faking. Then I can control the input separately from other logic (like reading an ADC input) or explicitly include certain files.
The way it is currently set up, it seems to be some sort of system test or library test, not a unit test.
the entire STM32Hal. So yes, thing you’re trying to unit test is not unit-testable that way.
On the other hand, when I move include/StaticData.hpp to lib/BasicSoftware/StaticData.hpp and replace the #include "main.hpp" with #include <stdint.h> so that uint16_t is found, and I build the firmware
Checking size .pio\build\robotdyn_blackpill_f303cc\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM: [ ] 0.1% (used 44 bytes from 40960 bytes)
Flash: [ ] 0.4% (used 972 bytes from 262144 bytes)
==================== [SUCCESS] Took 1.90 seconds ====================
it still builds of course, since the firwmare is using the BasicSoftware library and all the files in that folder are automatically in the include path – thus it makes no difference if StaticData.hpp is in include/ or in lib/BasicSoftware.
Further, doing env:testing → Advaned → Test then gives
Then this is not unit testing.
This should not be an issue. At work we unit test things like these without a problem.
Input voltages are mocked and then behavior can be verified by simulating different voltages.
Why should I have to adjust my architecture to enable unit testing?
That does not make sense.
You have shown yourself what happens if you don’t make the code unit testable, by including files of the framework that are not available for desktop. When your to-be-tested business logic unit is compiled during the unit tests and it requires, through the include chain, a #include "stm32_hal.h" and friends and that file isn’t there, compilation will fail and there’s no way around that if the file does not physically exist. That’s why we have many techniques in software design that battles this (Writing Testable Code. Many developers have a hate… | by Pedro Câmara | Feedzai Techblog | Medium).
But one can also attack this problem the other way around, as you’ve already noted
There are libraries / testing frameworks where you can mock functions. Then your business logic can still include all the microcontroller framework related stuff because those includes are now actually provided by a special testing library that is controllable in a way that you can make a certain function return what you want in the course of a specific unit test, to test how the business logic reacts to it.
So here we setup the Arduino mock library in such a way that when the business-logic calls digitalWrite() and digitalRead(), we control exactly what is returned and can also verify that a function was then called a certain number of times or with specific arguments for verification. The unit tests and the business logic still have #include <Arduino.h> in them because the ArduinoFake library provides that header and all the normal Arduino types and APIs, they are just all being redirected into controllable mock functions. The business logic doesn’t know it’s being mocked under-the-hood.
However, I’ve only ever seen such a mocking framework for Arduino with ArduinoFake, not for the STM32 HAL. And given that the STM32HAL is rather huge with many modules and functions, compared to the Arduino core, it would take some effort to write the mocking framework for it. Maybe other frameworks like google-test / gmock are more general in that regard, but I have no experience with those libraries. But people have definitely got it running with PlatformIO and native desktop. Maybe GMock is what you’re looking for, but I can’t confirm it.
Of course, one can also chose to run the embedded tests on the embedded device itself. PlatformIO has a readily-available example for that located at platformio-examples/unit-testing/stm32cube at develop · platformio/platformio-examples · GitHub specifically for the STM32Cube case. There, the entire HAL is available and compiled as normal, and can thus be used during the unit tests. But that will then also trigger something in hardware, like writing to a pin, instead of being mocked and internally not happening anything.
If you’d like to use gtest/gmock as your framework of choice, it makes sense to only test your business logic, i.e. keep any hardware specific stuff out of that code because you wouldn’t want to create a hardware abstraction layer in between. So, your business logic will have to be written in a way that it is unit testable, using the Dependency Injection Principle (interface-implementation scheme). This way, for your tests instead of the “real” objects you can inject the mocks which are fully configurable to your needs.
This concept is able to improve your code because it enables separation of concerns. But of course it will slow you down at first.
The implementation of the interface must be hidden from the test framework using lib_ignore. If you want, you can have a look here at how I did this. You will have to wrap all your libraries in interfaces. It is more work but for me it was absolutely worth it given that my project is complex.
