I’m trying to set up unit tests for an Arduino library which kind of works. However, I’d like to test if the library examples compile for a couple of boards.
To achieve this, I set up a unit test for each of the examples like this:
#include <unity.h>
#include "../examples/example1/example1.ino"
int main(int argc, char** argv)
{
setup();
UNITY_BEGIN();
RUN_TEST([] { TEST_PASS(); });
UNITY_END();
}
A pio test walks through all the examples, compiles and uploads the example. So far, so good. To save time in this step I’d prefer to only compile and check for compilation errors. So, I tried
'pio test --without-uploading --without-testing ’
which is only compiling. The problem is that it ignores tests which compile without error and only lists the ones with errors which is confusing.
Here the output of such a run:
C:\Users\lutz\source\teensy\01_Scratch\unit_test_testing\lib\myLib>pio test --without-uploading --without-testing --filter */test_example1
Verbosity level can be increased via `-v, -vv, or -vvv` option
Collected 2 tests
Processing compileExamples/test_example1 in t41 environment
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Building...
teensy_size: Memory Usage on Teensy 4.1:
teensy_size: FLASH: code:11052, data:3992, headers:8504 free for files:8102916
teensy_size: RAM1: variables:4672, code:8384, padding:24384 free for local variables:486848
teensy_size: RAM2: variables:12384 free for malloc/new:511904
----------------------------------------------------------- t41:compileExamples/test_example1 [PASSED] Took 1.86 seconds -----------------------------------------------------------
Processing compileExamples/test_example1 in uno environment
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Building...
src\myLib.cpp: In function 'void doBlink(unsigned int)':
src\myLib.cpp:9:9: error: 'digitalWriteFast' was not declared in this scope
digitalWriteFast(LED_BUILTIN, LOW);
^~~~~~~~~~~~~~~~
src\myLib.cpp:9:9: note: suggested alternative: 'digitalWrite'
digitalWriteFast(LED_BUILTIN, LOW);
^~~~~~~~~~~~~~~~
digitalWrite
src\myLib.cpp:15:9: error: 'digitalToggleFast' was not declared in this scope
digitalToggleFast(LED_BUILTIN);
^~~~~~~~~~~~~~~~~
src\myLib.cpp:15:9: note: suggested alternative: 'digitalRead'
digitalToggleFast(LED_BUILTIN);
^~~~~~~~~~~~~~~~~
digitalRead
*** [.pio\build\uno\src\myLib.cpp.o] Error 1
Building stage has failed, see errors above. Use `pio test -vvv` option to enable verbose output.
---------------------------------------------------------- uno:compileExamples/test_example1 [ERRORED] Took 1.16 seconds ----------------------------------------------------------
===================================================================================== SUMMARY =====================================================================================
Environment Test Status Duration
------------- ----------------------------- -------- ------------
uno compileExamples/test_example1 ERRORED 00:00:01.156
==================================================================== 1 test cases: 0 succeeded in 00:00:03.018 ====================================================================
C:\Users\lutz\source\teensy\01_Scratch\unit_test_testing\lib\myLib>
As expected the compilation passes for the Teensy and fails for the Uno. The summary at the end only prints the failed attempt for the uno and ignores the passed attempt for the Teensy. Don’t know if this is a bug or a feature. In any case it is confusing.
If needed, I can upload the full test project to GitHub