What does test_ignore affect?

PlatformIO Core
1

Based on the documentation it is unclear to me what the test_ignore build option (and pio test CLI flag) is looking at.

Is it file name, or is it maybe the unit test (test case to be precise) name? I was hoping it will work for file names but it doesn’t seem to work that way.

Use case: I can’t have common unit tests for native and other arduino-based platforms (arduino, esp32) - that’s because native platform expects main() function whereas arduino expects setup().

Based on my testing it doesn’t work for file names, as I have following layout:

tests/
integration_foo.cpp
test_foo.cpp

For native I want to run files starting with test_ (these include only my libraries) and for real microcontrollers I want to run integration_ files - there will be a code that will use real arduino dependency libraries.

I made following config in platformio.ini:

[env:esp32]
test_ignore =
    test_*
[env:native]
test_ignore =
    integration_*

But it doesn’t work - running pio test -e native still is compiling integration_foo.cpp.

2

How are the tests case names defined in it? As I read the docs

Ignore PIO Unit Testing tests where the name matches specified patterns.

So it’s a test name as shown in the reference screenshot, like test_function_calculator_addition

https://docs.platformio.org/en/latest/plus/unit-testing.html#demo

1 Like
3

The tests are defined as in the docs void functions, registered with RUN_TEST() macro - i believe this is the only format supported by unity.h.

Thank you for clarifying. I believe it might be a good idea to use some more explicit wording in the docs like test case name. Test alone might refer to test file, test suite (although in case of unity.h i believe it’s the same with file name?) or test case.

This means that I’ll have to resort to some macro magic to run test suites (integration on esp32, unit tests on native) only in proper environments. Does PIO automatically create a preprocessor constant where it stores environment name? (I know that I could do a workaround and use build_flags to specify constant for each build, but that’s an configuration extra step for each project).

4

The formerly accepted solution is no longer correct. test_ignore refers to test folders, not individual test functions inside your test code. The documentation is pretty inconsistent and vague about this and other topics, but at least this page is clear about this topic: Test Hierarchy — PlatformIO latest documentation

As for detecting which environment you’re in, you can sort of do that via #ifdef ARDUINO, as shown here: platformio-examples/unit-testing/calculator/test/test_common/test_calculator.cpp at develop · platformio/platformio-examples · GitHub

1 Like
5

This answer was written before the recent PIO Core 6 release and was very likely correct then – but indeed, not any more.

1 Like
6

Ah, I see. I just noted that in my answer.