Running tests on different target boards

I’m implementing a library which needs to work with many different targets and frameworks (well … initially only for a bunch of STM32 ARM Cortex “families” and CMSIS). One obvious approach is to add all the combinations to platformio.ini, but that doesn’t scale well.

My latest setup is for a single board, containing a few dozen [env:...] targets with individual little apps to test various aspects. I’m not using Unity tests but running, uploading, and then using a serial port to send requests and verify their output. With Unity, I’d have to mock everything, this way it’s just a matter of comparing the output with expected output from a text file. This uses a custom “match” target, added to PIO using extra_scripts.

IOW, there’s a library with “functionality being developed and extended”, and I can run a large automated series of tests by entering pio run -t upload -t match. Each [env:...] then sequentially builds, uploads, and runs, all fully automated. This lets me run single tests or all of them at once, and is very convenient.

Now I’m looking for a way to repeat such test sequences for other boards. The platformio.ini file contains this:

[env]
board = nucleo-f103rb
...

To override this, I could add this:

[platformio]
extra_configs = pio-config.ini

… and then create one directory for each target board, with a pio-config.ini file in it, which redefines (and overrides) the board = ... setting. That will lead to a potentially-large number of subdirectories, each with just a single file in it. Feels a bit complex to me.

I was hoping to find support for a PLATFORMIO_EXTRA_CONFIG environment variable, because then each variation could be a single file, e.g.

PLATFORMIO_EXTRA_CONFIG=targets/nucleo_l432kc pio run [etc ...]

Or a way to override the board = ... setting from the environment, but it looks like neither of these are currently supported.

Is there perhaps some other way to achieve this kind of functionality?

-jcw