PlatformIO Community

Problem with (native) unit testing on a large(r) project

After doing trials for several weeks with unit test, I came to the conclusion that there is a problem with native unit tests for larger projects. So I wrote this topic to describe it and maybe some day PlatformIO can come with a solution.

Say you are working on a larger project (>10K lines of code)

  • you organise your code into ‘libraries’ which are under ‘lib’. It cannot be avoided that some of these have hardware dependencies
  • but some are hardware independent, so you want to unit test them in ‘native
  • problem is that the native environment takes ALL of lib, and tries to compile it, (even if not being used by the test-application) and this build fails because of missing definitions in the native platform…

Simple example : say you have some lib using Arduino Serial.print()
You have another lib doing a circular buffer, with no hardware dependencies.
So you want to unit test the circular buffer, but when compiling all stuff in lib, it fails because Serial is undefined…

Kind regards, Pascal Roobrouck

So the ringbuffer should be its own lib then.

Alternatively as platformio-examples/unit-testing/arduino-mock at develop · platformio/platformio-examples · GitHub shows, inclusion of the ArduinoFake library provides missing Arduino functionality in native environments so that they can be tested relatively painlessly. I would even argue that that way, the logic closer to the hardware but still ‘only’ relying on Arduino APIs can be tested way better. It also includes FakeIt, a nice mocking software.

Alternatively, libraries can have an extraScript property in which sources can be dynamically excluded or included if they detect that they’re running natively. But alas, the same can be achieved using compile time macros in the code itself. Even simple macros like ARDUINO (which evaluatues to the Arduino IDE version number) can be used for guarding the pieces of code dependent on Arduino.

It is in its own subdirectory in lib folder, but the build for the test-application takes all code from lib, including other libraries which don’t make sense in the native environment.

Thanks for the tip on extraScript, I think this will allow to solve many things.

P

How about creating a “native” configuration environment, disable LDF for it. and manually specify lib_deps names? Also, the other solution could be using lib_ignore.

I thought lib_deps is only for external libraries,…
Can it be used for subfolders of /lib ?
Enumerating all libraries which are compatible with [native] would indeed be a solution.

Officially - yes, but you can also specify the name of the library. There is a special part of the code that verifies this. Maybe we should document this option. The upcoming PlatformIO Core 5.3 will not process dependencies that are not properly declared (owner/name or external source).

There is a better solution. You can add library.json to each lib/*component* and strictly configure compatible platforms. Please note that you need also to enable strict compatibility mode per a library/component or globally per configuration environment Library Dependency Finder (LDF) — PlatformIO latest documentation

Ok, interesting approach. When a lib/component is not compatible with the current platform, what happens ? It is just skipped during that build ? This would be an easy way to exclude hardware dependent libraries from compiling under [native] and would allow me to include mocks for [native] which are then excluded when building for a real target hw, eg [esp32]

Of course a lot can be done with preprocessor as well, but personally I do not like that as it makes the code less readible.

Yes, the libraries will be skipped.