Getting started with unit testing

I have recently been using Platformio to develop an ESP32 program. The program leverages the IoAbstraction library and I am trying to unit test my program, however when running a simple native unit test it fails to compile when I #include <MockIoAbstraction.h> indicating the library cannot be found.

The native calculator example (platformio-examples/unit-testing/calculator at develop · platformio/platformio-examples · GitHub) compiles and runs on my Windows 10 PC (compiler is Mingw).

When I add the IOAbstraction library using the Platformio IDE and add modify my plaformio.ini env as follows:
[env:native]
platform = native
test_ignore = test_embedded
lib_deps = davetcc/IoAbstraction@^1.6.7

the library is shown in the IDE under pio/libdeps/native folder.

I add the line #include <IoAbstraction.h> to test_desktop/test_calculator.cpp and attempt to run the test which fails to compile with the following result/error message:

Processing test_desktop in native environment
--------------------------------------------------------------------------------------------------------------------------------------------------------Building...
test\test_desktop\test_calculator.cpp:19:10: fatal error: MockIoAbstraction.h: No such file or directory

***************************************************************************
* Looking for MockIoAbstraction.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:MockIoAbstraction.h"
* Web  > https://platformio.org/lib/search?query=header:MockIoAbstraction.h
*
***************************************************************************

 #include <MockIoAbstraction.h>
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
*** [.pio\build\native\test\test_desktop\test_calculator.o] Error 1
============================================================== [FAILED] Took 0.83 seconds ==============================================================
Test          Environment    Status    Duration
------------  -------------  --------  ------------
test_common   native         PASSED    00:00:00.832
test_desktop  native         FAILED    00:00:00.830
======================================================== 1 failed, 1 succeeded in 00:00:01.663 ========================================================

I’m not sure what to do next, and looking for any suggestions as to how to get platformio to see the library and successfully compile this test. Thanks in advance for any support.

That topic might by a bit more complicated than you have anticipated.

The first error stems from the LDF rejecting the full inclusion of the library because it sees that it needs framework = arduino or mbed which is not set in the native environment, since there is no Arduino there normally. That is worked around by adding lib_compat_mode = off.

Next, you will see that even if you only include the IoAbstraction.h header, it will still ask for the Arduino headers in this library and its dependency.

Processing test_common in native environment
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Building…
In file included from .pio\libdeps\native\TaskManagerIO\src\TaskManagerIO.cpp:6:
.pio\libdeps\native\TaskManagerIO\src\TaskPlatformDeps.h:201:10: fatal error: Arduino.h: No such file or directory

.pio\libdeps\native\IoAbstraction\src/PlatformDetermination.h:33:10: fatal error: Arduino.h: No such file or directory

Etc. So while code for mocking exists natively in the library, it’s designed for unit tests running on the embedded device where it still has the Arduino core available to compile the rest of the library. Not pure desktop.

If you want exactly that and no desktop tests for that, you should be able to use it normally in the embedded unit tests.

However, there are still tricks to be done for native. By using ArduinoFake, we get a mockable version of the Arduino core, on top of the FakeIt framework and will provide the Arduino.h header.

This is also used in PlatformIO in the example platformio-examples/unit-testing/arduino-mock at develop · platformio/platformio-examples · GitHub.

Using that we can compile the “Arduino Framework” for native and setup the unit tests to make the Arduino core functions return what we want when our business logic calls them, and then test the results of th business logic.

However, when using this on top of IoAbstraction.h, we still get a fail: It won’t be able to find the Wire library or Wire.h. This is missing in the current version.

Processing test_common in native environment
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Building…
In file included from .pio\libdeps\native\IoAbstraction\src\EepromAbstractionWire.cpp:7:
.pio\libdeps\native\IoAbstraction\src/EepromAbstractionWire.h:38:10: fatal error: Wire.h: No such file or directory

But, using a bit of looking around and figuring out the lib yourself, one is able to come up with the missing Wire mocks. After some fixup for the min macro, the IoAbsraction library also compiles natively on desktop then.

I’ve created the repository GitHub - maxgerhardt/pio-unit-test-mock-ioabstraction: PlatformIO example using ArduinoFake and IoAbstraction for unit testing on the desktop with a test example that compiles for desktop and uses the IOabstraction library. First it uses the direct mock, then it gives a small but complet business logic test using a switch.

The technique there is that the business logic accepts that you can set the IoAbstractionRef. Usually this is your e.g. I2C expander device object or something, but it can be substituted with a MockedIoAbstraction. Hence we can simulate some input / stimulus and check the reaction of the business logic.

The IoAbstraction library can also be mocked in a more “hardcore” way. Using the expanded ArduinoFake library, we can let the IoAbstraction library think and use a “I2C expander” via the Wire library. We can then mock the return values and behavior of the Wire.xyz() calls. This is like simulating what I2C data is read over the I2C bus, at that level. Then the IoAbstraction library handles that as normally and as if it got it from a real I2C device.

That way is however really complicated, since you must now know what I2C packet data to return at which exact points. If one just cares about whether the business logic gets some input stimuli, the way above is way easier. But if the reaction of the IoAbstraction library and your business logic is supposed to be tested against abitrary I2C packets on the bus, that must be used. But that’s then also kinda like testing the IoAbstraction itself which I assume is already well-tested.

An example of just a mock-test for Wire is also included.

Processing test_desktop in native environment
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Building...
Testing...
test\test_desktop\test_calculator.cpp:192:test_function_calculator_addition     [PASSED]
test\test_desktop\test_calculator.cpp:193:test_function_calculator_subtraction  [PASSED]
test\test_desktop\test_calculator.cpp:194:test_function_calculator_multiplication       [PASSED]
test\test_desktop\test_calculator.cpp:195:test_ioabstraction_mock       [PASSED]
test\test_desktop\test_calculator.cpp:196:test_switchinput_mock [PASSED]
test\test_desktop\test_calculator.cpp:197:test_simple_arduino_mock      [PASSED]
test\test_desktop\test_calculator.cpp:198:test_i2c_wire_mock    [PASSED]

-----------------------
7 Tests 0 Failures 0 Ignored
OK
===================== [PASSED] Took 7.48 seconds =====================
Test          Environment    Status    Duration
------------  -------------  --------  ------------
test_common   native         PASSED    00:00:01.468
test_desktop  native         PASSED    00:00:07.476
==================== 2 succeeded in 00:00:08.943 ==================== 

Thanks for such a detailed and specific response. You are correct that this is more complicated than I anticipated. I greatly appreciated the example you created, I have downloaded and will work my way through it with the great notes you provided in your response.