How to test code that depends on Arduino libraries

Advanced Solutions Unit Testing
1

Hello, this more of a discussion question than a “help me with my code” question. So, get a drink, because this is a long one.

I’m going to briefly describe my project just to give some context: the project has several private libraries that are basically wrappers for Arduino libraries, for example, an HTTPClient wrapper that prepares a message before sending it via the Arduino lib. And the project has an state machine implementation that makes calls to those wrapper libraries.

Now I want to test this project with PlatformIO provided tools. For this I’m using googletest. I was able to create tests for the state machine using mocks for my private libraries without much problem, but I’m stuck at testing the wrapper libraries that are dependant on the Arduino libraries.

Here is an example of what I’m trying to do: in my code I have a function that does different things depending on the return value of the begin function

//production code
#include <HTTPClient.h>

HTTPClient client;

int functionToBeTested()
{
    if (client.begin("http://host.url"))
    {
        //do stuff
       return succes_code;
    }
    // do other stuff
    return error_code;
}

I want to create one test for the succes path and another for the error path, for that I tried to mock the begin function like this:

// mock code
#include <gmock/gmock.h>
#include <HTTPClient.h>

class HTTPClientMock : public HTTPClient
{
public:
    MOCK_METHOD(bool, begin, (String));
};

But it seems I’ve found a limitation of the googletest framework. The problem is that gmock is only capable of mocking virtual functions, which is not the case for functions provided by the Arduino libraries. So now I’m not sure how to proceed.

My first question is: does the way I’m thinking about tests makes sense? I’m not very familiar with how these tests more dependant on hardwareof are usually done, so, maybe my understading is not very correct.

Second: assuming this approach is doable, how do I solve this problem with mocking non-virtual functions?

In the gtest documentation, the solution provided is Hi-perf dependency injection, but this solution involves a lot of changes in the production code, so I’m not very inclined on using it.

Another possibility would be creating a fake for each Arduino library I’m using, which provides the same functions, but not the implementation. I just don’t know exactly how I would get this to “substitute” the production code with the testing code. Is using #ifdef macros in the productio code the usual way of doing this?

//production code
#ifdef PIO_UNIT_TESTING
#include <HTTPClient.h>

HTTPClient client;
#else
#include "myHTTPClinetFake.h"

HTTPClient fake client
#endif

Or is there a way to do the substitution at linking time by setting some variable in the platformio.ini?

This approach leaves me with another question. Using mocks I can do EXPECT_CALL(httpClientMock, begin(String(addrBuffer))).WillOnce(Return(true)); to ensure the test will cover the path I want it to cover. How would I do that (replicate the WillOnce(Return(true)) part) with a fake implementation?

Any insigths or reading material regarding this questions are greatly appreciated.

2 Likes
2

My advice is: skip it. Test your “business” code. Don’t the test the framework/library.

If your code “touches” the Arduino/vendor libs → make a thin wrapper, that you can mock for your own tests.

The problem is that:

  1. Not many people actually write tests, especially in the context of Arduno.
  2. The Arduino libs are not written with testing in mind, especially unit testing.
  3. You shouldn’t mock “not your own code” anyway.

Ad3. What if you mock a library, and the library changes its working? Or what if you just don’t understand the library correctly? You will have a successful test for the app that gives you invalid results.

You can mock your own code for your tests, but the glue with other people’s code should be tested functionally → without mocking. If possible.

Hence my advice. Need to use some other people’s HttpClient? Create your own HttpClient! It should be using the “vendor” HttpClient. It should be tested functionally. And you can design it to be easily mockable in your own use cases, so that your code – that needs a HttpClient – can just mock it.

1 Like
3

Thanks for the reply @dvdnwk!

From your answer, my understanding of unit testing isn’t all that correct :sweat_smile:

So in my case, that’d be the state machine.

I was already doing that without realizing. I have the library wrappers, which I was mocking to test the state machine.

So in the case of the HTTPClient library, I should just run the code normally and check that requests are being sent and received, correct?

1 Like
4

In case of Arduino → I’ve never done such automated tests. I mean, I just test it manually, that it works as expected, and that’s it.

Ideally, you would test it functionally – like for example: if you do servo.write(), you could check if servo.read() gives the proper value. But how do you test the hardware actually? How do you check if the servo really does, what the lib claims it should be causing? :wink:

That’s why I just said: don’t :wink: I personally gave up.

I just make my abstractions that use the “real libs” under the hood. So that I can mock my abstractions, and use my abstractions in my “business logic” (or let’s just call it: “my code”).

Example:

  • This is my DigitalPin “interface” (abstract base class)
  • When "my code"needs to use a digital pin, it requires an implementation of the interface – for example LedOnDigitalPin uses the interface
  • This is the “real” DigitalPin implementation – it implements my interface, it uses the “real” Arduino lib under the hood, to do the “real work”
  • This is a “mock” of DigitalPin implementation
  • This is a LedOnDigitalPinTest – as you can see, I can test “my code” without actually needing any hardware, thanks to that thin layer of abstraction (DigitalPin interface) between “my code” (the LedOnDigitalPin) and “the lib” (Arduino digitalWrite, etc functions)
  • There is no DigitalPinTest – it would require me to somehow test if the Arduino lib is actually doing its job. This is what I’m skipping. I just trust, that if the “manual tests” on the real hardware show it is working, then it is working. If Arduino lib is upgraded and changes it’s behaviour, my code might break. Shit happens. I may try to test some parts of Arduino lib for a challenge, but for now I don’t really care. If I were to write a DigitalPinTest, mocking Arduino functions would give me nothing – change in the Arduino lib would still give me false–positive results. Hence, no point in mocking Arduino libs. The test would have to be functional. Somehow xd

Mind that I’m a OOP and testing freak. it probably is over-engineered. It’s supposed to be. It was (is) a learning experience to me.

2 Likes
5

I know it has been months, but I could only come back to this now, and your explanation finally made sense. I’m testing only the sate machine, aka, the business code. And using abstract classes for the library wrappers, to be able to mock them.

It involved quite a bit of refactoring, but I think my code is generally better now :blush: So thanks for taking the time to explain everything!

1 Like