How to create coverage reports after tests run?

Advanced Solutions Unit Testing
1

I added

build_flags =
   ${env.build_flags}
   -lgcov
   --coverage
   -fprofile-abs-path

Now need to create html/lcov reports. Technically, that looks like “run script after tests”. But, surprisingly, this is not possible.

Currently ended with custon target:

import os
import shutil

Import("env")

out_dir = 'coverage'
env_name = env['PIOENV']
root = env.subst('$PROJECT_DIR')

def create_out_dir(target, source, env):
    if os.path.exists(out_dir):
        shutil.rmtree(out_dir)
    os.makedirs(out_dir)

env.AddCustomTarget(
    name="coverage_report",
    dependencies=None,
    actions=[
        create_out_dir,
        f"platformio test -e {env_name}",
        f"lcov -d .pio/build/{env_name}/ -c -o {out_dir}/lcov.info",
        f"lcov --extract {out_dir}/lcov.info '{root}/src/*' '{root}/include/*' -o {out_dir}/filtered_lcov.info",
        f"genhtml -o {out_dir}/ --demangle-cpp {out_dir}/filtered_lcov.info"
    ],
    title="Coverage Report",
    description="Run tests and generate coverage report"
)

But would like to have somethiing less “hacky”, IMHO custom target is overkill for such obvious demand. Could you advice anything?

2

I also need this. I am generating the test- coverage with gcovr:

gcovr --sort uncovered-number --sort-reverse --html-nested -e .pio* -o ./test/coverage/main.html

This command should run after all tests have run.
Currently I switch to the terminal, repeat the latest command.

But automating it would be nice.

I will take a look at AddCustomTarget to see if this would work for me. All unit testing is already setup in a dedicated environment.

3

Since the documentation doesn’t contain anything - at least I haven’t found anything :wink: - on the subject, it looks like a missing feature to me.

4

I have not tested but there is a “private” target named __test. Could you test this snippet?

def after_test(source, target, env):
    print("after_test")
    # do some actions

env.AddPostAction("__test", after_test)

See Pre & Post Actions — PlatformIO latest documentation

Also, you can try env.AddPostAction("$PROGPATH", callback...)

5

I’ve added

extra_scripts = post:extra_script.py

to the [env] section of my platformio.ini and added extra_script.py to the main folder of my project containing this

Import("env")

print("Current CLI targets", COMMAND_LINE_TARGETS)
print("Current Build targets", BUILD_TARGETS)
  
def after_test(source, target, env):
    print("#########  after_test ###########")
    # do some actions

env.AddPostAction("__test", after_test)

Neither running pio test -vvv nor running pio run --target __test -vvv shows the expected output

######### after_test ###########

but they show

Current CLI targets ['upload', '__test']
Current Build targets ['upload', '__test']

respectively

Current CLI targets ['__test']
Current Build targets ['__test']
6

Seems this doesn’t work.

I’ve seen this approach in issue tracker. IMHO looks more cryptic than custom target.

  1. IMHO, feature “do something after test” is missed (may be even “after test success”).
  2. Probably, coverage could have better support, but this is subjest to discuss, and anyway (1) is needed.

In ideal world, I would like to have this things:

  • Generating coverage report
  • select format (lconv / HTML)
  • filter dirs - usually only src/include of project needed (no deps / no sys headers)

That’s only preliminary opinion, “after test hook” will be enough to make things useable without pain.