I updated the regenerate_coverage.sh script in that minimal example so it loops until the problem happens.
This probably won’t make it much easier to investigate, but should make it easier to see.
EDIT: I also just added a hard limit to 50 iterations and a report of the number of iterations to make the bug appear (or not), which averages around 4 runs (but can be anything between 1 and 15) on my machine.
We don’t provide any random build flags or commands. You can even compare the verbose output from the multiple “run -v” iterations. Could be an issue with OS/compiler version?
Well, the issue appeared with platformio v6.0, and could be resolved by going back to v5.2.5 (up until the point that libraries started loosing backwards-compatibility and it became hard to maintain).
It doesn’t seem to depend on the gcc/OS versions as it appears on several combinations (my machine, CI and other people on the project), which all use different OS, and with several different gcc versions.
I understand that this is not a straight up platformio issue, but something changed in the way platformio does things which made it happen. I am also still investigating configration solutions on my end.
Just found some interesting piece of information regarding the way gcda files are generated:
Specifically this part caught my eyes:
At running time the statistic data is gathered and stored in memory. Some exit callback is registered and is called to write the data to the .gcda file when the program terminates. This means if you call abort() instead of exit() in your program, no .gcda file would be generated.
Is it possible that the fact we build a custom main() for each test instead of using the one from googletest changes the way the program terminates?
Thanks, an interesting note. This is a GoogleTest’s default main handler googletest/gtest_main.cc at main · google/googletest · GitHub
Sorry, I finally found the issue. Let me think about how to fix it.
Fixed in
Could you re-run pio upgrade --dev. Does it produce all coverage files now?
Please note that you will need to update your “main” function of the google tests and return the 0 code instead of GoogleTest’s return code (which always is negative when tests failed). We updated “Get Started” guide for GoogleTest GoogleTest — PlatformIO latest documentation
Hello.
The coverage reports are generated properly with pio 6.1.0rc1. I still have some cleanup to do to get this to production, but that is awesome.
Thank you very much!
Wow! What great news! Sorry again for the delay in fixing this problem. Glad that we finally solved all issues that turn away people from upgrading to PlatformIO Core 6.0
Happy coding with PlatformIO! 