Target header file is ignored. Instead header file from external library with same name is compiled

dmelendrez · October 8, 2023, 12:58am

Dear community

Presenting my case will be extremely challenging because it involves a HUGE project I have been working on for the past 3 years or so, involving dozens of files and nearly 10,000 lines of code. I will try my best since due to IP issues I cannot disclose much information about its real contents.

My project, based on a big custom wrapper library that merges and “packs” several peripherals and functionalities, had been working FLAWLESSLY until the past 2 days when it decided to completely break compilation and the project is unusable. After compiling everything turns red. I am in a panic state right now. I am unsure if a recent update in the pio core broke something.

The problem is the following:

My custom wrapper Class library (_WrapperLib") that lives inside the lib folder contains a header file called definitions.h. I use an external library that contains a header file with exactly the same name: definitions.h.

The external libraries are declared in the platformio.ini file using:

lib_extra_dirs = $PROJECT_DIR/../../libs

each and every one of my custom libraries including the wrapper library and a bunch of custom classes is also declared in the .ini file as:

build_flags = 
	-I lib/WrapperLib
	-I lib/Class1
	-I lib/Class2 
        ...
	-I lib/ClassN

(my project heavily uses inheritance to control a ton of peripherals attached to the board under development)

When opening the file definitions.h from the wrapper class everything inside the preprocessor directives is greyed out:

   #ifndef DEFINITIONS_H
   #define DEFINITIONS_H

   // All definitions are greyed out and thus this file is getting ignored!

   #endif

Please note the different shades. Also, I am using the #pragma once directive in all files!

when right-clicking the definitions.h text in the VSCode header that calls this header file and I select the “Go to Definition…” option it opens the header file from the EXTERNAL LIBRARY! Not the one from my own Wrapper Class.

Why is this happening? It used to be fine and this suddenly happened.

I have uninstalled VScode + pio wiping everything out. Running the compilation with NO additional extensions and nothing. Compilation fails EVERY TIME. I have tried on 3 different PCs and I am facing the same problem.

Any ideas would be very much appreciated.

maxgerhardt · October 8, 2023, 11:19am

If it broke because you changed changed the code, I hope you used a version control system such as git so you can easily revert to the code state before the code changes
If it broke because the PlatformIO core was updated, you can arbitrarily use any PlatformIO core version, by turning off PIO core updates in the VSCode extension’s settings and manually doing pip install "platformio==6.1.11" (see versions) with the pip executable as it shows up in CLI → pio system info.

None of that should be cause for any sort of panic.

Stuff like this should never ever be necessary. If these are in the lib/ folder, which is PlatformIO’s library search folder, then PlatformIO’s library dependency finder (LDF) should detect the used libraries (as they are used in src/) and will automatically add them to the include path, along with any other compile options which may be specified in the library’s library.json file, if one does exist. It should also detect that library’s dependencies (e.g., another library).

If any of the libraries that PlatformIO’s LDF found are wrong for your case, simply use lib_ignore to exclude them from the build in a certain environment. You can see the full library depedency graph at the beginning of compilation in a verbose build ( project tasks → Advanced → Verbose Build).

So in this case it thinks that DEFINITIONS_H is already defined, either by another file that was #include <..> before that, or by the file that includes this header, e.g.

#include <some_file_that_defines_definitions_h>
#include <screenshoted_header_file.h> // none of the code in this file will be active if DEFINITIONS_H is defined at this point already.

Do I understand it correctly that you have multiply libraries that all have their own definitions.h file? That is a very dangerous game to play because you have to make sure that when you include the file by name, the compiler will pick the right one out of N available. This depends on many factors.

First, if the definitions.h file is in the same folder as the .c or .cpp files that this header is supposed to be used by, you should always be using

#include "definitions.h"

and never ever

#include <definitions.h>

because with double quotes, the compiler will search the local directory first (stackoverflow, gcc docs)). In the other case, it depends on the order of the -I flags that either you set via build_flags or PlatformIO generates by using the LDF system which one gets discovered first – a bit of a gamble. You can see the exact order of include flags given in project tasks → Advanced → Verbose Build, and maybe when you just add -v to the build_flags, GCC will output verbose information on why it picked a certain definition.h in one folder over the definition.h in another folder.

If your program does not explicitly rely in some magical manner on all of these files being called definitions.h, the absolutely dead simplest way to eliminate this problem would be to use unique file names, such as module_a_definitions.h, module_b_definitions.h, .

dmelendrez · October 8, 2023, 2:18pm

Dear Max

Thank you so much for the extremely detailed answer. Apologies for the “panicking” confession Fortunately, yes. I do have backups in both GitHub and in Dropbox.

I indeed use quotation marks when adding “internal” headers. I have: #include "definitions.h" in my wrapper Class.

Stuff like this should never ever be necessary. If these are in the lib/ folder, which is PlatformIO’s library search folder, then PlatformIO’s library dependency finder (LDF) should detect the used libraries (as they are used in src/ )…

I will remove all build_flags (commenting those lines for now) and see what happens. I remember having to do this a couple of years ago since the project would not compile under any condition. I had this suggestion from the pio Forum back in 2020 and it made it work. Perhaps things have changed in the past 3 years.

Do I understand it correctly that you have multiply libraries that all have their own definitions.h file?

I use an external/contributed library that is shared with another part of the project hence why it does not belong to the internal folders of this specific project. I call this and other libraries using lib_extra_dirs = $PROJECT_DIR/../../libs in the platformio.ini file. This Class library also contains a “definitions.h” file which I cloned from another repo from GitHub. What baffles me is that it used to work just fine despite having the same name

You can see the exact order of include flags given in project tasks → Advanced → Verbose Build, and maybe when you just add -v to the build_flags , GCC will output verbose information on why it picked a certain definition.h in one folder over the definition.h in another folder…

I will definitely enable this option. It should give me some pointers (pun intended) on how the files are being read and compiled

If your program does not explicitly rely in some magical manner on all of these files being called definitions.h , the absolutely dead simplest way to eliminate this problem would be to use unique file names, such as module_a_definitions.h , module_b_definitions.h , .

I will change the name of the “offending” header file and report my results

In the meantime, thank you again for your valuable help

dmelendrez · October 8, 2023, 3:44pm

I can confirm that with your guidance I was able to solve my issue.

Basically, I removed the build_flags, then I changed the name of the header file "definitions.h" from the contributed/external library, cleaned the environment removing the .pio folder and recompiled everything.

I am still quite certain that a recent upgrade in the platformio core changed something in the way files are read and compiled. Trust me when I say that I did not change anything in my code that could have broken the compilation. I exclusively modified some variables and functions inside my code.

Thanks to @maxgerhardt once again