Include search path

marco.palumbi · December 5, 2017, 11:02am

I have two questions:

is there a way to add an include search path(-Imy/filepath) for a specific (or better a group of) c/cpp file?
is there any way to have the -Imy/filepath to be added before the system include sarch paths? i need to override a specific include file. in detail I wish to include my version of pthread.h instead of the one provided by the sdk that under:

platform=espressif32
board=esp32dev
framework=arduino

it is simply void

ivankravets · December 5, 2017, 5:37pm

http://docs.platformio.org/en/latest/projectconf/section_env_build.html#build-flags

marco.palumbi · December 16, 2017, 5:21pm

I read carefully that page several times before asking my question.

unfortunately build-flags apply to ALL the environment.

my question was if is there a way to specify different sets of flags for different group of files in the same build environment and not for different target environment.
may be I want to specify different include search path for different group of source files to avoid filename conflicts if i am going to integrate some libraries, or i need to specify a log related define (es. -DLOG_DEBUG) only for a specific source…

second question is unfortunately unanswered

thank you anyway

ivankravets · December 16, 2017, 5:42pm

You can split your group of files to component/library and place it to project/lib folder. Please read lib/readme.txt in this folder. Then create library.json file in library root folder and customize build flags using Redirecting...

Another option is to use Redirecting... and …

def callback(source, target, env):
    env.Append(CPPPATH=["/my/custom/inc"])

env.AddPreAction("$BUILD_DIR/src/custom_cppath.cpp.o", callback)

You can automate whole build project.

Timvrakas · March 20, 2020, 4:48am

Hi Ivan,
Could you give guidance on how to use library build flags to add include directives for only the compilation units of that library?

{
"name": "FreeRTOS",
    "build":{
        "flags":
        [
            "-Isrc",
            "-Isrc/Source/include",
            "-Isrc/Source/portable/GCC/ARM_CM4F/",
            "-Iconfig"
        ],
"libArchive": false
}
}

works, but then those files are included for all complilation units for the while project.

ivankravets · March 20, 2020, 7:09pm

Do you mean to export flags globally to the framework and other project libraries?

Timvrakas · March 21, 2020, 12:08am

The flags I have specified above are being applied to the compilation of all the source files for all the libraries, and all of the project source.
I’d like the library to export a few of the include files (those that define the interface to the library), but there are a number of internal header files that I don’t want to be included for the whole project, buy do need to be included in the compilation of the .c files of the library itself.

Am I explaining it well?

ivankravets · March 21, 2020, 9:58am

It looks like a known issue

github.com/platformio/platformio-core

Set global flags using internal library extra script

opened 12:15PM - 27 Nov 18 UTC

closed 03:55PM - 13 Apr 22 UTC

ivankravets

docs ldf

# Solution 1 Use `extraScript` and pass flags using SCons environment. See ex…amples https://docs.platformio.org/en/latest/librarymanager/config.html#extrascript # Solution 2 Please set build flags per project using https://docs.platformio.org/en/latest/projectconf/section_env_build.html#build-flags ------ - https://community.platformio.org/t/library-json-global-build-flag/3814/4

Timvrakas · March 21, 2020, 8:26pm

Thanks for giving me breadcrumbs to follow.
It looks like my issue is related to that one, but not exactly the same. When I dump the env and global_env from the library script, things look alright. The local env contains the CPPPATH files that I included with the flags directive. The global_env only includes the framework include paths.

When I dump the env and global_env from the project build script (with extra_script = pre:script.py), they both only contain the CPPPATH paths from the framework.

However, when I build with -v, the library include paths are still being applied to all the files in the project, including to other libraries…

Timvrakas · March 21, 2020, 8:32pm

Here is a excerpted compile log demonstrating the above. I will try to demonstrate in a fresh project.

CONFIGURATION: https://docs.platformio.org/page/boards/atmelsam/adafruit_feather_m4.html
PLATFORM: Atmel SAM 4.1.0 > Adafruit Feather M4 Express
HARDWARE: SAMD51J19A 120MHz, 192KB RAM, 512KB Flash
DEBUG: Current (atmel-ice) External (atmel-ice, jlink)
PACKAGES:
 - framework-arduino-samd-adafruit 1.5.10
 - framework-cmsis 1.40500.0 (4.5.0)
 - framework-cmsis-atmel 1.2.0 
 - toolchain-gccarmnoneeabi 1.70201.0 (7.2.1)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
>>>>>LOCAL ENV (LIB)<<<<<
[ 'C:\\Users\\timv\\Git\\SpaceSalmon\\src\\board\\guppy',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino',
  'C:\\Users\\timv\\.platformio\\packages\\framework-cmsis\\CMSIS\\Include',
  'C:\\Users\\timv\\.platformio\\packages\\framework-cmsis-atmel\\CMSIS\\Device\\ATMEL',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB\\Adafruit_TinyUSB_ArduinoCore',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB\\Adafruit_TinyUSB_ArduinoCore\\tinyusb\\src',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\variants\\feather_m4',
  'C:\\Users\\timv\\Git\\SpaceSalmon\\lib\\SSISD\\diskio',
  'C:\\Users\\timv\\Git\\SpaceSalmon\\lib\\SSISD\\sd_mmc',
  'C:\\Users\\timv\\Git\\SpaceSalmon\\lib\\SSISD\\FatFs\\source',
  'C:\\Users\\timv\\Git\\SpaceSalmon\\lib\\SSISD\\misc',
  'C:\\Users\\timv\\Git\\SpaceSalmon\\lib\\SSISD\\include',
  'C:\\Users\\timv\\Git\\SpaceSalmon\\lib\\SSISD\\hri',
  'C:\\Users\\timv\\Git\\SpaceSalmon\\lib\\SSISD\\hal\\include',
  'C:\\Users\\timv\\Git\\SpaceSalmon\\lib\\SSISD\\hpl\\port',
  'C:\\Users\\timv\\Git\\SpaceSalmon\\lib\\SSISD\\hal\\utils\\include']
<<<<<GLOBAL ENV (LIB)>>>>>
[ 'C:\\Users\\timv\\Git\\SpaceSalmon\\src\\board\\guppy',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino',
  'C:\\Users\\timv\\.platformio\\packages\\framework-cmsis\\CMSIS\\Include',
  'C:\\Users\\timv\\.platformio\\packages\\framework-cmsis-atmel\\CMSIS\\Device\\ATMEL',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB\\Adafruit_TinyUSB_ArduinoCore',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB\\Adafruit_TinyUSB_ArduinoCore\\tinyusb\\src',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\variants\\feather_m4']
Found 16 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <SSISD> (C:\Users\timv\Git\SpaceSalmon\lib\SSISD)
|   |-- <FreeRTOS> (C:\Users\timv\Git\SpaceSalmon\lib\FreeRTOS)
|-- <FreeRTOS> (C:\Users\timv\Git\SpaceSalmon\lib\FreeRTOS)
|-- <SPI> 1.0 (C:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\libraries\SPI)
|   |-- <Adafruit Zero DMA Library> 1.0.4 (C:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\libraries\Adafruit_ZeroDMA)
|-- <ssi_adc> (C:\Users\timv\Git\SpaceSalmon\lib\ssi_adc)
|   |-- <FreeRTOS> (C:\Users\timv\Git\SpaceSalmon\lib\FreeRTOS)
|-- <Eigen> #f351b6a [git+https://github.com/bolderflight/Eigen] (C:\Users\timv\Git\SpaceSalmon\.pio\libdeps\guppy\Eigen)
|-- <ArduinoJson> 6.12.0 (C:\Users\timv\Git\SpaceSalmon\lib\ArduinoJson)
Building in release mode
>>>>>LOCAL ENV (PROJECT)<<<<<
[ 'C:\\Users\\timv\\Git\\SpaceSalmon\\src\\board\\guppy',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino',
  'C:\\Users\\timv\\.platformio\\packages\\framework-cmsis\\CMSIS\\Include',
  'C:\\Users\\timv\\.platformio\\packages\\framework-cmsis-atmel\\CMSIS\\Device\\ATMEL',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB\\Adafruit_TinyUSB_ArduinoCore',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB\\Adafruit_TinyUSB_ArduinoCore\\tinyusb\\src',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\variants\\feather_m4']
<<<<<GLOBAL ENV (PROJECT)>>>>>
[ 'C:\\Users\\timv\\Git\\SpaceSalmon\\src\\board\\guppy',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino',
  'C:\\Users\\timv\\.platformio\\packages\\framework-cmsis\\CMSIS\\Include',
  'C:\\Users\\timv\\.platformio\\packages\\framework-cmsis-atmel\\CMSIS\\Device\\ATMEL',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB\\Adafruit_TinyUSB_ArduinoCore',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\cores\\arduino\\TinyUSB\\Adafruit_TinyUSB_ArduinoCore\\tinyusb\\src',
  'C:\\Users\\timv\\.platformio\\packages\\framework-arduino-samd-adafruit\\variants\\feather_m4']
arm-none-eabi-g++ -o .pio\build\guppy\src\board\guppy\System.cpp.o -c -fno-rtti -fno-exceptions -std=gnu++11 -fno-threadsafe-statics -mfloat-abi=hard -mfpu=fpv4-sp-d16 -mlong-calls -Os -ffunction-sections -fdata-sections -Wall -mthumb -nostdlib --param max-inline-insns-single=500 -mcpu=cortex-m4 -Wno-expansion-to-defined -DPLATFORMIO=40301 -DARDUINO_ARCH_SAMD -DARDUINO_FEATHER_M4 -DADAFRUIT_FEATHER_M4_EXPRESS -D__SAMD51J19A__ -D__SAMD51__ -D__FPU_PRESENT -DARM_MATH_CM4 -DENABLE_CACHE -DVARIANT_QSPI_BAUD_DEFAULT=50000000 -DGUPPY -DF_CPU=120000000L -DUSBCON -DARDUINO=10805 -DUSB_VID=0x239A -DUSB_PID=0x8031 "-DUSB_PRODUCT=\"Adafruit Feather M4\"" -DUSB_MANUFACTURER=\"Adafruit\" -DUSB_CONFIG_POWER=100 -Isrc -Ilib\ArduinoJson\src -I.pio\libdeps\guppy\Eigen 
-Ilib\ssi_adc\src -IC:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\libraries\SPI -IC:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\libraries\Adafruit_ZeroDMA -IC:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\libraries\Adafruit_ZeroDMA\utility -Ilib\SSISD -Ilib\FreeRTOS\src -Ilib\FreeRTOS\src\Source\include -Ilib\FreeRTOS\src\Source\portable\GCC\ARM_CM4F -Ilib\FreeRTOS\config -Ilib\SSISD\diskio -Ilib\SSISD\sd_mmc -Ilib\SSISD\FatFs\source -Ilib\SSISD\misc -Ilib\SSISD\include -Ilib\SSISD\hri -Ilib\SSISD\hal\include -Ilib\SSISD\hpl\port -Ilib\SSISD\hal\utils\include -Isrc\board\guppy -IC:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\cores\arduino -IC:\Users\timv\.platformio\packages\framework-cmsis\CMSIS\Include -IC:\Users\timv\.platformio\packages\framework-cmsis-atmel\CMSIS\Device\ATMEL -IC:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\cores\arduino\TinyUSB -IC:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\cores\arduino\TinyUSB\Adafruit_TinyUSB_ArduinoCore -IC:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\cores\arduino\TinyUSB\Adafruit_TinyUSB_ArduinoCore\tinyusb\src -IC:\Users\timv\.platformio\packages\framework-arduino-samd-adafruit\variants\feather_m4 src\board\guppy\System.cpp

Timvrakas · March 21, 2020, 8:47pm

Now I’m quite confused. I found this line

github.com

platformio/platformio-core/blob/00a9a2c04d3cfb6bffffd6a6cad97b92ae9cd979/platformio/builder/tools/platformio.py#L147


      
              **{
                  key: project_lib_builder.env.get(key)
                  for key in ("LIBS", "LIBPATH", "LINKFLAGS")
                  if project_lib_builder.env.get(key)
              }
          )
          
          
projenv = env.Clone()
          
          
# CPPPATH from dependencies
          projenv.PrependUnique(CPPPATH=project_lib_builder.env.get("CPPPATH"))
          # extra build flags from `platformio.ini`
          projenv.ProcessFlags(env.get("SRC_BUILD_FLAGS"))
          
          
is_test = "__test" in COMMAND_LINE_TARGETS
          if is_test:
              projenv.BuildSources(
                  "$BUILD_TEST_DIR", "$PROJECT_TEST_DIR", "$PIOTEST_SRC_FILTER"
              )
          if not is_test or env.GetProjectOption("test_build_project_src"):
              projenv.BuildSources(

which suggests that this is not a bug, but by design. I think I can fix the issue by overriding what files are included at what stage, but I’m not sure I understand why all library header files would be prepended into the project env.

EDIT/MORE:

Alright, a lot of me posting in one day, but I’ve made progress.

I can trim down my include search paths for the whole project by adding this to the post.py script:

Import('projenv')
projenv.ProcessUnFlags("\
-Ilib/SSISD/src/diskio \
-Ilib/SSISD/src/sd_mmc \
-Ilib/SSISD/src/FatFs/source \
-Ilib/SSISD/src/misc \
-Ilib/SSISD/src/include \
-Ilib/SSISD/src/hri \
-Ilib/SSISD/src/hal/include \
-Ilib/SSISD/src/hpl/port \
-Ilib/SSISD/src/hal/utils/include")

Those directories contain headers that are needed to compile objects in the library, but aren’t required for any code in the project (or libraries that depend on SSISD) to compile. So I can remove them from the include path list for most of the project.

I’m starting to realize that this is somewhat niche requirement. The less hacky alternative is to change the header file include directives from, for example:

#include "hpl_adc_sync.h"

to:

#include "hal/include/hpl_adc_sync.h"

This works, and prevents the header conflicts I’m worried about, but it requires modifying almost every file, and for external libraries that’s not a great option.

I think some way to specify this via JSON, instead of via script, would be nice. for example:

{
    "name": "SSISD",
    "build":{
        "flags":
        [
            "-Isrc/diskio",
            "-Isrc/sd_mmc",
            "-Isrc/FatFs/source",
            "-Isrc/include",
            "-Isrc/hri",
            "-Isrc/hal/include",
            "-Isrc/hpl/port",
            "-Isrc/hal/utils/include"
        ],
       "flags_external:
       [
       "-Isrc",
       "-Isrc/misc"
       ]
    }
}

Where the flags are used when compiling objects within the library, and external flags are exported to the build environment of the entire project.

ivankravets · March 22, 2020, 11:04am

Yes, this is correct workaround. Also, you can pass these includes directly via platformio.ini.

Timvrakas · March 22, 2020, 10:38pm

I’d like to be able to set up this configuration from the library itself rather than from the global config/script. But for now this solution works. The other option would be to just hardcore all my include paths, which might be the right way to do this anyway.

Thanks for all the help!