Conflicting build systems when finding components

Development Platforms
,
1

Hey wise people,

I am sure I am being super stupid, but cmake is a mortal enemy of mine and I am trying to hack together some stuff so that I can properly get to grips with the espidf build system as well as modularising my code and adding some build time ergonomics (at least that is the plan)! :frowning:

I am using frameworks espidf and arduino in my platformIO.ini. When I run idf.py reconfigure and idf.py build my project successfully compiles, but when I compile with pio run -e <MY_ENV> it fails unable to find build_info.h in main.cpp, however, idf seems able to find it…..?

For context, the build_info.h is supposed to be a generated file that I create, kinda like the following:

I made a cmake “module” in components/ as directed in espidf. Slung a CMakeLists.txt in it: which looks kinda like this (this is only supposed to be a test at the moment whilst I get to grips with things):

# Hacky Cmake Test to Generate build_info.h

# 1. Register component (no include dirs yet)

# 2. Generated header goes in build dir (NOT source dir)

set(BUILD_INFO_HEADER ${CMAKE_CURRENT_BINARY_DIR}/build_info.h)

idf_component_register(

    # SRCS ${BUILD_INFO_HEADER}

    SRCS dummy.cpp




)





#/workspace/build/esp-idf/build_info/build_info.h

message(STATUS " BUILD_INFO_HEADER has value ${BUILD_INFO_HEADER} ")




# 3. Generate it using a CMake script (portable)

add_custom_command(

    OUTPUT ${BUILD_INFO_HEADER}

    COMMAND ${CMAKE_COMMAND}

        -DOUTPUT_FILE=${BUILD_INFO_HEADER}

        -DPROJECT_ROOT=${PROJECT_ROOT}

        -P ${CMAKE_CURRENT_SOURCE_DIR}/generate_build_info.cmake

    # DEPENDS ${CMAKE_SOURCE_DIR}/.git/HEAD

    COMMENT "Generating build_info.h"

)





set_source_files_properties(${BUILD_INFO_HEADER} PROPERTIES GENERATED TRUE)




# 4. Create a target for it

add_custom_target(build_info_runner DEPENDS ${BUILD_INFO_HEADER})




# 5. Ensure this component depends on it

add_dependencies(${COMPONENT_LIB} build_info_runner)




# 6. Expose the generated header to dependents

target_include_directories(${COMPONENT_LIB} INTERFACE

    ${CMAKE_CURRENT_BINARY_DIR}

)

This calls a .cmake script which simply does a kinda python subprocesses thing to grab the current git hash at the project root. I have ommitted this since I didn’t want to overwhelm people and am convinced this isn’t the issue since it works under espidf.

Then I need to declare that I depend upon this components/build_info. So, in <PROJECT_ROOT>/src/CMakeLists.txt:

# Register this as an ESP-IDF component

idf_component_register(

    SRCS ${app_sources}

    INCLUDE_DIRS .

    REQUIRES build_info 

)

TLDR: Can’t understand why idf.py commands build the project just fine with the above set up with a declared dependency on a component as outlinde in the espidf build system docs, but platformIO then can’t find the generated files in main.cpp (or anywhere) where they are included.

2

If you have a minimal reproducable ESP-IDF project that builds with idf.py but doesn’t with PlatformIO, please provide the exact project files. If there are no mistakes in the platformio.ini, then it qualifies as a bug for https://github.com/platformio/platform-espressif32/issues/.

3

Hey, thanks for the pointer…

After tearing my hair out a fair bit I am very stuck with how to work with the cmake and espidf. There were some errors in what I originally wrote, namely after a day of getting confused and going around in circles, the cmake apparently isn’t working on either pio or with idf.py (sorry for communicating the wrong thing).

I am trying to just get a simple working example set up so I can experiment and get a hands on feel for this. I have read the platformIO docs for espidf framework and the espidf ones as well as the dev talk on the build system on YouTube. I wonder if I have missed something crucial or perhaps I just don’t get this.

Currently, my situation is this:

  1. I have a pio project with src_dir pointing to main. I renamed it so that I could build the project in espidf too (since espidf seems to insist on its own pre-defined layout)

    [platformio]

src_dir = main

  2. In PROJECT_DIR/main I have a load of subfolders, each pertaining to different peripherals and FreeRTOS task set up. They are very ugly and do a lot of #include ../blah/blah.h. I also have PROJECT_DIR/main/main.cpp. I want to get away from this ugly layout and modularise things properly into espidf components.

  3. My CMakeLists.txt at the root of the project looks like this:

    cmake_minimum_required(VERSION 3.16.0)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)

project(<PROJECT_NAME>)



set(PROJECT_ROOT ${CMAKE_CURRENT_SOURCE_DIR} CACHE PATH "Root of the project") # This is currently being used as the working dir of one of my build time generating .cmake scripts

  4. The CMakeLists.txt in main/ looks like this:

    

FILE(GLOB_RECURSE app_sources ${COMPONENT_DIR}/*.*)

message(STATUS "All application sources: ${app_sources}")




idf_component_register(

    SRCS ${app_sources}

    INCLUDE_DIRS "."

    REQUIRES build_info

)

  5. in PROJECT_DIR/components I have a component called build_info. It contains a CMakeLists.txt. It looks like this:

    set(GENERATED_BUILD_DIR ${CMAKE_CURRENT_BINARY_DIR})
set(BUILD_INFO_HEADER ${GENERATED_BUILD_DIR}/build_info.h)

idf_component_register(
    SRCS dummy.cpp
    INCLUDE_DIRS ${GENERATED_BUILD_DIR}
)

# Always-run generator
add_custom_target(generate_build_info ALL
    COMMAND ${CMAKE_COMMAND}
        -DOUTPUT_FILE=${BUILD_INFO_HEADER}
        -DPROJECT_ROOT=${PROJECT_DIR}
        -P ${CMAKE_CURRENT_SOURCE_DIR}/generate_build_info.cmake
    BYPRODUCTS ${BUILD_INFO_HEADER}   # optional, helps CMake know the file exists
    VERBATIM
)

# Ensure component builds after header is generated
add_dependencies(${COMPONENT_LIB} generate_build_info)

# Expose include directory for consumers
target_include_directories(${COMPONENT_LIB} PUBLIC ${GENERATED_BUILD_DIR})

# Mark the file as GENERATED so CMake doesn’t warn
set_source_files_properties(${BUILD_INFO_HEADER} PROPERTIES GENERATED TRUE)

# Export path for other components
set(BUILD_INFO_HEADER_PATH ${BUILD_INFO_HEADER} CACHE INTERNAL "Path to build_info.h")

  6. The key bit is that I am trying to do a basic example to get to grips with this build system. The idea is that this will run the platform independent .cmake file every time at BUILD time not CONFIGURATION time. This is because the build_info.h header is supposed to include details like git commit and the datetime of compilation and make it available to anything that depends on it.For reference the .cmake file looks like this:

  7. 
execute_process(
    COMMAND git rev-parse --short HEAD
    WORKING_DIRECTORY ${PROJECT_ROOT}
    OUTPUT_VARIABLE GIT_COMMIT
    OUTPUT_STRIP_TRAILING_WHITESPACE
    ERROR_QUIET
)

# Get build-time timestamp (THIS is now build-time, not configure-time)
# this does appear to get generated at build time and not configure time AS LONG AS THE target file is not there
string(TIMESTAMP BUILD_TIME "%Y-%m-%d %H:%M:%S")

message(STATUS "Running build_info generation in script mode. Targetting git command in ${PROJECT_ROOT}")
message(STATUS "Output file: ${OUTPUT_FILE}")
message(STATUS "CMAKE_SOURCE_DIR: ${CMAKE_SOURCE_DIR}")

if(NOT GIT_COMMIT)
    set(GIT_COMMIT "UNKNOWN")
endif()

file(WRITE ${OUTPUT_FILE}
"#pragma once\n"
"#define BUILD_GIT_COMMIT \"${GIT_COMMIT}\"\n"
"#define BUILD_TIME \"${BUILD_TIME}\"\n"
)

  8. If we build in a devcontainer using idf.py and look at the build/ dir and at the compile commands.json we can see that associated to main.cpp is an -I/workspace/build/esp-idf/build_info:

    -I/workspace/build/esp-idf/build_info -mlongcalls -Wno-frame-address  -fno-builtin-memcpy -fno-builtin-memset -fno-builtin-bzero -fno-builtin-stpcpy -fno-builtin-strncpy -ffunction-sections -fdata-sections -Wall -Werror=all -Wno-error=unused-function -Wno-error=unused-variable -Wno-error=unused-but-set-variable -Wno-error=deprecated-declarations -Wextra -Wno-error=extra -Wno-unused-parameter -Wno-sign-compare -Wno-enum-conversion -gdwarf-4 -ggdb -Og -fno-shrink-wrap -fmacro-prefix-map=/workspace=. -fmacro-prefix-map=/home/vscode/esp/esp-idf=/IDF -fstrict-volatile-bitfields -fno-jump-tables -fno-tree-switch-conversion -std=gnu++2b -fno-exceptions -fno-rtti -fuse-cxa-atexit -o esp-idf/main/CMakeFiles/__idf_main.dir/main.cpp.obj -c /workspace/main/main.cpp",

  "file": "/workspace/main/main.cpp"

  9. If we look in /workspace/build/esp-idf/build_info we can find build_info.h with full read and write permission for everyone. When we view it, it is correct:

    vscode@ef9b1cc41195:/workspace$ cat build/esp-idf/build_info/build_info.h
    #pragma once
    #define BUILD_GIT_COMMIT “c7a5c10”
    #define BUILD_TIME “2026-03-25 10:34:55”
    vscode@ef9b1cc41195:/workspace$

and yet when we try and compile:

main/main.cpp:19:10: fatal error: build_info.h: No such file or directory




Looking for build_info.h dependency? Check our library registry!



CLI  > platformio lib search “header:build_info.h”

Web  > https://registry.platformio.org/search?q=header:e[me[Kbuild_info.h






19 | #include “build_info.h”
|          ^~~~~~~~~~~~~~
compilation terminated.

I’m at a bit of a loss. If the cmake is correct and the dependency is registered and I have propagated the include dir and the file is there then….I don’t understand what could be causing this?

I have tried quite a bit of googling but can’t understand how this is possible…the file is being generated and it is there…so how can it be unavailable to the include?