No Such File or Directory with ESP32 Project

ajs4392 · March 9, 2021, 3:59am

Hi, first I just want to thank everyone for their work on PlatformIO. It’s a really great tool and I enjoy learning how to use it.

I’m trying to incorporate PlatformIO into a Samsung SmartThings project for a friend (link to source code). When I build the project with pio run, I get the following error message:

components/ser2sock/ser2sock.cpp:53:10: fatal error: alarmdecoder_main.h: No such file or directory

As you’ll see in the source code, that file exists in the include directory. I’m not very well-versed in building C++ projects. Does anyone know what may be causing this? It seems that it could be some sort of circular dependency since the libraries in lib_dir are including files from the include directory, though this could be totally incorrect.

platformio.ini

[platformio]
src_dir = main
lib_dir = components

[env:esp32dev]
platform = espressif32@1.12.4
board = esp32dev
framework = espidf
monitor_speed = 115200

Build Error

Processing esp32dev (platform: espressif32@1.12.4; board: esp32dev; framework: espidf)
----------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (1.12.4) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-espidf 3.40001.200521 (4.0.1) 
 - tool-cmake 3.16.4 
 - tool-esptoolpy 1.20600.0 (2.6.0) 
 - tool-ninja 1.7.1 
 - toolchain-esp32ulp 1.22851.191205 (2.28.51) 
 - toolchain-xtensa32 2.80200.200827 (8.2.0)
Reading CMake configuration...
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 6 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <otaupdate>
|-- <pushover>
|-- <ser2sock>
|-- <stsdk>
|   |-- <otaupdate>
|-- <twilio>
|-- <alarmdecoder-api>
Building in release mode
Archiving .pio/build/esp32dev/bootloader/esp-idf/efuse/libefuse.a
Indexing .pio/build/esp32dev/bootloader/esp-idf/efuse/libefuse.a
Archiving .pio/build/esp32dev/bootloader/esp-idf/log/liblog.a
Indexing .pio/build/esp32dev/bootloader/esp-idf/log/liblog.a
Archiving .pio/build/esp32dev/bootloader/esp-idf/xtensa/libxtensa.a
Indexing .pio/build/esp32dev/bootloader/esp-idf/xtensa/libxtensa.a
Compiling .pio/build/esp32dev/lib94b/otaupdate/ota_util.o
components/otaupdate/ota_util.cpp:48:10: fatal error: alarmdecoder_main.h: No such file or directory

***************************************************************************
* Looking for alarmdecoder_main.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:alarmdecoder_main.h"
* Web  > https://platformio.org/lib/search?query=header:alarmdecoder_main.h
*
***************************************************************************

 #include "alarmdecoder_main.h"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Compiling .pio/build/esp32dev/lib866/pushover/pushover.o
Compiling .pio/build/esp32dev/lib342/ser2sock/ser2sock.o
components/pushover/pushover.cpp:54:10: fatal error: alarmdecoder_main.h: No such file or directory

***************************************************************************
* Looking for alarmdecoder_main.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:alarmdecoder_main.h"
* Web  > https://platformio.org/lib/search?query=header:alarmdecoder_main.h
*
***************************************************************************

 #include "alarmdecoder_main.h"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Compiling .pio/build/esp32dev/lib50f/stsdk/caps_battery.o
Compiling .pio/build/esp32dev/lib50f/stsdk/caps_button.o
*** [.pio/build/esp32dev/lib94b/otaupdate/ota_util.o] Error 1
*** [.pio/build/esp32dev/lib866/pushover/pushover.o] Error 1
components/ser2sock/ser2sock.cpp:53:10: fatal error: alarmdecoder_main.h: No such file or directory

***************************************************************************
* Looking for alarmdecoder_main.h dependency? Check our library registry!
*
* CLI  > platformio lib search "header:alarmdecoder_main.h"
* Web  > https://platformio.org/lib/search?query=header:alarmdecoder_main.h
*
***************************************************************************

 #include "alarmdecoder_main.h"
          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
*** [.pio/build/esp32dev/lib342/ser2sock/ser2sock.o] Error 1
============================================= [FAILED] Took 7.09 seconds =============================================
The terminal process "platformio 'run'" terminated with exit code: 1.

Terminal will be reused by tasks, press any key to close it.

maxgerhardt · March 9, 2021, 10:38am

I see your component folder is your library folder per config

github.com

ajschmidt8/AlarmDecoder-STSDK/blob/014aaea3c947bf77caecde6c16e48579a035771f/platformio.ini#L1-L3


      
          [platformio]
          src_dir = main
          lib_dir = components

When libraries are build, they do not see the include/ folder of the main project, if not explicitly configured for it (include/ only seen by files in src/ per default). For that you have to add build_flags = -I include to make it globally visible.

Further I saw that PIO wasn’t able to detect the otaupdate’s library dependency on alarmdecoder-api, so a more aggressive lib_ldf_mode must be used, lib_ldf_mode = chain+.

Doing that gets the project to the linking stage, but, …

Linking .pio\build\esp32dev\firmware.elf
c:/users/max/.platformio/packages/toolchain-xtensa32@2.80200.200827/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32dev\libff9\libotaupdate.a(ota_util.o):(.literal.ota_https_read_version_info+0x4): undefined reference to `_binary_update_root_pem_start'
c:/users/max/.platformio/packages/toolchain-xtensa32@2.80200.200827/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32dev\libff9\libotaupdate.a(ota_util.o):(.literal.ota_https_update_device+0x54): undefined reference to `_binary_update_public_key_pem_end'
c:/users/max/.platformio/packages/toolchain-xtensa32@2.80200.200827/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: .pio\build\esp32dev\libff9\libotaupdate.a(ota_util.o):(.literal.ota_https_update_device+0x58): undefined reference to `_binary_update_public_key_pem_start'
collect2.exe: error: ld returned 1 exit status
*** [.pio\build\esp32dev\firmware.elf] Error 1

It can’t find the binary data for update_root.pem and update_public_key.pem. Those have to be added via a embed_txtfiles directives and the CMakeLists.txt. However, a technicality of that you’re using ESP-IDF as components by declaring that to the lib_dir is that we have to write those directives into main/CMakeLists.txt since for libraries, the CMakeLists.txt is not process (and in your fork also deleted).

So with an addition of

target_add_binary_data(${COMPONENT_TARGET} "../components/otaupdate/update_public_key.pem" TEXT)
target_add_binary_data(${COMPONENT_TARGET} "../components/otaupdate/update_root.pem" TEXT)

at the end of main/CMakeLists.txt and board_build.embed_txtfiles those files are included.

Finally, you have to reference partition file partitions.2MB.csv in the platformio.ini, otherwise it won’t get used. See docs.

All that with a final platformio.ini of

[platformio]
src_dir = main
lib_dir = components

[env:esp32dev]
platform = espressif32@1.12.4
board = esp32dev
framework = espidf
monitor_speed = 115200
build_flags = -I include
lib_ldf_mode = chain+
board_build.embed_txtfiles =
  components/otaupdate/update_public_key.pem
  components/otaupdate/update_root.pem
board_build.partitions = partitions.2MB.csv

and recompiling…

Linking .pio\build\esp32dev\firmware.elf
Building .pio\build\esp32dev\firmware.bin
Retrieving maximum program size .pio\build\esp32dev\firmware.elf
Checking size .pio\build\esp32dev\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  13.0% (used 42628 bytes from 327680 bytes)
Flash: [=======   ]  70.1% (used 1077182 bytes from 1536000 bytes)
esptool.py v2.6
======================== [SUCCESS] Took 54.07 seconds ========================

works.

But again because you’re using ESP-IDF components as libraries (and thus this ignores the KConfig and the CMakeLists.txt, this is not optimal. I’ll see if there’s a prettier way.

maxgerhardt · March 9, 2021, 10:59am

Yeah I think a way more natural way to add PIO build support with less intrusive changes (like removing CMakeLists.txt files etc) and better compatibility (components are now components and not libraries), is doing

[platformio]
src_dir = main
include_dir = main

[env:esp32dev]
platform = espressif32@1.12.4
board = esp32dev
framework = espidf
monitor_speed = 115200
build_flags = -I main
board_build.partitions = partitions.2MB.csv
board_build.embed_txtfiles =
  components/otaupdate/update_public_key.pem
  components/otaupdate/update_root.pem
  components/pushover/pushover_root.pem
  components/twilio/twilio_root.pem
  components/twilio/sendgrid_root.pem
  components/stsdk/device_info.json
  components/stsdk/onboarding_config.json

See my fork at GitHub - maxgerhardt/AlarmDecoder-STSDK: AlarmDecoder IoT direct connected SmartThings SDK device.

The stsdk referenced $ENV{STDK_CORE_PATH} which I didn’t want to put in my environment variables, so I just cloned the repo locally in the project and modified the path it to make it truly stand-alone – one might want to undo or rethink that though too.

Compilation works ofc, too.

Checking size .pio\build\esp32dev\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  14.2% (used 46548 bytes from 327680 bytes)
Flash: [=======   ]  74.2% (used 1140466 bytes from 1536000 bytes)

And now it also shows properly in the menuconfig.

(and under components config)

ajs4392 · March 9, 2021, 2:02pm

@maxgerhardt, awesome! thank you for your help here. Your fork built for me as well. Appreciate your thorough reply. I will have to review everything to make sure I understand it all.