Problem using a custom ESP32 Framework

PlatformIO Core
1

I’m trying to use a custom ESP32 framework that I’ve created for one of my projects but also use the standard ESP32 framework for other projects. I’m running PlatformIO under Windows. I created the custom ESP32 framework by building my own Arduino ESP32 libraries and copying those over the existing Arduino ESP32 framework files, modifying the package.json file (to new version number). I then created a new repository to put this custom framework in. The problem is when installing the custom framework it initially overwrites the standard Arduino ESP32 framework and when I switch to a project that uses the standard framework it renames my custom framework to something like framework-arduinoespressif32@src-d361349f05f9ed119dcb8461c90e20e7 where that is the full commit hash from the last commit to the repository. I’ll post a reply here (since I don’t get the code formatting button when creating a topic) with the platform.ini and package.json contents that I am using.

2

Well lovely, I don’t seem to get the code formatting option here in the reply either, oh well. Here is the platform.ini my project is using for the custom ESP32 framework:

; PlatformIO Project Configuration File
;
; Build options: build flags, source filter
; Upload options: custom upload port, speed and extra flags
; Library options: dependencies, extra library storages
; Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html
[platformio]
default_envs = Inkplate6
description = Inkplate Weather Station Project
[env:Inkplate6]
platform = espressif32
board = Inkplate6
board_build.partitions = dual.csv
framework = arduino
; Uncomment the following line to build with custom Arduino Espressif32 framework with LWIP & BLE buffers in PSRAM
platform_packages = platformio/framework-arduinoespressif32 @ https://github.com/rspurlock/framework-arduinoespressif32#v4.4
; Uncomment the following line to set the ESP32 core debug level
;build_flags = -DCORE_DEBUG_LEVEL=5
monitor_speed = 115200
monitor_filters = esp32_exception_decoder
lib_deps =
  https://github.com/rspurlock/Inkplate-Arduino-library
  bblanchon/ArduinoJson@^6.21.2

Here is the updated package.json file that I updated before uploading my custom framework to the repository:

{
  "name": "framework-arduinoespressif32",
  "version": "4.4",
  "description": "Arduino Wiring-based Framework for the Espressif ESP32, ESP32-S and ESP32-C series of SoCs",
  "keywords": [
    "framework",
    "arduino",
    "espressif",
    "esp32"
  ],
  "license": "LGPL-2.1-or-later",
  "repository": {
    "type": "git",
    "url": "https://github.com/rspurlock/framework-arduinoespressif32"
  }
}

I have even tried updating the .piopm file to this to see if it made a difference (it didn’t).

{"type": "tool", "name": "framework-arduinoespressif32", "version": "4.4", "spec": {"owner": "rspurlock", "id": null, "name": "framework-arduinoespressif32", "requirements": null, "uri": null}}

The worst thing this problem causes is that with the framework name being as long as it is it causes the Windows command line for building to be longer than 32k characters which fails to create a process, i.e. you can no longer build until you delete the long named framework and let it download again (which goes over the top of the existing standard framework that is there).

3

Could you replace with platform_packages = framework-arduinoespressif32?

4

Thanks Ivan, I deleted both arduinoespressif32 folders in the packages folder and updated the line in platformio.ini for the project using the custom framework. When I saved the platform.ini it downloaded the custom framework but it did it to an arduinoepressif32 folder (without the version number in the folder name). Then switching to a project without the custom framework it installed the standard Epressif32 framework and renamed the custom folder to framework-arduinoespressif32@src-d361349f05f9ed119dcb8461c90e20e7 again. Is there something else I can try? I tried to see if there was some Python code downloading/changing the folder name but I wasn’t able to find anything.

5

I don’t think that’s a problem, that’s just how PlatformIO works. Even when your platform folder is renamed, when you build the PlatformIO project that references your custom package, it will properly use the paths to where those files are stored.

6

But it is a problem under Windows as with that long a folder name the command line exceeds 32k characters and builds no longer work. I was hoping there was some way to do something like the SAMD framework does and have the folder name something like arduinoespressif32@version (instead of with the full commit ID, even the shorted 6 digit commit ID might make it functional on Windows).

7

When the command line exceeds 32K characters, PlatformIO is supposed to save the command to a file and have the compiler execute the commands stored in that file; that bypasses that limit. That’s supposed to happen automatically. You’re saying in your case it doesn’t and it breaks compilation?

8

Yes, that appears to be the case. I’ll try to get more information on the failure to build and share that here later today. Thanks for the update on how it is supposed to work.

9

You can cross check that it’s building on Linux (which doesn’t have that path length limitation) by e.g. uploading the project to github and using Github Actions (save https://docs.platformio.org/en/latest/integration/ci/github-actions.html#using-cmd-run-command as .github/workflows/build.yml) to make a Linux machine build it.

10

Hi Max, sorry for the delay. After a bit of digging the problem actually lies in the xtensa-esp32-elf-g++ compiler. It apparently can’t handle really long include paths and fails when attempting to spawn an additional process. I was able to save the temporary file used by PlatformIO and re-create the same failure using an “extra script” to invoke the compiler with the saved temp file (it generates the same CreateProcess failure). I then edited the temp file include paths to remove the @src-commitID and doing that allows the Xtensa compiler to function properly (so it’s not actually a PlatformIO issue but a tools issue under Windows). Is there anyway that I can get the folder name shortened to something else besides using the long commit ID so that the compiler tool doesn’t have the problem actually compiling my project? Thanks for helping me debug this issue.

11

Might be related to this:

12

This could be related but I’ve narrowed it down to the Xtensa compiler failing trying to spawn a process (probably related to include path length). I’m trying to get a custom ESP32 Arduino framework that I built installed in a shorter folder name so the include path isn’t so long. I only want/need the custom framework on one project and trying to switch between projects causes the framework folder to be renamed to include a long name with something like a GUID appended to it (which then causes the long include path failure when switching back to the project using that custom framework).